C#: End-to-End PowerBI Embedding - "App Owns Data"

In working with a number of developer-heavy customers, there seems to be a strong demand for taking the reports and dashboard developed in PowerBI and publishing them in either Intranet or Internet web pages.  This concept is currently known as PowerBI Embedded and comes in several different options for developers to implement.

Licensing and Disclaimer

Before we start, I will note that there are definitely licensing concerns and implications to choices made as a part of this tutorial.  While I could likely articulate the current rules surrounding PowerBI licensing, there are a variety of options and I am not truly qualified to explain all of these nuances. . . .so I won't even try.

PLEASE seek assistance from your Microsoft account team if you choose to Embed PowerBI in your applications to determine what licensing model works best for your organization.

Additionally, technologies frequently change.  I will make every attempt to update this tutorial as functionality and technology changes, however for the most current up-to-date information, you should always refer to the Microsoft PowerBI developer documentation, and never any individual tutorial (including this one).

Step 0: Make Some Choices

Before we begin, lets discuss some of the options available to developers who are trying to embed their report.  At it's simplest, developers can place their PowerBI reports in a web page by using an HTML IFRAME on their site and ensuring they have a pointer to the correct PowerBI report.  Here are a simplified list of options:

Option A - Publish to Web

The simplest way to get a PowerBI report into your website is to use the "Publish to Web" option in PowerBI.com.  Just publish your report from the PowerBI client application, and from the File Menu choose "Publish to Web".  This is an entirely public method of publishing though, and there is NO way to lock down ANYTHING.

Option B - User Owns Data

The second option is currently termed "User Owns Data", and in this model, each user who logs into your website with a valid Azure AD account will be allowed to see what they're permitted to in any given PowerBI Report.  Great for Intranet sharing where you know all of your users and trust their Azure AD sources.

Option C - App Owns Data - FOCUS OF THIS TUTORIAL

The third option is currently known as "App Owns Data" and leverages what is essentially a single service account to publish the report as that service account user.  Great for situations where you are publishing a report on a public website where users will not be logging in before they see your PowerBI report. 

Assuming you chose to go with Option C above, the rest of this tutorial will walk through how to implement this.  You can also follow the tutorial linked in the Option C section above, but these steps should simplify the official tutorial some more.

Step 1: Develop a PowerBI Report

While it may seem obvious, this first step is important to leveraging some of the tools to their greatest extent.  You can change your specific report later, however for the purpose of following the PowerBI Embedding wizard, it's easiest to have a .pbix file on hand as you embed your first report.
The easiest way to develop your first report is to download the PowerBI client and create a report to embed.  If you haven't done so already, you may download the PowerBI client from powerbi.com.

Step 2: Obtain a Service Account

This approach assumes that a service account user will be logging into PowerBI on behalf of your application, therefore, you will need to have a fully licensed PowerBI user [with password] created. Speak with your organization's Azure AD administrator and have them authorize the new user for PowerBI in the Office Portal (portal.office.com). 

Step 2a -Test Logging In with Service Account

Just to be sure your user was provisioned correctly (and has access to PowerBI), PLEASE login to Powerbi.com with your new user and ensure they can use or see a report (ideally the one you created above in Step 1).

Step 3: Use the Wizard!

The PowerBI product group recently developed an amazing wizard to walk users through the process of creating and embedding reports in your website.  Navigate to the wizard at https://app.powerbi.com/embedsetup and choose the option labeled "Embed for Customers".  If you are using PowerBI Government, the same wizard is available to you at https://app.powerbigov.us/embedsetup.

The following are some important notes for as you walk through the wizard:

Step	Note
Step 1 - Sign In	Sign in with your new service account. If you do not login with your service account, your "user" will never have the chance to accept terms and conditions (Step 5), and you will get some strange unauthorized error in your code.
Step 2 - Register Your Application	The PowerBI report you are creating needs to be registered in Azure Active Directory so that OAuth can be used to verify your user has rights to the report being published. The official Microsoft tutorial goes into great detail on what this means and why, but for now, just give your application a name so that it gets permission to use the embedded objects.
Step 3 - Create a Workspace	Even though you probably already have a PowerBI workspace, just let the application create one for you. Your code can be updated later.
Step 4 - Import Content	Same as above. Even though you've probably already created a report, just for now, provide it a .PBIX file so that it creates another one for the sample source code that comes out of the wizard.
Step 5 - Grant Permissions	Since this service account is acting like a user, and every user needs to accept terms and conditions when using Azure AD for Oauth. . . .you'll need to allow your application and user to be related with one another in this step.

Upon completion, you should see an option to download sample code.  Download this .zip file and unzip it somewhere you can open it in Visual Studio 2015 or higher.

Step 4: Modify the Sample Solution Code

Open the .sln file you just downloaded in Visual Studio and navigate to the web.config.

Step 4a - Add Password to Web Config

The solution should have everyhing you need. . . EXCEPT a password.  While we should NOT publish this code (nor check it into source control) with a password in our code, for now we just want to be sure everything is working.  Add your password to the AppSettings section where you see the blank.

Step 4b -  Restore NuGet Packages

From the Tools Menu, navigate to the NuGet Package Manager and choose the option to restore packages not currently in the solution.

Step 4c - Update NuGet Packages (Optional)

If you will be publishing your work to a modern web server, you will likely want to update your packages to ensure any vulnerability and server-side incompatibilities are addressed before being published.  

Step 4d - Build and Run

If it worked, you should see a set of 3 buttons. . . if you pick Embed Report, the following should show without an error:

Step 5: Secure Your Identity

Now that we know it works, you will now want to go through and hide your service account's login in something other than plain text in the web.config file.

In the next article we will go through the options for securing your identity if you are using an Azure Web Service.  Stay tuned!

Schematron: Validating NIEM Documents Against Non-Conformant Code Lists

Schematron rules and assertions are based upon XPath statements, which allow for a number of powerful XML querying capabilities. Two XPath capabilities leveraged and outlined in this section are doc() and XPath predicates which allow us to validate data captured in an NIEM XML instance against external code list of any kind.

Lets assume a scenario where we would like to validate an exchange document’s category against a predefined list of enumerated values.  This list is maintained by an outside party in a format other than NIEM and changes on a fairly regular basis. 

Traditionally, a NIEM practitioner would take this list and define an enumeration within an extension schema to enforce this code list.  Each time the third party makes a change to that code list, an updated NIEM extension schema would be created and redistributed.  This maintenance-intensive process could become overwhelming therefore the team chose instead to simply adopt the third-party list and keep it in the following non-conformant format relying instead on Schematron to perform the validation:

<?xml version="1.0" encoding="UTF-8"?>
<!-- List of Valid code Values -->
<CategoryList>
  <Category>a</Category>
  <Category>b</Category>
</CategoryList>

As shown in the above, the valid categories include the values “a” and “b”.  An example of a NIEM-conformant XML payload would look something like the following:

<ns:SomeDocument 
    xmlns:nc="http://niem.gov/niem/niem-core/2.0"    
    xmlns:ns="http://www.niematron.org/SchematronTestbed"
    schemaLocation"http://www.niematron.org/SchematronTestbed  ./SomeDocument.xsd">
  <nc:DocumentCategoryText>A</nc:DocumentCategoryText>
  <!-- Remaining Document Elements Omitted -->
</ns:SomeDocument>

In this example, the developers would like to perform the validation ignoring case, therefore the Schematron rule to validate the nc:DocumentCategoryText against the third-party-provided list would look something like the following:

<pattern id="eDocumentCategory">
  <title>Verify the document category matches the external list of valid categories.</title>
  <rule context="/ns:SomeDocument">
    <let name="sText" value="lower-case(nc:DocumentCategoryText)"/>
    <assert test="count(doc('./CategoryList.xml')/CategoryList/Category[. = $sText]) &gt; 0">
      Invalid document category.
    </assert>
  </rule>
</pattern>

Lets look at some of the key statements in the above Schematron example breaking it into individual parts. 

• lower-case(nc:DocumentCategoryText) – This statement encapsulated in a <let> tag converts the text in the NIEM payload to lower case thereby ignoring deviations from the code list due to case.  It is then stored in a temporary variable named $sText.
• doc('.CategoryList.xml')/… – This effectively points the parser at the third-party provided file (in this example assumed to be in the same directory as the .sch file) so that elements from that file can be referenced using the XPath in addition to elements in the source payload document. 
• …/Category[. = $sText] – The usage of the square brackets ([ and ])  in  an XPath statement is considered a predicate.  Any number of predicate statements can be made to help filter values contained within an XPath, but in this case, the expression tells the parser to select all of the Category elements with the value contained in the variable $sText.
• count(…) &gt; 0 – The XQuery count function returns the number of elements contained in the XPath.  If no match to the category existed, the count would return a value of zero, therefore we want to ensure the value is greater than zero meaning a match existed in the external code list.

Schematron: Correct nc:DateRepresentation Usage

The inherent flexibility of NIEM proves to be an incredibly beneficial when used correctly, however this benefit can also be one of its largest banes.  Sometimes this flexibility can lead to confusion when implementers attempt to deploy a NIEM exchange which is “valid” according to the XSD, yet not what the recipient is expecting. 

One such example is NIEM’s usage of substitution groups where a variety of data elements are legal according to the schema, but rarely are all of these legal options accounted for by the recipient’s adapter.  Take NIEM’s DateType as an example.  It employs the explicit substitution group (abstract data element) of nc:DateRepresentation which can be one of several different data types.  This representation can be replaced with a date (2009-01-01), a date/time (2009-01-01T12:00:00), a month and a year (01-2009), etc. 

Lets assume for a minute that a document has two different dates: a document filed date, and a person’s birth date.  The publisher’s intention is that filed date be a “timestamp” which includes both a date and a time, while the birth date is simply a date including a month, day and year.  A valid sample XML payload would look something like the following:

<?xml version="1.0" encoding="UTF-8"?>
<ns:SomeDocument>
  <nc:DocumentFiledDate>
    <nc:DateTime>2009-01-01T01:00:00</nc:DateTime>
  </nc:DocumentFiledDate>
  <nc:Person>
    <nc:PersonBirthDate>
      <nc:Date>1970-01-01</nc:Date>
    </nc:PersonBirthDate>
  </nc:Person>
</ns:SomeDocument>

The Schematron code to enforce the publisher’s intentions could appear as the following:

<pattern id="eDocumentDateTime">
  <title>Verify the document filed date includes a date/Time</title>
  <rule context="ns:SomeDocument/nc:DocumentFiledDate">
    <assert test="nc:DateTime">
      A date and a time must be provided as the document filed date.
    </assert>
  </rule>
</pattern>
<pattern id="ePersonBirthDate">
  <title>Ensure the person's birth date is an nc:Date.</title>
  <rule context="ns:SomeDocument/nc:Person/nc:PersonBirthDate">
    <assert test="nc:Date">
      A person's birth date must be a full date.
    </assert>
  </rule>
</pattern>

This is a great example of how Schematron can help clarify a publisher’s intent as NEIM-conformant services are developed and deployed.

Link: Schematron Tutorials

There are a number of solid Schematron tutorials posted on the internet.  Two of most comprehensive ones available are:

• The Oasis-Hosted Coverpages.com Schematron Article.
• Roger Costello’s Tutorial 

We won’t plagiarize their incredible wealth of Schematron material here, so it is highly recommend developers jump to and use these online tutorials.

Schematron: Recommended Reading

What books exists out there that detail Schematron and how to use it?

Best one I've seen to date is "Schematron" from O'Reilly Media available in electronic (Adobe PDF) format for $9.99 USD. Provides a great overview and set of examples using Schematron, albeit exclusively in hierarchically linked XML documents. It does assume the reader already has a strong understanding of XML and XSLT. If you are a bit rusty on XSLT or XQuery, you'll want to pick up a book on that as well.

XML Editor Support for Schematron

XML editors are one of the primary tools in a practitioner's toolbox and as such, are the first place many of us look for assistance using a complex validation engine such as Schematron. Currently today editors on the market have different levels of support for Schematron; as it is at the core a multi-pass validation engine. The following table should provide you with links to where tutorials for some of the most common XML editors are located:

Editor	Native Schematron Validation?	Native Schematron Editing?	Tutorial
oXygen	Yes	Yes	http://www.oxygenxml.com/validation.html
XMLSpy	No	Yes	http://xml.sys-con.com/node/40656
Editix	Yes	Yes	None available

Update: 2009-10-05 - Adding Hyperlink to Editor websites.