This is a community project that is not supported by Microsoft.
❗ This version of PBI Inspector only supports the new enhanced metadata file format (PBIR), see https://learn.microsoft.com/en-gb/power-bi/developer/projects/projects-report. For the older PBIR-legacy file format, please use the previous version of PBI Inspector available at https://github.com/NatVanG/PBI-Inspector.
To support the new enhanced report format (PBIR), a new "part" custom command has been introduced which helps to navigate to or iterate over the new metadata file format's parts such as "Pages", "Visuals", "Bookmarks" etc. Rules defined against the new format are not backward compatible with the older PBIR-legacy format and vice versa.
Thanks to Michael Kovalsky and Rui Romano for their feedback on this project. Thanks also to Luke Young for creating the PBI Inspector logo.
Please report issues here.
- Intro
- Releases
- Base rules
- Run from the Graphical user interface
- Run from the Command line
- Interpreting results
- Azure DevOps integration
- Running reports on reports
- Custom rules guide
- Patching
- Examples
- Known issues
- Report an issue
So we've DevOps, MLOps and DataOps... but why not VisOps? How can we ensure that business intelligence charts and other visuals within report pages are published in a consistent, performance optimised and accessible state? For example, are local report settings set in a consistent manner for a consistent user experience? Are visuals deviating from the specified theme by, say, using custom colours? Are visuals kept lean so they render quickly? Are charts axes titles displayed? etc.
With Microsoft Power BI, visuals are placed on a canvas and formatted as desired, images may be included and theme files referenced. Testing the consistency of the visuals output has thus far typically been a manual process. The Power BI file format (.pbip) was introduced then recently enhanced to enable pro developer application lifecycle management and source control. PBI Inspector provides the ability to define fully configurable testing rules powered by Greg Dennis's Json Logic .NET implementation, see https://json-everything.net/json-logic.
See releases for the Windows application and Command Line interface (CLI) at: https://github.com/NatVanG/PBI-InspectorV2/releases
While PBI Inspector supports custom rules, it also includes the following base rules defined at https://github.com/NatVanG/PBI-InspectorV2/blob/part-concept/Rules/Base-rules.json, some rules allow for user parameters:
- Remove custom visuals which are not used in the report (no user parameters)
- Reduce the number of visible visuals on the page (set parameter
paramMaxVisualsPerPageto the maximum number of allowed visible visuals on the page) - Reduce the number of objects within visuals (override hardcoded
6parameter value the maximum number of allowed objects per visuals) - Reduce usage of TopN filtering visuals by page (set
paramMaxTopNFilteringPerPage) - Reduce usage of Advanced filtering visuals by page (set
paramMaxAdvancedFilteringVisualsPerPage) - Reduce number of pages per report (override hardcoded
10parameter value the maximum number of allowed pages per report) - Avoid setting ‘Show items with no data’ on columns (no user parameters)
- Tooltip and Drillthrough pages should be hidden (no user parameters)
- Ensure charts use theme colours (no user parameters)
- Ensure pages do not scroll vertically (no user parameters)
- Ensure alternativeText has been defined for all visuals (disabled by default, no user parameters)
To modify parameters, save a local copy of the Base-rules.json file at https://github.com/NatVanG/PBI-InspectorV2/blob/part-concept/Rules/Base-rules.json and point PBI Inspector to the new file.
To disable a rule, edit the rule's json to specify "disabled": true. At runtime PBI Inspector will ignore any disabled rule.
Running PBIRInspectorWinForm.exe presents the user with the following interface:
- Browse to your local PBI Desktop File, either the *.pbip file or its parent folder.
- Either use the base rules file included in the application or select your own.
- Use the "Browse" button to select an output directory to which the results will be written. Alternatively, select the "Use temp files" check box to write the resuls to a temporary folder that will be deleted upon exiting the application.
- Select output formats, either JSON or HTML or both. To simply view the test results in a formatted page select the HTML output.
- Select "Verbose" to output both test passes and fails, if left unselected then only failed test results will be reported.
- Select "Run". The test run log messages are displayed at the bottom of the window. If "Use temp files" is selected (or the Output directory field is left blank) along with the HTML output check box, then the browser will open to display the HTML results.
- Any test run information, warnings or errors are displayed in the console output textbox.
All command line parameters are as follows:
-pbip filepath: Required. The path to the *.pbip file.
-pbix filepath: Not currently supported.
-rules filepath: Required. The filepath to the rules file. Save a local copy of the Base Rules file and modify as required.
-verbose true|false: Optional, false by default. If false then only rule violations will be shown otherwise all results will be listed.
-output directorypath: Optional. If -formats is set to either JSON, HTML or PNG, writes results to the specified directory, any existing files will be overwritten. If not supplied then a temporary directory will be created in the user's temporary files folder.
-formats CONSOLE,JSON,HTML,PNG,ADO: Optional. Comma-separated list of output formats.
- CONSOLE (default) writes results to the console output. If "-formats" is not specified then "CONSOLE" will be used by default.
- JSON writes results to a Json file.
- HTML writes results to a formatted Html page. If no output directory is specified and the HTML format is specified, then a browser page will be opened to display the HTML results. When specifying "HTML" format, report page wireframe images will be created so there is no need to also include the "PNG" format.
- PNG draws report pages wireframes clearly showing any failing visuals.
- ADO outputs Azure DevOps compatible task commands for use in a deployment pipeline. Task commands issued are "task.logissue" and "task.complete", see https://learn.microsoft.com/en-us/azure/devops/pipelines/scripts/logging-commands?view=azure-devops&tabs=bash#task-commands. PBI Inspector rules definition can be given a "logType" attribute of either "warning" or "error" which will be passed to the Azure DevOps task command as follows:
##vso[task.logissue type=warning|error]. When specifying "ADO" all other output format types will be ignored.
Commmand line examples:
- Run "Base-rules.json" rule definitions against PBI report file at "Sales.Report and return results in Json and HTML formats:
PBIRInspectorCLI.exe -pbip "C:\Files\Sales.pbip" -rules ".\Files\Base-rules.json" -output "C:\Files\TestRun" -formats "JSON,HTML"
- Run "Base-rules.json" rule definitions against PBI report file at "Sales.Report and return results to the console only:
PBIRInspectorCLI.exe -pbip "C:\Files\Sales.pbip" -rules ".\Files\Base-rules.json" -output "C:\Files\TestRun" -formats "Console"
- Run "Base-rules.json" rule definitions against PBI report file at "Sales.Report and return results as Azure DevOps compatible log and tasks commands (see https://learn.microsoft.com/en-us/azure/devops/pipelines/scripts/logging-commands?view=azure-devops&tabs=bash#task-commands):
PBIRInspectorCLI.exe -pbip "C:\Files\Sales.pbip" -rules ".\Files\Base-rules.json" -formats "ADO"
If a verbose output was requested, then results for both test passes and failures will be reported. The JSON output is intended to be consumed by a subsequent process, for example a Power BI report may be created that uses the JSON file as a data source and visualises the PBI Inspector test results. The HTML page is a more readable format for humans which also includes report page wireframe images when tests are at the report page level. These images are intended to help the user identify visuals that have failed the test such as in the example image below. The PBI Inspector logo is also displayed at the centre of each failing visuals as an additional identification aid when the wireframe is busy.
Visuals with a dotted border are visuals hidden by default as the following example:
For a tutorial on how to run PBI Inspector as part of an Azure DevOps pipeline job (alongside Tabular Editor's BPA rules), see https://learn.microsoft.com/en-us/power-bi/developer/projects/projects-build-pipelines.
❗ Please note that to work with PBI Inspector V2 the YAML file referenced in the tutorial needs to be updated as follows ContinuousIntegration-Rules-PBIR.yml.
Custom rules are defined in a JSON file as an array of rule objects as follows:
{
"rules": [
...
]
}Each rule object has the following properties:
{
"id": "A unique identifier of your choice for the rule",
"name": "A name that is shown in HTML results with wireframe images.",
"description": "Details to help you and others understand what this rule does",
"disabled": true|false(default),
"part": "Optional. One of Report|Pages|PagesHeader|AllPages|Visuals|AllVisuals|Bookmarks|BookmarksHeader|AllBookmarks. If the part specified is an array with multiple items (such as "Pages"), the rule will apply to each array item in sequence."
"test": [
//test logic
,
//optional data variables
,
//expected result
],
"patch":
[
//optional patch logic to fix the issue
]
}For example (without the optional patch logic), the following rule checks that, for each Page, certain charts have both axes titles displayed. The rule returns the names of failing visuals in an array.
{
"id": "SHOW_AXES_TITLES",
"name": "Show visual axes titles",
"description": "Check that certain charts have both axes title showing, returns an array of visual names that fail the test.",
"part": "Pages",
"disabled": false,
"test": [
{
"map": [
{
"filter": [
{
"part": "Visuals"
},
{
"and": [
{
"in": [
{
"var": "visual.visualType"
},
[
"lineChart",
"barChart",
"columnChart",
"clusteredBarChart",
"stackedBarChart"
]
]
},
{
"or": [
{
"==": [
{
"var": "visual.objects.categoryAxis.0.properties.showAxisTitle.expr.Literal.Value"
},
"false"
]
},
{
"==": [
{
"var": "visual.objects.valueAxis.0.properties.showAxisTitle.expr.Literal.Value"
},
"false"
]
}
]
}
]
}
]
},
{
"var": "name"
}
]
},
{
},
[]
]
}Optionally a rule can now also define a patch to fix items (e.g. visuals) failing the test. For example a patch for the test above is as follows. The patch iterates through the failing visual names returned and fixes the "Visuals" part of the report definition by setting the "showAxisTitle" property to "true" for both the category and value axes:
"patch": [
"Visuals",
[
{
"op": "replace",
"path": "/visual/objects/categoryAxis/0/properties/showAxisTitle/expr/Literal/Value",
"value": "true"
},
{
"op": "replace",
"path": "/visual/objects/valueAxis/0/properties/showAxisTitle/expr/Literal/Value",
"value": "true"
}
]
]A patch definition has the following structure:
"patch": [
"One of Report|Pages|PagesHeader|AllPages|Visuals|AllVisuals|Bookmarks|BookmarksHeader|AllBookmarks",
[patch logic operator array]
]The patch logic operator array is defined as per the JSON Patch specification at https://tools.ietf.org/html/rfc6902. PBI Inspector uses the .NET implementation of JSON Patch, for see https://docs.json-everything.net/patch/basics/.
Therefore the full rule example including the patch is as follows:
{
"id": "SHOW_AXES_TITLES",
"name": "Show visual axes titles",
"description": "Check that certain charts have both axes title showing.",
"part": "Pages",
"disabled": true,
"applyPatch": true,
"test": [
{
"map": [
{
"filter": [
{
"part": "Visuals"
},
{
"and": [
{
"in": [
{
"var": "visual.visualType"
},
[
"lineChart",
"barChart",
"columnChart",
"clusteredBarChart",
"stackedBarChart"
]
]
},
{
"or": [
{
"==": [
{
"var": "visual.objects.categoryAxis.0.properties.showAxisTitle.expr.Literal.Value"
},
"false"
]
},
{
"==": [
{
"var": "visual.objects.valueAxis.0.properties.showAxisTitle.expr.Literal.Value"
},
"false"
]
}
]
}
]
}
]
},
{
"var": "name"
}
]
},
{
},
[]
],
"patch": [
"Visuals",
[
{
"op": "replace",
"path": "/visual/objects/categoryAxis/0/properties/showAxisTitle/expr/Literal/Value",
"value": "true"
},
{
"op": "replace",
"path": "/visual/objects/valueAxis/0/properties/showAxisTitle/expr/Literal/Value",
"value": "true"
}
]
]
}Here's another custom rule with patch example that sets the report's default page. First we test the report's activePageName, if it's incorrect we replace it:
{
"id": "ACTIVE_PAGE",
"name": "Ensure report's active page index is set to the correct page",
"description": "",
"part": "PagesHeader",
"applyPatch": true,
"test": [
{
"var": "activePageName"
},
"ReportSection89a9619c7025093ade1c"
],
"patch": [
"PagesHeader",
[
{
"op": "replace",
"path": "/activePageName",
"value": "ReportSection89a9619c7025093ade1c"
}
]
]
}For full rule file examples see:
- Currently page wireframes are only created in a 16:9 aspect ratio so custom report page sizes including tooltip pages may not render as expected as shown in the following tooltip page example. See tooltip page example below:
- Currently page wireframes do not faithfully represents the report page layout when visual groups are present.
Please report issues at https://github.com/NatVanG/PBI-InspectorV2/issues.