In the age of self-service analytics, the key expectation from any BI platform is to offer the powerful capabilities that exists in enterprise-scale environments to even the individual tool user. One such platform is PowerBI. True to its name it provides powerful and cool features simplified for individual users.
Can you imagine a decade ago how huge and complex data warehousing efforts were? Data integration and consolidation of data from varied systems into one single source of truth to support analytics?
Microsoft has enabled a similar architecture with the same power of data consolidation to a common model but achievable by individual users unlike an army working on a similar data warehousing effort.
This is based on apowerful combination of PowerBI Dataflow, the self-service data prep tool and the PowerBI shared datasets known as the Common Data Model. Refer to the architecture diagram from the Microsoft site (https://docs.microsoft.com/en-us/power-bi/transform-model/service-dataflows-overview)
PowerBI Dataflow is a low-code/no-code tool that lets you pull data from any type of data sources that is supported by PowerBI. With simplicity of any data prep tool, all the transformations needed can be done to all the sourced datasets and loaded as entities into what is a Common Data Model (CDM). CDM is another powerful feature to enable a common semantic model for datasets for consumption across the organization.
One of the usage scenarios for this architecture is to enable self-service analytics on a Data Governance platform like Collibra.
Dataflows can be used to connect to the Collibra REST API interfaces to pull together key metadata entities into a common data model for a ‘Collibra Data Mart’. This then becomes a “live”(meta)data mart for analysis using the PowerBI reports and dashboards. Thus, powerful Data Governance Dashboards can be enabled. As future extensions, analysis can be done across Collibra instances extending to even other Data Governance platforms in the Enterprise.
The data captured as part of this ‘Collibra Data Mart’ is stored in an Azure Data Lake Storage making it extensible and scalable. With a PowerBI Premium license a storage of 100TB/node is available enabling historical analysis.
To know more about how to build such “Collibra Data Marts” and other solution accelerators from Lucid, contact us at collibrainfo@lucidtechsol.com.
