Enabling traceability of data from dashboard to system-of-record for a Multi-National Oil and Gas Corporation headquartered in the US

Client: A Multi-National Oil and Gas Corporation headquartered in the US

Business Problem:

The client had critical dashboards built on Tableau that connected to several of their enterprise data stores on SAP HANA, SQL server and other relational databases. It was important to understand the data lineage for each of the dashboard elements to its system-of-record to evaluate its data quality and do impact analysis. The client wanted to do a four-week proof-of-concept exercise to evaluate options.

Solution:

Lucid’s integration templates helped the client with the data lineage from Tableau report attributes to their data sources in SAP HANA and SQL server. The client’s data source was in SQL Server. Data from here and the client’s SAP R/3 solution were fed into an SAP HANA database. Reports were built in Tableau to use the data stored in SAP HANA.

The solution was a combination of Collibra Connect templates to extract and load metadata from various systems and a custom tool to discover & load business terms into a glossary from a data dictionary that was managed through a combination of spreadsheets and ER Studio.

  • Tableau REST APIs were used to extract workbook and dashboard metadata
  • SAP HANA system tables were used to extract tables/columns metadata and view xmls that were parsed to load the view to table mapping
  • SQL server system tables to extract all SQL server database objects metadata

Technology: Collibra DGC ver 5.0.2,  Collibra Connect with Collibra DGC Connector v1.3, SAP HANA ver2.0, Tableau ver 10.1.2, SQL server 2016

About the Author: Site Admin

Associate Tech Lead
Email: paramagurus@lucidtechsol.com
Linkedin: http://linkedin.com/in/paramagurus