Metadata Management as an afterthought

As Organizations realize Analytics is fundamental to their survival, they are in a frenzy to feed all possible data into their analytical platforms. The analytical platforms are maturing in capability to manage variety, volume and velocity of the data that they are being fed with and churn out the needed analysis for the end-users. All of this looks hunky-dory and suddenly we have these situations. A metrics on the Sales dashboard shows some unexplainable value and the Sales manager is keen to know the reason. He wants to know what the source of data for this metric is. A data mart contains ‘Personally Identifiable Information’ of customers and the business analyst needs to know where they are in order to request restricted access to the end users. Unfortunately Business does not seem have this information anywhere. They look to IT for this information. Now who in IT, IT support or IT Development team? IT support did not bother to collect this information when the development team rolled out the analytical application into production. The IT development team did not create this information in the first place as it was not a must-have for the platform development. What is this ‘information’ that is everybody looking for to solve their puzzles?

This is Metadata or the ‘data about data’. This is the information that helps you identify the system-of-record for an element in a report, helps you understand the sensitivity of the data elements in a data mart or the basic definition for the data element that is needed to interpret the same. We have three flavors of Metadata, the Business, the Technical and the Operational metadata. The operational metadata becomes relevant and important for running the analytical platforms and the focus is only after the platform goes live. However, the Business and the Technical metadata are best captured and managed during the development of the platform. In all, they make up the foundation for Data Governance, the key goal of which is to ensure the right data is available to the right user at the right time.

It is time organizations realize that Metadata Management needs the same level of focus as Data Management or rather look at it as a key component of Data Management. Left to be an afterthought, it surely brings down the usability and the overall value of the analytical platform.
In the next set of blogs, we will look at detailed aspects of metadata types, ‘what’ and ‘how’ of its management…

About the Author: Anand Govindarajan

Anand Govindarajan

Chief Data Architect
Email: anandg@lucidtechsol.com
Linkedin: http://in.linkedin.com/in/anandgovindarajan/