Data Quality Management for a Canadian Federal Department

Business Problem: The client needed to identify bad quality data and cleanse data issues during migration from Mainframe to Oracle

Our Solution: The work involved developing DataStage jobs with Data Rules Stage to identify & cleanse data issues. Data Rules developed in Information Analyzer were reused in DataStage to identify & cleanse the data issues. Each record undergoes a multi-phase cleanse process, based on the various types of data issues. The migration of the cleansed Mainframe data to Oracle using DataStage jobs, was tracked with detailed audit information generated by the DataStage jobs. Code generators were developed in Java to generate the DataStage jobs used for various types of data cleansing activities.

Technology: IBM InfoSphere Information Server 8.7, DataStage, Oracle 11i, Data Rules Stage, Java, MS Excel 2010

About the Author: Site Admin