Data integration — quicker and cheaper.
We have built our own accelerators to deliver high-quality data integration services, quicker at a lower cost. Some of our accelerators are:
Shared containers/mapplets built with reusable logic.
For example a shared container to enforce primary key, unique key, foreign key constraints for ETL against IBM Pure Data Analytics (Netezza) system is used as the product does not enforce such constraints at the DB level.
Generic ETL jobs/templates, which use runtime metadata to extract data from source and load to target.
These are multi instance jobs which can work based on user defined SQL (or table metadata) as source and run “n” instances in parallel to quickly load target. These work across different ETL toolsets as well.
Data validation routines designed to validate and report any referential integrity violations.
They function in a batch mode, especially on MPP systems (like IBM Pure Data Analytics – Netezza). Since the validations are done at non-real time, it improves the load performance and the violated records are flagged off from further processing.