As a veteran of the early data warehousing years, I’ve come to think of Data Warehousing for IT and Business Intelligence for the business, with ETL (Extract, Transform, Load) belonging in the realm of IT and programming. Have times changed!!
In today’s dynamic world, the definitions that govern data transformation are not static. They change frequently as new codes, new regulations, new business scenarios are defined. Whether for data warehousing or business process integration, some data transformation definitions are best managed by business people.
For example, a trading transaction may need to be transformed into multiple general ledger records, based on certain codes or code combinations. New codes are always being introduced, which means that some trading transactions will fail the transformation process because the rule governing the new code doesn’t exist. If you’re thinking in code, this means putting an application change request in and waiting, waiting, waiting…until a developer can use an ETL tool to update the transformation definition.
If you’re thinking in rules, transactions with unknown codes or code combinations are sent to a queue where business analysts create new rules to govern the code in the future. Business Users create new and update ETL Definitions as needed, without custom programming.
For truly dynamic business process integration and data transformation, think in rules.