Buzzwords: “ETL” (and “ELT”)

11. June 2008 by Rick Abbott.

Extract, Transform, and Load (ETL) is another one of those terms that was very useful as recently as 3-5 years ago, but now has lost some of it’s value for some of the same reasons “data warehouse” has become archaic - it describes a component of the architecture that has sprawled outside the bounds defined by this term. Ten years ago, ETL accurately described the majority of the processing required to load data into a data warehouse, data mart, etc… The term mimics the process used to accomplish this task, namely “extracting”(or receiving) data from source systems, “transforming” the data by applying business rules, data cleansing, and other manipulations on the data sets, and then “loading” the data into the target data store. In a predominately batch world, this is a fine way to handle the loading process, and in fact a significant amount of data is still being loaded in this manner today.

But things are changing - the analytical world today is not so straightforward, and as a result the loading requirements have become much more complex. And in fact the biggest change is the move away from a physical “load” of data, to a virtual integration of data. Virtual data integration (VDI) is not always the answer, but in a business climate that increasingly rewards real-time feedback, VDI provides visibility into the business operations as they occur, not days or even hours later.

One final word about “ELT” - extract, load, and transform. I’ve heard some vendors (and analysts) talk up ELT as a new and improved ETL, but in my mind this is just an architectural choice. It makes little difference if you load the data before transforming, or the other way around.

Posted in Industry Buzzwords, Technical Focus | No Comments »