3 Steps To a High Quality ETL Code

If you’d want to find any news on ETL, don’t waste your time. It is taken as an outdated topic. They even predict it will be dismissed in the very near future with data virtualisation, data federation and whatnot. If you’d ask me, I’m thrilled with ETL. Until the world has its data warehouses massively implemented for the last 20 years, ETL will continue to be an important part of data integration.

The more information companies obtain, the more ETL development is involved and the more important the quality of ETL code is.

How easily the system will be maintained?

Does ETL perform in its optimal way?

How easily we can trace our data flow on its way from a data source to a target system?

This is all about ETL code quality, which people try to keep up with development standards and processes. For the last year we’ve taken a deep dive into the issue, talking to our clients and ETL professionals, starting conversations and gathering opinions of ETL folks from all over the world. We even launched a survey to find out how important ETL code quality issue is for real people who deal with ETL day-to-day.

So far, listening to people we see that ETL code quality is a live issue for many and here are our observations on what is going on:

  • everyone understands that to have standards for development is vital and the more managerial position is the more importance a person pays to formalisation of standards.
  • people tend to agree that it takes efforts to enforce adherence to standards whilst development.
  • when talking about standards many refer to velocity, but velocity itself is a methodology that can have slightly different interpretation by different people and itself it gives generic methodology of development rather than a step by step guidance
  • there’s still uncertainty at what stage and by whom the code review shall be performed: developers? QA? Support team?
  • people try to make their code review processes easier and more effective. Some try to automate it with the help of writing scripts, some minimise the number of standards in their check-lists to make them feasible.

In Acctiva we have found the way how to help people to deal with it. We have chosen Undraleu software for our clients to automate their ETL code review processes seamlessly. It is about best practices of ETL code writing, agility in review processes and about reducing a hassle with the help of Undraleu. How does it sound?

If you are interested in ETL code quality as we are, we’re happy to show you our gemstone DEMO REQUEST