Machine learning will displace “extract, transform and load” in business intelligence and data integration
Conclusion: Organisations continue to emphasise their competitive differentiation based on the data they hold, and the insights gained from analysing this valuable resource. The rate at which organisations are shifting from traditional process-based to insight-oriented differentiation is being further accelerated by the adoption of Cloud-based data analytics services.
The combined result is an increasing portion of enterprise project activity that can be classified as extract, transform and load (ETL).
Despite ETL being the mainstay of data integration for decades, the cost of specialised skills and significant manual effort expended on integrating disparate data sources is now coming into sharp focus. In response, organisations are rightly seeking lower-cost solutions for data integration.
Although ETL exists in the form of at least one tool in almost every enterprise, the cost of ETL as a proportion of data analytics projects means organisations must decrease reliance on traditional ETL tools in favour of automated solutions that exploit machine learning techniques to reduce the need for ETL developers.
About The Advisor
Sam Higgins was an IBRS advisor between 2017 and 2020 with over 20 years of both tactical and strategic experience in the application of information and communications technology (ICT) to achieve business outcomes from large complex organisations. Through previous roles as a leading ICT executive, strategist, architect, industry analyst, program consultant and advisor, Sam has developed an extensive knowledge of key markets including as-a-Service (Cloud) computing, enterprise architecture (including service-orientation and information management), enterprise applications and development, business intelligence; along with ICT management and governance practices such as ICT planning, strategic sourcing, portfolio and project management. Sam’s knowledge of service-oriented architecture and associated business models is widely recognised, and he was a contributing author on the Paul Allen book Service-orientation: Winning Strategies and Best Practices, released in 2006 by Cambridge University Press. As the former Research Director for Longhaus he undertook the first in depth research into the implications of cloud computing and other “as-a-Service” ICT offerings on the Australian and near shore markets. The 2010 report entitled, Defining cloud computing highlights provider gaps in the Australian ICT market, was widely reported in both the online ICT industry press and mainstream media.