The only constant is change is ever so true with Information Technology. IT systems and solutions are created in the context of the evolutionary period in IT resources. From a number of implementations we derive technical know-how, patterns, best practices, and wisdom. Sometimes the evolutionary cycles take us to newer possibilities that invalidate the current common wisdom. So is with business intelligence.
It has been almost two years since I wrote the demise and restoration of data warehousing. The brave new world is now established.
Business Intelligence aided by data warehousing has been my area of expertise for a while now. When we approach a new solution in this space, the patterns are often brought into discussions. The patterns such as source systems, ETL, staging, data warehouse, data marts, operational data stores, single version of truth - many technical jargon and marketing catchphrases. These form the traditional structure of business intelligence implementations. The traditional approach and structures, tried and tested as it may be, also binds us down to inefficiencies and impossibilities of the past and sometimes hinder us from looking into the possibilities of the present.
A few years back there was a hype about pervasive business intelligence - a capability to apply business intelligence techniques to everything and everywhere. The vision and hype was good, but the capabilities were limited. Today, we might be at a ripe stage to think about pervasive business intelligence, and bring back the hype. Business intelligence that transcends the traditional structures of data warehousing.
In my opinion the data warehousing should get extinct soon. Even the term data integration. From today’s data warehousing patterns, what will remain are data movement, data persistence and data access.
Business Intelligence, to retain the respect for the term, will have to incorporate techniques and principles from more advanced and recent developments in data mining, data discovery and data visualisation. But then that is what business intelligence, in a wider sense, all about. Today’s data analysis will remain as a smaller subset among the larger capabilities in data mining.
Now, all these - movement/persistence/access/mining/discovery/visualisation - are known as data science as well. Is business intelligence data science? Or data science business intelligence? Or a more apt question would be - when will data “science” become “business” intelligence? Soon.