In the past decade enterprises saw a widespread adoption of business intelligence and data warehousing solutions to understand the patterns hidden in data, fragmented across various source systems. Most of these efforts built business intelligence solutions based on structured transactional data generated by business operations. However, information is also exchanged over various unstructured and semi-structured formats such as emails, instant messages, documents, spreadsheets, presentations, news, blogs etc. Such information is not captured by traditional business intelligence and data warehousing solutions, partly because of BI/DW tools and techniques were conceived to deal with structured data.
Today we see enterprises taking a closer look at enterprise content which comprises of unstructured and semi-structured, context-sensitive and time-sensitive data. Enterprise Content Management systems are implemented alongside Enterprise Resource Planning systems. Still, the relation between these two systems is often a mystery to various stakeholders. In most cases the missing component is the thread that interweaves and integrates these systems, various formats of data, and various techniques for retrieval, aggregation and analysis of different representations of data.
‘In an ideal enterprise all available information is structured and stored in databases and managed by tools. Unfortunately this is not the case even with the most technologically advanced enterprises. Enterprise Information is an organizational asset built on all the data available in an enterprise, regardless of structure, format, representation, storage, lifetime value or any such attribute. Data upon processing generates information that is useful for further consumption such as analysis, decision-making, planning, and predictions.
Information Technology has grown over the years solving problems, and initial solutions often tend to be more specific to problems at hand. Solutions to build information out of structured and unstructured data are therefore vastly dissimilar. An attempt to build Enterprise Information would then require either a re-engineering of existing systems or an integration of data outside external systems based on a set of associative and semantic rules.
Application of Enterprise Information into actionable items requires distribution of information for consumption by right entity – and individual, a group or a system. As the information generation and distribution becomes increasingly efficient, the requirements for such information also become frequent, and near real-time. In an increasingly competitive world, real-time information may also become one of the business differentiators.
Real-time applications in enterprises were meant to be transaction-processing systems. Building real time business intelligence based on transactional data followed. But when we bring in non-transactional, ad-hoc, unstructured data into the scope, the phrase real-time can have a broader definition.
Different aspects of actionable information in real time are detailed in the following sections.
Real-time data integration is required to get supply data to reports and dashboards that require real-time changes.
A true real-time business intelligence report tends to be complex with limited practical uses in most industries. A near real-time report is generally with a few updates every hour is realistic in most scenarios. Such reports with a wider coverage in organization hierarchy are sometimes known as operational business intelligence reports. Similarly there are operational dashboards with trickle-feed data and little historical information, and business intelligence dashboards which are typically visualization of business intelligence reports.
The data integration strategies differ from operational, analytic and strategic reporting. In today’s world integration of operational data is typically achieved using web services, messaging, and APIs while bulk data for business intelligence and analytics (including historical) are integrated using specialized data integration / ETL tools.
Real-time data integration often ends at bringing data to a data store or portal. Distribution (or dissemination) of processed or analyzed real-time data to a wide enterprise audience is another aspect of a real-time solution. Real-time data access addresses how data is made available over a variety of channels for information, consumption and decision-making.
Typical channels of information dissemination are portals, portlets/widgets, email, and feeds. During distribution one should also address requirements such as authentication, authorization, encryption, access control, opt-in/opt-out, distribution list, subscriptions etc.
Search is used to discover information online. Search algorithms/components built with chronology, metadata and online indexing help users to understand the trends, patterns and inter-relationships within the data. A live index may also act as a change data capture mechanism for live data integration.
Live collaboration enables end-users to collaborate, share information, interpret, discuss, comment, record and build knowledge which is not (or cannot be) generated by information systems. Such features help users to add a human-element to an analytical and decision-making system.