If you’re a manager within operations and frustrated about the lack of transparency you have around your operating environment or your lack of autonomy to include new data on your existing reports or dashboards, then this post is for you.
This universal problem is centred around your company Business Intelligence (BI) solution serving the needs of the masses and as a result, is generic in serving specific needs. As a manager, you need to make decisions specific to your operating environment and even though your company BI solution may be comprehensive in providing business information, it will always be limited in providing all the information you need as you and your team continue pushing boundaries to make better decisions.
Through most business improvement initiatives and certainly any form of innovation, you can almost guarantee that the new data you collect will not be available for reporting or analyses from the company BI solution. In larger companies, the process to update the data warehouse can take considerable effort and time.
Traditional BI serves generic needs
Having been an operations analyst for a number of years, BI was certainly something I was exposed to often. As an analyst, it was daily practice to query and extract data from various sources including the company data warehouse, cleanse data, find common fields to combine key data sets (data integration), perform a variety of transformations to tease data into a required format at which point I could finally perform calculations to suit the task at hand. The steps in the preparation process described above could range from less than an hour to days depending on the size and complexity of the analysis requirements.
What I noticed over time was that each BI system I was using whilst working for several large corporates, although very comprehensive, they all served generic needs. I often found myself combining data from sources outside of the data warehouse to update a report or dashboard so management could be better informed. I remember once trying to follow a process to include new data in the existing data warehouse and hearing from my colleagues at the time that “the last time we tried to have the data warehouse updated, it took 8 months”.
Operations managers need agility in their BI solutions driven by their world of day-to-day tactical decisions. Engineers and analysts need the ability to quickly include new insights on existing reporting and dashboard solutions to accelerate business improvement. These are specific needs and are not considered in the generic operational profile of a company-wide BI solution.
So how can management and knowledge workers for business operations utilise a generic BI solution to cater to specific needs?
Welcome to Operational Intelligence
Operational Intelligence is a derivative of BI, with BI being the technologies, applications and practices for the collection, integration, analysis and presentation of business information. Operational Intelligence is BI for business operations.
The key differentiator of Operational Intelligence over BI is the ability for operational teams to quickly augment company-wide data warehouse data with data from operational data sources. This ability gives operational management complete autonomy over what they see on their business reports and dashboards, so ERP information alongside field device information for example. There are also a range of new BI tools available to knowledge workers to better serve their needs and accelerate the time it takes to perform analyses.
Video: Demonstration of an Operational Intelligence solution
BI Tools of Today
Self-service data preparation and data discovery tools. If you haven’t heard of these terms and you’re a knowledge worker working out of Excel, Google them now. BI tools leverage off existing BI systems and empower knowledge workers, especially engineers and analysts, with the ability to quickly turnaround insights for operational management. These tools are the equivalent of construction workers using a jackhammer (new BI Tools) over a pickaxe (MSExcel) to break through stubborn material (data preparation).
Companies like Paxata, Tableau and TIBCO are really making the lives of knowledge workers easier through their self-service platforms. Whilst these tools can replicate data preparation requirements for repetitive use, they’re usually limited to the individual user and not built for collaboration i.e. their true strength lies in data preparation for visualisation and analyses. When you have reporting or dashboard requirements, its best to utilise a more robust solution that makes operational data sets available to all knowledge workers and therefore can be used at scale. Data marts are a more robust solution and are a fundamental component of Operational Intelligence.
Data marts: The secret sauce of Operational Intelligence
A data mart stores data in a way that serves the specific needs of a user group, by extracting data from various data sources and leveraging off existing BI systems. Data marts should be the best friend of every operations manager by providing complete autonomy over any reporting, dashboard or analytical requirement.
Technically, data marts act in a similar way to a data warehouse, in that they extract data from multiple sources, transform data into a required format and then load data into a series of tables suitable for consumption. The majority of consumption is still around reporting or analyses by knowledge workers however in business today, consumption also includes the supply of prepared data sets to intelligent models (predictive and simulation models) for real time analytics and decision making. This process of extract, transform and load is commonly referred to as ETL in the IT community and is a fundamental process of BI.
A well-designed data mart will serve data to consumers (knowledge workers) as close to the end-state of their reporting or analyses requirements to significantly reduce, and in some cases eliminate, data preparation requirements. Data preparation is an overhead for knowledge workers and in most cases, a source of waste for business improvement. The main benefits of a well-designed data mart include an increase in productivity and efficiency of knowledge workers, an increase in the performance of applications (less processing requirements) and a reduction in the time it takes to discover new business improvement opportunities.
If you’re a manager in operations and find the use of generic business reports frustrating by not serving your specific needs, you need to invest in an Operational Intelligence solution that includes a data mart. A data mart should be one of the key data assets for every department and most teams.
But my team performance is measured by generic business reports?
This is true, you need a single source of truth to standardise how business performance is measured and even with the introduction of an Operational Intelligence solution this is unlikely to change.
The link between a data mart and the company approved business reports is your company BI solution. This link is more of a ‘loose coupling’ and is what provides the flexibility to customise your data mart to serve specific needs whilst still being relevant in the broad context of the business.
As a manager of operations, this means your team can act on specific insights from your Operational Intelligence solution, providing the agility you require, whilst conforming to the standard way your company measures business performance.
Governance between OI and BI systems
Finally, it must be made clear that IT and Operations must collaborate to ensure best practices are made available to the rest of the business and that consumers of data are using trustworthy sources.
Companies should review their business policies around data consumption and reporting practices to ensure the right balance between flexibility and standardisation is achieved. Flexibility will create an environment for innovation, a culture most companies are striving to achieve today, whilst standardisation is still critically important so results achieved can be sustained over the longer term. Like all things in life, a healthy balance is required to operate optimally, and this is no different for business.
Author:
Aaron Sharpe,
Principal Consultant – Operational Intelligence
Data Engineering