Business Intelligence: Data Soup – From Business Intelligence to Analytics

Business Intelligence Series

The days when everything was reduced to simple terminology like reports or queries are gone. One can see it in the market trends related to reporting or data, as well in the jargon soup the IT people use on the daily basis – Business Intelligence (BI), Data Mining (DM), Analytics, Data Science, Data Warehousing (DW), Machine Learning (ML), Artificial Intelligence (AI) and so on. What’s more confusing for the users and other spectators is the easiness with which all these concepts are used, sometimes interchangeably, and often it feels like nothing makes sense.

BI is used nowadays to refer to the technologies, architectures, methodologies, processes and practices used to transform data into what is desired as meaningful and useful information.  From its early beginnings in the 60s, the intelligence from Business Intelligence (BI) refers to the ability to apprehend the interrelationships of the facts to be processed (aka data) in such a way as to guide action towards a desired goal.

The main purpose of BI was and is to guide actions and provide a solid basis for decision making, aspect not necessarily reflected in the way organizations use their BI infrastructure. Except basic operational/tactical/strategic reports and metrics that reflect to a higher or lower degree organizations’ goals, BI often fails to provide the expected value. The causes are multiple ranging from an organizations maturity in devising a strategy and dividing it into SMART goals and objectives, to the misuse of technologies for the wrong purposes.

Despite the basic data analysis techniques, the rich visualizations and navigation functionality, BI fails often to deliver by itself more than ordinary and already known information. Information becomes valuable when it brings novelty, when it can be easily transformed into knowledge, or even better, when knowledge is extracted directly. To address the limitations of the BI a series of techniques appeared in parallel and coined in the 90s as Data Mining.

Mining is the process of obtaining something valuable from a resource. What DM tries to achieve as process is the extraction of knowledge in form patterns from the data by categorizing, clustering, identifying dependencies or anomalies. When compared with data analysis, the main characteristics of DM is the fact that is used to test models and hypotheses, and that it uses a set of semiautomatic and automatic out-of-the-box statistics packages, AI or predictive algorithms with applicability in different areas – Web,  text, speech, business processes, etc.

DM proved to be useful by allowing to build models rooted in historical data, models which allowed predicting outcome or behavior, however the models are pretty basic and there’s always a threshold beyond which they can’t go. Furthermore, the costs of preparing the data and of the needed infrastructure seem to be high compared with the benefits data mining provides. There are scenarios in which DM proves to bring benefit, while in others it raises more challenges than can solve. Privacy, security, misuse of information and the blind use of techniques without understanding the data or the models behind, are just some of such challenges.  

Information seems too common, while knowledge can become expensive to obtain. The middle way between the two found its future into another buzzword – analytics – the systematic analysis of data or statistics using specific mathematical methods. Analytics combine the agility of data analysis techniques with the power of predictive and prescriptive techniques used in DM in discovering patterns into the data. Analytics attempts to identify why it happens by using a chain of inferences resulted from data’s analyzing and understanding. From another perspective analytics seems to be a rebranded and slightly enhanced version of BI.

Business Process Management: Business Process (Definitions)

"A business process is a collection of activities that takes one or more kinds of input and creates an output that is of value to the customer. A business process has a goal and is affected by events occurring in the external world or in other processes." (James A Champy & Michael M Hammer, "Reengineering the Corporation", 1993)

"A process is a set of linked activities that take an input and transform it to create an output. Ideally, the transformation that occurs in the process should add value to the input and create an output that is more useful and effective to the recipient either upstream or downstream."

(Henry J Johansson, "Business process reengineering: Breakpoint strategies for market dominance", 1993)

"Major operational activities or processes supported by a source system, such as orders, from which data can be collected for the analytic purposes of the data warehouse. Choosing the business process is the first of four key steps in the design of a dimensional model." (Ralph Kimball & Margy Ross, "The Data Warehouse Toolkit" 2nd Ed., 2002)

"The sequence of activities 'enclosing' the production process. These activities are common to all types of products and services, and include defining the job, negotiation with the customer, and reporting project status." (Richard D Stutzke, "Estimating Software-Intensive Systems: Projects, Products, and Processes", 2005)

"The subject areas of a business. The method by which a business is divided up. In a data warehouse, the subject areas become the fact tables." (Gavin Powell, "Beginning Database Design", 2006)

"A structured description of the activities or tasks that have to be done to fulfill a certain business need. The activities or tasks might be manual steps (human interaction) or automated steps (IT steps)." (Nicolai M Josuttis, "SOA in Practice", 2007)

"A structured and measured, managed, and controlled set of interrelated and interacting activities that uses resources to transform inputs into specified outputs." (Nathalíe Galeano, "Competency Concept in VO Breeding Environment", 2008) 

"The codification of rules and practices that constitute a business." (Judith Hurwitz et al, "Service Oriented Architecture For Dummies" 2nd Ed., 2009)

"The defined method for a range of activities that organizations perform. A business process can include anything from the steps needed to make a product to how a supply is ordered or how an invoice is created." (Tony Fisher, "The Data Asset", 2009)

"A structured description of the activities or tasks that have to be done to fulfill a certain business need. The activities or tasks might be manual steps (human interaction) or automated steps (IT steps)." (David Lyle & John G Schmidt, "Lean Integration", 2010)

"An activity as carried out by business people, including the mechanisms involved. This is in the domain of Row Two, the Business Owner’s View. Alternatively, the architect in Row Three sees a system process which is about the data transformations involved in carrying out a business process. In either case, processes can be viewed at a high level or in atomic detail." (David C Hay, "Data Model Patterns: A Metadata Map", 2010)

"A collection of activities performed to accomplish a clearly defined goal." (Linda Volonino & Efraim Turban, "Information Technology for Management" 8th Ed., 2011)

"A collection of activities designed to produce a specific output for a particular customer or market." (International Qualifications Board for Business Analysis, "Standard glossary of terms used in Software Engineering", 2011)

"A process that is intended to contribute to the overall value of an enterprise. The complex interactions between people, applications, and technologies designed to create customer value. A process is composed of activities." (DAMA International, "The DAMA Dictionary of Data Management", 2011)

"A business process is a series of steps required to execute a function that is important to an organization. Business processes include things like taking an order or setting up an account or paying a claim. In process analysis, business processes are the focus of opportunities for improvement. Organizations usually have a set of key processes that require support from other areas, like information technology." (Laura Sebastian-Coleman, "Measuring Data Quality for Ongoing Improvement ", 2012)

 "A holistic management approach for the detection, analysis, modeling, implementation, improvement and governance of the activities within or between enterprises." (Michael Fellmann et al, "Supporting Semantic Verification of Process Models", 2012)

"An activity (or set of activities) that is managed by an organization to produce some result of value to that organization, its customers, its suppliers, and/or its partners." (Graham Witt, "Writing Effective Business Rules", 2012)

"The codification of rules and practices that constitute a business." (Marcia Kaufman et al, "Big Data For Dummies", 2013)

"A coordinated set of collaborative and transactional work activities carried out to complete work steps." (Robert F Smallwood, "Information Governance: Concepts, Strategies, and Best Practices", 2014)

"The defined method for a range of activities that organizations perform. A business process can include anything from the steps needed to make a product to how a supply is ordered or how a decision is made." (Jim Davis & Aiman Zeid, "Business Transformation", 2014)

"A set of activities that teams within an organization carry out to accomplish a specific goal." (David K Pham, "From Business Strategy to Information Technology Roadmap", 2016)

"The business activities executed to deliver products or services to external customers. Business process is supported by and consumes IT-services to achieve their objectives." (by Brian Johnson & Leon-Paul de Rouw, "Collaborative Business Design", 2017)

"At its most generic, any set of activities performed by a business that is initiated by an event, transforms information, materials or business commitments, and produces an output. Value chains and large-scale business processes produce outputs that are valued by customers. Other processes generate outputs that are valued by other processes." (Appian)