Business Intelligence: Data Modeling (Part IV: From Data to Storytelling II)

Business Intelligence Series

Being snapshots in people and organizations’ lives, data arrive to tell a story, even if the story might not be worth telling or might be important only in certain contexts. In fact each record in a dataset has the potential of bringing a story to life, though business people are more interested in the hidden patterns and “stories” the data reveal through more or less complex techniques. Therefore, data are usually tortured until they confess something, and unfortunately people stop analyzing the data with the first confession(s). 

Even if it looks like torture, data need to be processed to reveal certain characteristics, trends or patterns that could help us in sense-making, decision-making or similar specific business purposes. Unfortunately, the volume of data increases with an incredible velocity to which further characteristics like variety, veracity, volume, velocity, value, veracity and variability may add up. 

The data in a dashboard, presentation or even a report should ideally tell a story otherwise the data might not be worthy looking at, at least from some people’s perspective. Probably, that’s one of the reason why man dashboards remain unused shortly after they were made available, even if considerable time and money were invested in them. Seeing the same dull numbers gives the illusion that nothing changed, that nothing is worth reviewing, revealing or considering, which might be occasionally true, though one can’t take this as a rule! Lot of important facts could remain hidden or not considered. 

One can suppose that there are businesses in which something important seldom happens and an alert can do a better job than reviewing a dashboard or a report frequently. Probably an alert is a better choice than reporting metrics nobody looks at! 

Organizations usually define a set of KPIs (key performance indicators) and other types of metrics they (intend to) review periodically. Ideally, the numbers collected should define and reflect the critical points (aka pain points) of an organization, if they can be known in advance. Unfortunately, in dynamic businesses the focus can change considerably from one day to another. Moreover, in systemic contexts critical points can remain undiscovered in time if the set of metrics defined doesn’t consider them adequately. 

Typically only one’s experience and current or past issues can tell what one should consider or ignore, which are the critical/pain points or important areas that must be monitored. Ideally, one should implement alerts for the critical points that require a immediate response and use KPIs for the recurring topics (though the two approaches may overlap). 

Following the flow of goods, money and other resources one can look at the processes and identify the areas that must be monitored, prioritize them and identify the metrics that are worth tracking, respectively that reflect strengths, weaknesses, opportunities, threats and the risks associated with them. 

One can start with what changed by how much, what caused the change(s) and what further impact is expected directly or indirectly, by what magnitude, respectively why nothing changed in the considered time unit. Causality diagrams can help in the process even if the representations can become quite complex. 

The deeper one dives and the more questions one attempts to answer, the higher the chances to find a story. However, can we find a story that’s worth telling in any set of data? At least this is the point some adepts of storytelling try to make. Conversely, the data can be dull, especially when one doesn’t track or consider the right data. There are many aspects of a business that may look boring, and many metrics seem to track the boring but probably important aspects. 

Previous Post <<||>> Next Post

Business Intelligence: Dashboards (Part I: Dashboards Are Dead & Other Crap)

Business Intelligence Series

I find annoying the posts that declare that a technology is dead, as they seem to seek the sensational and, in the end, don't offer enough arguments for the positions taken; all is just surfing though a few random ideas. Almost each time I klick on such a link I find myself disappointed. Maybe it's just me - having too great expectations from ad-hoc experts who haven't understood the role of technologies and their lifecycle.

At least until now dashboards are the only visual tool that allows displaying related metrics in a consistent manner, reflecting business objectives, health, or other important perspective into an organization's performance. More recently notebooks seem to be getting closer given their capabilities of presenting data visualizations and some intermediary steps used to obtain the data, though they are still far away from offering similar capabilities. So, from where could come any justification against dashboard's utility? Even if I heard one or two expert voices saying that they don't need KPIs for managing an organization, organizations still need metrics to understand how the organization is doing as a whole and taken on parts. 

Many argue that the design of dashboards is poor, that they don't reflect data visualization best practices, or that they are too difficult to navigate. There are so many books on dashboard and/or graphic design that is almost impossible not to find such a book in any big library if one wants to learn more about design. There are many resources online as well, though it's tough to fight with a mind's stubbornness in showing no interest in what concerns the topic. Conversely, there's also lot of crap on the social networks that qualify after the mainstream as best practices. 

Frankly, design is important, though as long as the dashboards show the right data and the organization can guide itself on the respective numbers, the perfectionists can say whatever they want, even if they are right! Unfortunately, the numbers shown in dashboards raise entitled questions and the reasons are multiple. Do dashboards show the right numbers? Do they focus on the objectives or important issues? Can the number be trusted? Do they reflect reality? Can we use them in decision-making? 

There are so many things that can go wrong when building a dashboard - there are so many transformations that need to be performed, that the chances of failure are high. It's enough to have several blunders in the code or data visualizations for people to stop trusting the data shown.

Trust and quality are complex concepts and there’s no standard path to address them because they are a matter of perception, which can vary and change dynamically based on the situation. There are, however, approaches that allow to minimize this. One can start for example by providing transparency. For each dashboard provide also detailed reports that through drilldown (or also by running the reports separately if that’s not possible) allow to validate the numbers from the report. If users don’t trust the data or the report, then they should pinpoint what’s wrong. Of course, the two sources must be in synch, otherwise the validation will become more complex.

There are also issues related to the approach - the way a reporting tool was introduced, the way dashboards flooded the space, how people reacted, etc. Introducing a reporting tool for dashboards is also a matter of strategy, tactics and operations and the various aspects related to them must be addressed. Few organizations address this properly. Many organizations work after the principle "build it and they will come" even if they build the wrong thing!

Previous Post <<||>> Next Post

Business Intelligence: Key Performance Indicators (Between Certainty and Uncertainty)

Business Intelligence Series

Despite the huge collection of documented Key Performance Indicators (KPIs) and best practices on which KPIs to choose, choosing a reliable set of KPIs that reflect how the organization performs in achieving its objectives continues to be a challenge for many organizations. Ideally, for each objective there should be only one KPIs that reflects the target and the progress made, though is that realistic?

Let's try to use the driver's metaphor to exemplify several aspects related to the choice of KPIs. A driver's goal is to travel from point A to point B over a distance d in x hours. The goal is SMART (Specific, Measurable, Achievable, Relevant, and Time-bound) if the speed and time are realistic and don't contradict Physics, legal or physical laws. The driver can define the objective as "arriving on time to the destination". 

One can define a set of metrics based on the numbers that can be measured. We have the overall distance and the number of hours planned, from which one can derive an expected average speed v. To track a driver's progress over time there are several metrics that can be thus used: e.g., (1) the current average speed, (2) the number of kilometers to the destination, (3) the number of hours estimated to the destination. However, none of these metrics can be used alone to denote the performance alone. One can compare the expected with the current average speed to get a grasp of the performance, and probably many organizations will use only (1) as KPI, though it's needed to use either (2) or (3) to get the complete picture. So, in theory two KPIs should be enough. Is it so?

When estimating (3) one assumes that there are no impediments and that the average speed can be attained, which might be correct for a road without traffic. There can be several impediments - planned/unplanned breaks, traffic jams, speed limits, accidents or other unexpected events, weather conditions (that depend on the season), etc. Besides the above formula, one needs to quantify such events in one form or another, e.g., through the perspective of the time added to the initial estimation from (3). However, this calculation is based on historical values or navigator's estimation, value which can be higher or lower than the final value. 

Therefore, (3) is an approximation for which is needed also a confidence interval (± t hours). The value can still include a lot of uncertainty that maybe needs to be broken down and quantified separately upon case to identify the deviation from expectations, e.g. on average there are 3 traffic jams (4), if the road crosses states or countries there may be at least 1 control on average (5), etc. These numbers can be included in (3) and the confidence interval, and usually don't need to be reported separately, though probably there are exceptions. 

When planning, one needs to also consider the number of stops for refueling or recharging the car, and the average duration of such stops, which can be included in (3) as well. However, (3) slowly becomes  too complex a formula, and even if there's an estimation, the more facts we're pulling into it, the bigger the confidence interval's variation will be. Sometimes, it's preferable to have instead two-three other metrics with a low confidence interval than one with high variation. Moreover, the longer the distance planned, the higher the uncertainty. One thing is to plan a trip between two neighboring city, and another thing is to plan a trip around the world. 

Another assumption is that the capability of the driver/car to drive is the same over time, which is not always the case. This can be neglected occasionally (e.g. one trip), though it involves a risk (6) that might be useful to quantify, especially when the process is repeatable (e.g. regular commuting). The risk value can increase considering new information, e.g. knowing that every a few thousand kilometers something breaks, or that there's a traffic fine, or an accident. In spite of new information, the objective might also change. Also, the objective might suffer changes, e.g. arrive on-time safe and without fines to the destination. As the objective changes or further objectives are added, more metrics can be defined. It would make sense to measure how many kilometers the driver covered in a lifetime with the car (7), how many accidents (8) or how many fines (9) the driver had. (7) is not related to a driver's performance, but (8) and (9) are. 

As can be seen, simple processes can also become very complex if one attempts to consider all the facts and/or quantify the uncertainty. The driver's metaphor applies to a simple individual, though once the same process is considered across the whole organization (a group of drivers), the more complexity is added and the perspective changes completely. E.g., some drivers might not even reach the destination or not even have a car to start with, and so on. Of course, with this also the objectives change and need to be redefined accordingly. 

The driver's metaphor is good for considering planning activities in which a volume of work needs to be completed in a given time and where a set of constraints apply. Therefore, for some organizations, just using two numbers might be enough for getting a feeling for what's happening. However, as soon one needs to consider other aspects like safety or compliance (considered in aggregation across many drivers), there might be other metrics that qualify as KPIs.

It's tempting to add two numbers and consider for example (8) and (9) together as the two are events that can be cumulated, even if they refer to different things that can overlap (an accident can result in a fine and should be counted maybe only once). One needs to make sure that one doesn't add apples with juice - the quantified values must have the same unit of measure, otherwise they might need to be considered separately. There's the tendency of mixing multiple metrics in a KPI that doesn't say much if the units of measure of its components are not the same. Some conversions can still be made (e.g. how much juice can be obtained from apples), though that's seldom the case.

Previous Post <<||>> Next Post

Business Intelligence: Key Performance Indicators (An Introduction)

Key Performance Indicators (KPIs) are quantifiable measurements (aka metrics) that reflect the critical success factor of an organization in respect to their strategic goals and objectives. They allow measuring the progress toward reaching the defined goals and, to some degree, forecasting the further  evolution. They help keeping the focus on the goals, increases awareness in what concerns the goals and provide visibility into the business.

As they reflect an organization’s objectives, KPIs need to be anchored and aligned with them. If there’s no association with an objective then one doesn’t deal with a KPI but with other form of performance metric. Therefore KPIs need to change with the objectives, they are not fix.

One important requirement for a KPI is to be defined using SMART (specific, measurable, attainable, relevant, time-bound) criteria. Thus a KPI needs to be clear and unambiguous (specific), needs to measure the progress against a goal (measurable), needs to be realistic (attainable), needs to be relevant for the business and its current strategy (relevant), and needs to specify when the result(s) can be achieved (time-bound). To the SMART criteria some consider also the requirement for a KPI to be periodically and consistently evaluated and reviewed (trackable) and agreed between the parties afected by it (agreed).

A KPI needs to be visible within an organization, understandable and non-redundant. Even if KPIs are a tool for the upper management, their definition and impact needs to be visible and understood by all the people working with it, even if this can lead to unexpected behavior. The requirement for non-redundancy implies a partition of the KPIs to limit the cases in which two or more KPIs provide the same information.

A KPI needs to be supported by actions and needs to trigger actions. It’s nice to have KPIs reported periodically to the upper management, though as long no action is triggered, there’s no value in it. A KPI is kind of reinforcement for questions like: “why are we doing good/bad?”. The negative variations must trigger some form of action, however also the positive variation could involve further analysis to understand what caused the improvement.

The variation of a KPI needs to be supported by facts – each variation needs to be explainable in one form or another. A number without a story remains a number that can or not be trusted. Therefore, it might be needed to have further metrics or reports that support the KPIs, that can be used to identify the sources for variation, in order to understand the data.

Last but not the least KPIs need to be documented. The documentation needs to include at minimum a rough definition that includes the rationale, the boundary as well the critical values, metric’s owners, unit of measure, etc. In addition, one can add historical information about the KPI in respect to when and what caused variations, respectively how the variations were brought under control.

KPIs vary from an organization to another, the variation in not only influenced by the different goals organizations might have, but also based on the fact that organizations tend to measure different things, often the wrong things. It’s in general recommended to have a small number of KPIs that reflect in one dasboard how the business is doing and what is important for the business.

KPIs provide a basis for change by providing insights into what needs to change to improve some aspects of the business. When adequately defined and measured, KPIs provide a good perspective over an organization’s effort in achieving its goals and objectives, and therefore a good tool for monitoring and stirring organization’s strategy.

Previous Post <<||>> Next Post

Strategic Management: Defining the Strategy

In a previous post an organization’s strategy was defined as a set of coordinated and sustainable actions following a set of well-defined goals, actions devised into a plan and designed to create value and overcome an organization’s challenges. In what follows are described succinctly the components of the strategy.

A strategy’s definition should start with the identification of organization’s vision, where the organization wants to be in the future, its mission statement, a precise description of what an organization does in turning the vision from concept to reality, its values - traits and qualities that are considered as representative, and its principles – the guiding laws and truths for action. All these components have the purpose at defining at high-level the where (the vision), the why (the mission), the what (the core values) and by which means (the principles) of the strategy.

One of the next steps that can be followed in parallel is to take inventory of the available infrastructure: systems, processes, procedures, practices, policies, documentation, resources, roles and their responsibilities, KPIs and other metrics, ongoing projects and initiatives. Another step resumes in identifying the problems (challenges), risks and opportunities existing in the organization as part of a SWOT analysis adjusted to organization’s internal needs. One can extend the analysis to the market and geopolitical conditions and trends to identify further opportunities and risks. Within another step but not necessarily disconnected from the previous steps is devised where the organization could be once the problems, risks, threats and opportunities were addressed.

Then the gathered facts are divided into two perspectives – the “IS” perspective encompasses the problems together with the opportunities and threats existing in organization that define the status quo, while the “TO BE” perspective encompasses the wished state. A capability maturity model can be used to benchmark an organization’s current maturity in respect to industry practices, and, based on the wished capabilities, to identify organization’s future maturity.

Based on these the organization can start formulating its strategic goals – a set of long-range aims for a specific time-frame, from which are derived a (hierarchical) set of objectives, measurable steps an organization takes in order to achieve the goals. Each objective carries with it a rational, why the objective exists, an impact, how will the objective change the organization once achieved, and a target, how much of the objective needs to be achieved. In addition, one can link the objectives to form a set of hypothesis - predictive statements of cause and effect that involve approaches of dealing with the uncertainty. In order to pursue each objective are devised methods and means – the tactics (lines of action) that will be used to approach the various themes. It’s important to prioritize the tactics and differentiate between quick winners and long-term tactics, as well to define alternative lines of actions.

Then the tactics are augmented in a strategy plan (roadmap) that typically covers a minimum of 3 to 5 years with intermediate milestones. Following the financial cycles the strategy is split in yearly units for each objective being assigned intermediate targets. Linked to the plan are estimated the costs, effort and resources needed. Last but not the least are defined the roles, management and competency structures, with their responsibilities, competencies and proper level of authority, needed to support strategy’s implementation. Based on the set objectives are devised the KPIs used to measure the progress (success) and stir the strategy over its lifecycle.

By addressing all these aspects is created thus a first draft of the strategy that will need several iterations to mature, further changes deriving from the contact with the reality.

Strategic Management: Strategic Planning (Definition)

"[…] strategic planning […] is the continuous process of making present entrepreneurial (risk-taking) decisions systematically and with the greatest knowledge of their futurity; organizing systematically the efforts needed to carry out these decisions; and measuring the results of these decisions against the expectations through organized, systematic feedback." (Peter F Drucker, "Management: Tasks, Responsibilities, Practices", 1973)

"The process of determining how a problem or opportunity may be responded to. Involves identifying problems or opportunities, analyzing relevant characteristics of the circumstances, organizing the formal response, deputizing a leader to head the response effort, and supervising the person(s) selected." (Robert McCrie, "Security Operations Management" 2nd Ed., 2006)

"Written record of a strategic plan, usually consisting of an overview, strategy charter, description of the current environment, research findings, tactics, roles and accountabilities, key performance indicators, and recommended next steps." (Teri Lund & Susan Barksdale, "10 Steps to Successful Strategic Planning", 2006)

"The implementation of an organization's objectives. Strategic planning decisions will have long-term impacts on the organization while operational decisions are day-to-day in nature." (Jae K Shim & Joel G Siegel, "Budgeting Basics and Beyond", 2008)

"The selection of short- and long-term objectives and the drawing up of tactical and strategic plans to achieve those objectives. After deciding on a set of strategies to be followed, the organization needs more specific plans, such as locations, methods of financing, and hours of operation. As these plans are made, they will be communicated throughout the organization. When implemented, the plans will serve to coordinate the efforts of all parts of the organization toward the company's objectives." (Jae K Shim & Joel G Siegel, "Budgeting Basics and Beyond", 2008)

"A deliberative, disciplined approach to producing fundamental decisions and actions that shape and guide what an organization (or other entity) is, what it does, and why it does it.” (John M Bryson, 2011)

"A long-range plan that serves as a business’s road map for the future. It includes the product lines and services, the number of employees, technology requirements, industry trends, competitor analysis, revenue and profitability goals, types of customers, and long-range marketing plans." (Gina Abudi & Brandon Toropov, "The Complete Idiot's Guide to Best Practices for Small Business", 2011)

"A series of processes in which an organization selects and arranges its businesses or services to keep the organization viable even when unexpected events distrupt one or more of its business's markets, products, or services." (Linda Volonino & Efraim Turban, "Information Technology for Management" 8th Ed., 2011)

"A high-level document that explains the organization's vision and mission, plus the approach that will be adopted to achieve this mission and vision, including the specific goals and objectives to be achieved during the period covered by the document." (Project Management Institute, "The Standard for Portfolio Management" 3rd Ed., 2012)

"The process by which an organization envisions its future and develops the necessary goals and procedures to achieve that vision." (Joan C Dessinger, "Fundamentals of Performance Improvement" 3rd Ed., 2012)

"A systematic process of envisioning a desired future and translating this vision into broadly defined goals or objectives and a sequence of steps to achieve them." (Robert F Smallwood, "Information Governance: Concepts, Strategies, and Best Practices", 2014)

"The process by which organizations identify a desired outcome, the resources required to support that outcome, and the plan to achieve the outcome. Typically, strategic planning is an important step in identifying the creation of new competitive advantages." (Evan Stubbs, "Big Data, Big Innovation", 2014)

"A process of selecting from alternative courses of action, matching that with the available resources, and combining these in a way that will most effectively achieve the objective; Intended action toward an organizational goal or objective." (Ken Sylvester, "Negotiating in the Leadership Zone", 2015)

"A formalised step-by-step set of procedures for coordinating the strategy process." (Duncan Angwin & Stephen Cummings, "The Strategy Pathfinder" 3rd Ed., 2017)

"A document used to communicate with the organization the organization’s goals, the actions needed to achieve those goals, and all the other critical elements developed during the planning exercise." (William Stallings, "Effective Cybersecurity: A Guide to Using Best Practices and Standards", 2018)

Business Intelligence: Indicators (Just the Quotes)

"If we view organizations as adaptive, problem-solving structures, then inferences about effectiveness have to be made, not from static measures of output, but on the basis of the processes through which the organization approaches problems. In other words, no single measurement of organizational efficiency or satisfaction - no single time-slice of organizational performance can provide valid indicators of organizational health." (Warren G Bennis, "General Systems Yearbook", 1962)

"The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor." (Donald T Campbell, "Assessing the impact of planned social change", 1976)

"Indicators tend to direct your attention toward what they are monitoring. It is like riding a bicycle: you will probably steer it where you are looking. If, for example, you start measuring your inventory levels carefully, you are likely to take action to drive your inventory levels down, which is good up to a point. But your inventories could become so lean that you can’t react to changes in demand without creating shortages. So because indicators direct one’s activities, you should guard against overreacting. This you can do by pairing indicators, so that together both effect and counter-effect are measured. Thus, in the inventory example, you need to monitor both inventory levels and the incidence of shortages. A rise in the latter will obviously lead you to do things to keep inventories from becoming too low." (Andrew S Grove, "High Output Management", 1983)

"So because indicators direct one’s activities, you should guard against overreacting. This you can do by pairing indicators, so that together both effect and counter-effect are measured. […] In sum, joint monitoring is likely to keep things in the optimum middle ground." (Andrew S Grove, "High Output Management", 1983)

"The first rule is that a measurement - any measurement - is better than none. But a genuinely effective indicator will cover the output of the work unit and not simply the activity involved. […] If you do not systematically collect and maintain an archive of indicators, you will have to do an awful lot of quick research to get the information you need, and by the time you have it, the problem is likely to have gotten worse." (Andrew S Grove, "High Output Management", 1983)

"The number of possible indicators you can choose is virtually limitless, but for any set of them to be useful, you have to focus each indicator on a specific operational goal. […] Put another way, which five pieces of information would you want to look at each day, immediately upon arriving at your office?" (Andrew S Grove, "High Output Management", 1983)

"All good KPIs that I have come across, that have made a difference, had the CEO’s constant attention, with daily calls to the relevant staff. [...] A KPI should tell you about what action needs to take place. [...] A KPI is deep enough in the organization that it can be tied down to an individual. [...] A good KPI will affect most of the core CSFs and more than one BSC perspective. [...] A good KPI has a flow on effect." (David Parmenter, "Pareto’s 80/20 Rule for Corporate Accountants", 2007)

"If the KPIs you currently have are not creating change, throw them out because there is a good chance that they may be wrong. They are probably measures that were thrown together without the in-depth research and investigation KPIs truly deserve." (David Parmenter, "Pareto’s 80/20 Rule for Corporate Accountants", 2007)

"Key performance indicators (KPIs) are the vital navigation instruments used by managers to understand whether their business is on a successful voyage or whether it is veering off the prosperous path. The right set of indicators will shine light on performance and highlight areas that need attention. ‘What gets measured gets done’ and ‘if you can’t measure it, you can’t manage it’ are just two of the popular sayings used to highlight the critical importance of metrics. Without the right KPIs managers are sailing blind." (Bernard Marr, "Key Performance Indicators (KPI): The 75 measures every manager needs to know", 2011)

"KRAs and KPIs KRA and KPI are two confusing acronyms for an approach commonly recommended for identifying a person’s major job responsibilities. KRA stands for key result areas; KPI stands for key performance indicators. As academics and consultants explain this jargon, key result areas are the primary components or parts of the job in which a person is expected to deliver results. Key performance indicators represent the measures that will be used to determine how well the individual has performed. In other words, KRAs tell where the individual is supposed to concentrate her attention; KPIs tell how her performance in the specified areas should be measured. Probably few parts of the performance appraisal process create more misunderstanding and bewilderment than do the notion of KRAs and KPIs. The reason is that so much of the material written about KPIs and KRAs is both." (Dick Grote, "How to Be Good at Performance Appraisals: Simple, Effective, Done Right", 2011)

"A statistical index has all the potential pitfalls of any descriptive statistic - plus the distortions introduced by combining multiple indicators into a single number. By definition, any index is going to be sensitive to how it is constructed; it will be affected both by what measures go into the index and by how each of those measures is weighted." (Charles Wheelan, "Naked Statistics: Stripping the Dread from the Data", 2012)

"Even if you have a solid indicator of what you are trying to measure and manage, the challenges are not over. The good news is that 'managing by statistics' can change the underlying behavior of the person or institution being managed for the better. If you can measure the proportion of defective products coming off an assembly line, and if those defects are a function of things happening at the plant, then some kind of bonus for workers that is tied to a reduction in defective products would presumably change behavior in the right kinds of ways. Each of us responds to incentives (even if it is just praise or a better parking spot). Statistics measure the outcomes that matter; incentives give us a reason to improve those outcomes." (Charles Wheelan, "Naked Statistics: Stripping the Dread from the Data", 2012)

"Once these different measures of performance are consolidated into a single number, that statistic can be used to make comparisons […] The advantage of any index is that it consolidates lots of complex information into a single number. We can then rank things that otherwise defy simple comparison […] Any index is highly sensitive to the descriptive statistics that are cobbled together to build it, and to the weight given to each of those components. As a result, indices range from useful but imperfect tools to complete charades." (Charles Wheelan, "Naked Statistics: Stripping the Dread from the Data", 2012)

"Defining an indicator as lagging, coincident, or leading is connected to another vital notion: the business cycle. Indicators are lagging or leading based on where economists believe we are in the business cycle: whether we are heading into a recession or emerging from one." (Zachary Karabell, "The Leading Indicators: A short history of the numbers that rule our world", 2014)

"[…] economics is a profession grounded in the belief that 'the economy' is a machine and a closed system. The more clearly that machine is understood, the more its variables are precisely measured, the more we will be able to manage and steer it as we choose, avoiding the frenetic expansions and sharp contractions. With better indicators would come better policy, and with better policy, states would be less likely to fall into depression and risk collapse." (Zachary Karabell, "The Leading Indicators: A short history of the numbers that rule our world", 2014)

"Our needs going forward will be best served by how we make use of not just this data but all data. We live in an era of Big Data. The world has seen an explosion of information in the past decades, so much so that people and institutions now struggle to keep pace. In fact, one of the reasons for the attachment to the simplicity of our indicators may be an inverse reaction to the sheer and bewildering volume of information most of us are bombarded by on a daily basis. […] The lesson for a world of Big Data is that in an environment with excessive information, people may gravitate toward answers that simplify reality rather than embrace the sheer complexity of it." (Zachary Karabell, "The Leading Indicators: A short history of the numbers that rule our world", 2014)

"Statistics are meaningless unless they exist in some context. One reason why the indicators have become more central and potent over time is that the longer they have been kept, the easier it is to find useful patterns and points of reference." (Zachary Karabell, "The Leading Indicators: A short history of the numbers that rule our world", 2014)

"The indicators - through no particular fault of anyone in particular - have not kept up with the changing world. As these numbers have become more deeply embedded in our culture as guides to how we are doing, we rely on a few big averages that can never be accurate pictures of complicated systems for the very reason that they are too simple and that they are averages. And we have neither the will nor the resources to invent or refine our current indicators enough to integrate all of these changes." (Zachary Karabell, "The Leading Indicators: A short history of the numbers that rule our world", 2014)

"We don’t need new indicators that replace old simple numbers with new simple numbers. We need instead bespoke indicators, tailored to the specific needs and specific questions of governments, businesses, communities, and individuals." (Zachary Karabell, "The Leading Indicators: A short history of the numbers that rule our world", 2014)

"Yet our understanding of the world is still framed by our leading indicators. Those indicators define the economy, and what they say becomes the answer to the simple question 'Are we doing well?'" (Zachary Karabell, "The Leading Indicators: A short history of the numbers that rule our world", 2014)

"[…] an overall green status indicator doesn’t mean anything most of the time. All it says is that the things under measurement seem okay. But there always will be many more things not under measurement. To celebrate green indicators is to ignore the unknowns. […] The tendency to roll up metrics into dashboards promotes ignorance of the real situation on the ground. We forget that we only see what is under measurement. We only act when something is not green." (Sriram Narayan, "Agile IT Organization Design: For Digital Transformation and Continuous Delivery", 2015)

"Financial measures are a quantification of an activity that has taken place; we have simply placed a value on the activity. Thus, behind every financial measure is an activity. I call financial measures result indicators, a summary measure. It is the activity that you will want more or less of. It is the activity that drives the dollars, pounds, or yen. Thus financial measures cannot possibly be KPIs." (David Parmenter, "Key Performance Indicators: Developing, implementing, and using winning KPIs" 3rd Ed., 2015)

"Key performance indicators (KPIs) are those indicators that focus on the aspects of organizational performance that are the most critical for the current and future success of the organization." (David Parmenter, "Key Performance Indicators: Developing, implementing, and using winning KPIs" 3rd Ed., 2015)

"Key Performance Indicators (KPIs) in many organizations are a broken tool. The KPIs are often a random collection prepared with little expertise, signifying nothing. [...] KPIs should be measures that link daily activities to the organization’s critical success factors (CSFs), thus supporting an alignment of effort within the organization in the intended direction." (David Parmenter, "Key Performance Indicators: Developing, implementing, and using winning KPIs" 3rd Ed., 2015)

"Most organizational measures are very much past indicators measuring events of the last month or quarter. These indicators cannot be and never were KPIs." (David Parmenter, "Key Performance Indicators: Developing, implementing, and using winning KPIs" 3rd Ed., 2015)

"We need indicators of overall performance that need only be reviewed on a monthly or bimonthly basis. These measures need to tell the story about whether the organization is being steered in the right direction at the right speed, whether the customers and staff are happy, and whether we are acting in a responsible way by being environmentally friendly. These measures are called key result indicators (KRIs)." (David Parmenter, "Key Performance Indicators: Developing, implementing, and using winning KPIs" 3rd Ed., 2015)

"Indicators represent a way of 'distilling' the larger volume of data collected by organizations. As data become bigger and bigger, due to the greater span of control or growing complexity of operations, data management becomes increasingly difficult. Actions and decisions are greatly influenced by the nature, use and time horizon (e.g., short or long-term) of indicators." (Fiorenzo Franceschini et al, "Designing Performance Measurement Systems: Theory and Practice of Key Performance Indicators", 2019)

"Indicators take on the role of real 'conceptual technologies', capable of driving organizational management in intangible terms, conditioning the 'what' to focus and the 'how'; in other words, they become the beating heart of the management, operational and technological processes." (Fiorenzo Franceschini et al, "Designing Performance Measurement Systems: Theory and Practice of Key Performance Indicators", 2019)

"Monitoring a process requires identifying specific activities, responsibilities and indicators for testing effectiveness and efficiency. Effectiveness means setting the right goals and objectives, making sure that they are properly accomplished (doing the right things); effectiveness is measured comparing the achieved results with target objectives. On the other hand, efficiency means getting the most (output) from the available (input) resources (doing things right): efficiency defines a link between process performance and available resources." (Fiorenzo Franceschini et al, "Designing Performance Measurement Systems: Theory and Practice of Key Performance Indicators", 2019)

"People do care about how they are measured. What can we do about this? If you are in the position to measure something, think about whether measuring it will change people’s behaviors in ways that undermine the value of your results. If you are looking at quantitative indicators that others have compiled, ask yourself: Are these numbers measuring what they are intended to measure? Or are people gaming the system and rendering this measure useless?" (Carl T Bergstrom & Jevin D West, "Calling Bullshit: The Art of Skepticism in a Data-Driven World", 2020)

"A KPI is a performance measure that demonstrates how effectively an organisation is achieving its critical objectives. They are used to track performance over a period of time to ensure the organisation is heading in the desired direction, and are quantifiable to guide whether activities need to be dialled up or down, resources adjusted or management resource focused on understanding what is in play that may be holding back the organisation." (Ian Wallis, "Data Strategy: From definition to execution", 2021)

"The KPI juggernaut has been misused and abused in too many organisations to the extent it has devalued the concept of KPIs. KPIs used well - the ten things that really matter to an organisation - can, in my experience, be a real galvanising force to get focus and attention put in those areas which really can make a difference. The rest is a distraction, there through some misplaced view that more adds value when actually it detracts through losing the focus from where it needs to be." (Ian Wallis, "Data Strategy: From definition to execution", 2021)

Business Intelligence: Reporting (Just the Quotes)

"A man's judgment cannot be better than the information on which he has based it. Give him no news, or present him only with distorted and incomplete data, with ignorant, sloppy, or biased reporting, with propaganda and deliberate falsehoods, and you destroy his whole reasoning process and make him somewhat less than a man." (Arthur H Sulzberger, [speech] 1948)

"The secret language of statistics, so appealing in a fact-minded culture, is employed to sensationalize, inflate, confuse, and oversimplify. Statistical methods and statistical terms are necessary in reporting the mass data of social and economic trends, business conditions, 'opinion' polls, the census. But without writers who use the words with honesty and understanding and readers who know what they mean, the result can only be semantic nonsense." (Darell Huff, "How to Lie with Statistics", 1954)

"To be worth much, a report based on sampling must use a representative sample, which is one from which every source of bias has been removed." (Darell Huff, "How to Lie with Statistics", 1954)

"It is probable that one day we shall begin to draw organization charts as a series of linked groups rather than as a hierarchical structure of individual 'reporting' relationships." (Douglas McGregor, "The Human Side of Enterprise", 1960)

"[...] as the planning process proceeds to a specific financial or marketing state, it is usually discovered that a considerable body of 'numbers' is missing, but needed numbers for which there has been no regular system of collection and reporting; numbers that must be collected outside the firm in some cases. This serendipity usually pays off in a much better management information system in the form of reports which will be collected and reviewed routinely." (William H. Franklin Jr., Financial Strategies, 1987)

"Intangible assets [...] surpass physical assets in most business enterprises, both in value and contribution to growth, yet they are routinely expensed in the financial reports and hence remain absent from corporate balance sheets. This asymmetric treatment of capitalizing (considering as assets) physical and financial investment while expensing intangibles leads to biased and deficient reporting of firms’ performance and value." (Baruch Lev, "Intangibles: Management, Measurement, and Reporting", 2000)

"Project planning is the key to effective project management. Detailed and accurate planning of a project produces the managerial information that is the basis of project justification (costs, benefits, strategic impact, etc.) and the defining of the business drivers (scope, objectives) that form the context for the technical solution. In addition, project planning also produces the project schedules and resource allocations that are the framework for the other project management processes: tracking, reporting, and review." (Rob Thomsett, "Radical Project Management", 2002)

"Many management reports are not a management tool; they are merely memorandums of information. As a management tool, management reports should encourage timely action in the right direction, by reporting on those activities the Board, management, and staff need to focus on. The old adage 'what gets measured gets done' still holds true." (David Parmenter, "Pareto’s 80/20 Rule for Corporate Accountants", 2007)

"Reporting to the Board is a classic 'catch-22' situation. Boards complain about getting too much information too late, and management complains that up to 20% of their time is tied up in the Board reporting process. Boards obviously need to ascertain whether management is steering the ship correctly and the state of the crew and customers before they can relax and 'strategize' about future initiatives. The process of assessing the current status of the organization from the most recent Board report is where the principal problem lies. Board reporting needs to occur more efficiently and effectively for both the Board and management." (David Parmenter, "Pareto’s 80/20 Rule for Corporate Accountants", 2007)

"Readability in visualization helps people interpret data and make conclusions about what the data has to say. Embed charts in reports or surround them with text, and you can explain results in detail. However, take a visualization out of a report or disconnect it from text that provides context (as is common when people share graphics online), and the data might lose its meaning; or worse, others might misinterpret what you tried to show." (Nathan Yau, "Data Points: Visualization That Means Something", 2013)

"Another way to secure statistical significance is to use the data to discover a theory. Statistical tests assume that the researcher starts with a theory, collects data to test the theory, and reports the results - whether statistically significant or not. Many people work in the other direction, scrutinizing the data until they find a pattern and then making up a theory that fits the pattern." (Gary Smith, "Standard Deviations", 2014)

"These practices - selective reporting and data pillaging - are known as data grubbing. The discovery of statistical significance by data grubbing shows little other than the researcher’s endurance. We cannot tell whether a data grubbing marathon demonstrates the validity of a useful theory or the perseverance of a determined researcher until independent tests confirm or refute the finding. But more often than not, the tests stop there. After all, you won’t become a star by confirming other people’s research, so why not spend your time discovering new theories? The data-grubbed theory consequently sits out there, untested and unchallenged." (Gary Smith, "Standard Deviations", 2014)

"A dashboard is like the executive summary of a report. We read executive summaries and skip the body of the report if the summary is more or less in line with our expectations. Trouble is, measurement is never exhaustive. It is only when we dive in that we realize what areas may have been missed." (Sriram Narayan, "Agile IT Organization Design: For Digital Transformation and Continuous Delivery", 2015)

"'Getting it right the first time' is a rare achievement, and ascertaining the organization’s winning KPIs and associated reports is no exception. The performance measure framework and associated reporting is just like a piece of sculpture: you can be criticized on taste and content, but you can’t be wrong. The senior management team and KPI project team need to ensure that the project has a just-do-it culture, not one in which every step and measure is debated as part of an intellectual exercise." (David Parmenter, "Key Performance Indicators: Developing, implementing, and using winning KPIs" 3rd Ed., 2015)

"In order to get measures to drive performance, a reporting framework needs to be developed at all levels within the organization." (David Parmenter, "Key Performance Indicators: Developing, implementing, and using winning KPIs" 3rd Ed., 2015)

"Statistics, because they are numbers, appear to us to be cold, hard facts. It seems that they represent facts given to us by nature and it’s just a matter of finding them. But it’s important to remember that people gather statistics. People choose what to count, how to go about counting, which of the resulting numbers they will share with us, and which words they will use to describe and interpret those numbers. Statistics are not facts. They are interpretations. And your interpretation may be just as good as, or better than, that of the person reporting them to you." (Daniel J Levitin, "Weaponized Lies", 2017)

Business Intelligence: Metric (Definitions)

"(1) The degree to which a product, process, or project possesses some attribute of interest. (2) A measured quantity (such as size, effort, duration, or quality). (3) The distance between two points in a vector space." (Richard D Stutzke, "Estimating Software-Intensive Systems: Projects, Products, and Processes", 2005)

"A summarizable numerical value used to monitor business activity; it is also known as a fact." (Reed Jacobsen & Stacia Misner, "Microsoft SQL Server 2005 Analysis Services Step by Step", 2006)

"A metric is a measurement. When a plan is put into place, a way to measure the outcome is needed. When a market share forecast is created and the outcomes are measured at a future date, the planned metric is compared with the actual metric to determine the degree to which the metric was met. From this data, strategies can be revised and tactical options can be reconsidered." (Steven Haines, "The Product Manager's Desk Reference", 2008)

"A numerical value describing a procedure, process, product attribute, or goal. A distinction is made between basic metrics (that can be measured directly) and derived metrics which result from mathematical operations using basic metrics." (Lars Dittmann et al, "Automotive SPICE in Practice", 2008)

"a measurement of some parameter, usually used in the assessment of a technology, approach, or design." (Bruce P Douglass, "Real-Time Agility: The Harmony/ESW Method for Real-Time and Embedded Systems Development", 2009)

"A metric is a standard unit of measure, such as meter or mile for length, or gram or ton for weight, or, more generally, part of a system of parameters, or systems of measurement, or a set of ways of quantitatively and periodically measuring, assessing, controlling, or selecting a person, process, event, or institution, along with the procedures to carry out measurements and the procedures for the interpretation of the assessment in the light of previous or comparable assessments." (Mark S Merkow & Lakshmikanth Raghavan, "Secure and Resilient Software Development", 2010)

"Groupings of data, or numbers, that reflect specific measures or subjects." (Annetta Cortez & Bob Yehling, "The Complete Idiot's Guide To Risk Management", 2010)

"a calculated value based on measurements used to monitor and control a process or business activity. Most metrics are ratios comparing one measurement to another." (DAMA International, "The DAMA Dictionary of Data Management", 2011)

"A specific, measurable standard against which actual performance is compared." (Linda Volonino & Efraim Turban, "Information Technology for Management" 8th Ed., 2011) 

"Generally, a unit of measure selected used to monitor and control a process." (DAMA International, "The DAMA Dictionary of Data Management", 2011)

"In a data warehouse, numeric facts that measure a business characteristic of interest to the end user." (Carlos Coronel et al, "Database Systems: Design, Implementation, and Management" 9th Ed., 2011)

"Measurement of a particular characteristic of a task (for example, duration, effort, quality, cost, value delivered, or customer satisfaction)." (Charles Cooper & Ann Rockley, "Managing Enterprise Content: A Unified Content Strategy" 2nd Ed., 2012)

"1. A value from measuring a certain program or component attribute. Finding metrics is a task for static analysis. 2. A measurement scale and the method used for measurement." (Tilo Linz et al, "Software Testing Foundations" 4th Ed., 2014)

"A method of measuring something. It provides quantifiable data used to gauge the effectiveness of a process; metrics are commonly used to measure the effectiveness of a help desk." (Darril Gibson, "Effective Help Desk Specialist Skills", 2014)

"A value that you use to study some aspect of a project. A metric can be an attribute (such as the number of bugs) or a calculated value (such as the number of bugs per line of code)." (Rod Stephens, "Beginning Software Engineering", 2015)

"A measurement used to support the monitoring of a key performance indicator (KPI). A metric can have targets and can be used as a service level." (by Brian Johnson & Leon-Paul de Rouw, "Collaborative Business Design", 2017)

"Facts and figures representing the effectiveness of business processes that organizations track and monitor to assess the state of the company." (Jonathan Ferrar et al, "The Power of People: Learn How Successful Organizations Use Workforce Analytics To Improve Business Performance", 2017)

"A metric is the measurement of a particular characteristic of a company’s performance or efficiency. Metrics are the variables whose measured values are tied to the performance of the organization. They are also known as the performance metrics because they are performance indicators." (Amar Sahay, "Business Analytics" Vol. I, 2018)

"A measurable quantity that indicates progress toward some goal." (O Sami Saydjari, "Engineering Trustworthy Systems: Get Cybersecurity Design Right the First Time", 2018)

"Any number (often one calculated using two or more input numbers) used to evaluate some part of an organization's performance." (Marci S. Thomas & Kim Strom-Gottfried, "Best of Boards" 2nd Ed., 2018)

"Metrics are agreed-upon measures used to evaluate how well the organization is progressing toward the Portfolio, Large Solution, Program, and Team’s business and technical objectives." (Dean Leffingwell, "SAFe 4.5 Reference Guide: Scaled Agile Framework for Lean Enterprises" 2nd Ed., 2018)

"In a machine learning context, a metric is a measure of how good or bad a particular model is at its task. In a software context, a metric is a measure defined for an application, program, or function." (Alex Thomas, "Natural Language Processing with Spark NLP", 2020)

"A business calculation defined by an expression built with functions, facts, attributes, or other metrics." (Microstrategy)

"A measurement scale and the method used for measurement" (ISO 14598)

"Quantifiable measures used to track, monitor, and gauge the results and success of various business processes. Metrics are meant to communicate a company’s progression toward certain long and short term objectives. This often requires the input of key stakeholders in the business as to which metrics matter to them." (Insight Software)

"Tools designed to facilitate decision making and improve performance and accountability through collection, analysis, and reporting of relevant performance-related data." (NIST SP 800-55)

Business Intelligence: Dashboards (Definitions)

"A dashboard is a visual display of the most important information needed to achieve one or more objectives; consolidated and arranged on a single screen so the information can be monitored at a glance." (Stephen Few, "Dashboard Confusion", Intelligent Enterprise, 2004)

Dashboard reports: "Highly summarized, often graphical, representations of the state of the business that are often used by executives and strategic decision makers." (Reed Jacobsen & Stacia Misner, "Microsoft SQL Server 2005 Analysis Services Step by Step", 2006)

Dashboard: "A means of providing information in a straightforward way. Like the part in a car it is named after, a business dashboard allows executives to see key metrics about anything from monthly sales to manufacturing downtime." (Tony Fisher, "The Data Asset", 2009)

Dashboard (also called performance dashboard): "The presentation of key business measurements on a single interface designed for quick interpretation, often using graphics. The most effective dashboards are supported by a full data mart that enables drilling down into more detailed data to better understand the indicators." (Laura Reeves, "A Manager's Guide to Data Warehousing", 2009)

Dashboard: "A visual display mechanism to enable business users at every level to receive the information they need to make better decisions that improve business performance." (Paulraj Ponniah, "Data Warehousing Fundamentals for IT Professionals", 2010)

Dashboard: "A BI tool that provides a comprehensive, at-a-glance view of corporate performance with graphical presentations, resembling a dashboard of a car. These graphical presentations show performance measures, trends, and exceptions, and integrate information from multiple business areas." (Linda Volonino & Efraim Turban, "Information Technology for Management" 8th Ed., 2011)

Dashboard: "A technique to represent vast amounts of decision-support information at an amalgamated level using tabular and graphic representation, such as graphs and traffic lights." (Paul C Dinsmore et al, "Enterprise Project Governance", 2012)

Dashboards: "Business intelligence tools that display performance indicators, present data and information at both summary and detailed levels, and assist decision-makers employing them to act on the information they present." (Joan C Dessinger, "Fundamentals of Performance Improvement" 3rd Ed., 2012)

Dashboard: "A view that displays ranges of data in a graphical format. Key performance indicators (KPIs) or any element can be displayed in a dashboard. Each element is represented by a gauge that displays the data ranges that are defined. Links to comments, trend data, and element properties can also be provided." (Jim Davis & Aiman Zeid, "Business Transformation", 2014)

"[...] dashboards indicate the status of a performance metric at a given point in time. [...] dashboards are used to represent actual granular data, they contain data that is more recent than that of scorecards." (Saumya Chaki, "Enterprise Information Management in Practice", 2015)

Data dashboard: "A management-level online report capturing data conditions and trends."(Gregory Lampshire, "The Data and Analytics Playbook", 2016)

"A dashboard is a visual display of data used to monitor conditions and/or facilitate understanding."

(Steve Wexler et al, "The Big Book of Dashboards: Visualizing Your Data Using Real-World Business Scenarios", 2017)

"A dashboard is a reporting tool that consolidates, aggregates and arranges measurements, metrics (measurements compared to a goal) and sometimes scorecards on a single screen so information can be monitored at a glance. Dashboards differ from scorecards in being tailored to monitor a specific role or generate metrics reflecting a particular point of view; typically they do not conform to a specific management methodology." (Information Management) [also (Intrafocus)] 

"Dashboards are a reporting mechanism that aggregate and display metrics and key performance indicators (KPIs), enabling them to be examined at a glance by all manner of users before further exploration via additional business analytics (BA) tools." (Gartner)

Business Intelligence: Key Performance Indicator (Definitions)

"A performance measure that is indicative of the organization's performance within a specific area." (William A Giovinazzo, "Internet-Enabled Business Intelligence", 2002)

"A key performance indicator is a metric that provides business users with an indication of the current and historical performance of an aspect of the business." (Claudia Imhoff et al, "Mastering Data Warehouse Design", 2003)

"A measurement of business operations that compares a value at a specified point in time to a predetermined goal and, optionally, determines a trend direction. Often, a KPI is displayed using a graphical image such as a stoplight or a gauge using colors and relative indicators according to predetermined business rules." (Reed Jacobsen & Stacia Misner, "Microsoft SQL Server 2005 Analysis Services Step by Step", 2006) 

"An important set of metrics (see Metrics) used to determine how well a product is performing in the market." (Steven Haines, "The Product Manager's Desk Reference", 2008)

"Financial and non-financial metrics used to assess the strategic performance of an organization." (Ralph Kimball, "The Data Warehouse Lifecycle Toolkit", 2008)

"Quantifiable, measurable objectives agreed to beforehand and that reflect the critical success factors of an organization." (Tilak Mitra et al, "SOA Governance", 2008)

"A piece of information that an organization considers a crucial reflection of how well it's doing." (Ken Withee, "Microsoft Business Intelligence For Dummies", 2010)

"Financial and nonfinancial metrics used by an organization to define and evaluate how successful it is, typically in terms of making progress toward its goals." (Janice M Roehl-Anderson, "IT Best Practices for Financial Managers", 2010) 

"A business calculation (metric) with associated target values or ranges that allows macro level insights into the business process to manage profitability and monitor strategic impact." (DAMA International, "The DAMA Dictionary of Data Management", 2011)

"In business intelligence, refers to quantifiable measurements (numeric or scale-based) that assess a company’s effectiveness or success in reaching strategic and operational goals. Examples of KPI are product turnovers, sales by promotion, sales by employee, earnings per share, etc." (Carlos Coronel et al, "Database Systems: Design, Implementation, and Management 9th Ed", 2011)

"Metrics that measure the actual performance of critical aspects of IT, such as critical projects and applications, servers, the network, and so forth, against predefined goals and objectives." (Linda Volonino & Efraim Turban, "Information Technology for Management" 8th Ed., 2011)

"A measure used to quantify performance and outcomes." (Carl F Lehmann, "Strategy and Business Process Management", 2012)

"Quantitative performance measures that define the critical success factors of an organization, help the organization measure progress toward its goals and objectives, and identify areas for organizational performance and improvement." (Joan C Dessinger, "Fundamentals of Performance Improvement" 3rd Ed., 2012)

"A high-level measurement meant to indicate how well an individual or group is performing a set of activities that is considered critical to the overall success of an endeavor." (Project Management Institute, "Navigating Complexity: A Practice Guide", 2014)

"A measure that indicates the achievement of a specific objective." (Sally-Anne Pitt, "Internal Audit Quality", 2014)

"A measurement that shows whether an organization is progressing toward its stated goals." (Jim Davis & Aiman Zeid, "Business Transformation: A Roadmap for Maximizing Organizational Insights", 2014)

"Quantitative and measurable statement used to judge whether or not a goal has been reached; linked to a measurement and to the means of evaluation." (Gilbert Raymond & Philippe Desfray, "Modeling Enterprise Architecture with TOGAF", 2014)

"A set of metrics directly linked to the desired corporate objective (e.g., shareholder value) and explicitly integrated into the firm's incentive compensation system." (Thomas C Wilson, "Value and Capital Management", 2015)

"Most frequently referred to as KPIs. Metrics that indicate the performance of the business." (Brittany Bullard, "Style and Statistics", 2016)

"A set of business metrics used to determine whether a person, product, group, or division is successful." (Pamela Schure & Brian Lawley, "Product Management For Dummies", 2017)

"A variable or metric against which the success of a function or business is judged." (Jonathan Ferrar et al, "The Power of People: Learn How Successful Organizations Use Workforce Analytics To Improve Business Performance", 2017)

"A quantifiable measure used to evaluate the success of an organization, strategic initiative, employee, etc., in meeting the objectives for performance." (H James Harrington & William S Ruggles, "Project Management for Performance Improvement Teams", 2018)

"Quantifiable measurements, agreed to beforehand, that reflect the critical success factors of an organization." (William Stallings, "Effective Cybersecurity: A Guide to Using Best Practices and Standards", 2018)

"Used to assess and measure the performance of a specific business task. For example sales results in terms of order rates over a quarterly (3 month) period." (BCS Learning & Development Limited, "CEdMA Europe", 2019)

"KPIs are metrics defined to measure business performance of an enterprise. This term is related to BPM." (Martin Oberhofer et al, "The Art of Enterprise Information Architecture", 2010)

"A key performance indicator (KPI) is a high-level measure of system output, traffic or other usage, simplified for gathering and review on a weekly, monthly or quarterly basis." (Gartner)

"An acronym for Key Performance Indicator. These are key indicators to the health of the business." (BI System Builders)

"A business calculation that allows macro level insights into the business process to manage profitability." (Information Management)

"A type of performance measurement an organization may use to evaluate its success." (Board International)

"Business metrics used to evaluate factors that are crucial to organizational success." (Insight Software)

"Personalized performance metrics and benchmarks that drive the financial and operational success of the company." (Appian)

"A predefined measure that is used to track performance of a strategic goal, objective, plan, initiative, or business process. A KPI is evaluated against a target. An explicit and measurable value taken directly from a data source. Key performance indicators (KPIs) are used to measure performance in a specific area, for example, revenue per customer." (Microsoft)

"An indicator gauging how well a company progresses in numerous areas such as finance, customer service, and product availability and distribution." (Microstrategy)

"Key Performance Indicator - is a critical measurement of the performance of essential tasks, operations, or processes in a company. A KPI will usually unambiguously reveal conditions or performance that is outside the norm and that signals a need for managerial intervention." (Targit)

"Key performance indicators or KPIs […] are visual indicators in the form of color-coded shapes that are tied to a pre-defined, critical threshold. When the threshold is crossed, the KPI’s function is to alert key personnel so that they can take the necessary action." (Logi Analytics) [source]

"Key performance indicators (KPIs) are business metrics used by corporate executives and other managers to track and analyze factors deemed crucial to the success of an organization." (Techtarget) [source] 

Business Intelligence: Between Products, Partners, People and Processes

Business Intelligence Series

In the previous post, “BI between Potential, Reality, Quality and Stories”, I was commenting five of the important findings of a study led by KPMG in respect to the state of art in BI initiatives. My comments were centered mainly on the first 3 of the 4Ps (Products, People, Partners, respectively Processes) considered in ITSM (IT Service Management). The connection to IT Service Management isn’t accidental, BI being also an organizational capability. Many of the aspects related to the 4Ps perspectives, reveal the maturity of an organization in leveraging its BI infrastructure.  In this post I would like to consider BI landscape from these 4 perspectives.

Products

Products or technology perspective has within BI context a dual nature. First of all we have to consider the BI infrastructure – the whole set of BI tools we have at disposal for our shiny reports. Secondly, because the BI infrastructure doesn’t stand on itself, we have to consider also IT infrastructure on which BI infrastructure is based upon – a full range of ISs (Information Systems) in which data are entered, processed, transported and consumed before they are used by the BI tools. For Data Quality issues, we often have to consider the broader perspective, and tackle the problems at the source. Otherwise we might arrive to treat the symptoms and not the causes. It’s important to note that the two layers or perspectives are interconnected, the consequences being bidirectional.

A typical BI infrastructure revolves around several databases, maybe one or more data warehouses and data marts, and one or more reporting systems. Within the most basic scenario, the data flow is unidirectional from databases to data warehouse/marts, reports being built on top of the data warehouse/marts or directly on the IS’ databases. In more complex scenarios, the data can flow between the various ISs when they were integrated, and even between data warehouses/marts, within a unidirectional or bidirectional flow.  Unless the reports are based directly on the ISs’ databases, such architectures lead to data duplication, conversions between complex schemas, delays between the various layers, to mention just a few of the most important implications. In some point in time the complexity falls down on you.

One of the problems I met is that a considerable percent of the IS are not developed to address BI requirements. It starts with data validation, with the way data are modeled, structured, formatted and made available for BI consumption. If you want to increase the quality of your data, you have sooner or later to address them. It’s important thus the degree to which the systems are designed to cover the BI needs in particular, and decision making in general. This presumes that BI requirements need to be addressed in early phases of implementations, software design or when tools are consider for purchase.

In addition many ISs come with their own (standard) reports or reporting frameworks, becoming thus part of your BI infrastructure, intended or unintended. Even if such reports are intended to cover basic immediate reporting requirements, they not always so easy to consume, the logic behind them is not visible, are hard to extend, are not always tested, the additional reports built in other tools need to be synchronized with them, etc.

Partners

We gather huge volumes of data, we are drowning in it; we want to take decision rooted in data and get visibility into the past, actual and future state of business. How can we achieve that if we don’t have the knowledge and human resources to achieve that? “Partners” is the magic word – external suppliers specialized, in theory, to provide this kind of services: BI analysts and developers, business analysts, data miners, and other IT professionals work together in order to build your BI infrastructure. One detail many people forget is that BI tools provide potentiality; are the skills and knowledge of those working with them that transforms that potentiality into success. On their capabilities depends the success of such projects. Not to forget that BI projects are similar to other IT projects, falling under same type of fallacies plus a few other fallacies of their own derived from exploratory and complex nature of BI projects.

There is a dual nature also in “partners” perspective – except the external perspective which concerns the external partners and the IT department or the business as a whole, there is also the internal perspective in which the IT department plays again a central role. I heard it often loudly affirmed that the other departments are customers of the IT department, or the reciprocal. I have seen also this conception brought to extreme, in which the IT had no word to say in what concerns the IT infrastructure in general, respectively the BI infrastructure in particular. As long the IT department isn’t treated as a business partner, an organization will be more likely sabotaged from inside. Sabotage it’s a word too strong maybe, though it kind of reflects the state of art.

People

Same as partners, people perspective includes a considerable variety of types: IT staff, executives, managers, end-users and other types of stakeholders, each of them with a word to say, grouped in various groups of interests that don’t always converge, situations in which politics plays a major role. It’s actually interesting to see how the decision for a given BI solution is made, how the solution takes its place into the landscape, how it’s used and misused, how personalities and knowledge harness it or stand in the way. I feel that there are organizations (people) which do BI just for the sake of doing something, copying sometimes recipes of success, without uniting the dots, without clear goals and strategy. There are people who juggle with numbers and BI concepts without knowing their meaning and what they involve. This aspect is reflected in how BI tools are selected, implemented and used.

Having the best tools, consultants and highest data quality, won’t guarantee the success of BI initiative without users’ acceptance, without teaching them how to make constructive use of tools and data, on how to use and built models in order to solve the problems the business is confronted with, on how to address strategic, tactical and operational requirements. The transformation from a robot to a knowledge worker doesn’t happen over night. Is needed to make people aware of the various aspects of BI – data quality, process and data ownership, on how models can be used and misused, on how models evolve or become obsolete, how the BI infrastructure has to evolve with the business’ dynamics. There are so many aspects that need to be considered. It’s a continuous learning process.

Processes

In processes' perspective can be depicted a dual nature too. First of all we have to consider the processes which are used to manage efficiently and effectively the whole BI infrastructure. They are widely discussed in various methodologies like ITIL, whose implementation is thoroughly documented. Secondly, it’s the reflection of departmental processes within the various data perspectives – how they are measured, and how the measurements are further used for continuous improvement. 

Considering that this aspect is correlated with an organization’s capability model, I don’t think that many organizations go/rich that far. Sure the trend is to define meaningful KPIs, growth, health and other type of metrics, but the question is – are you using those metrics constructively, are you aligning them with your strategic, tactic and operational goals? I think there is lot of potential in this, though in order to measure processes accordingly is imperative to have also the system designed for this purpose. Back to technological perspective…

 

Previous Post <<||>> Next Post

Business Intelligence: Reports Types

    Have you ever wondered how many types of reports are there? In Information Systems (IS) nomenclature I found the following different types of reports considered:

    Standard reports – reports that are coming with a software application/package, as opposed of custom reports, reports created on customers’ request.

    Ad-hoc reports – reports built usually to satisfy one-time requests, though they can easily evolve to a standard report.

    Graphic reports- reports providing graphical visualization of data with the help of charts

    Transactional reports (OLAP reports) – reports built in transactional systems, containing up-to-date data.

    Analytic reports (OLAP reports) – reports built in an OLAP environment, containing data desynchronized from the OLTP environment, the data being refreshed on a periodic basis.

    Predictive reports – reports relying on powerful DM models and predictive technique.

    Parameterized reports – reports whose output is based on a set of predefined parameters.

    Linked reports – reports that provide an access point to other reports.

    Snapshot reports – reports that contain data retrieved at a specific point of time.

    Cached reports – reports saved in order to improve the performance by reducing the number of requests to the database/report engine.

    Click-through reports – reports whose display is based on interactive data selection

    Drilldown reports – a set of reports on the same topic showing data at different levels of details, the navigation being made from higher to lower level of details.

    Drill-through reports – reports accessible through a hyperlink from the original report.

    Sub-reports – a report contained in the body of another report, allowing for example the display of parent/child or header/lines relations.

    Metric-based reports – reports supposed to encompass the various types of business metrics; they can be further categorized in:
Health Metrics – reports designed to show the health of a system in terms of its usage and the adherence to the processes defined.
Growth Metrics - reports designed to show the growth of a system in terms of data, transaction or amount volume.
KPI (Key Process Indicator) reports – reports designed to measure an organization progress towards set organizational goals.
LPI (Lean Process Indicator) reports – reports designed to reflect business’ progress toward Lean Management organizational goals.

    Dashboards – reports offering an eye-bird view of several key performance indicators.

    Another characterization of reports can be based on the functional department for which the report is created, thus we can speak of financial reports, operational reports, sourcing reports, (global) supply chain reports, marketing reports, maintenance reports, etc.

Note:
    The term of financial report might refer in special to financial statements.

David Parmenter - Collected Quotes

"All good KPIs that I have come across, that have made a difference, had the CEO’s constant attention, with daily calls to the relevant staff. [...] A KPI should tell you about what action needs to take place. [...] A KPI is deep enough in the organization that it can be tied down to an individual. [...] A good KPI will affect most of the core CSFs and more than one BSC perspective. [...] A good KPI has a flow on effect." (David Parmenter, "Pareto’s 80/20 Rule for Corporate Accountants", 2007)

"If the KPIs you currently have are not creating change, throw them out because there is a good chance that they may be wrong. They are probably measures that were thrown together without the in-depth research and investigation KPIs truly deserve." (David Parmenter, "Pareto’s 80/20 Rule for Corporate Accountants", 2007)

"Many management reports are not a management tool; they are merely memorandums of information. As a management tool, management reports should encourage timely action in the right direction, by reporting on those activities the Board, management, and staff need to focus on. The old adage “what gets measured gets done” still holds true." (David Parmenter, "Pareto’s 80/20 Rule for Corporate Accountants", 2007)

"Reporting to the Board is a classic 'catch-22' situation. Boards complain about getting too much information too late, and management complains that up to 20% of their time is tied up in the Board reporting process. Boards obviously need to ascertain whether management is steering the ship correctly and the state of the crew and customers before they can relax and 'strategize' about future initiatives. The process of assessing the current status of the organization from the most recent Board report is where the principal problem lies. Board reporting needs to occur more efficiently and effectively for both the Board and management." (David Parmenter, "Pareto’s 80/20 Rule for Corporate Accountants", 2007)

"Financial measures are a quantification of an activity that has taken place; we have simply placed a value on the activity. Thus, behind every financial measure is an activity. I call financial measures result indicators, a summary measure. It is the activity that you will want more or less of. It is the activity that drives the dollars, pounds, or yen. Thus financial measures cannot possibly be KPIs." (David Parmenter, "Key Performance Indicators: Developing, implementing, and using winning KPIs" 3rd Ed., 2015)

"'Getting it right the first time' is a rare achievement, and ascertaining the organization’s winning KPIs and associated reports is no exception. The performance measure framework and associated reporting is just like a piece of sculpture: you can be criticized on taste and content, but you can’t be wrong. The senior management team and KPI project team need to ensure that the project has a just-do-it culture, not one in which every step and measure is debated as part of an intellectual exercise." (David Parmenter, "Key Performance Indicators: Developing, implementing, and using winning KPIs" 3rd Ed., 2015)

"In order to get measures to drive performance, a reporting framework needs to be developed at all levels within the organization." (David Parmenter, "Key Performance Indicators: Developing, implementing, and using winning KPIs" 3rd Ed., 2015)

"Key performance indicators (KPIs) are those indicators that focus on the aspects of organizational performance that are the most critical for the current and future success of the organization." (David Parmenter, "Key Performance Indicators: Developing, implementing, and using winning KPIs" 3rd Ed., 2015)

"Key Performance Indicators (KPIs) in many organizations are a broken tool. The KPIs are often a random collection prepared with little expertise, signifying nothing. [...] KPIs should be measures that link daily activities to the organization’s critical success factors (CSFs), thus supporting an alignment of effort within the organization in the intended direction." (David Parmenter, "Key Performance Indicators: Developing, implementing, and using winning KPIs" 3rd Ed., 2015)

"Most organizational measures are very much past indicators measuring events of the last month or quarter. These indicators cannot be and never were KPIs." (David Parmenter, "Key Performance Indicators: Developing, implementing, and using winning KPIs" 3rd Ed., 2015)

"The traditional balanced-scorecard (BSC) approach uses performance measures to monitor the implementation of the strategic initiatives, and measures are typically cascaded down from a top-level organizational measure such as return on capital employed. This cascading of measures from one another will often lead to chaos, with hundreds of measures being monitored by staff in some form of BSC reporting application." (David Parmenter, "Key Performance Indicators: Developing, implementing, and using winning KPIs" 3rd Ed., 2015)

"We need indicators of overall performance that need only be reviewed on a monthly or bimonthly basis. These measures need to tell the story about whether the organization is being steered in the right direction at the right speed, whether the customers and staff are happy, and whether we are acting in a responsible way by being environmentally friendly. These measures are called key result indicators (KRIs)." (David Parmenter, "Key Performance Indicators: Developing, implementing, and using winning KPIs" 3rd Ed., 2015)

"Every day spent producing reports is a day less spent on analysis and projects." (David Parmenter)