Showing posts with label project failure. Show all posts
Showing posts with label project failure. Show all posts

09 April 2024

Business Intelligence: Why Data Projects Fail to Deliver Real-Life Impact (Part IV: Making It in the Statistics)

Business Intelligence
Business Intelligence Series

Various sources (e.g., [1], [2], [3]) advance the failure rates for data projects somewhere between 70% and 85%, rates which are a bit higher than the failure of standard projects estimated at 60-75% but not by much. This means that only 2-3 out of 10 projects will succeed and that’s another reason to plan for failure, respectively embrace the failure

Unfortunately, the statistics advanced on project failure have no solid fundament and should be regarded with circumspection as long the methodology and information about the population used for the estimates aren’t shared, though they do reflect an important point – many data projects do fail! It would be foolish to think that your project will not fail just because you’re a big company, and you have the best resources, and you have a proven rate of success, and you took all the precautions for the project not to fail.

Usually at the end of a project the team meets together to document the lessons learned in the hope that the next projects will benefit from them. The team did learn something, though as the practice shows even if the team managed to avoid some issues, other issues will impact the next similar project, leading to similar variances. One can summarize this as "on the average the impact of new issues and avoided known issues tends to zero out" or "on average, the plusses and minuses balance each other across projects". It’s probably a question of focus – if organizations focus too much on certain aspects, other aspects are ignored and/or unseen. 

So, your first data project will more likely fail. The question is: what do you do about it? It’s important to be aware of why projects and data projects fail, though starting to consider and monitor each possible issue can prove to be ineffective. One can, however, create a risk register from the list and estimate the rates for each of the potential failures, respectively focus on only the top 3-5 which have the highest risk. Of course, one should reevaluate the estimates on a regular basis though that’s Risk Management 101. 

Besides this, one should focus on how the team can make the project succeed. When adopting a technology, methodology or set of processes, it’s recommended to start with a proof-of-concept (PoC). To make the PoC a helpful experience it’s probably important to start with a topic that’s not too big to handle, but that also involves some complexity that would allow the organization to evaluate the targeted set of tools and technologies. It can also be a topic for which other organizations have made important progress, respectively succeed. The temptation is big to approach the most stringent issues in the organization, respectively to build something big that can have an enormous impact for the organization. Jumping too soon into such topics can just increase the chances of failure. 

One can also formulate the goals, objectives and further requirements in a form that allows the organization to build upon them even if the project fails. A PoC is about learning, building a foundation, doing the groundwork, exploring, mapping the unknown, and identifying what's still missing to make progress, respectively closing the full circle. A PoC is less about overachievement and a big impact, which can happen, though is a consequence of the good work done in the PoC. 

The bottom line, no matter whether you succeed or fail, once you start a project, you’ll still make it in the statistics! More important is what you’ve learnt after the first data project, respectively how you can use the respective knowledge in further projects to make a difference!

Previous Post <<||>> Next Post

References:
[1] Harvard Business Review (2023) Keep Your AI Projects on Track, by Iavor Bojinov (link)
[2] Cognilytica (2023) The Shocking Truth: 70-80% of AI Projects Fail! (link)
[3] VentureBeat (2019) Why do 87% of data science projects never make it into production? (link)

08 April 2024

Business Intelligence: Why Data Projects Fail to Deliver Real-Life Impact (Part III: Failure through the Looking Glass)

Business Intelligence
Business Intelligence Series

There’s a huge volume of material available on project failure – resources that document why individual projects failed, while in general projects fail, why project members, managers and/or executives think projects fail, and there seems to be no other pleasant activity at the end of a project than to theorize why a project failed, the topic culminating occasionally with the blaming game. Success may generate applause, though is failure that attracts and stirs the most waves (irony, disapproval, and other similar behavior) and everybody seems to be an expert after the consumed endeavor. 

The mere definition of a project failure – not fulfilling project’s objectives within the set budget and timeframe - is a misnomer because budgets and timelines are estimated based on the information available at the beginning of the project, the amount of uncertainty for many projects being considerable, and data projects are no exceptions from it. The higher the uncertainty the less probable are the two estimates. Even simple projects can reveal uncertainty especially when the broader context of the projects is considered. 

Even if it’s not a common practice, one way to cope with uncertainty is to add a tolerance for the estimates, though even this practice probably will not always accommodate the full extent of the unknown as the tolerances are usually small. The general expectation is to have an accurate and precise landing, which for big or exploratory projects is seldom possible. 

Moreover, the assumptions under which the estimates hold are easily invalidated in praxis – resources’ availability, first time right, executive’s support to set priorities, requirements’ quality, technologies’ maturity, etc. If one looks beyond the reasons why projects fail in general, quite often the issues are more organizational than technological, the lack of knowledge and experience being one of the factors. 

Conversely, many projects will not get approved if the estimates don’t look positive, and therefore people are pressured in one way or another to make the numbers fit the expectations. Some projects, given their importance, need to be done even if the numbers don’t look good or can’t be quantified correctly. Other projects represent people’s subsistence on the job, respectively people self-occupation to create motion, though they can occasionally have also a positive impact for the organizations. These kinds of aspects almost never make it in statistics or surveys. Neither do the big issues people are afraid to talk about. Where to consider that in the light of politics and office’s grapevine the facts get distorted.

Data projects reflect all the symptoms of failure projects have in general, though when words like AI, Statistics or Machine Learning are used, the chances for failure are even higher given that the respective fields require a higher level of expertise, the appropriate use of technologies and adherence to the scientific process for the results to be valid. If projects can benefit from general receipts, respectively established procedures and methods, their range of applicability decreases when the mentioned areas are involved. 

Many data projects have an exploratory nature – seeing what’s possible - and therefore a considerable percentage will not reach production. Moreover, even those that reach that far might arrive to be stopped or discarded sooner or later if they don’t deliver the expected value, and probably many of the models created in the process are biased, irrelevant, or incorrectly apply the theory. Where to add that the mere use of tools and algorithms is not Data Science or Data Analysis. 

The challenge for many data projects is to identify which Project Management (PM) best practices to consider. Following all or no practices at all just increases the risks of failure!

Previous Post <<||>> Next Post

06 April 2024

Business Intelligence: Why Data Projects Fail to Deliver Real-Life Impact (Part I: First Thoughts)

Business Intelligence
Business Intelligence Series

A data project has a set of assumptions and requirements that must be met, otherwise the project has a high chance of failing. It starts with a clear idea of the goals and objectives, and they need to be achievable and feasible, with the involvement of key stakeholders and the executive without which it’s impossible to change the organization’s data culture. Ideally, there should also be a business strategy, respectively a data strategy available to understand the driving forces and the broader requirements. 

An organization’s readiness is important not only in what concerns the data but also the things revolving around the data - processes, systems, decision-making, requirements management, project management, etc. One of the challenges is that the systems and processes available can’t be used as they are for answering important business questions, and many of such questions are quite basic, though unavailability or poor quality of data makes this challenging if not impossible. 

Thus, when starting a data project an organization must be ready to change some of its processes to address a project’s needs, and thus the project can become more expensive as changes need to be made to the systems. For many organizations the best time to have done this was when they implemented the system, respectively the integration(s) between systems. Any changes made after that come in theory with higher costs derived from systems and processes’ redesign.

Many projects start big and data projects are no exception to this. Some of them build a costly infrastructure without first analyzing the feasibility of the investment, or at least whether the data can form a basis for answering the targeted questions. On one side one can torture any dataset and some knowledge will be obtained from it (aka data will confess), though few datasets can produce valuable insights, and this is where probably many data projects oversell their potential. Conversely, some initiatives are worth pursuing even only for the sake of the exposure and experience the employees get. However, trying to build something big only through the perspective of one project can easily become a disaster. 

When building a data infrastructure, the project needs to be an initiative given the transformative potential such an endeavor can have for the organization, and the different aspects must be managed accordingly. It starts with the management of stakeholders’ expectations, with building a data strategy, respectively with addressing the opportunities and risks associated with the broader context.

Organizations recognize that they aren’t capable of planning and executing such a project or initiative, and they search for a partner to lead the way. Becoming overnight such a partner is more than a challenge as a good understanding of the industry and the business is needed. Some service providers have such knowledge, at least in theory, though the leap from knowledge to results can prove to be a challenge even for experienced service providers. 

Many projects follow the pattern: the service provider comes, analyzes the requirements, builds something wonderful, the solution is used for some time and then the business realizes that the result is not what was intended. The causes are multiple and usually form a complex network of causality, though probably the most important aspect is that customers don’t have the in-house technical resources to evaluate the feasibility of requirements, solutions, respectively of the results. Even if organizations involve the best key users, are needed also good data professionals or similar resources who can become the bond between the business and the services provider. Without such an intermediary the disconnect between the business and the service provider can grow with all the implications. 

Previous Post <<||>> Next Post

02 February 2021

Data Migrations (DM): Conceptualization I (Goals, Objectives & Requirements)

Data Migration
Data Migrations Series

One of the nowadays’ challenges is finding the right mix of technologies that allows building a solution for a business need. There are so many choices and the responsible person is easily tempted to use one of the trending technologies just because he wants to learn something new or the technologies seem to fit into the bigger picture, which probably in many cases it would be acceptable. Unfortunately, there’s also the tendency of picking a technology without looking at what functionality it provides, respectively whether the functionality meets intended solutions’ requirements. Moreover, the requirements are sometimes barely defined at the appropriate level of detail, fact that makes from the implementation project a candidate for failure. Sometimes even the goals and objectives aren’t clearly stated, fact that can make a project’s success easily questionable from the beginning. 

A goal is a general statement that reflects the desired result toward which an organization’s effort needs to be directed. For example, a Data Migration (DM)’s primary goal can be formulated as 'to make available all the master and transactional data needed by the business from the legacy systems to the target system(s) within expected timeline and quality with a minimal disruption for the business'. 

An objective is a break down of the goal into several components that should foster a clear understanding on how the goal will be achieved. Ideally the objectives should be SMART (specific, measurable, attainable, relevant, time-bound), even if measurable objectives are sometimes hard to define properly. One can consider them as the tactics used in achieving the goal. For example, the above formulated goal can be broken down into the following objectives:

  • Build a DM concept/strategy
  • Build a flexible and performant infrastructure for DM that can be adapted to further requirements
  • Provide a basis for further DMs
  • Align DM and main project’s requirements and activities
  • Provide an interface and support for the Data Management areas
  • Foster trust, transparency and awareness 
  • Address internal/external compliance requirements
  • Document and communicate accountability for the various activities
  • Cleanse and enrich the data needed by the target system 
  • Archive the DM and project data 

One can attempt defining the objectives directly from the goal(s), though unless one is aware of all the implication a DM has, more likely one will be forced to define and evaluate the individual functional and nonfunctional requirements for the DM first, and attempt consolidating the requirements into a set of objectives. In the end it can be a combination of both, in which some objectives are first formulated, the requirements are defined and evaluated, respectively the objectives are refined to accommodate the requirements. 

ISO 9126, an international standard for the evaluation of software quality, defines about 45-50 attributes that can be used for addressing the requirements of software solutions, attributes that reflect functionality, reliability, usability, efficiency, and maintainability characteristics. One can start with such a list and identify how important are the respective attributes for the solution.  The next step would be to document the requirements into a consolidated list by providing a short argumentation for their use, respectively how they will be addressed as part of the solution. The process can prove to be time-consuming, however it is a useful exercise that usually needs to be done only once and be reviewed occasionally.

The list can be created independently of any other documentation or be included directly into a concept or strategy. The latter will assure in theory that the document provides a unitary view of the migration, considering that each new or obsolete requirement can impact the concept. 

Previous Post <<||>> Next Post

09 January 2021

ERP Implementations: It’s all about Partnership I

ERP Implementation

Unless the organization (customer) implementing an ERP system has a strong IT team and the knowledge required for the implementation is available already in-house, the resources need to be acquired from the market, and probably the right thing to do is to identify a certified implementer (partner) which can fill the knowledge and skillset gaps, respectively which can help splitting the risks associated with such an implementation.

In theory, the customer provides knowledge about its processes, while the partner comes with expertise about the system to be implemented and further technologies, industry best practices, project methodologies, etc. Further on, the mix is leveraged to harness the knowledge and reach project’s objectives. 

In praxis however finding an implementer which can act as partner might be more challenging than expected. This because the implementer needs to understand customer’s business and where it’s heading, bridge the gap between functional requirements and system’s functionality, advise on areas of improvement, prepare the customer for the project and lead the customer through the changes, respectively establish a basis for the future. Some of the implications are seldom made explicit even if they are implied by what is needed by the project. 

Technology is seldom the issue in an ERP implementation, the challenges residing in handing the change and the logistics required. There are so many aspects to be considered and handled, and this can be challenging for any implementer no matter how long has been on the market or how experienced the resources are. Somebody needs to lead the change and the customer seldom has the knowledge to handle the change. In some cases, the implementer must make the customer aware of the implications, while in others needs to take the initiative and lead the change, though the customer needs to play along, which can be challenging also. 

Many aspects need to be handled at management level from a strategical point of view on customer’s side. It starts with assuring that the most important aspects of the business where considered, that the goals and objectives are clear, that the proper environment is created, and ends with the timely decision-making, with assuring that the resources are available when needed, that the needed organization structures and roles are in place, that the required knowledge is available before, during and after implementation, that the potential brought by the ERP system is harnessed for the years to come. 

A partnership allows in theory splitting the implementation risks as ERP implementations have a high rate of failure. Quite often the outcomes of such projects don’t meet the expectations, the systems being in extremis unusable or a bottleneck for the organization. Ideally one should work with the partner(s) and attempt solving the issues, split eventually the incurred cost overruns, find a middle way. Most of the times it’s recommended to find a solution together rather than coming to a litigation. 

Given the complex dependencies existing between the various parts of the project, the causes that lead to poor implementations are difficult to prove, as there are almost always grey areas. Moreover, the litigations can require a considerable time and resources to settle. These can be just extreme situations, and as long one has a good partner, there’s no need to think that far. On the other side, even if undesirable, one must be prepared also for such outcomes, even if the countermeasures may involve an additional effort. Therefore, one must address such issues in contracts by establishing the areas of accountability/responsibilities for each party, document adequately the requirements and further (important) communication, make sure that the deliverables have the expected quality, etc.

Previous Post <<||>> Next Post

04 August 2020

Project Management: Redefining Projects' Success I

Mismanagement

A project is typically considered as successful if has met the beforehand defined objectives within the allocated budget, timeframe and expected quality levels. Any negative deviation from any of these equates with a project failure. In other words, the success or failure of a project is judged as black or white with no grays in between, which is utopic, especially for mid to big software projects, typically associated with lot of uncertainty. According to this definition a project which had a delay of a few months, or the budget was overrun by 10%, or the users got only 90% from the planned functionality, or any combination of these negative deviations, can be considered as failed.

If a small project needed 6 instead of 3 months to complete, which is normal for projects with reduced priority, as long the project costs haven’t changed, then the increase in duration can be ignored. In exchange, 3 months of a delay for a 2 years project is normal, especially when the project is complex. Even if additional costs incurred within this timeframe, as long they are a small percentage in comparison with the overall project costs, then the impact can be acceptable for the business. On the other side, when the delays have an exponential growth with further implications, then the problem changes dramatically.

Big projects have typically a strategic importance. It’s the case of ERP implementations, which besides the technology changes have in theory have the potential to transform an organization pushing it to reach further performance levels. Such projects are estimated to take on average one to two years for a medium organization, however the delays can easily reach 50% to 100% from the initial estimation. Independently of what caused the delay, as long the organization achieved the intended goals and can cover project’s costs, one can say that the project made a (positive) difference.

Independently of project’s size, if 90% of the important functionality is available, then more likely the 10% can be covered in a first step with manual work, following in time to further invest into the system as part of a continuous improvement process. It’s maybe not ideal for the users, however the approach incorporates also the learning curve of working with the system and understanding ist possibilities and limitations. Of course, when the percentage of the available functionality decreases below a given limit, system’s acceptance is endangered, which users eventually start looking for alternatives.

There are also projects which opened the door to new possibilities and which require more investments to leverage the full capabilities. Some ERP implementations have this potential, despite overruns. Some of such investments are entitled while others are not. Related to this last category, there are projects which are on time, on budget, and the deliverables satisfy the quality criteria and objectives, however they make no difference for the organization despite the important investments made. Sure, some of the projects from this category are a must (e.g. updates, upgrade, technology changes), however there are also projects which can be considered as self-occupational hazard. In extremis such projects run in the background and cost organizations lot of energy and resources, while their effects are questionable.

At least from these examples the definition of a project's success needs to be changed or maybe standardized to consider not only intrinsic but also extrinsic aspects. In theory, that is the role of a Project Management Office (PMO), however it’s challenging to find an evaluation methodology that fits all needs. Further on, from same considerations, benchmarking projects across organizations and industries can prove to be a foolhardy attempt.

30 January 2020

Project Management: Methodologies (The Good, the Bad and the Ugly)

Mismanagement

The Good
: Nowadays there're several Project Management (PM) methodologies to choose from to address a project’s specifics and, when adapted and applied accordingly, a methodology can enable projects to be run and brought under control.

The Bad: Even if the theoretical basis of PM methodologies has been proved and perfected over the years, projects continue to fail at a disturbing rate. Of course, the reasons behind their failure are multiple, though often the failure reasons are rooted in how PM methodologies are taught, understood and implemented.

Same as a theoretical course in cooking won’t make one a good cook, a theoretical course in PM won’t make one a good Project Manager or knowledgeable team member in applying the learned methodology. Surprisingly, the expectation is exactly that – the team member got a training and is good to go. Moreover, people believe that managing a software project is like coordinating the building of a small treehouse. To some degree there are many similarities though the challenges typically lie in details, and these details often escape a standard course.

To bridge the gap between theory and practice is needed time for the learner to grow in the role, to learn the does and don’ts, and, most important, to learn how to use the tools at hand efficiently. The methodology is itself a tool making use of further tools in its processes – project plans, work breakdown structures, checklists, charters, reports, records, etc. These can be learned only through practice, hopefully with some help (aka mentoring) from an experienced person in the respective methodology, either the Project Manager itself, a trainer or other team member. Same as one can’t be thrown into the water and expected to traverse the Channel Tunnel, you can’t do that with a newbie.

There’s a natural fallacy to think that we've understood more than we have. We can observe our understanding's limits when we are confronted with the complexities involved in handing PM activities. A second fallacy is not believing other people’s warnings against using a tool or performing an activity in a certain way. A newbie’s mind has sometimes the predisposition of a child to try touching a hot stove even if warned against it. It’s part of the learning process, though some persist in such behavior without learning much. What’s even more dangerous is a newbie pretending to be an expert and this almost always ends badly.

The Ugly appears when the bad is brought to extreme, when methodologies are misused for the wrong purposes to the degree that they destroy anything in their way. Of course, a pool can be dug by using a spoon but does it make sense to do that? Just because a tool can be used for something it doesn’t mean it should be used for it as long there are better tools for the same. It seems a pretty logical thing though the contrary happens more often than we’d like. It starts with the preconception that one should use the tool one knows best, ignoring in the process the fit for purpose condition. What’s even more deplorable is breaking down a project to fit a methodology while ignoring the technical and logistical aspects.

Any tool can lead to damages when used excessively, in wrong places, at the wrong point in time or by the wrong person. Like the instruments in an orchestra, when an instrument plays the wrong note, it dissonates from the rest. When more instruments play wrongly, then the piece is unrecognizable. It’s the role of the bandmaster to make the players touch the right notes at the right time.

07 May 2019

Project Management: Agility under Eyeglasses II

Misanagement


Employees are used to follow procedures and processes, and when they aren’t available insecurity rules - each day there’s another idea advanced how things are supposed to work. Practically, the Agile approaches (incl. Agile Prince2) focus on certain aspects and ignore specific Project Management activities that need to be performed inside of a project – releasing resources for the project, getting users on-board, getting management’s buy-in, etc. Therefore, they need to be used with a methodology that offers the lacking processes. Problematic is when is considered that the Agile approaches are self-consistent and the Project Management practices and principles don’t apply anymore.

It’s true that the Agile methods attempt reconciling disciplined project execution with creativity and innovation, however innovation is needed typically in design (incl. prototyping) , while in programing there isn’t lot of room for creativity per se. The real innovation appears when the customer lists the functionality it needs from a system and the vendor, after analyzing all the related requirements, is capable to evaluate and propose a solution from the industry trending technologies. That’s innovation and not changing controls in user interfaces!

User stories are good for situations in which an organization doesn’t know what’s doing or the tasks have a deep segmentation and specialization. Starting from user stories and building upwards to processes can prove to be a waste of time the customer pays for, while the approach leaves few room for innovation. In big projects it’s also difficult to sense the contradictions from user stories or their duplication. Even if the user stories allow maybe (but not necessarily) a better effort estimate the level of detail can become overwhelming for any skilled solution architect.

It’s also true that an agile approach needs a culture with certain characteristics. A culture can’t be changed with one project or several projects running in parallel. Typically, is recommended to start with a pilot test, assert organization’s readiness, disseminate knowledge, start several small to medium projects and build from there. For sure starting a big project with an agile methodology  will involve more challenges to the extent the challenges will push back.

One sign of agility is when self-organizing teams emerge within projects, however it takes time and training to build such teams. The seeds must be planted long before, for such teams to emerge. The key is being able of working in such teams. In extremis, conflicts appear when multiple self-organizing teams appear, each with its own political agenda, agendas that don’t necessarily match project manager or stakeholders’ agendas, and from here a large range of potential conflicts.

The psychological effect of tight sprints (iterations) and daily status meetings for the whole duration of a project is not to neglect. It builds unnecessary stress and, unless the planning reaches perfection, the programmer or consultant will often find himself in the position to be in defensive. The frequent meetings can easily become a source for nuisance and in extremis can lead to extreme behavior that can easily affect the productivity and involved persons’ health.

Personally, I wouldn’t recommend using an Agile methodology for a big project like an ERP implementation unless it was adequately adapted to organization’s needs. This doesn’t necessarily mean that the Agile methods aren’t suitable for big projects, it means that the risks are high because in big projects there’s the chance for all these mentioned issues to occur.

Despite the weak points of the Agile methods, when adequately applied, they have the chance of better performing than the “traditional” approaches. Even if people tend to see more the negative sides there’s lot of potential in being agile.

Project Management: Agility under Eyeglasses I

Mismanagement

There are more and more posts in the cyberspace voicing against the agile practices, the way they are understood and implemented by organizations. Some try to be hilarious [5]; others try to keep the scholastic seriousness [1] [2] [3] [4], and all of them make some valid points. In each remark there’re some seeds of truth, even if context-dependent.

Personally, I embrace an agile approach when possible, however I find it difficult to choose between the agile methodologies available on the market because each of them introduces some concepts that contradict what it means to be agile – to respond promptly to business needs. It doesn’t mean that one must consider each requirement, but that’s appropriate to consider those which have business justification. Moreover, organizations need to adapt the methodologies to their needs, and seldom vice-versa.
Considering the Agile Manifesto, it’s difficult to take as serious statements that lack precision, formulations like “we value something over something else” are more of a wish than principles. When people don’t understand what the agile “principles” mean, one occasionally hears statements like “we need no documentation”, “we need no project plan”, “the project plan is not important”, “Change Management doesn’t apply to agile projects” or “we need only high-level requirements because we’ll figure out where we’re going on the way”. Because of the lack of precision, a mocker can variate the lesser concept to null and keep the validity of the agile “principles”.
The agile approaches seem to lack control. If you’re letting the users in charge of the scope then you risk having a product that offers a lot though misses the essential, and thus unusable or usable to a lower degree. Agile works good for prototyping something to show to the users or when the products are small enough to easily fit within an iteration, or when the vendor wants to gain a customer’s trust. Therefore, agile works good with BI projects that combine in general all three aspects.
An abomination is the work in fix sprints or iterations of one or a few weeks, and then chopping the functionality to fit the respective time intervals. If you have the luck of having sign-offs and other activities that steal your time, then the productive time reduces up to 50% (the smaller the iterations the higher the percentage). What’s even unconceivable is that people ignore the time spent with bureaucracy. If this way of working repeats in each iteration then the project duration multiplies by a factor between 2 or 4, the time spent on Project Management increasing by the same factor. What’s not understandable is that despite bureaucracy the adherence to delivery dates, budget and quality is still required.
Sometimes one has the feeling that people think that software development and other IT projects work like building a house or like the manufacturing of a mug. You choose the colors, the materials, the dimensions and voila the product is ready. IT projects involve lot of unforeseen and one must react agilely to it. Here resides one of the most important challenges.   
Communication is one important challenge in a project especially when multiple interests are involved. Face-to-face conversation is one of the nice-to-have items on the wish list however in praxis isn’t always possible. One can’t expect that all the resources are available to meet and decide. In addition, one needs to document everything from meeting minutes, to Business Cases and requirements. A certain flexibility in changing the requirements is needed though one can’t change them arbitrarily, there must be a concept behind otherwise the volume of overwork can easily make the budget for a project explode exponentially.
||>> Next Post (continuation) 
Resources:
[1] Harvard Business Review (2018) Why Agile Goes Awry - and How to Fix It, by Lindsay McGregor & Neel Doshi (Online) Available from: https://hbr.org/2018/10/why-agile-goes-awry-and-how-to-fix-it
[2] Forbes (2012) The Case Against Agile: Ten Perennial Management Objections, by Steve Denning  (Online) Available from:
https://www.forbes.com/sites/stevedenning/2012/04/17/the-case-against-agile-ten-perennial-management-objections/#6df0e6ea3a95 
[3] Springer (2018) Do Agile Methods Work for Large Software Projects?, by Magne Jørgensen  (Online) Available from:
https://link.springer.com/chapter/10.1007/978-3-319-91602-6_12
[4] Michael O Church (2015) Why “Agile” and especially Scrum are terrible  (Online) Available from:
https://michaelochurch.wordpress.com/2015/06/06/why-agile-and-especially-scrum-are-terrible/
[5] Dev.to (2019) Mockery of agile, by Artur Martsinkovskyi (Online) Available from: https://dev.to/arturmartsinkovskyi/mockery-of-agile-5bdf

24 April 2019

Project Management: The Butterflies of Project Management

Mismanagement

Expressed metaphorically as "the flap of a butterfly’s wings in Brazil set off a tornado in Texas”, in Chaos Theory the “butterfly effect” is a hypothesis rooted in Edward N Lorenz’s work on weather forecasting and used to depict the sensitive dependence on initial conditions in nonlinear processes, systems in which the change in input is not proportional to the change in output.  

Even if overstated, the flapping of wings advances the idea that a small change (the flap of wings) in the initial conditions of a system cascades to a large-scale chain of events leading to large-scale phenomena (the tornado) . The chain of events is known as the domino effect and represents the cumulative effect produced when one event sets off a chain of similar events. If the butterfly metaphor doesn’t catch up maybe it’s easier to visualize the impact as a big surfing wave – it starts small and increases in size to the degree that it can bring a boat to the shore or make an armada drown under its force. 

Projects start as narrow activities however the longer they take and the broader they become tend to accumulate force and behave like a wave, having the force to push or drawn an organization in the flood that comes with it. A project is not only a system but a complex ecosystem - aggregations of living organisms and nonliving components with complex interactions forming a unified whole with emergent behavior deriving from the structure rather than its components - groups of people tend to  self-organize, to swarm in one direction or another, much like birds do, while knowledge seems to converge from unrelated sources (aka consilience). 

 Quite often ignored, the context in which a project starts is very important, especially because these initial factors or conditions can have a considerable impact reflected in people’s perception regarding the state or outcomes of the project, perception reflected eventually also in the decisions made during the later phases of the project. The positive or negative auspices can be easily reinforced by similar events. Given the complex correlations and implications, aspects not always correct perceived and understood can have a domino effect. 

The preparations for the project start – the Business Case, setting up the project structure, communicating project’s expectation and addressing stakeholders’ expectations, the kick-off meeting, the approval of the needed resources, the knowledge available in the team, all these have a certain influence on the project. A bad start can haunt a project long time after its start, even if the project is on the right track and makes a positive impact. In reverse, a good start can shade away some mishaps on the way, however there’s also the danger that the mishaps are ignored and have greater negative impact on the project. It may look as common sense however the first image often counts and is kept in people’s memory for a long time. 

As people are higher perceptive to negative as to positive events, there are higher the chances that a multitude of negative aspects will have bigger impact on the project. It’s again something that one can address as the project progresses. It’s not necessarily about control but about being receptive to the messages around and of allowing people to give (constructive) feedback early in the project. It’s about using the positive force of a wave and turning negative flow into a positive one. 

Being aware of the importance of the initial context is just a first step toward harnessing waves or winds’ power, it takes action and leadership to pull the project in the right direction.

21 April 2019

Project Management: Planning Correctly Misundersood II

Mismanagement

Even if planning is the most critical activity in Project Management it seems to be also one of the most misunderstood concepts. Planning is critical because it charters the road ahead in terms of what, when, why and who, being used as a basis for action, communication, for determining the current status in respect to the initial plan, as well the critical activities ahead.

The misunderstandings derive maybe also from the fact that each methodology introduces its own approach to planning. PMI as traditional approach talks about baseline planning with respect to scope schedule and costs, about management plans, which besides the theme covered in the baseline, focus also on quality, human resources, risks, communication and procurement, and separate plans can be developed for requirements, change and configuration management, respectively process improvement. To them one can consider also action and contingency planning.

In Prince2 the product-based planning is done at three levels – at project, stage, respectively team level – while separate plans are done for exceptions in case of deviations from any of these plans; in addition there are plans for communication, quality and risk management. Scrum uses an agile approach looking at the product and sprint backlog, the progress being reviewed in stand-up meetings with the help of a burn-down chart. There are also other favors of planning like rapid application planning considered in Extreme Programming (XP), with an open, elastic and undeterministic approach. In Lean planning the focus is on maximizing the value while minimizing the waste, this being done by focusing on the value stream, the complete list of activities involved in delivering the end-product, value stream's flow being mapped with the help of visualization techniques such as Kanban, flowcharts or spaghetti diagrams.

With so many types of planning nothing can go wrong, isn’t it? However, just imagine customers' confusion when dealing with a change of methodology, especially when the concepts sound fuzzy and cryptic! Unfortunately, also the programmers and consultants seem to be bewildered by the various approaches and the philosophies supporting the methodologies used, their insecurity bringing no service for the project and customers’ peace of mind. A military strategist will more likely look puzzled at the whole unnecessary plethora of techniques. On the field an army has to act with the utmost concentration and speed, to which add principles like directedness, maneuver, unity, economy of effort, collaboration, flexibility, simplicity and sustainability. It’s what Project Management fails to deliver.

Similarly to projects, the plan made before the battle seldom matches the reality in the field. Planning is an exercise needed to divide the strategy in steps, echelon and prioritize them, evaluate the needed resources and coordinate them, understand the possible outcomes and risks, evaluate solutions and devise actions for them. With a good training, planning and coordination, each combatant knows his role in the battle, has a rough idea about difficulties, targets and possible ways to achieve them; while a good combatant knows always the next action. At the same time, the leader must have visibility over fight’s unfold, know the situation in the field and how much it diverged from the initial plan, thus when the variation is considerable he must change the plan by changing the priorities and make better use the resources available.

Even if there are multiple differences between the two battlefields, the projects follow the same patterns of engagement at different scales. Probably, Project Managers can learn quite of a deal by studying the classical combat strategists, and hopefully the management of projects would be more effective and efficient if the imperatives of planning, respectively management, were better understood and addressed.

25 December 2012

Project Management: Project Failure (Just the Quotes)

"Rushing into action, you fail.
Trying to grasp things, you lose them.
Forcing a project to completion,
you ruin what was almost ripe."
(Lao Tzu, "Tao Te Ching", cca. 6th-century BC)

"In many ways, project management is similar to functional or traditional management. The project manager, however, may have to accomplish his ends through the efforts of individuals who are paid and promoted by someone else in the chain of command. The pacing factor in acquiring a new plant, in building a bridge, or in developing a new product is often not technology, but management. The technology to accomplish an ad hoc project may be in hand but cannot be put to proper use because the approach to the management is inadequate and unrealistic. Too often this failure can be attributed to an attempt to fit the project to an existing management organization, rather than molding the management to fit the needs of the project. The project manager, therefore, is somewhat of a maverick in the business world. No set pattern exists by which he can operate. His philosophy of management may depart radically from traditional theory." (David I Cleland & William R King, "Systems Analysis and Project Management", 1968)

"But the greater the primary risk, the safer and more careful your secondary assumptions must be. A project is only as sound as its weakest assumption, or its largest uncertainty." (Robert Heller, "The Naked Manager: Games Executives Play", 1972)

"Faced with a decision, always ask one implacable question: If this project fails, if the worst comes to the worst, what will be the result? If the answer is total corporate disaster, drop the project. If the worst possible outcome is tolerable, say, break-even, the executive has the foundation of all sound decision making - a fail-safe position." (Robert Heller, "The Naked Manager: Games Executives Play", 1972)

"The Eighth Truth of Management is: if you are doing something wrong, you will do it badly. The reverse of this truth is that, if your decision is blindingly right, you will execute it well - or appear to do so, which is much the same thing. But any executive can massacre his own nonsensical project." (Robert Heller, "The Naked Manager: Games Executives Play", 1972)

"Yet intelligent men plump for one project rather than another on the strength of a difference of a few decimal points in the rate of return calculated over the next decade. All such mind-stretching calculation comes under the lash of the Seventh Truth of Management: if you need sophisticated calculations to justify an action, it is probably wrong (the sophisticated calculations, anyway, are all too often based on simple false assumptions)." (Robert Heller, "The Naked Manager: Games Executives Play", 1972)

"Poor management can increase software costs more rapidly than any other factor. Particularly on large projects, each of the following mismanagement actions has often been responsible for doubling software development costs." (Barry Boehm, "Software Engineering Economics", 1981)

"[…] the longer one works on […] a project without actually concluding it, the more remote the expected completion date becomes. Is this really such a perplexing paradox? No, on the contrary: human experience, all-too-familiar human experience, suggests that in fact many tasks suffer from similar runaway completion times. In short, such jobs either get done soon or they never get done. It is surprising, though, that this common conundrum can be modeled so simply by a self-similar power law." (Manfred Schroeder, "Fractals, Chaos, Power Laws Minutes from an Infinite Paradise", 1990)

"Even when you have skilled, motivated, hard-working people, the wrong team structure can undercut their efforts instead of catapulting them to success. A poor team structure can increase development time, reduce quality, damage morale, increase turnover, and ultimately lead to project cancellation." (Steve McConnell, "Rapid Development", 1996)

"Software projects fail for one of two general reasons: the project team lacks the knowledge to conduct a software project successfully, or the project team lacks the resolve to conduct a project effectively." (Steve C McConnell, "Software Project Survival Guide", 1997)

"Today, excellent companies realize that project failures have more to do with behavioral shortcomings - poor employee morale, negative human relations, low productivity, and lack of commitment." (Harold Kerzner, "In search of excellence in project management", 1998)

"Success in all types of organization depends increasingly on the development of customized software solutions, yet more than half of software projects now in the works will exceed both their schedules and their budgets by more than 50%." (Barry Boehm, "Software Cost Estimation with Cocomo II", 2000)

"Choosing a proper project strategy can mean the difference between success and failure." (James P Lewis, "Project Planning, Scheduling, and Control" 3rd Ed., 2001)

"Note that a project always begins as a concept, and a concept is usually a bit fuzzy. Our job as a team is to clarify the concept, to turn it into a shared understanding that the entire team will accept. It is failure to do this that causes many project failures." (James P Lewis, "Project Planning, Scheduling, and Control" 3rd Ed., 2001)

"Project failures are not always the result of poor methodology; the problem may be poor implementation. Unrealistic objectives or poorly defined executive expectations are two common causes of poor implementation. Good methodologies do not guarantee success, but they do imply that the project will be managed correctly." (Harold Kerzner, "Strategic Planning for Project Management using a Project Management Maturity Model", 2001)

"Projects often fail at the beginning, not the end." (James P Lewis, "Project Planning, Scheduling, and Control" 3rd Ed., 2001)

"Success or failure of a project depends upon the ability of key personnel to have sufficient data for decision-making. Project management is often considered to be both an art and a science. It is an art because of the strong need for interpersonal skills, and the project planning and control forms attempt to convert part of the 'art' into a science." (Harold Kerzner, "Strategic Planning for Project Management using a Project Management Maturity Model", 2001)

"Today, most project management practitioners focus on planning failure. If this aspect of the project can be compressed, or even eliminated, then the magnitude of the actual failure, should it occur, would be diminished. A good project management methodology helps to reduce planning failure. Today, we believe that planning failure, when it occurs, is due in large part to the project manager’s inability to perform effective risk management." (Harold Kerzner, "Strategic Planning for Project Management using a Project Management Maturity Model", 2001)

"When unmeetable expectations are formed, failure is virtually assured, since we have defined failure as unmet expectations. This is called a planning failure and is the difference between what was planned to be accomplished and what was, in fact, achievable. The second component of failure is poor performance or actual failure. This is the difference between what was achievable and what was actually accomplished. […] Perceived failure is the net sum of actual failure and planning failure. […] Planning failure is again assured even if no actual failure occurs. In both of these situations (overplanning and underplanning), the actual failure is the same, but the perceived failure can vary considerably." (Harold Kerzner, "Strategic Planning for Project Management using a Project Management Maturity Model", 2001)

"You need to identify and terminate infeasible projects early. Sending a message to project managers that project termination threatens their career will tempt them to continue projects that should die” (Barry Bohem,"Project termination doesn't equal project failure", Computer, 34 (9),  2001)

"Projects fail because of context, not because of content.[...] the traditional emphasis in project management on the technical issues of the project (content) has led to a legacy of an extremely poor set of tools, techniques, and tips for managing the complex of people, political, and other 'softer' issues that make up the context of the project." (Rob Thomsett, "Radical Project Management", 2002)

"Agile development methodologies promise higher customer satisfaction, lower defect rates, faster development times and a solution to rapidly changing requirements. Plan-driven approaches promise predictability, stability, and high assurance. However, both approaches have shortcomings that, if left unaddressed, can lead to project failure. The challenge is to balance the two approaches to take advantage of their strengths and compensate for their weaknesses." (Barry Boehm & Richard Turner, "Observations on balancing discipline and agility", Agile Development Conference, 2003)

"A project is composed of a series of steps where all must be achieved for success. Each individual step has some probability of failure. We often underestimate the large number of things that may happen in the future or all opportunities for failure that may cause a project to go wrong. Humans make mistakes, equipment fails, technologies don't work as planned, unrealistic expectations, biases including sunk cost-syndrome, inexperience, wrong incentives, contractor failure, untested technology, delays, wrong deliveries, changing requirements, random events, ignoring early warning signals are reasons for delays, cost overruns and mistakes. Often we focus too much on the specific project case and ignore what normally happens in similar situations (base rate frequency of outcomes- personal and others)." (Peter Bevelin, "Seeking Wisdom: From Darwin to Munger", 2003)

"If you've been in the software business for any time at all, you know that there are certain common problems that plague one project after another. Missed schedules and creeping requirements are not things that just happen to you once and then go away, never to appear again. Rather, they are part of the territory. We all know that. What's odd is that we don't plan our projects as if we knew it. Instead, we plan as if our past problems are locked in the past and will never rear their ugly heads again. Of course, you know that isn't a reasonable expectation." (Tom DeMarco & Timothy Lister, "Waltzing with Bears: Managing Risk on Software Projects", 2003)

"Many things can put a project off course: bureaucracy, unclear objectives, and lack of resources, to name a few. But it is the approach to design that largely determines how complex software can become. When complexity gets out of hand, developers can no longer understand the software well enough to change or extend it easily and safely. On the other hand, a good design can create opportunities to exploit those complex features." (Eric Evans, "Domain-Driven Design: Tackling complexity in the heart of software", 2003)

"A fundamental reason for the difficulties with modern engineering projects is their inherent complexity. The systems that these projects are working with or building have many interdependent parts, so that changes in one part often have effects on other parts of the system. These indirect effects are frequently unanticipated, as are collective behaviors that arise from the mutual interactions of multiple components. Both indirect and collective effects readily cause intolerable failures of the system. Moreover, when the task of the system is intrinsically complex, anticipating the many possible demands that can be placed upon the system, and designing a system that can respond in all of the necessary ways, is not feasible. This problem appears in the form of inadequate specifications, but the fundamental issue is whether it is even possible to generate adequate specifications for a complex system." (Yaneer Bar-Yam, "Making Things Work: Solving Complex Problems in a Complex World", 2004)

"The collapse of a particular project may appear to have a specific cause, but an overly high intrinsic complexity of these systems is a problem common to many of them. A chain always breaks first in one particular link, but if the weight it is required to hold is too high, failure of the chain is guaranteed."  (Yaneer Bar-Yam, "Making Things Work: Solving Complex Problems in a Complex World", 2004)

"As hard as it is to find good ideas, it's even more difficult to manage them. While the project is humming along, vision document in place and a strong creative momentum moving forward, there is another level of thinking that has to occur: how will the designs and ideas translate into decisions? Even if good designs and ideas are being investigated, and people are excited about what they're working on, the challenge of convergence toward specifications remains. If a shift of momentum toward definitive design decisions doesn't happen at the right time and isn't managed in the right way, disaster waits. For many reasons, project failure begins here." (Scott Berkun, "The Art of Project Management", 2005)

"Failure usually results from a lack of a common approach to accomplish the work as a team. Inadequate leadership fails to create the environment in which teams can flourish. Furthermore, potential team members are seldom trained in how to share their efforts to accomplish team goals. The team may also assume they know more about teamwork than they actually do. So we need to be able to differentiate between superficial teamwork and the real thing." (Kevin Forsberg et al, "Visualizing Project Management: Models and frameworks for mastering complex systems" 3rd Ed., 2005)

"Project failures can frequently be traced to unrealistic technical, cost, or schedule targets. Such targets may be entirely arbitrary or based on bad assumptions - setting team members up for failure. Furthermore, the goals that motivate one team member may not motivate another member. All tasks don’t have to be inherently motivating - that’s not sensible. But there have to be motivating factors, if by nothing more than participating in goal determination. This also helps ensure adequate opportunity and risk identification, analysis, and management." (Kevin Forsberg et al, "Visualizing Project Management: Models and frameworks for mastering complex systems" 3rd Ed., 2005)

"Projects are complex non-linear systems and have significant inertia. If you wait to see acute problems before taking action, you will be too late and may make things worse." (Scott Berkun, "Making Things Happen: Mastering Project Management", 2005)

"The appropriate models help avoid costly errors that can lead to failure. One of the major sources of project failure is f lawed requirements and scope management. Models of the project environment, therefore, need to address the development and management of project requirements. Continuing to work on the project solution with an insufficient understanding of stakeholder requirements and a deficient requirements development process often leads to expensive time delays and redesigns. This doesn’t have to be the case. A strong requirements development and management process model can provide that ounce of prevention." (Kevin Forsberg et al, "Visualizing Project Management: Models and frameworks for mastering complex systems" 3rd Ed., 2005)

"Any effort at large-scale reorganization - that is, any project spanning more than two years and, more generally, anything that has not already been done - is inevitably doomed to failure." (Corinne Maier, "Bonjour Laziness: Why Hard Work Doesn't Pay", 2007)

"In software management, coordination is not an afterthought or an ancillary matter; it is the heart of the work, and deciding what tools and methods to use can make or break a project. But getting sidetracked in managing those tools is a potent temptation." (Scott Rosenberg, "Dreaming in Code", 2007)

"As a general rule, implementations do not just spontaneously combust. Failures tend to stem from the aggregation of many issues. Although some issues may have been known since the early stages of the project (for example, the sales cycle or system design), implementation teams discover the majority of problems during the middle of the implementation, typically during some form of testing." (Phil Simon, "Why New Systems Fail: An Insider’s Guide to Successful IT Projects", 2010)

"Implementing new systems is not like baking a cake. Organizations cannot follow a recipe with the following ingredients: three consultants, six weeks of testing, two training classes, and a healthy dose of project management. Nor do projects bake for six months until complete, after which time everyone enjoys a delicious piece of cake. For all sorts of reasons, a well-conceived and well-run project may fail, whereas a horribly managed project may come in under budget, ahead of schedule, and do everything that the vendor promised at the onset." (Phil Simon, "Why New Systems Fail: An Insider’s Guide to Successful IT Projects", 2010)

"Pre-implementation, post-implementation, and ongoing data audits are invaluable tools for organizations. Used judiciously by knowledgeable and impartial resources, audits can detect, avoid, and minimize issues that can derail an implementation or cause a live system to fail. Rather than view them as superfluous expenses, organizations would be wise to conduct them at key points throughout the system’s life cycle." (Phil Simon, "Why New Systems Fail: An Insider’s Guide to Successful IT Projects", 2010)

"The best managed project may fail, whereas a horribly managed project may come in under budget, ahead of schedule, and do everything that the vendor promised at the onset. In reality, however, organizations are unlikely to find themselves in one of these extreme scenarios. On a fundamental level, successfully activating and utilizing a new system is about minimizing risk from day one until the end of the project and beyond. The organization that can do this stands the best chance of averting failure." (Phil Simon, "Why New Systems Fail: An Insider’s Guide to Successful IT Projects", 2010)

"Understanding the causes of system failures may help organizations avoid them, although there are no guarantees." 
(Phil Simon, "Why New Systems Fail: An Insider’s Guide to Successful IT Projects", 2010)

"Stakeholder management to me is key, as success or failure is in the eye of the beholder. Time, cost and quality fall prey to the perceptions of the key stakeholders, who may have nothing to do with the running of the project." (Peter Parkes, "NLP for Project Managers", 2011)

"Projects fail from under-communicating, not over-communicating. Even if resource constraints preclude the dependency that you want from being delivered any sooner, clarifying priorities and expectations enables you to plan ahead and work through alternatives." (Edmond Lau, "The Effective Engineer: How to Leverage Your Efforts In Software Engineering to Make a Disproportionate and Meaningful Impact", 2015)

"Reading reviews of failure can be a dispiriting exercise. It can also create a distorted perception of reality. Reform of the implementation of large programs and projects should not just be based on a litany of what has gone wrong. Many things go right and, for that very reason, go unnoticed." (Peter Shergold, "Learning from Failure", 2015)

"Complexity is one of the causes of failing projects; failing to split the project into smaller tasks also causes software quality issues or project failure. Complexity can also be caused by the size of the  project; if the project is too big, there is a huge possibility that the project may become complex and complicated." (Abu S Mahfuz,"Software Quality Assurance", 2016)

"A correct and sanity-checked judgment of both the financial and logistic feasibility of a project is absolutely critical to its eventual outcome; get it wrong and a failed project is almost guaranteed. In support of this statement we can refer to the contents of a number of serious ­academic studies which support the idea that two of the chief causes of the failure, particularly the financial failure, of major infrastructure ­projects are optimism bias and strategic misrepresentations. […] Optimism bias means that the original project sponsors and planners fooled themselves that the project would be easy and could be completed within a 'back-of-the-envelope' budget, perhaps because they didn’t have the experience to know it would be difficult and expensive, or perhaps they didn’t want to know. Strategic misrepresentations mean that even if they did know it would be more difficult and expensive than their published estimate, they lied about it so that the Public, the Banks, and Politicians, would support the idea." (Tony Martyr, "Why Projects Fail", 2018)

"No project should be allowed to proceed without clear specification and acceptance  criteria, that are understood by all participants." (Tony Martyr, "Why Projects Fail", 2018)

"[...] consistently good project results are hard to come by, yet most organisations continue to think they’re doing a great job. It’s got to the stage where project failure has become so commonplace that we’ve started to see it as success, or we just aren’t seeing clearly at all." (Tony Martyr, "Why Projects Fail", 2018)

"Every year more than two-thirds of projects are considered failures, and most organisations would not be surprised by this statistic. In most cases, however, failure was the result of not making a hard decision."  (Colin D Ellis, "The Project Book", 2019)

"Organisations whose IT projects failed usually deployed recognisable project management methodologies; the reasons for failure were invariably to do with failures of project governance rather than simply of operational management." (Alan Calder, "ISO/IEC 38500: A pocket guide" 2nd Ed, 2019)

"Part of the problem is that we take project failure personally, seeing it as a stain on our reputation. It’s worth remembering that while a project may fail, this doesn’t make you a failure as a leader. In fact, the research shows that those who embrace failure become much more resilient and make better decisions as a result, so in that sense failure can only be a good thing." (Colin D Ellis, "The Project Book", 2019)

"Remember, though, there are only two reasons for project failure: poor project sponsorship and poor project management. And given that the buck stops with you, you could argue there’s only one reason for project failure." (Colin D Ellis, "The Project Book", 2019)

"A project is usually considered a failure if it is late, is over budget, or does not meet the customer’s expectations. Without the control that project management provides, a project is more likely to have problems with one of these areas. A problem with only one constraint (scope, schedule, cost, resources, quality, and risk) can jeopardize the entire project." (Sandra F Rowe, "Project Management for Small Projects" 3rd Ed., 2020)

19 December 2012

Project Management: Quality (Just the Quotes)

"Managers jeopardize product quality by setting unreachable deadlines. They don’​​​​​​t think about their action in such terms; they think rather that what they’​​​​​​re doing is throwing down an interesting challenge to their workers, something to help them strive for excellence." (Tom DeMarco & Timothy Lister, "Peopleware: Productive Projects and Teams", 1987)

"People under time pressure don’​​​​​​t work better - ​​​​​​they just work faster. In order to work faster, they may have to sacrifice the quality of the product and of their own work experience." (Tom DeMarco & Timothy Lister, "Peopleware: Productive Projects and Teams", 1987)

"The [software] builders’​​​​​​ view of quality, on the other hand, is very different. Since their self-esteem is strongly tied to the quality of the product, they tend to impose quality standards of their own. The minimum that will satisfy them is more or less the best quality they have achieved in the past. This is invariably a higher standard than what the market requires and is willing to pay for." (Tom DeMarco & Timothy Lister, "Peopleware: Productive Projects and Teams", 1987)

"Even when you have skilled, motivated, hard-working people, the wrong team structure can undercut their efforts instead of catapulting them to success. A poor team structure can increase development time, reduce quality, damage morale, increase turnover, and ultimately lead to project cancellation." (Steve McConnell, "Rapid Development", 1996)

"Issues of quality, timeliness and change are the conditions that are forcing us to face up to the issues of enterprise architecture. The precedent of all the older disciplines known today establishes the concept of architecture as central to the ability to produce quality and timely results and to manage change in complex products. Architecture is the cornerstone for containing enterprise frustration and leveraging technology innovations to fulfill the expectations of a viable and dynamic Information Age enterprise." (John Zachman, "Enterprise Architecture: The Issue of The Century", 1997)

"The aim of leadership should be to improve the performance of man and machine, to improve quality, to increase output, and simultaneously to bring pride of workmanship to people. Put in a negative way, the aim of leadership is not merely to find and record failures of men, but to remove the causes of failure: to help people to do a better job with less effort." (W Edwards Deming, "Out of the Crisis", 2000)

"Stakeholder management to me is key, as success or failure is in the eye of the beholder. Time, cost and quality fall prey to the perceptions of the key stakeholders, who may have nothing to do with the running of the project." (Peter Parkes, "NLP for Project Managers", 2011)

"Complexity is one of the causes of failing projects; failing to split the project into smaller tasks also causes software quality issues or project failure. Complexity can also be caused by the size of the  project; if the project is too big, there is a huge possibility that the project may become complex and complicated." (Abu S Mahfuz,"Software Quality Assurance", 2016)

"A project is usually considered a failure if it is late, is over budget, or does not meet the customer’s expectations. Without the control that project management provides, a project is more likely to have problems with one of these areas. A problem with only one constraint (scope, schedule, cost, resources, quality, and risk) can jeopardize the entire project." (Sandra F Rowe, "Project Management for Small Projects" 3rd Ed., 2020)

06 April 2012

Business Intelligence: Between Potential, Reality, Quality and Stories

Business Intelligence
Business Intelligence Series

Have you ever felt that you are investing quite a lot of time, effort, money and other resources into your BI infrastructure, and in the end you don’t meet your expectations? As it seems you’re not the only one. The “Does your business intelligence tell you the whole story” paper released in 2009 by KPMG provides some interesting numbers to support that:
1. “More than 50% of business intelligence projects fail to deliver the expected benefit” (BI projects failure)
2. “Two thirds of executives feel that the quality of and timely access to data is poor and inconsistent” (reports and data quality)
3. “Seven out of ten executives do not get the right information to make business decisions.” (BI value)
4. “Fewer than 10% of organizations have successfully used business intelligence to enhance their organizational and technological infrastructures”  (BI alignment)
5. “those with effective business intelligence outperform the market by more than 5% in terms of return on equity” (competitive advantage)

The numbers reflect to some degree also my expectations, though they seem more pessimistic than I expected. That’s not a surprise, considering that such studies can be strongly biased, especially because in them are reflected expectations, presumptions and personal views over the state of art within an organization.

KPMG builds on the above numbers and several other aspects that revolve around the use of governance and alignment in order to increase the value provided by BI to the business, though I feel that they are hardly scratching the surface. Governance and alignment look great into studies and academic work, though they alone can’t bring success, no matter how much their importance and usage is accentuated. Sometimes I feel that people hide behind big words without even grasping the facts. The importance of governance and alignment can’t be neglected, though the argumentation provided by KPMG isn’t flawless. There are statements I can agree with, and many which are circumstantial. Anyway, let’s look a little deeper at the above numbers.

I suppose there is no surprise concerning the huge rate of BI projects’ failure. The value is somewhat close to the rate of software projects’ failure. Why would make a BI project an exception from a typical software project, considering that they are facing almost the same environments and challenges?  In fact, given the role played by BI in decision making, I would say that BI projects are more sensitive to the various factors than a typical software project.  

It doesn’t make sense to retake the motives for which software projects fail, but some particular aspects need to be mentioned. KPMG insists on the poor quality of data, on the relevance and volume of reports and metrics used, the lack of reflecting organization’s objectives, the inflexibility of data models, lack of standardization, all of them reflecting in a degree or other on the success of a BI project. There is much more to it!

KPMG refers to a holistic approach concentrated on the change of focus from technology to the actual needs, a change of process and funding.  A reflection of the holistic approach is also the view of the BI infrastructure from the point of view of the entire IT infrastructure, of the organization, network of partners and of the end-products – mainly models and reports. Many of the problems BI initiatives are confronted with refer to the quality of data and its many dimensions (duplicates, conformity, consistency, integrity, accuracy, availability, timeliness, etc.) , problems which could be in theory solved in the source systems, mainly through design. Other problems, like dealing with complex infrastructures based on more or less compatible IS or BI tools, might involve virtualization, consolidation or harmonization of such solutions, plus the addition of other tools.

Looking at the whole organization, other problems appear: the use of reports and models without understanding the whole luggage of meaning hiding behind them, the different views within the same data and models, the difference of language, problems, requirements and objectives, the departmental and organizational politics, the lack of communication, the lack of trust in the existing models and reports, and so on. What all these points have in common are people! The people are the maybe the most important factor in the adoption and effective usage of BI solutions. It starts with them – identifying their needs, and it ends with them – as end users. Making them aware of all contextual requirements, actually making them knowledge workers and not considering them just simple machines could give a boost to your BI strategy.

Partners doesn’t encompass just software vendors, service providers or consultants, but also the internal organizational structures – teams, departments, sites or any other similar structure. Many problems in BI can be tracked down to partners and the ways a partnership is understood, on how resources are managed, how different goals and strategies are harmonized, on how people collaborate and coordinate. Maybe the most problematic is the partnership between IT and the other departments on one side, and between IT and external partners on the other side. As long IT is not seen as a partner, as long IT is skip from the important decisions or isn’t acting as a mediator between its internal and external partners, there are few chances of succeeding. There are so many aspects and lot of material written on this topic, there are models and methodologies supposed to make things work, but often between theory and practice there is a long distance.

How many of the people you met were blaming the poor quality of the data without actually doing something to improve anything? If the quality of your data in one of your major problems then why aren’t you doing something to improve that?  Taking the ownership over your data is a major step on the way to better data quality, though a data management strategy is needed. This involve the design of a framework that facilitates data quality and data consumption, the design and use of policies, practices and procedures to properly manage the full data lifecycle. Also this can be considered as part of your BI infrastructure, and given the huge volume, the complexity and diversity of data, is nowadays a must for an organization.

The “right information” is an evasive construct. In order to get the right information you must be capable to define what you want, to design your infrastructure with that in mind and to learn how to harness your data. You don’t have to look only at your data and information but also at the whole DIKW pyramid. The bottom line is that you don’t have to build only a BI infrastructure but a knowledge management infrastructure, and methodologies like ITIL can help you achieve that, though they are not sufficient. Sooner or later you’ll arrive to blame the whole DIKW pyramid - the difficulty of extracting information from data, knowledge from information, and the ultimate translation into wisdom. Actually that’s also what the third and fourth of the above statements are screaming out loud – it’s not so easy to get information from the silos of data, same as it’s not easy to align the transformation process with organizations’ strategy.

Also timeliness has a relative meaning. It’s true that nowadays’ business dynamics requires faster access to data, though it requires also to be proactive, many organizations lacking this level of maturity. In order to be proactive it’s necessary to understand your business’ dynamics thoroughly, that being routed primarily in your data, in the tools you are using and the skill set your employees acquired in order to move between the DIKW layers. I would say that the understanding of DIKW is essential in harnessing your BI infrastructure.

KPMG considers that the 5% increase in return on equity associated with the effective usage of BI is a positive sign, not necessarily. The increase can be associated with hazard or other factors as well, even if it’s unlikely probable to be so. The increase it’s quite small when considered with the huge amount of resources spent on BI infrastructure. I believe that BI can do much more for organizations when harnessed adequately. It’s just a belief that needs to be backed up by numbers, hopefully that will happen someday, soon.

Previous Post <<||>> Next Post
Related Posts Plugin for WordPress, Blogger...

About Me

My photo
IT Professional with more than 24 years experience in IT in the area of full life-cycle of Web/Desktop/Database Applications Development, Software Engineering, Consultancy, Data Management, Data Quality, Data Migrations, Reporting, ERP implementations & support, Team/Project/IT Management, etc.