Showing posts sorted by date for query Project Management. Sort by relevance Show all posts
Showing posts sorted by date for query Project Management. Sort by relevance Show all posts

18 May 2025

#️⃣Software Engineering: Mea Culpa (Part VII: A Look Forward)

Software Engineering Series
Software Engineering Series

I worked for more than 20 years in various areas related to ERP systems - Data Migrations, Business Intelligence/Analytics, Data Warehousing, Data Management, Project Management, (data) integrations, Quality Assurance, and much more, having experience with IFS IV, Oracle e-Business Suite, MS Dynamics AX 2009 and during the past 3-7 years also with MS Dynamics 365 Finance, SCM & HR (in that order). Much earlier, I started to work with SQL Server (2000-2019), Oracle, and more recently with Azure Synapse and MS Fabric, writing over time more than 800 ad-hoc queries and reports for the various stakeholders, covering all the important areas, respectively many more queries for monitoring the various environments. 

In the areas where I couldn’t acquire experience on the job, I tried to address this by learning in my free time. I did it because I take seriously my profession, and I want to know how (some) things work. I put thus a lot of time into trying to keep actual with what’s happening in the MS Fabric world, from Power BI to KQL, Python, dataflows, SQL databases and much more. These technologies are Microsoft’s bet, though at least from German’s market perspective, all bets are off! Probably, many companies are circumspect or need more time to react to the political and economic impulses, or probably some companies are already in bad shape. 

Unfortunately, the political context has a broad impact on the economy, on what’s happening in the job market right now! However, the two aspects are not the only problem. Between candidates and jobs, the distance seems to grow, a dense wall of opinion being built, multiple layers based on presumptions filtering out voices that (still) matter! Does my experience matter or does it become obsolete like the technologies I used to work with? But I continued to learn, to keep actual… Or do I need to delete everything that reminds the old?

To succeed or at least be hired today one must fit a pattern that frankly doesn’t make sense! Yes, soft skills are important though not all of them are capable of compensating for the lack of technical skills! There seems to be a tendency to exaggerate some of the qualities associated with skills, or better said, of hiding behind big words. Sometimes it feels like a Shakespearian inaccurate adaptation of the stage on which we are merely players.

More likely, this lack of pragmatism will lead to suboptimal constructions that will tend to succumb under their own structure. All the inefficiencies need to be corrected, or somebody (or something) must be able to bear their weight. I saw this too often happening in ERP implementations! Big words don’t compensate for the lack of pragmatism, skills, knowledge, effort or management! For many organizations the answer to nowadays problems is more management, which occasionally might be the right approach, though this is not a universal solution for everything that crosses our path(s).

One of society’s answers to nowadays’ problem seems to be the refuge in AI. So, I wonder – where I’m going now? Jobless, without an acceptable perspective, with AI penetrating the markets and making probably many jobs obsolete. One must adapt, but adapt to what? AI is brainless even if it can mimic intelligence! Probably, it can do more in time to the degree that many more jobs will become obsolete (and I’m wondering what will happen to all those people). 

Conversely, to some trends there will be probably other trends against them, however it’s challenging to depict in clear terms the future yet in making. Society seems to be at a crossroad, more important than mine.

Previous Post <<||>> Next Post

26 April 2025

🏭🗒️Microsoft Fabric: Deployment Pipelines [Notes]

Disclaimer: This is work in progress intended to consolidate information from various sources for learning purposes. For the latest information please consult the documentation (see the links below)! 

Last updated: 26-Apr-2025

[Microsoft Fabric] Deployment Pipelines

  • {def} a structured process that enables content creators to manage the lifecycle of their organizational assets [5]
    • enable creators to develop and test content in the service before it reaches the users [5]
      • can simplify the deployment process to development, test, and production workspaces [5]
      • one Premium workspace is assigned to each stage [5]
      • each stage can have 
        • different configurations [5]
        • different databases or different query parameters [5]
  • {action} create pipeline
    • from the deployment pipelines entry point in Fabric [5]
      • creating a pipeline from a workspace automatically assigns it to the pipeline [5]
    • {action} define how many stages it should have and what they should be called [5]
      • {default} has three stages
        • e.g. Development, Test, and Production
        • the number of stages can be changed anywhere between 2-10 
        • {action} add another stage,
        • {action} delete stage
        • {action} rename stage 
          • by typing a new name in the box
        • {action} share a pipeline with others
          • users receive access to the pipeline and become pipeline admins [5]
        • ⇐ the number of stages are permanent [5]
          • can't be changed after the pipeline is created [5]
    • {action} add content to the pipeline [5]
      • done by assigning a workspace to the pipeline stage [5]
        • the workspace can be assigned to any stage [5]
    • {action|optional} make a stage public
      • {default} the final stage of the pipeline is made public
      • a consumer of a public stage without access to the pipeline sees it as a regular workspace [5]
        • without the stage name and deployment pipeline icon on the workspace page next to the workspace name [5]
    • {action} deploy to an empty stage
      • when finishing the work in one pipeline stage, the content can be deployed to the next stage [5] 
        • deployment can happen in any direction [5]
      • {option} full deployment 
        • deploy all content to the target stage [5]
      • {option} selective deployment 
        • allows select the content to deploy to the target stage [5]
      • {option} backward deployment 
        • deploy content from a later stage to an earlier stage in the pipeline [5] 
        • {restriction} only possible when the target stage is empty [5]
    • {action} deploy content between pages [5]
      • content can be deployed even if the next stage has content
        • paired items are overwritten [5]
    • {action|optional} create deployment rules
      • when deploying content between pipeline stages, allow changes to content while keeping some settings intact [5] 
      • once a rule is defined or changed, the content must be redeployed
        • the deployed content inherits the value defined in the deployment rule [5]
        • the value always applies as long as the rule is unchanged and valid [5]
    • {feature} deployment history 
      • allows to see the last time content was deployed to each stage [5]
      • allows to to track time between deployments [5]
  • {concept} pairing
    • {def} the process by which an item in one stage of the deployment pipeline is associated with the same item in the adjacent stage
      • applies to reports, dashboards, semantic models
      • paired items appear on the same line in the pipeline content list [5]
        • ⇐ items that aren't paired, appear on a line by themselves [5]
      • the items remain paired even if their name changes
      • items added after the workspace is assigned to a pipeline aren't automatically paired [5]
        • ⇐ one can have identical items in adjacent workspaces that aren't paired [5]
  • [lakehouse]
    • can be removed as a dependent object upon deployment [3]
    • supports mapping different Lakehouses within the deployment pipeline context [3]
    • {default} a new empty Lakehouse object with same name is created in the target workspace [3]
      • ⇐ if nothing is specified during deployment pipeline configuration
      • notebook and Spark job definitions are remapped to reference the new lakehouse object in the new workspace [3]
      • {warning} a new empty Lakehouse object with same name still is created in the target workspace [3]
      • SQL Analytics endpoints and semantic models are provisioned
      • no object inside the Lakehouse is overwritten [3]
      • updates to Lakehouse name can be synchronized across workspaces in a deployment pipeline context [3] 
  • [notebook] deployment rules can be used to customize the behavior of notebooks when deployed [4]
    • e.g. change notebook's default lakehouse [4]
    • {feature} auto-binding
      • binds the default lakehouse and attached environment within the same workspace when deploying to next stage [4]
  • [environment] custom pool is not supported in deployment pipeline
    • the configurations of Compute section in the destination environment are set with default values [6]
    • ⇐ subject to change in upcoming releases [6]
  • [warehouse]
    • [database project] ALTER TABLE to add a constraint or column
      • {limitation} the table will be dropped and recreated when deploying, resulting in data loss
    • {recommendation} do not create a Dataflow Gen2 with an output destination to the warehouse
      • ⇐ deployment would be blocked by a new item named DataflowsStagingWarehouse that appears in the deployment pipeline [10]
    • SQL analytics endpoint is not supported
  • [Eventhouse]
    • {limitation} the connection must be reconfigured in destination that use Direct Ingestion mode [8]
  • [EventStream]
    • {limitation} limited support for cross-workspace scenarios
      • {recommendation} make sure all EventStream destinations within the same workspace [8]
  • KQL database
    • applies to tables, functions, materialized views [7]
  • KQL queryset
    • ⇐ tabs, data sources [7]
  • [real-time dashboard]
    • data sources, parameters, base queries, tiles [7]
  • [SQL database]
    • includes the specific differences between the individual database objects in the development and test workspaces [9]
  • can be also used with

    References:
    [1] Microsoft Learn (2024) Get started with deployment pipelines [link]
    [2] Microsoft Learn (2024) Implement continuous integration and continuous delivery (CI/CD) in Microsoft Fabric [link]
    [3] Microsoft Learn (2024)  Lakehouse deployment pipelines and git integration (Preview) [link]
    [4] Microsoft Learn (2024) Notebook source control and deployment [link
    [5] Microsoft Learn (2024) Introduction to deployment pipelines [link]
    [6] Environment Git integration and deployment pipeline [link]
    [7] Microsoft Learn (2024) Microsoft Learn (2024) Real-Time Intelligence: Git integration and deployment pipelines (Preview) [link]
    [8] Microsoft Learn (2024) Eventstream CI/CD - Git Integration and Deployment Pipeline [link]
    [9] Microsoft Learn (2024) Get started with deployment pipelines integration with SQL database in Microsoft Fabric [link]
    [10] Microsoft Learn (2025) Source control with Warehouse (preview) [link

    Resources:

    Acronyms:
    CLM - Content Lifecycle Management
    UAT - User Acceptance Testing

    25 April 2025

    💫🗒️ERP Systems: Microsoft Dynamics 365's Business Process Catalog (BPC) [Notes]

    Disclaimer: This is work in progress intended to consolidate information from the various sources and not to provide a complete overview of all the features. Please refer to the documentation for a complete overview!

    Last updated: 25-Apr-2025

    Business Process Catalog - End-to-End Scenarios

    [Dynamics 365] Business Process Catalog (BPC)

    • {def} lists of end-to-end processes that are commonly used to manage or support work within an organization [1]
      • agnostic catalog of business processes contained within the entire D365 solution space [3]
        • {benefit} efficiency and time savings [3]
        • {benefit} best practices [3]
        • {benefit} reduced risk [3]
        • {benefit} technology alignment [3]
        • {benefit} scalability [3]
        • {benefit} cross-industry applicability [3]
      • stored in an Excel workbook
        • used to organize and prioritize the work on the business process documentation [1]
        • {recommendation} check the latest versions (see [R1])
      • assigns unique IDs to 
        • {concept} end-to-end scenario
          • describe in business terms 
            • not in terms of software technology
          • includes the high-level products and features that map to the process [3]
          • covers two or more business process areas
          • {purpose} map products and features to benefits that can be understood in business contexts [3]
        • {concept} business process areas
          • combination of business language and basic D365 terminology [3]
          • groups business processes for easier searching and navigation [1]
          • separated by major job functions or departments in an organization [1]
          • {purpose} map concepts to benefits that can be understood in business context [3]
          • more than 90 business process areas defined [1]
        • {concept} business processes
          • a series of structured activities and tasks that organizations use to achieve specific goals and objectives [3]
            • efficiency and productivity
            • consistency and quality
            • cost reduction
            • risk management
            • scalability
            • data-driven decision-making
          • a set of tasks in a sequence that is completed to achieve a specific objective [5]
            • define when each step is done in the implementation [5] [3]
            • define how many are needed [5] [3]
          • covers a wide range of structured, often sequenced, activities or tasks to achieve a predetermined organizational goal
          • can refer to the cumulative effects of all steps progressing toward a business goal
          • describes a function or process that D365 supports
            • more than 700 business processes identified
            • {goal} provide a single entry point with links to relevant product-specific content [1]
          • {concept} business process guide
            • provides documentation on the structure and patterns of the process along with guidance on how to use them in a process-oriented implementation [3]
            • based on a catalog of business process supported by D365 [3]
          • {concept} process steps 
            • represented sequentially, top to bottom
              • can include hyperlinks to the product documentation [5] 
              • {recommendation} avoid back and forth in the steps as much as possible [5]
            • can be
              • forms used in D365 [5]
              • steps completed in LCS, PPAC, Azure or other Microsoft products [5]
              • steps that are done outside the system (incl. third-party system) [5]
              • steps that are done manually [5]
            • are not 
              • product documentation [5]
              • a list of each click to perform a task [5]
          • {concept} process states
            • include
              • project phase 
                • e.g. strategize, initialize, develop, prepare, operate
              • configuration 
                • e.g. base, foundation, optional
              • process type
                • e.g. configuration, operational
        • {concept} patterns
          • repeatable configurations that support a specific business process [1]
            • specific way of setting up D365 to achieve an objective [1]
            • address specific challenges in implementations and are based on a specific scenario or best practice [6]
            • the solution is embedded into the application [6]
            • includes high-level process steps [6]
          • include the most common use cases, scenarios, and industries [1]
          • {goal} provide a baseline for implementations
            • more than 2000 patterns, and we expect that number to grow significantly over time [1]
          • {activity} naming a new pattern
            • starts with a verb
            • describes a process
            • includes product names
            • indicate the industry
            • indicate AppSource products
        • {concept} reference architecture 
          • acts as a core architecture with a common solution that applies to many scenarios [6]
          • typically used for integrations to external solutions [6]
          • must include an architecture diagram [6]
      • {concept} process governance
        • {benefit} improved quality
        • {benefit} enhanced decision making
        • {benefit} agility adaptability
        • {benefit{ Sbd alignment
        • {goal} enhance efficiency 
        • {goal} ensure compliance 
        • {goal} facilitate accountability 
        • {concept} policy
        • {concept} procedure
        • {concept} control
      • {concept} scope definition
        • {recommendation} avoid replicating current processes without considering future needs [4]
          • {risk} replicating processes in the new system without re-evaluating and optimizing [4] 
          • {impact} missed opportunities for process improvement [4]
        • {recommendation} align processes with overarching business goals rather than the limitations of the current system [4]
      • {concept} guidance hub
        • a central landing spot for D365 guidance and tools
        • contains cross-application documentations
    • {purpose} provide considerations and best practices for implementation [6]
    • {purpose} provide technical information for implementation [6]
    • {purpose} provide link to product documentation to achieve the tasks in scope [6]
    Previous Post <<||>> Next Post 

    References:
    [1] Microsoft Learn (2024) Dynamics 365: Overview of end-to-end scenarios and business processes in Dynamics 365 [link]
    [2] Microsoft Dynamics 365 Community (2023) Business Process Guides - Business Process Guides [link]
    [3] Microsoft Dynamics 365 Community (2024) Business Process Catalog and Guidance - Part 2 Introduction to Business Processes [link]
    [4] Microsoft Dynamics 365 Community (2024) Business Process Catalog and Guidance - Part 3: Using the Business Process Catalog to Manage Project Scope and Estimation [link]
    [5] Microsoft Dynamics 365 Community (2024) Business Process Catalog and Guidance - Part 4: Authoring Business Processes [link]
    [6] Microsoft Dynamics 365 Community (2024) Business Process Catalog and Guidance - Part 5:  Authoring Business Processes Patterns and Use Cases [link]
    [7] Microsoft Dynamics 365 Community (2024) Business Process Catalog and Guidance  - Part 6: Conducting Process-Centric Discovery [link]
    [8] Microsoft Dynamics 365 Community (2024) Business Process Catalog and Guidance  - Part 7: Introduction to Process Governance [link]

    Resources:
    [R1] GitHub (2024) Business Process Catalog [link]
    [R2] Microsoft Learn (2024) Dynamics 365 guidance documentation and other resources [link]
    [R3] Dynamics 365 Blog (2025) Process, meet product: The business process catalog for Dynamics 365 [link]

    Acronyms:
    3T - Tools, Techniques, Tips
    ADO - 
    BPC - Business Process Catalog
    D365 - Dynamics 365
    LCS - Lifecycle Services
    PPAC - Power Platform admin center
    RFI - Request for Information
    RFP - Request for Proposal

    19 April 2025

    🧮ERP: Implementations (Part XVI: It’s All About Politics)

    ERP Implementations Series
    ERP Implementations Series

    An ERP implementation takes place within a political context and politics can make or break implementations. Politics occurs whenever individuals or organization groups interact to make decisions that affect parts or the whole organization. Besides decision-making there are further components that revolve around the various types of resources allocation and management, resulting in power dynamics that shape and pull organizations in politically charged directions.

    Given the deep implications of ERP systems, probably in no other type of projects the political aspects are that visible and stringent to all employees to the degree that they pull decisions in one direction independently of the actual requirements. It may seem incredible, though there are cases in which ERP systems were selected just because the organization’s CEO played golf with the vendor’s CEO. In the end, the gaps between systems should be minimal nowadays, at least in theory, isn’t it?

    Of course, just because one meets certain strange behaviors, it doesn’t mean that this is common practice! There are higher chances of selecting an inadequate system just because the sales representative did a good job and convinced the audience that the system can do anything they want. It probably does if coins are used for each missing feature, and in the long term it can be a lot of coins. Conversely, even if a system satisfies nowadays’ requirements, it doesn’t mean it will continue to do the same with future requirements. Only the future can tell whether the choice of a system over the others was a good one.

    The bigger the gaps between the various interests, the more difficult it becomes to pull the project in the right direction. Probably the best way to demonstrate why one system is better than another is by bringing facts and focusing on the main requirements of the organization. This supposes the existence of an explicit list of requirements with a high-level description of how they can be addressed by the future system. This might not be enough, though it’s a good start, a good basis for discussion, for making people aware of the implications. However, doing this exercise for 2-3 or more systems is not cost effective, as such analysis can become time-consuming and expensive.

    One way to address political resistance is by discussing openly with the stakeholders and addressing their concerns, arguing why the system is a good choice, what can be done to address the gaps, and so on. It will not always be enough, though it’s important to establish common ground for further discussions. Further on, it’s important to keep the same openness and disposition for communication given that the further the project progresses, the higher the likelihood of other concerns to appear. It’s a never-ending story if there are gaps between needs and what the system provides.

    It's important to establish clear and honest communication with the stakeholders, informing them proactively about the challenges faced, independently in which area they are faced. Conversely, too much communication can be disruptive and can create other challenges. One way to cope with this is by identifying the communication needs of each stakeholder and trying to identify what’s the volume of information, respectively the communication needs of each of them. That’s project management 1:1.

    The Project Manager and his team should ideally anticipate and address the potential conflicts timely, before they propagate and reach a broader audience. It’s questionable how much can be achieved proactively, especially when the project keeps everybody busy. The tendency is to answer politics with politics, though brainstorming sessions, open communication and a few other approaches can reach deeper where politics can’t.

    16 April 2025

    🧮ERP: Implementations (Part XIV: A Never-Ending Story)

    ERP Implementations Series
    ERP Implementations Series

    An ERP implementation is occasionally considered as a one-time endeavor after which an organization will live happily ever after. In an ideal world that would be true, though the work never stops – things that were carved out from the implementation, optimizations, new features, new regulations, new requirements, integration with other systems, etc. An implementation is thus just the beginning from what it comes and it's essential to get the foundation right – and that’s the purpose of the ERP implementation – provide a foundation on which something bigger and solid can be erected. 

    No matter how well an ERP implementation is managed and executed, respectively how well people work towards the same goals, there’s always something forgotten or carved out from the initial project. Usually, the casual suspects are the integrations with other systems, though there can be also minor or even bigger features that are planned to be addressed later, if the implementation hasn’t consumed already all the financial resources available, as it's usually the case. Some of the topics can be addressed as Change Requests or consolidated on projects of their own. 

    Even simple integrations can become complex when the processes are poorly designed, and that typically happens more often than people think. It’s not necessarily about the lack of skillset or about the technologies used, but about the degree to which the processes can work in a loosely coupled interconnected manner. Even unidirectional integrations can raise challenges, though everything increases in complexity when the flow of data is bidirectional. Moreover, the complexity increases with each system added to the overall architecture. 

    Like a sculpture’s manual creation, processes in an ERP implementation form a skeleton that needs chiseling and smoothing until the form reaches the desired optimized shape. However, optimization is not a one-time attempt but a continuous work of exploring what is achievable, what works, what is optimal. Sometimes optimization is an exact science, while other times it’s about (scientifical) experimentation in which theory, ideas and investments are put to good use. However, experimentation tends to be expensive at least in terms of time and effort, and probably these are the main reasons why some organizations don’t even attempt that – or maybe it’s just laziness, pure indifference or self-preservation. In fact, why change something that already works?

    Typically, software manufacturers make available new releases on a periodic basis as part of their planning for growth and of attracting more businesses. Each release that touches used functionality typically needs proper evaluation, testing and whatever organizations consider as important as part of the release management process. Ideally, everything should go smoothly though life never ceases to surprise and even a minor release can have an important impact when earlier critical functionality stopped working. Test automation and other practices can make an important difference for organizations, though these require additional effort and investments that usually pay off when done right. 

    Regulations and other similar requirements must be addressed as they can involve penalties or other risks that are usually worth avoiding. Ideally such requirements should be supported by design, though even then a certain volume of work is involved. Moreover, the business context can change unexpectedly, and further requirements need to be considered eventually. 

    The work on an ERP system and the infrastructure built around it is a never-ending story. Therefore, organizations must have not only the resources for the initial project, but also what comes after that. Of course, some work can be performed manually, some requirements can be delayed, some risks can be assumed, though the value of an ERP system increases with its extended usage, at least in theory. 

    🧮ERP: Implementations (Part XIII: On Project Management)

    ERP Implementations Series
    ERP Implementations Series

    Given its intrinsic complexity and extended implications, an ERP implementation can be considered as the real test of endurance for a Project Manager, respectively the team managed. Such projects typically deal with multiple internal and external parties with various interests in the outcomes of the project. Moreover, such projects involve multiple technologies, systems, and even methodologies. But, more importantly, such projects tend to have specific characteristics associated with their mass, being challenging to manage within the predefined constraints: time, scope, costs and quality.

    From a Project Manager’s perspective what counts is only the current project. From a PMO perspective, one project, independent of its type, must be put within the broader perspective, while looking at the synergies and other important aspects that can help the organization. Unfortunately, for many organizations all begins and ends with the implementation, and this independently of the outcomes of the project. Often failure lurks in the background and usually there can be small differences that in the long term have a considerable impact. ERP implementations are more than other projects sensitive on the initial conditions – the premises under which the project starts and progresses. 

    One way of coping with this inherent complexity is to split projects into several phases considered as projects or subprojects in their own boundaries. This allows organizations to narrow the focus and split the overall work into more manageable pieces, reducing to some degree the risks while learning in the process about organization’s capabilities in addressing the various aspects. Conversely, the phases are not necessarily sequential but often must overlap to better manage the resources and minimize waste. 

    Given that an implementation project can take years, it’s normal for people to come and go, some taking over work from colleagues, with or without knowledge transfer. The knowledge is available further on, as long as the resources don’t leave the organization, though knowledge transfer can’t be taken for granted. It’s also normal for resources to suddenly not be available or disappear, increasing the burden that needs to be shifted on others’ shoulders. There’s seldom a project without such events and one needs to make the best of each situation, even if several tries and iterations are needed in the process.

    Somebody needs to manage all this, and the weight of the whole project falls on a PM’s shoulders. Managing by exception and other management principles break under the weight of implementation projects and often it’s challenging to make progress without addressing this. Fortunately, PMs can shift the burden on Key Users and other parties involved in the project. Splitting a project in subprojects can help set boundaries even if more management could occasionally be involved. Also having clear responsibilities and resources who can take over the burdens when needed can be a sign of maturity of the teams, respectively the organization. 

    Teams in Project Management are often compared with teams in sports, though the metaphor is partially right when each party has a ball to play with, while some of the players or even teams prefer to play alone at their own pace. It takes time to build effective teams that play well together, and the team spirit or other similar concepts can't fill all the gaps existing in organizations! Training in team sports has certain characteristics that must be mirrored in organizations to allow for teams to improve. Various parties expect from the PM to be the binder and troubleshooter of something that should have been part of an organization’s DNA! Bringing external players to do the heavy lifting may sometimes work, though who’ll do the lifting after the respective resources are gone? 

    Previous Post <<||>> Next Post

    15 March 2025

    💫🗒️ERP Systems: Microsoft Dynamics 365's Business Performance Analytics (BPA) [notes]

    Disclaimer: This is work in progress intended to consolidate information from the various sources and not to provide a complete overview of all the features. Please refer to the documentation for a complete overview!

    Last updated: 15-Mar-2025

    [Dynamics 365] Business Performance Analytics (BPA)

    • {def} centralized reporting hub within D365 F&O designed to streamline insights and help organizations make faster, data driven decisions [3]
      • solution designed to transform organization's data into actionable insights [1]
      • provides an intuitive out-of-box data model along with familiar tools like Microsoft Excel and Power BI for self-service analytics [4]
        • data extracted from D365 is classified in BPA in the form of value chains
          • ⇐ a group of business processes on top of the value chain [4]
    • {benefit} allows to simplify data insights by providing a unified view of business data across entities in near real time [4]
    • {benefit} allows to streamline financial and operations reporting to reduce the cycle times [4]
    • {benefit} allows users of all technical abilities to quickly access and analyze data to facilitate data driven decisions [4]
    • {benefit} provides auditors with direct access to financial data, making the audit process more efficient
    • {benefit} enables ease of use through familiar apps like Excel and Power BI, in addition to AI driven insights and automation in this platform that can be scalable and extendable [4]
    • {feature} extends into Microsoft Fabric
      • {benefit} provide a scalable, secure environment for handling large data sets and ensuring insights are always powered by the latest technology [3]
    • {feature} ETL process 
      • involves extracting data from finance and operations database, transforming and loading it into Dataverse [4]
        • each of the entities required for the generation of the dimensional model for the value chains that were mentioned earlier, they are backed by the underlying tables in finance and operations database [4]
      • installed in Dataverse, virtual  entities that are created will then pull in the data into the managed data lake [4]
      • the data is then transformed to generate the dimensional  model which is then pushed into the embedded Power BI workspace in the form of analytical tables [4]
      • BPA consumes this data from Power BI workspace to render the power BI reports [4]
      • this data can also be extended to Fabric if there is a need to consolidate data from multiple sources [4]
    • {feature} reports 
      • designed to provide a detailed overview of an organization's financial health [8]
      • further reports will be added to expand the coverage for the value chains [8]
      • out-of-box reports can't be modified
        • ⇐ users cannot rename, delete or edit these type of reports [8]
        • there’s the option to duplicate the base report and edit the version thus created [8]
      • can be shared with other users who have access to BPA 
        • ⇐ they can receive an in-app notification [8]
        • can be shared over email with another user by entering user’s email address [8] 
        • one can configure whether the recipient can edit or view the report [8]
      •   {feature} allows to create a new Power BI or Excel report from scratch [8]
        • {option} start with a blank report or duplicate an existing report [8]
    • {feature} data refresh
      • automatic data refreshes run currently two times a day [4]
        • at 12:00 AM and 12:00 PM UTC
        • the volume of data is also constrained by the storage capacity of the A3 SKU for Power BI Embedded [1]
          • future release, may support additional data reporting capacity [1]
            • ⇐ so that larger data sets can be reported and analyzed [1]
        • the target is to have refreshes every hour or less [3]
      • data volume will be initially for about eight quarters of data [4]
      • extensibility will be supported with bring your own Fabric [4]
    • architecture
      • SaaS solution
        • {capability} immediate deployment 
          • businesses can start to analyze data and generate insights with minimal setup [1]
        • {capability} comprehensive reporting and dashboards
          • provides access to a wide range of preconfigured reports that cover multiple business functions [1]
        • {capability} near-real-time analytics 
          • future releases will offer more frequent data refreshes to enable near-real-time data analysis and reporting
        • {capability} predictive insights 
          • future releases will introduce predictive analytics capabilities that enable businesses to 
            • forecast trends
            • identify risks
            • seize opportunities [1]
        • {capability} user-friendly interface 
          • intuitive design ⇒ minimal training
            • fosters broader adoption 
            • enables a data-driven culture across the organization [1]
        • {capability} cost-effectiveness
          • available as part of D365 license
            • ⇒ provides advanced analytics without requiring significant investments in IT infrastructure [1]
      • DaaS solution
        • {capability} organizations can integrate its data models with their existing data warehousing infrastructure in Microsoft Fabric [1]
          • maximizes the value of existing data solutions [1]
          • positions businesses for future enhancements [1]
        • {capability} unified and scalable data models
          • customers can build custom models on top of a unified framework
            • ensures consistency and scalability across data sets [1]
        • {capability} future-proofing with automatic upgrades
          • data models integrate seamlessly with future D365 updates
            • reduces manual maintenance and ensures access to the latest features [1]
        • {capability} consistency and standardization
          • data models provide consistency and standardization across data sources
            • ensure high data quality and integrity [1]
        • {capability} advanced analytics and AI 
          • by customizing the data models, organizations can take advantage of advanced analytics and AI capabilities [1]
            • deeper insights without having to develop them from scratch [1]
        • {capability} enhanced data governance
          • unified data models support better data governance by providing standardized data definitions, relationships, and hierarchies [1]
            • ensure consistency and quality across the organization [1]
      • requires an integrated Power Platform environment [5]
        • must be integrated with the Microsoft Entra tenant [5]
      • uses shared Dataverse entitlements [1]
        • includes access to the data lake [1]
    • setup
      • dimensions
        • the selection of dimensions might affect the dimension groups that are created using these dimensions and the users who are assigned there [7]
          • e.g. legal entity, business unit
      • dimension groups
        • users can select specific values for the legal entity, or add a range of values [7]
          • selecting an invalid combination of dimension values, the dimension group will filter out all the records on the report [7]
        • {warning} assigning too many dimension groups to a user, slows the load for that user [7]
      • roles
        • determine which reports the user can access [7]
    • security
      • secure data through role-based access control on top of the value chains [7]
      • the first user who signs into the app is assigned the BPA admin role [7]
        • allows a user to access the administrator section of the BPA [7]
          • where the security can be set up [7]
        • has automatically assigned 
          • Microsoft report viewer role 
          • the All Access Dimension group [7]
            • allow the admin to see the data  in all the reports across all the dimensions [7]
      • {feature} dimension-based role-level security
        • ensures that users only see the data relevant to them based on their role
          •  confidently share reports without duplicating them
            • ⇐ data is automatically filtered by organization's security policies [3]
        • simple but powerful way to maintain control while providing access for teams that love working in Excel [3]
    • accessibility
      • can be accessed through either 
        • Power Platform
          • admins can access BPA app through PowerApps' makeup portal [6]
        • Dynamics 365
          • through the BPA preview shortcut in the homepage or the default dashboard [6]
          • for end users, the BPA preview shortcut is provided when they have certain duties associated to their role(s) [6]
    • licensing
      • included in D365 F&O license [4]
    • requirements
      • requires a tier two environment and Dynamics 365 finance version 1.0.38 or later [5]
    • {project} timeline
      • [2025 wave 1] backup and restore custom reports and analytics
        • {benefit} support better lifecycle management and empower customers to develop on sandbox instances before publishing to production [3]
      • 2025: available in all regions where F&O is available [3]
      • Oct-2024: GA

    References:
    [1] Microsoft Learn (2024) Dynamics 365 Finance: What is Business performance analytics? [link]
    [2] Microsoft Learn (2025) Business performance analytics (BPA) with Dynamics 365 Finance [link]
    [3] Dynamics 365 Finance - Business Performance Analytics 2025 Release Wave 1 Release Highlights [link]
    [4] Dynamics 365 Community (2024) Dynamics 365 Bites: Business Performance Analytics Part 1 [link]
    [5] Dynamics 365 Community (2024) Dynamics 365 Bites: Business Performance Analytics Part 2 [link]
    [6] Dynamics 365 Community (2024) Dynamics 365 Bites: Business Performance Analytics Part 3 [link]
    [7] Dynamics 365 Community (2024) Dynamics 365 Bites: Business Performance Analytics Part 4 [link]   
    [8] Dynamics 365 Community (2024) Dynamics 365 Bites: Business Performance Analytics Part 5 [link]
    [9] Microsoft Learn (2024) Dynamics 365: Business performance analytics introduction [link

    Acronyms:
    AI - Artificial Intelligence
    BPA - Business Performance Analytics
    D365 F&O - Dynamics 365 for Finance and Operations
    DaaS - Data-as-a-Service
    ETL - Extract, Transfer, Load
    GA - General Availability
    MF - Microsoft Fabric
    PP - Public Preview
    SaaS - Software-as-a-Service
    SKU - Stock Keeping Unit
    UTC - Coordinated Universal Time

    08 March 2025

    #️⃣Software Engineering: Programming (Part XVI: The Software Quality Perspective and AI)

    Software Engineering Series
    Software Engineering Series

    Organizations tend to complain about poor software quality developed in-house, by consultancy companies or third parties, without doing much in this direction. Unfortunately, this agrees with the bigger picture reflected by the quality standards adopted by organizations - people talk and complain about them, though they aren’t that eager to include them in the various strategies, or even if they are considered, they are seldom enforced adequately!

    Moreover, even if quality standards are adopted, and a lot of effort may be spent in this direction (as everybody has strong opinions and there are many exceptions), as projects progress, all the good intentions come to an end, the rules fading on the way either because are too strict, too general, aren’t adequately prioritized or communicated, or there’s no time to implement (all of) them. This applies in general to programming and to the domains that revolve around data – Business Intelligence, Data Analytics or Data Science.

    The volume of good quality code and deliverables is not only a reflection of an organization’s maturity in dealing with best practices but also of its maturity in handling technical debt, Project Management, software and data quality challenges. All these aspects are strongly related to each other and therefore require a systemic approach rather than focusing on the issues locally. The systemic approach allows organizations to bridge the gaps between business areas, teams, projects and any other areas of focus.

    There are many questionable studies on the effect of methodologies on software quality and data issues, proclaiming that one methodology is better than the other in addressing the multifold aspects of software quality. Besides methodologies, some studies attempt to correlate quality with organizations’ size, management or programmers’ experience, the size of software, or whatever characteristic might seem to affect quality.

    Bad code is written independently of companies’ size or programmer's experience, management or organization’s maturity. Bad code doesn’t necessarily happen all at once, but it can depend on circumstances, repetitive team, requirements and code changes. There are decisions and actions that sooner or later can affect the overall outcome negatively.

    Rewriting the code from scratch might look like an approachable measure though it’s seldom the cost-effective solution. Allocating resources for refactoring is usually a better approach, though this tends to increase considerably the cost of projects, and organizations might be tempted to face the risks, whatever they might be. Independently of the approaches used, sooner or later the complexity of projects, requirements or code tends to kick back.

    There are many voices arguing that AI will help in addressing the problems of software development, quality assurance and probably other areas. It’s questionable how much AI will help to address the gaps, non-concordances and other mistakes in requirements, and how it will develop quality code when it has basic "understanding" issues. Even if step by step all current issues revolving around AI will be fixed, it will take time and multiple iterations until meaningful progress will be made.

    At least for now, AI tools like Copilot or ChatGPT can be used for learning a programming language or framework through predefined or ad-hoc prompts. Probably, it can be used also to identify deviations from best practices or other norms in scope. This doesn’t mean that AI will replace for now code reviews, testing and other practices used in assuring the quality of software, but it can be used as an additional method to check for what was eventually missed in the other methods.

    AI may also have hidden gems that when discovered, polished and sized, may have a qualitative impact on software development and software. Only time will tell what’s possible and achievable.

    22 February 2025

    🧩IT: The Annotated Laws that Govern IT Professionals' Lives - Part I

    "A bad idea executed to perfection is still a bad idea." (Norman R Augustine) [Augustine's Law]

    "Bad code executed by powerful machines is still bad code." [sql-troubles]

    "A great many problems do not have accurate answers, but do have approximate answers, from which sensible decisions can be made." (Berkeley's Law)

    "It's easier to take/sell approximations as accurate answers than to find accurate answers. In time people will see no difference in between." [sql-troubles]

    "About the time you finish doing something, you know enough to start." (James C Kinser) [Kinser's Law]

    "By the time you finish something, the problem changed." [sql-troubles]

    "People will more likely repeat their known mistakes than trying something new." [sql-troubles]

    "The ofter a method failed, the higher the chances for it to succeed when used by somebody else." [sql-troubles]

    "People tend to reuse a method that previously failed (multiple times) than try something new." [sql-troubles]

    "By the time we start something, somebody else solved already the problem." [sql-troubles]

    "Adding manpower to a late software project makes it later." (Fred P Brooks, "The Mythical Man-Month: Essays", 1975) [Brook's Law]

    "Adding manpower seldom solves a problem that requires intelligent effort." [sql-troubles]

    "The easiest way to make a project on time is to to move the deadline as suited." [sql-troubles]

    "An object will fall so as to do the most damage." [Law of selective gravity]

    "A bug will appear to do the most damage." [sql-troubles]

    "Anything can be made to work if you fiddle with it long enough." (Wyszkowski's second law)
    "Some problems do require infinite time." [sql-troubles]

    "Build a system that even a fool can use, and only a fool will want to use it." [Shaw's principle]

    "Doing it the hard way is always easier." (Murphy's paradox)

    "Doing it the easy way is always harder." [sql-troubles]

    "Don't force it - get a bigger hammer." [Anthony's law of force]

    "Don't optimize it, get a more powerful machine." [sql-troubles]

    "Every solution breeds new problems." [Murphy's laws]

    "Every new problem multiplies the possible solutions." [sql-troubles]

    "It's easier to change the problem to fit the solution." [sql-troubles]

    "Everyone has a scheme that will not work." [Howe's law]

    "Any scheme can work by accident." [sql-troubles]

    "It takes more than an accident for a scheme to work." [sql-troubles]

    "Everything goes wrong all at once." (Quantized revision of Murphy's law)

    "Small events converge toward bigger events." [sql-troubles]

    "Things already went wrong before we observe them as such." [sql-troubles]

    "If a problem causes many meetings, the meetings eventually become more important than the problem." (Arthur Bloch, "Murphy's Law (Price/Stern/Sloan", 1977) (Hendrickson’s Law)

    "More meetings tend to create more problems." [sql-troubles]

     "Fewer meetings tend to create more problems." [sql-troubles]

    "If a project is not worth doing at all, it's not worth doing well." (Gordon's first law)

    "The more a project is not worth doing, the more attention will attract."  [sql-troubles]

    "If an experiment works, something has gone wrong." [Finagle's first law]

    "If anything can go wrong, it will." [Murphy's laws]

    "Things go wrong at a faster pace than one can find solutions." [sql-troubles]

    "If there are two or more ways to do something, and one of those ways can result in a catastrophe, then someone will do it." [Murphy's Laws]

    "It's enough one way, for things to result in catastrophes." [sql-troubles]

    "Sometimes it's better to do nothing than make things worse." [sql-troubles]

    "Once all the known wrong solutions were exhausted, one discovers a new wrong solution." [sql-troubles]

    "If they know nothing of what you are doing, they suspect you are doing nothing." (Robert J Graham et al, "The Complete Idiot's Guide to Project Management", 2007)  [Graham's Law]

    "People are good at ignoring the obvious." [sql-troubles]

    "The more one explains, the more one is misunderstood." [sql-troubles] 

    "If you mess with a thing long enough, it'll break." [Schmidt's law]

    "Things break by design." [sql-troubles]

    "One can learn to break things, by simply playing with them." [sql-troubles] 

    "It's easier to break than design things. One can find thousands ways on how to break the same thing." [sql-troubles] 

    "In any collection of data, the figure most obviously correct, beyond all need of checking, is the mistake." (Finagle's third law)

    "In any collection of data there's at least a mistake." [sql-troubles]

    "In any given set of circumstances, the proper course of action is determined by subsequent events." [McDonald's corollary to Murphy's laws]

    "In crises that force people to choose among alternative courses of action, most people will choose the worst one possible." (Rudin's law)

    "People go wrong with confidence." [sql-troubles]

    "The more alternatives, the higher the chances to go wrong." [sql-troubles] 

    "Information necessitating a change of design will be conveyed to the designer after - and only after - the plans are complete." [First law of revision:]

    "In simple cases, presenting one obvious right way versus one obvious wrong way, it is often wiser to choose the wrong way so as to expedite subsequent revision." (First corollary

    "The designer will get ahead of the design." [sql-troubles] 

    "It is impossible to make anything foolproof because fools are so ingenious." (Murphy's second corollary)

    "It works better if you plug it in." (Sattinger's law)

    "It works longer if you don't plug it in." [sql-troubles]

    "It's not a question of IF the car will break down, but WHEN it will break down." (Murphy's theory of automobiles)

    "It's not a question of IF a program will break down, but when the code will break down." [sql-troubles]

    "The longer a program runs smoothly, the higher the chances that will break down soon." [sql-troubles]

    "Left to themselves, things tend to go from bad to worse." (Murphy's first corollary)

    "The more on tries to fix things, the faster everything goes worse." [sql-troubles]

    "Logic is a systematic method of coming to the wrong conclusion with confidence." (Manly's maxim)

     "One doesn't need logic to arrive at the right conclusion." [sql-troubles]

    "Matter will be damaged in direct proportion to its value." (Murphy's constant)

    "Most problems have either many answers or no answer. Only a few problems have a single answer." [Berkeley's Law]

    "It's better to have a multitude of approximate solutions than one correct solution." [sql-troubles]

    "Negative expectations yield negative results. Positive expectations yield negative results." (Non-reciprocal law of expectations)

    "Negative results yield when there are no expectations." [sql-troubles]

    "No matter how many things have gone wrong, there remains at least one more thing that will go wrong." (Murphy's law of the infinite)

    "Things can go wrong in a multitude of ways." [sql-troubles]

    "No matter how minor the job is, it's still over $50." (Murphy's rule of auto repair)

    "No matter what the experiment's result, there will always be someone eager to: (i) misinterpret it, (ii) fake it, or (c) believe it supports his own pet theory." (Finagle's second law)
    "It's easier to fake the experiment to get the right results." [sql-troubles]
    "Nothing ever goes away." (Commoner's second law of ecology)
    "Things do go away, but tend to come back." [sql-troubles]

    "Nothing is as easy as it looks." (Murphy's first corollary)

    "All things look simple until one dives deeper." [sql-troubles]

    "Nothing is ever so bad that it can't get worse." (Gattuso's extension of Murphy's Law)

    "Once a job is fouled up, anything done to improve it only makes it worse." (Finagle's fourth law)

    "Once a mistake is corrected, a second mistake will become apparent." (Murphy's law of revision)

    "Correcting mistakes introduces other mistakes." [sql-troubles]

    "The chief cause of problems is solutions." [Sevareid's Law]

    "The more time you spend in reporting on what you are doing, the less time you have to do anything. Stability is achieved when you spend all your time doing nothing but reporting on the nothing you are doing." [Cohn's Law]

    "Reporting increases the needs for more information. The less one reports, the lower the need for further information." [sql-troubles]

    "The more innocuous the modification appears to be, the further its influence will extend and the more plans will have to be redrawn." [H B Fyfe's second law of revision]

    "The only thing more costly than stretching the schedule of an established development program is accelerating it, which is itself the most costly action known to man." (Norman R Augustine, "Augustine's Laws", 1983) [Law of economic unipolarity]

    "The other line moves faster." (Etorre's observation)

    "The other team moves faster." [sql-troubles]

    "If you change lines, the one you just left will start to move faster than the one you are now in." (O'Brien's variation

    "If you change a line, the whole codes breaks." [sql-troubles]

    "The longer you wait in line, the greater the likelihood that you are in the wrong line." (The Queue Principal)

    "The longer you wait for a deliverable, the greater the likelihood that it contains bugs." [sql-troubles]

    "The perceived usefulness of an article is inversely proportional to its actual usefulness once bought and paid for." (Glatum's law of materialistic acquisitiveness)

    "The probability of anything happening is in inverse ratio to its desirability." (Gumperson's law)

    "The solution to a problem changes the problem." [Peers's Law]

    "A problem to a solution changes thr solution." [sql-troubles]

    "The tasks to do immediately are the minor ones; otherwise, you’ll forget them. The major ones are often better to defer. They usually need more time for reflection. Besides, if you forget them, they’ll remind you." [Wolf ’s Law of Management]

    "There are two states to any large project: Too early to tell and too late to stop." (Ernest Fitzgerald) [Fitzgerald's Law]

    "There is a solution to every problem; the only difficulty is finding it." [Evvie Nef's Law]

    "There is a solution to every problem we are not trying to solve." [sql-troubles]

    "Finding problems is easier than finding solutions." [sql-troubles]

    "One stumbles upon the same problen twice." [sql-troubles]

    "There is no mechanical problem so difficult that it cannot be solved by brute strength and ignorance. [William's Law]

    "There's no software problem so difficult that can't be solved by brute force and ignorance." [sql-troubles]

    "There's always one more bug." (Lubarsky's law of cybernetic entomology)

    "Software solutions diverge to a set of bugs." [sql-troubles

    "Things get worse under pressure." [Murphy's law of thermodynamics]

    "Things get worse also without pressure." [sql-troubles]

    "Things go right gradually, but things go wrong all at once." (Murphy's asymmetry principle)

    "Tolerances will accumulate unidirectionally toward maximum difficulty of assembly. (Klipstein's law)

    "Two wrongs are only the beginning." (Kohn's corollary to Murphy's law)

    "One wrong can be the beginning of another." [sql-troubles]

    "When all else fails, read the instructions." [Cahn's axiom]

    "Even if you read the instructions, things fall." [sql-troubles]

    "When an error has been detected and corrected, it will be found to have been correct in the first place." [Scott's second law]

    "Any two related problems may look the same when regarded from same perspective." [sql-troubles]

    "When in doubt, use a bigger hammer." [Dobbins’ Law]

    "When taking something apart to fix a minor malfunction, you will cause a major malfunction." (Murphy's second law of construction)

    "Whenever you set out to do something, something else must be done first." (Murphy's sixth corollary)

    "While the difficulties and dangers of problems tend to increase at a geometric rate, the knowledge and manpower qualified to deal with these problems tend to increase linearly." [Dror's First Law]

    "Beyond a point, the problems are so complex that people can't differentiate between geometric and linear rates." [sql-troubles]

     Previous Post <<||>> Next Post

    24 January 2025

    🧭Business Intelligence: Perspectives (Part XXIV: Building Castles in the Air)

    Business Intelligence Series
    Business Intelligence Series

    Business users have mainly three means of visualizing data – reports, dashboards and more recently notebooks, the latter being a mix between reports and dashboards. Given that all three types of display can be a mix of tabular representations and visuals/visualizations, the difference between them is often neglectable to the degree that the terms are used interchangeably. 

    For example, in Power BI a report is a "multi-perspective view into a single semantic model, with visualizations that represent different findings and insights from that semantic model" [1], while a dashboard is "a single page, often called a canvas, that uses visualizations to tell a story" [1], a dashboards’ visuals coming from one or more reports [2]. Despite this clear delimitation, the two concepts continue to be mixed and misused in conversations even by data-related professionals. This happens also because in other tools the vendors designate as dashboard what is called report in Power BI. 

    Given the limited terminology, it’s easy to generalize that dashboards are useless, poorly designed, bad for business users, and so on. As Stephen Few recognized almost two decades ago, "most dashboards fail to communicate efficiently and effectively, not because of inadequate technology (at least not primarily), but because of poorly designed implementations" [3]. Therefore, when people say that "dashboards are bad" refer to the result of poorly implementations, of what some of them were part of, which frankly is a different topic! Unfortunately, BI implementations reflect probably more than any other areas how easy is to fail!

    Frankly, here it is not necessarily the poor implementation of a project management methodology at fault, which quite often happens, but the way requirements are defined, understood, documented and implemented. Even if these last aspects are part of the methodologies, they are merely a reflection of how people understand the business. The outcomes of BI implementations are rooted in other areas, and it starts with how the strategic goals and objectives are defined, how the elements that need oversight are considered in the broader perspectives. The dashboards become thus the end-result of a chain of failures, failing to build the business-related fundament on which the reporting infrastructure should be based upon. It’s so easy to shift the blame on what’s perceptible than on what’s missing!

    Many dashboards are built because people need a sense of what’s happening in the business. It starts with some ideas based on the problems identified in organizations, one or more dashboards are built, and sometimes a lot of time is invested in the process. Then, some important progress is made, and all comes to a stale if the numbers don’t reveal something new, important, or whatever users’ perception is. Some might regard this as failure, though as long as the initial objectives were met, something was learned in the process and a difference was made, one can’t equate this with failure!

    It’s more important to recognize the temporary character of dashboards, respectively of the requirements that lead to them and build around them. Of course, this requires occasionally a different approach to the whole topic. It starts with how KPIs and other business are defined and organized, respectively on how data repositories are built, and it ends with how data are visualized and reported.

    As the practice often revealed, it’s possible to build castles in the air, without a solid foundation, though the expectation for such edifices to sustain the weight of businesses is unrealistic. Such edifices break with the first strong storm and unfortunately it's easier to blame a set of tools, some people or a whole department instead at looking critically at the whole organization!


    References:
    [1] Microsoft Learn (2024) Power BI: Glossary [link]
    [2] Microsoft Learn (2024) Power BI: Dashboards for business users of the Power BI service [link
    [3] Stephen Few, "Information Dashboard Design", 2006
    Related Posts Plugin for WordPress, Blogger...

    About Me

    My photo
    Koeln, NRW, Germany
    IT Professional with more than 25 years experience in IT in the area of full life-cycle of Web/Desktop/Database Applications Development, Software Engineering, Consultancy, Data Management, Data Quality, Data Migrations, Reporting, ERP implementations & support, Team/Project/IT Management, etc.