Disclaimer: This is work in progress intended to consolidate information from various sources for learning purposes. For the latest information please consult the documentation (see the links below)!
Last updated: 13-Apr-2025
[Microsoft Fabric] Continuous Integration & Continuous Deployment [CI/CD]- {def} development processes, tools, and best practices used to automates the integration, testing, and deployment of code changes to ensure efficient and reliable development
- can be used in combination with a client tool
- e.g. VS Code, Power BI Desktop
- don’t necessarily need a workspace
- developers can
- create branches
- commit changes to that branch locally
- push changes to the remote repo
- create a pull request to the main branch
- ⇐ all steps can be performed without a workspace [1]
- workspace is needed only as a testing environment [1]
- to check that everything works in a real-life scenario [1]
- addresses a few pain points [2]
- manual integration issues
- manual changes can lead to conflicts and errors
- slow down development [2]
- development delays
- manual deployments are time-consuming and prone to errors
- lead to delays in delivering new features and updates [2]
- inconsistent environments
- inconsistencies between environment cause issues that are hard to debug [2]
- lack of visibility
- can be challenging to
- track changes though their lifetime [2]
- understand the state of the codebase[2]
- {process} continuous integration (CI)
- {process} continuous deployment (CD)
- architecture
- {layer} development database
- {recommendation} should be relatively small [1]
- {layer} test database
- {recommendation} should be as similar as possible to the production database [1]
- {layer} production database
- data items
- items that store data
- items' definition in Git defines how the data is stored [1]
- {stage} development
- {best practice} back up work to a Git repository
- back up the work by committing it into Git [1]
- {prerequisite} the work environment must be isolated [1]
- so others don’t override the work before it gets committed [1]
- commit to a branch no other developer is using [1]
- commit together changes that must be deployed together [1]
- helps later when
- deploying to other stages
- creating pull requests
- reverting changes
- {warning} big commits might hit the max commit size limit [1]
- {bad practice} store large-size items in source control systems, even if it works [1]
- {recommendation} consider ways to reduce items’ size if they have lots of static [1] resources, like images [1]
- {action} revert to a previous version
- {operation} undo
- revert the immediate changes made, as long as they aren't committed yet [1]
- each item can be reverted separately [1]
- {operation} revert
- reverting to older commits
- {recommendation} promote an older commit to be the HEAD
- via git revert or git reset [1]
- shows that there’s an update in the source control pane [1]
- the workspace can be updated with that new commit [1]
- {warning} reverting a data item to an older version might break the existing data and could possibly require dropping the data or the operation might fail [1]
- {recommendation} check dependencies in advance before reverting changes back [1]
- {concept} private workspace
- a workspace that provides an isolated environment [1]
- ⇐ allows to work in isolation [1]
- {prerequisite} the workspace is assigned to a Fabric capacity [1]
- {prerequisite} access to data to work in the workspace [1]
- {step} create a new branch from the main branch [1]
- allows to have most up-to-date version of the content [1]
- can be used for any future branch created by the user [1]
- when a sprint is over, the changes are merged and one can start a fresh new task [1]
- switch the connection to a new branch on the same workspace
- approach can be used when is needed to fix a bug in the middle of a sprint [1]
- {validation} connect to the correct folder in the branch to pull the right content into the workspace [1]
- {best practice} make small incremental changes that are easy to merge and less likely to get into conflicts [1]
- update the branch to resolve the conflicts first [1]
- {best practice} change workspace’s configurations to enable productivity [1]
- connection between items, or to different data sources or changes to parameters on a given item [1]
- {recommendation} make sure you're working with the supported structure of the item you're authoring [1]
- if you’re not sure, first clone a repo with content already synced to a workspace, then start authoring from there, where the structure is already in place [1]
- {constraint} a workspace can only be connected to a single branch at a time [1]
- {recommendation} treat this as a 1:1 mapping [1]
- {stage} test
- {best practice} allows to simulate a real production environment for testing purposes [1]
- {alternative} simulate this by connecting Git to another workspace [1]
- factors to consider for the test environment
- data volume
- usage volume
- production environment’s capacity
- stage and production should have the same (minimal) capacity [1]
- using the same capacity can make production unstable during load testing [1]
- {recommendation} test using a different capacity similar in resources to the production capacity [1]
- {recommendation} use a capacity that allows to pay only for the testing time [1]
- allows to avoid unnecessary costs [1]
- {best practice} use deployment rules with a real-life data source
- {recommendation} use data source rules to switch data sources in the test stage or parameterize the connection if not working through deployment pipelines [1]
- {recommendation} separate the development and test data sources [1]
- {recommendation} check related items
- the changes made can also affect the dependent items [1]
- {recommendation} verify that the changes don’t affect or break the performance of dependent items [1]
- via impact analysis.
- {operation} update data items in the workspace
- imports items’ definition into the workspace and applies it on the existing data [1]
- the operation is same for Git and deployment pipelines [1]
- {recommendation} know in advance what the changes are and what impact they have on the existing data [1]
- {recommendation} use commit messages to describe the changes made [1]
- {recommendation} upload the changes first to a dev or test environment [1]
- {benefit} allows to see how that item handles the change with test data [1]
- {recommendation} check the changes on a staging environment, with real-life data (or as close to it as possible) [1]
- {benefit} allows to minimize the unexpected behavior in production [1]
- {recommendation} consider the best timing when updating the Prod environment [1]
- {benefit} minimize the impact errors might cause on the business [1]
- {recommendation} perform post-deployment tests in Prod to verify that everything works as expected [1]
- {recommendation} have a deployment, respectively a recovery plan [1]
- {benefit) allows to minimize the effort, respectively the downtime [1]
- {stage} production
- {best practice} let only specific people manage sensitive operations [1]
- {best practice} use workspace permissions to manage access [1]
- applies to all BI creators for a specific workspace who need access to the pipeline
- {best practice} limit access to the repo or pipeline by only enabling permissions to users [1] who are part of the content creation process [1]
- {best practice} set deployment rules to ensure production stage availability [1]
- {goal} ensure the data in production is always connected and available to users [1]
- {benefit} allows deployments run while while minimizing the downtimes
- applies to data sources and parameters defined in the semantic model [1]
- deployment into production using Git branches
- {recommendation} use release branches [1]
- requires changing the connection of workspace to the new release branches before every deployment [1]
- if the build or release pipeline requires to change the source code, or run scripts in a build environment before deployment, then connecting the workspace to Git won't help [1]
- {recommendation} after deploying to each stage, make sure to change all the configuration specific to that stage [1]
References:
[1] Microsoft Learn (2025) Fabric: Best practices for lifecycle management
in Fabric [link]
[2] Microsoft Learn (2025) Fabric: CI/CD for pipelines in Data Factory in
Microsoft Fabric [link]
[3] Microsoft Learn (2025) Fabric: Choose the best Fabric CI/CD workflow option for you [link]
[3] Microsoft Learn (2025) Fabric: Choose the best Fabric CI/CD workflow option for you [link]
Acronyms:
API - Application Programming Interface
BI - Business Intelligence
CI/CD - Continuous Integration and Continuous Deployment
VS - Visual Studio
BI - Business Intelligence
CI/CD - Continuous Integration and Continuous Deployment
VS - Visual Studio
