🏭Fabric

Fabric 

Delta Lake

  • real-time analytics
  • Direct Lake
    • semantic model
  • semantic link
  • delta tables

Lakehouse

  • sharing
  • high concurrency mode
  • CI/CD integration (Git + Deployment pipelines)
Notebooks
 

Data Warehouse (aka Fabric Warehouse)

  • security
  • restore
    • restore-in-place: Jagadish
    • table cloning to a previous point-in-time: 
    • log checkpointing: Conan
  • feature
    • table cloning
  • T-SQL language support  
    • SP_RENAME
    • TRIM
    • GENERATE_SERIES
Spark engine
  • starter pools
  • custom pools

Readiness

  • slides: Microsoft
  • Dynamics Data in Synapse Link for Dataverse
New Features 
  • Mirroring Azure SQL Database 
  • Azure Private Link Support for Microsoft Fabric 
  • Trusted workspace access: Microsoft
  • OneLake Shortcuts APIs 
  • VNET Data Gateway for Fabric and Power BI 
  • Copilot in Fabric
    • PP, Jan-2024: Xu
Useful
Books 
  • Learn KQL in One Month - [files]
  • Must Learn KQL - [code, ebook]
  • The Definitive Guide to KQL [code]
Sessions
  • Integrate your SAP data into Microsoft Fabric [link]



Resources
Events
  • Kusto Con: 2024
  • Virtual training days (EN, DE)
Certifications

DP 600: Implementing Analytics Solutions Using Microsoft Fabric [study guide]
  • Maintain a data analytics solution (25–30%)
    • Implement security and governance
      • Implement workspace-level access controls
      • Implement item-level access controls
      • Implement row-level, column-level, object-level, and file-level access control
      • Apply sensitivity labels to items
      • Endorse items
    • Maintain the analytics development lifecycle
      • Configure version control for a workspace
      • Create and manage a Power BI Desktop project (.pbip)
      • Create and configure deployment pipelines
      • Perform impact analysis of downstream dependencies from lakehouses, data warehouses, dataflows, and semantic models
      • Deploy and manage semantic models by using the XMLA endpoint
      • Create and update reusable assets, including Power BI template (.pbit) files, Power BI data source (.pbids) files, and shared semantic models
  • Prepare data (45–50%)
    • Get data
      • Create a data connection
      • Discover data by using OneLake data hub and real-time hub
      • Ingest or access data as needed
      • Choose between a lakehouse, warehouse, or eventhouse
      • Implement OneLake integration for eventhouse and semantic models
    • Transform data
      • Create views, functions, and stored procedures
      • Enrich data by adding new columns or tables
      • Implement a star schema for a lakehouse or warehouse
      • Denormalize data
      • Aggregate data
      • Merge or join data
      • Identify and resolve duplicate data, missing data, or null values
      • Convert column data types
      • Filter data
    • Query and analyze data
      • Select, filter, and aggregate data by using the Visual Query Editor
      • Select, filter, and aggregate data by using SQL
      • Select, filter, and aggregate data by using KQL
  • Implement and manage semantic models (25–30%)
    • Design and build semantic models
      • Choose a storage mode
      • Implement a star schema for a semantic model
      • Implement relationships, such as bridge tables and many-to-many relationships
      • Write calculations that use DAX variables and functions, such as iterators, table filtering, windowing, and information functions
      • Implement calculation groups, dynamic format strings, and field parameters
      • Identify use cases for and configure large semantic model storage format
      • Design and build composite models
    • Optimize enterprise-scale semantic models
      • Implement performance improvements in queries and report visuals
      • Improve DAX performance
      • Configure Direct Lake, including default fallback and refresh behavior
      • Implement incremental refresh for semantic models
DP 700:  Implementing Data Engineering Solutions Using Microsoft Fabric [study guide]
  • Implement and manage an analytics solution (30–35%)
    • Configure Microsoft Fabric workspace settings
      • Configure Spark workspace settings
      • Configure domain workspace settings
      • Configure OneLake workspace settings
      • Configure data workflow workspace settings
    • Implement lifecycle management in Fabric
      • Configure version control
      • Implement database projects
      • Create and configure deployment pipelines
    • Configure security and governance
      • Implement workspace-level access controls
      • Implement item-level access controls
      • Implement row-level, column-level, object-level, and folder/file-level access controls
      • Implement dynamic data masking
      • Apply sensitivity labels to items
      • Endorse items
      • Implement and use workspace logging
    • Orchestrate processes
      • Choose between a pipeline and a notebook
      • Design and implement schedules and event-based triggers
      • Implement orchestration patterns with notebooks and pipelines, including parameters and dynamic expressions
  • Ingest and transform data (30–35%)
    • Design and implement loading patterns
      • Design and implement full and incremental data loads
      • Prepare data for loading into a dimensional model
      • Design and implement a loading pattern for streaming data
    • Ingest and transform batch data
      • Choose an appropriate data store
      • Choose between dataflows, notebooks, KQL, and T-SQL for data transformation
      • Create and manage shortcuts to data
      • Implement mirroring
      • Ingest data by using pipelines
      • Transform data by using PySpark, SQL, and KQL
      • Denormalize data
      • Group and aggregate data
      • Handle duplicate, missing, and late-arriving data
    • Ingest and transform streaming data
      • Choose an appropriate streaming engine
      • Choose between native storage, followed storage, or shortcuts in Real-Time Intelligence
      • Process data by using eventstreams
      • Process data by using Spark structured streaming
      • Process data by using KQL
      • Create windowing functions
  • Monitor and optimize an analytics solution (30–35%)
    • Monitor Fabric items
      • Monitor data ingestion
      • Monitor data transformation
      • Monitor semantic model refresh
      • Configure alerts
    • Identify and resolve errors
      • Identify and resolve pipeline errors
      • Identify and resolve dataflow errors
      • Identify and resolve notebook errors
      • Identify and resolve eventhouse errors
      • Identify and resolve eventstream errors
      • Identify and resolve T-SQL errors
    • Optimize performance
      • Optimize a lakehouse table
      • Optimize a pipeline
      • Optimize a data warehouse
      • Optimize eventstreams and eventhouses
      • Optimize Spark performance
      • Optimize query performance

Acronyms:
BI - Business Intelligence 
CI/CD - Continuous Integration/Continuous Deployment
DAX - 
KQL - Kusto Query Language
GA - General Available
PP - Public Preview
PP0 - Private Preview
SQL - Structured Query Language
T-SQL - Transact-SQL

No comments:

Post a Comment

Subscribe to: Posts (Atom)
Related Posts Plugin for WordPress, Blogger...

About Me

My photo
Koeln, NRW, Germany
IT Professional with more than 25 years experience in IT in the area of full life-cycle of Web/Desktop/Database Applications Development, Software Engineering, Consultancy, Data Management, Data Quality, Data Migrations, Reporting, ERP implementations & support, Team/Project/IT Management, etc.