Showing posts with label Direct Lake. Show all posts
Showing posts with label Direct Lake. Show all posts

13 October 2026

🏭🗒️Microsoft Fabric: Direct Lake [Notes] 🆕

Disclaimer: This is work in progress intended to consolidate information from various sources for learning purposes. For the latest information please consult the documentation (see the links below)! 

Last updated: 13-Oct-2026

[MF] Direct Lake

  • {def} Power BI semantic model table storage mode option available in MF [1]
    • optimized for large volumes of data to be quickly loaded into memory from Delta tables available in the OneLake [1] 
    • leverages VertiPaq query engine [1]
      • {benefit} deliver query performance comparable to Import mode 
        • without the management overhead of data refresh cycles to load the entire data volume [1]
      • similar to Import mode
        • {difference}refreshes copies only metadata (aka framing)
          •  [Import mode] replicates the data and creates an entire cached copy of the data for the semantic model [1]
    •  depends on data preparation being done in the data lake [1]
    • {benefit} enables high-performance interactive analysis [1]
      •  ideal for semantic models connecting to large data sources
        • e.g. lakehouse, warehouse and other data source with Delta tables [1]
      • performance depends on well-tuned Delta tables [1]
    •  {benefit} performance
      • normally outperforms DirectQuery queries when loading and interacting with visuals in reports [1]
    • {benefit}maximizes ROI because 
      • analyzed data volumes can exceed capacity's max memory limits [1]
        • only the needed data to answer a query is loaded into memory [1]  
    • {benefit} minimizes data latencies 
      •  makes new data available without refresh schedules [1]
        • quickly and automatically synchronizing a semantic model with its sources [1]
  • {operation} refresh 
    • low-cost operation that analyzes the metadata of the latest version of the Delta tables and updates references to the latest files in OneLake
    • DL moves data preparation to OneLake
      • <-- uses the full breadth of Fabric technologies for data prep
        • incl. Spark jobs, T-SQL DML statements, dataflows, pipelines, etc.
  • {operation} transcoding
    • the process of loading data from OneLake only when a query requests a column for the first time 
    • {action} the semantic model receives a DAX (or MDX) query 
      • {step} determines what columns are needed to produce the result
      • {step} determines which columns are already resident in memory [2]
      • {step} loads the missing columns from OneLake [2]
        •  typically a fast operation, however it can depend on factors 
          •  e.g. the cardinality of data stored in the columns
        •  remains resident until there's reason for it to be removed (evicted) from memory
          • {reason} the model or table was refreshed after a Delta table update at the source [2]
          • {reason} no query used the column for some time [2]
          • {reason} other memory management reasons [2]
            •  e.g. memory pressure in the capacity due to other, concurrent operations [2]
      •  SKU determines the maximum available memory for each Direct Lake semantic model on the capacity [2]
  • {operation} framing 
    • an operation triggered by a refresh of a semantic model [2]
    • it usually takes only a few seconds to complete
      • <-- because it's a low-cost operation where the semantic model analyzes the metadata of the latest version of the Delta Lake tables and updates to reference the latest Parquet files in OneLake [2]
    • the process might evict resident table column segments and dictionaries from memory if the underlying data changed [2]
      • the point in time of the refresh becomes the new baseline for all future transcoding events [2]
        • DL queries only consider data in the Delta tables as of the time of the most recent framing operation [2]
        • DL tables are queried to return data based on the state of the Delta table at the point of the most recent successful framing operation [2]
          • that time isn't necessarily the latest state of the Delta tables [2]
    • the semantic model analyzes the Delta log of each Delta table during framing to drop only the affected column segments and to reload newly added data during transcoding [2]
      • {optimization} dictionaries usually aren't dropped when incremental framing takes effect, and new values are added to the existing dictionaries [2]
      • helps reduce the reload burden and benefits query performance [2]
        • when a Delta table receives no updates, no reload is necessary for columns already resident in memory [2]
        •  queries show far less performance impact after framing because incremental framing essentially enables the semantic model to update substantial portions of the existing in-memory data in place [2]
    • can fail if a Delta table exceeds the Fabric capacity guardrails [2]
      •  e.g. when a Delta table has more than 10,000 parquet files [2]
  •  {architecture} Direct Lake on OneLake
    •  isn't coupled with the SQL analytics endpoint [2]
    •  offers tighter integration with OneLake features such as OneLake security [2]
    •  offers more efficient DAX query plans because [2]
      • <-- isn’t required to check for SQL based security [2]
      •  fallback isn't supported [2]
  • {architecture} Direct Lake on SQL analytics endpoint
    •  a DAX query might use DirectQuery fallback [2]
      •  involves seamlessly switching to DirectQuery mode [2]
      •  DirectQuery fallback
        • retrieves data directly from the SQL analytics endpoint of the lakehouse or the warehouse [2]
      •  fallback occurs when SQL based security is detected in the SQL analytics endpoints
        •  a DirectQuery operation sends a query to the SQL analytics endpoint. Fallback operations might result in slower query performance [2]
  •  {setting} Keep your Direct Lake data up to date
    • performs automatic updates of Direct Lake tables [2]
      • via semantic model settings >> Refresh 
    • {enabled|default} ensures that data changes in OneLake are automatically reflected in the Direct Lake semantic model [2]
    •  the semantic model performs a framing operation whenever data modifications in the underlying Delta tables are detected [2]
    • {recommendation} leave the setting on, especially when you have a small or medium-sized semantic model
      • useful in case of low-latency reporting requirements and Delta tables are modified regularly [2]
    •  {disabled} it might be needed to disable automatic updates to allow completion of data preparation jobs or an ETL process before exposing any new data to consumers of the semantic model [2]
      •  users can trigger a refresh manually in the Fabric portal [2]
        •  via refresh schedule, or programmatically by using the Power BI REST API or TOM [2]

References: 
[1] Microsoft Learn (2026) Direct Lake overview [link]
[2] Microsoft Learn (2026) 

Acronyms:
API - Application Programming Interface
DL - Direct Lake 
MDX - Multidimensional Expressions
MF - Microsoft Fabric
REST
SKU - Stock Keeping Unit
TOM - Tabular Object Model
T-SQL - Transact SQL

 

Related Posts Plugin for WordPress, Blogger...

About Me

My photo
Koeln, NRW, Germany
IT Professional with more than 25 years experience in IT in the area of full life-cycle of Web/Desktop/Database Applications Development, Software Engineering, Consultancy, Data Management, Data Quality, Data Migrations, Reporting, ERP implementations & support, Team/Project/IT Management, etc.