Disclaimer: This is work in progress intended to consolidate information from various sources for learning purposes. For the latest information please consult the documentation (see the links below)!
Last updated: 7-Apr-2025
[Microsoft Fabric] User Data Functions (UDFs)- {def} a platform that allows customers to host and run custom logic on Fabric from different types of items and data sources [1]
- empower data developers to write custom logic and embed it into their Fabric ecosystem [1]
- the logic can include internal algorithms and libraries [1]
- {benefit} allows to handle complex transformations, optimizing performance, and integrating diverse data sources beyond the capabilities of low-code tools [5]
- via public libraries from PyPI
- via native Fabric integrations to
- connect to Fabric data sources
- e.g. warehouse, lakehouse or SQL Database
- invoke functions from other Fabric items
- e.g. notebooks, Power BI reports, data pipelines
- via edit functions directly in the Fabric portal [1]
- via in-browser tooling, VS Code extension
- supports the Python 3.11 runtime [1]
- {goal} reusability
- by creating libraries of standardized functionality [1]
- can be used in many solutions across the organization [1]
- {goal} customization
- the applications are tailored to customers’ needs [1]
- {goal} encapsulation
- functions can perform various tasks to build sophisticated workflows [1]
- {goal} external connectivity
- data functions can be invoked from external client applications using a REST endpoint [1]
- allows integrations with external systems [1]
- {feature} programming model
- SDK that provides the necessary functionality to author and publish runnable functions in Fabric [4]
- {benefit} allows to seamlessly integrate with other items in the Fabric ecosystem [4]
- e.g. Fabric data sources
- {concept} user data functions item
- contains one or many functions that can be invoked from the Fabric portal using the provided REST endpoint [4]
- from another Fabric item, or from an external application
- each function is a method in Python script that allows passing parameters and returning an output to the invoker [4]
- {component} fabric.functions library
- provides the code needed to create user data functions in Python [4]
- imported in the template by default [4]
- {method} fn.UserDataFunctions()
- provides the execution context [4]
- added at the beginning of the code file in all new user data functions items, before any function definitions [4]
- {concept} functions
- every function is identified with a @udf.function() decorator
- can be invoked individually from the portal or an external invoker [4]
- functions without the decorator can be invoked directly [4]
- {feature} invocation logs
- function invocations are logged
- {benefit} allows to check the status or perform debugging [3]
- {limitation} can take few minutes to appear [3]
- {recommendation} refresh the page after a few minutes [3]
- {limitation} daily ingestion limit: 250 MB [3]
- reset the next day
- a new log is made available [3]
- {limitation} logs are sampled while preserving a statistically correct analysis of application data [3]
- the sampling is done by User data functions to reduce the volume of logs ingested [3]
- if some logs are partially missing, it might be because of sampling [3]
- {limitation} supported log types: information, error, warning, trace [3]
- {feature} Manage connections
- {limitation} only supports connecting to Fabric-native data sources [2]
- {limitation} only available in a subset of Fabric regions [2]
- {limitation} editable by the owner only [2]
- can only be modified and published by the user who is the owner of the User Data Functions Fabric item [2]
- {limitation} adds further keywords to the list of reserved keywords
- namely: req, context, reqInvocationId
- can't be used as parameter names or function names [2]
References:
[1] Microsoft Learn (2025) Microsoft Fabric: What is Fabric User data
functions (Preview)? [link]
[2] Microsoft Learn (2025) Microsoft Fabric: Service details and limitations
of Fabric User Data Functions [link]
[3] Microsoft Learn (2025) Microsoft Fabric: User data functions invocation
logs (Preview) [link]
[6] Microsoft Learn (2025) Microsoft Fabric: Use the Functions
activity to run Fabric user data functions and Azure Functions [link]
Resources:
[R1] Microsoft Fabric Updates Blog (2025) Utilize User Data
Functions in Data pipelines with the Functions activity (Preview) [link]
Acronyms:
PyPI - Python Package Index
REST - Representational State
Transfer
SDK - Software Development Kit
UDF - User Data
Function
VS - Visual Studio
No comments:
Post a Comment