Disclaimer: This is work in progress intended to consolidate information
from various sources for learning purposes. For the latest information
please consult the documentation (see the links below)!
Last updated: 9-Mar-2025
|
|
Real-Time Intelligence architecture [4]
|
[Microsoft Fabric] Real-Time Intelligence [RTI]
- [def]
- {goal} provide a complete real-time SaaS platform within MF
-
{benefit} helps gain actionable insights from data, with the ability to
ingest, transform, query, visualize, and act on it in real time [4]
- {goal} provides a single place for data-in-motion
- {benefit} allows to pull event streams from Real Time Hub
-
provides a single data estate for data in motion simplifying the
ingestion, curation and processing of streaming data from Microsoft
and external sources [4]
-
empowers users to extract insights and visualize data in motion [1]
- {goal} enable rapid solution development
-
{benefit} provides a range of no-code, low-code and pro-code experiences
for various scenarios [4]
-
everything from business insight discovery to complex stream
processing, and application and model development [4]
- {goal} enable real-time AI insights
-
{benefit} scales beyond human monitoring and drive actions with built
in, automated capabilities [4]
- allows anyone in the organization to take advantage of [4]
- offers an end-to-end solution for
- event-driven scenarios
- ⇐ rather than schedule-driven solutions.
- streaming data
- data logs
-
{benefit} help customers accelerate speed and precision of business by
providing [4]
- {goal} operational efficiency
-
by allowing to streamline processes and make data driven decisions
with accurate, up to date information [4]
- {goal} end-to-end visibility
-
by allowing to gain a holistic understanding of business health and
discover actionable insights for timely action [4]
- {goal} competitive advantage
-
by allowing to quickly react to shifting market trends, identify
opportunities and mitigate risk in real time [4]
-
seamlessly connects time-based data from various sources using no-code
connectors [1]
- enables immediate
- visual insights
- geospatial analysis
- trigger-based reactions
- ⇐ all are part of an organization-wide data catalog [1]
-
⇐ time oriented data is difficult to manage, yet critical for success
[4]
-
{challenge} capture high throughput data from disparate sources in
real time [4]
- {challenge} model scenarios using event data [4]
-
{challenge} choose from an array of bespoke technologies and data
formats [4]
-
{challenge} leverage the power of AI against data in real time [4]
-
without the ability to leverage time oriented data, businesses are
vulnerable to risks [4]
- {risk} poor decision-making
- {risk} financial loss
- {risk} reduced operational efficiency
- {risk} impaired data integrity
- {risk} non-compliance
- {risk} negative user experience
- {capability} single unified SaaS solution
- in opposition to a fragmented, fragile tech stack
-
allows to ingest & process all event sources, in any data format
[4]
-
one can connect to diverse streaming sources and leverage no
code and low code experiences to process and route quickly [4]
-
via out of the box connectors for streaming and event data sources
[4]
-
events can be routed to other Fabric and 3rd party entities [4]
-
organizational BI reports can be enhanced with enriched data [4]
-
allows to analyze and transform data event streams using queries and
visual exploration to discover insights in real time [4]
- one can manage an unlimited amount of data [4]
- multiple databases can be monitored and managed at once [4]
- allows to act quickly on top of data
-
via triggers and alerts on changing data to respond automatically
and set action when specific conditions are detected [4]
-
helps drive actions on a per instance state that evolves over time
[4]
-
helps to act on data without needing a deep schema and semantic
modeling [4]
- {capability} accessible data and analytics tools
- in opposition to advanced skillsets required
- {capability} real-time stream processing
- in opposition to batch data processing
- handles
- data ingestion
- data transformation
- data storage
- data analytics
- data visualization
- data tracking
- AI
- real-time actions
- can be used for
- data analysis
- immediate visual insights
- centralization of data in motion for an organization
- actions on data
-
efficient querying, transformation, and storage of large volumes of
structured or unstructured data [1]
- helps evaluate data from
- IoT systems
- system logs
- free text
-
semi structured data, or contribute data for consumption by others in your
organization,
- provides a versatile solution
-
transforms the data into a dynamic, actionable resource that drives value
across the entire organization
-
its components are built on trusted, core Microsoft rather than
schedule-driven solutions
-
⇐ together they extend the overall Fabric capabilities to provide
event-driven solutions [1]
-
{feature} Real-Time hub
-
serves as a centralized catalog that facilitates the easy access,
addition, exploration, and data sharing [1]
- expands the range of data sources
-
⇐ it enables broader insights and visual clarity across various domains
[1]
- ensures that data is accessible to all [1]
- promoting quick decision-making and informed action
-
the sharing of streaming data from diverse sources unlocks the potential
to build BI solutions across the organization [1]
- use the data consumption tools to explore the data [1]
-
{feature} Real-Time dashboards
- come equipped with out-of-the-box interactions
-
{benefit} simplify the process of understanding data, making it
accessible to anyone who wants to make decision based on data in motion
using visual tools, Natural Language and Copilot [1]
- query the data in real-time as it’s being loaded [6]
- every time a query is run, it leverages the latest data available in an Eventhouse or OneLake [6]
- behave much like DirectQuery, but without the need to load data into a semantic model. [6]
- {feature} Activator
-
{benefit} allows to turn insights into actions by setting up alerts from
various parts of Fabric to react to data patterns or conditions in
real-time [1]
- takes events as they are being processed into Eventstreams or Eventhouses and connects them to downstream systems to make data actionable [6]
-
{feature}
Real-Time hub events
- a catalog of data in motionless
- contains:
- data streams
-
all data streams that are actively running in Fabric to which the user
has access to
-
once a stream of data is connected, the entire SaaS solution
becomes accessible [1]
- Microsoft sources:
-
easily discover streaming sources that the users have and quickly
configure ingestion of those sources into Fabric
-
e.g. Azure Event Hubs, Azure IoT Hub, Azure SQL DB CDC, Azure Cosmos
DB CDC, PostgreSQL DB CDC
- Fabric events
-
event-driven capabilities support real-time notifications and data
processing
- ⇒ one can monitor and react to events [1]
-
e.g. Fabric Workspace Item events, Azure Blob Storage events
-
⇐ the events can be used to trigger other actions or workflows [1]
-
e.g. invoking a data pipeline or sending a notification via
email.
-
the events can be sent to other destinations via eventstreams [1]
-
{feature}
Eventstreams
- event processing capabilities
- ⇐ behave like event listeners that wait for data to be pushed to them [6]
-
{benefit} allow to capture, transform, and route high volumes of real-time
events to various destinations with a no-code experience [1]
- support multiple data sources and data destinations [1]
-
{benefit} allow to do filtering, data cleansing, transformation, windowed
aggregations, and dupe detection, to land the data in the needed
shape [1]
-
one can use the content-based routing capabilities to send data to
different destinations based on filters [1]
-
derived eventstreams allows constructing new streams as a result of
transformations and/or aggregations that can be shared to consumers in
Real-Time hub [1]
-
{feature} Eventhouses
- the ideal analytics engine to process data in motion
- scalable ingestion engine with the ability to handle up to millions of events per hour [6]
-
tailored to time-based, streaming events with structured, semi structured,
and unstructured data [1]
-
data is automatically indexed and partitioned based on ingestion time
-
⇐ provides fast and complex analytic querying capabilities on
high-granularity data [1]
-
the stored data can be made available in OneLake for consumption by other
Fabric experiences [1]
-
⇐ the data is ready for lightning-fast query using various code,
low-code, or no-code options in Fabric [1]
-
the data can be queried in native KQL or in T-SQL in the KQL query set [1]
References:
[1] Microsoft Fabric (2024) What is Real-Time Intelligence? [
link]
[2] Microsoft Fabric (2024) Real-Time Intelligence documentation in
Microsoft Fabric [
link]
[3] Microsoft Fabric Updates Blog (2024) Fabric workloads are now
generally available! [
link]
[4] Microsoft Learn (2025) Real Time Intelligence L200 Pitch
Deck [
link]
[5] Microsoft Fabric Community (2024) Benefits of Migrating to Fabric RTI [
link]
[6] Microsoft Fabric Update Blog (2025) Operational Reporting with Microsoft Fabric Real-Time Intelligence [
link]
[7] Microsoft Learn (2025) Get started with Real-Time Intelligence in Microsoft Fabric [
link]
[8] Microsoft Learn (2025) Implement Real-Time Intelligence with Microsoft Fabric [
link]
Resources:
[R1] Microsoft Learn (2024) Microsoft Fabric exercises [
link]
[R2] Microsoft Learn (2024) Microsoft Fabric RTI Demo Application [
link] [
GitHub]
[R3] Microsoft Fabric Updates Blog (2024) Understanding Real-Time Intelligence usage reporting and billing [
link]
[R4] Microsoft Learn (2025) Fabric: What's new in Microsoft Fabric? [
link]
Acronyms:
AI - Artificial Intelligence
CDC - Change Data Capture
DB - database
IoT -
Internet of Things
KQL - Kusto Query Language
MF
- Microsoft Fabric
RTI - Real-Time Intelligence
SaaS -
Software-as-a-Service
SQL - Structured Query Language
