u/jujaraynaveen

Hey everyone,

If you've worked with Microsoft Fabric medallion architectures, you know the pain: a column gets renamed or dropped in Bronze or Silver, and a few days later, a business user reports that three Power BI dashboards are showing blanks. Tracking down what broke across the workspace can be a nightmare.

To solve this, I built Microsoft Fabric Schema Drift Detection— an autonomous system that detects schema drift across the entire medallion architecture, maps it using a column-level lineage graph, alerts the team, and drafts auto-fix pull requests.

Here is the GitHub repository if you want to check it out: https://github.com/naveenjujaray/microsoft-fabric-schema-drift-detection

🕵️ How It Works

Instead of just checking if a table schema changed, this tool maps cross-layer lineage all the way to reports:

>Bronze (Lakehouse) ➔ Silver (Lakehouse) ➔ Gold (Warehouse) ➔ Semantic Model ➔ Power BI Reports (PBIP)

1. Cross-Layer Lineage: If \silver.customers.email` is renamed to `email_address`, the tool walks the lineage graph to detect that it will break `gold.Dim_Customer.Email`, which breaks the semantic model `Customer.Email`, which ultimately breaks the visual in your "Customer Detail" Power BI report.`

2. DAX Parsing: It parses DAX measures (like \SUM(Sales[Freight])`) to flag downstream breaks if upstream columns are dropped or modified.`

3. AI Impact Analysis: An optional Claude-powered agent reviews the drift, judges the business severity (e.g., info vs. critical), and drafts the Pull Request description.

4. Auto-Fix PRs: The tool automatically drafts a new Git branch and applies find/replace TMDL fixes to your Power BI Project (PBIP) files in Git, opening a PR to fix the broken references automatically.

5. Multi-Channel Alerts: Sends rich, native notifications to MS Teams (Adaptive Cards), Slack (Block Kit), and Outlook (HTML emails).

🏛️ Running It (Simulate Mode)

One of my main goals was to make this easy to test without needing a live Fabric tenant or incurring capacity costs. The tool includes a **simulate mode** that spins up a local DuckDB replica of AdventureWorksLT to mock the medallion layers:

# Clone the repository
git clone https://github.com/naveenjujaray/microsoft-fabric-schema-drift-detection.git
cd microsoft-fabric-schema-drift-detection

# Install requirements
pip install -r requirements.txt

# Run the 60-second demo simulation
python main.py --mode simulate --once --dry-run

This will snapshot baselines, inject 5 kinds of schema drift, and print out the entire detection report, the PR it would open, and all the Teams/Slack/Outlook payloads.

🏭 Running inside Fabric

It also fully integrates into a real Fabric tenant. You can run the code inside a Fabric Notebook (using the built-in notebook identity for auth) and schedule it as a Data Pipeline that fails on critical drift to serve as a CI gate.

Would love to hear your thoughts, feedback, or any features you'd like to see added!