u/Due-Advantage297

I’ve been building a medallion architecture pipeline in Databricks.

For this project, I built Marathos Atlas event using :
- Medallion architecture / Bronze , Silver and Gold layers.
- Pyspark and Lakeflow pipelines
- Unity Catalog
- Streaming ingestion , which makes the streaming tables for Bronze and silver
- Data cleaning and transformation
- Dimensional modeling
-Gold views for analytics
- Databricks Dashboard for insights (KPI’s)
- Genie space for stakeholders or users to query the data

What would you do differently in this project?