NL dbt meetup: From Fabric to Fantastic: How dbt Makes Your Lakehouses and Warehouses Shine

FROM FABRIC TO FANTASTIC Sam Debruyn 🇳🇱 dbt Meetup September 2024 HOW DBT MAKES LAKEHOUSES AND WAREHOUSES SHINE Who am I? Sam Debruyn 📍 Heist-op-den-Berg, BE 💼 Consultant / Data & Cloud Architect 5⃣ years in data 🔟 years in software / architecture / cloud 🫶 dbt, Microsoft, modern data stack What we'll talk about Medallion What is Microsoft Fabric? Medallion on Fabric dbt on Fabric Demo The 3 Layers of the Medallion Architecture The 3 Layers of the Medallion Architecture The 3 Layers of the Medallion Architecture Curated/gold Purpose : high-quality data supporting business reporting, advanced analytics. Pre-aggregated and tailored to analytical needs. Fragmentation in the modern data stack Introducing Microsoft Fabric Data Integration Data Factory Data Engineering Synapse Data Warehouse Synapse Data Science Synapse Real Time Analytics Synapse Business Intelligence Power BI Observability Data Activator Unified data foundation OneLake UNIFIED SaaS experience Security and governance Compute and storage Business model Single… • Onboarding and trials • Sign-on • Navigation model • UX model • Workspace organization • Collaboration experience • Data Lake • Storage format • Data copy for all engines • Security model • CI/CD • Monitoring hub • Data Hub • Governance & compliance Bring all these analytics services together in a single end-to-end analytics platform that can go from the data lake to the business user Lakehouse/Warehouse Where does dbt fit in? Compatibility dbt-fabric: a quick lookback The future for dbt on Fabric Official Microsoft support ✅ tsql-utils: support for dbt-utils, dbt-date, dbt-expectations, dbt-audit-helper Nested CTEs: enable unit testing & more macros dbt-fabricspark: Spark SQL talking to the Livy endpoint in your Lakehouse … Questions? sam@debruyn.dev https://debruyn.dev