Fabric Capacities: Divide and Conquer

FABRIC CAPACITIES Sam Debruyn Power BI & Fabric Summit February 2025 DIVIDE & CONQUER 2025 2025 This Summit presented to you by Special thanks to Fabric and Power BI Team at Who am I? Sam Debruyn 📍 Heist-op-den-Berg, BE 💼 Consultant / Data & Cloud Architect 5⃣ years in data 🔟 years in software / architecture / cloud 🫶 Fabric, Azure, modern data stack What we'll talk about Intro Why multiple Workspaces Capacities Throttling Avoiding throttling Conclusion Setting the stage… Fabric medallion architecture example The 3 Layers of the Medallion Architecture The 3 Layers of the Medallion Architecture The 3 Layers of the Medallion Architecture Curated/gold Purpose : high-quality data supporting business reporting, advanced analytics. Pre-aggregated and tailored to analytical needs. Overview Overview: entire platform (example) Did I invent this? No, this is also how Microsoft recommends it Easy to extend Workspaces & Capacities Fabric concepts: Workspaces & Capacities Capacity • pool of Capacity Units • matches a certain amount of compute power • to be spread amongst one or more Workspaces Workspace • logical grouping of items • Lakehouses, Warehouses, Reports, KQL, … • possible access control boundary Why should you create separate Workspaces? Workspace Configuration Workspace Configuration Workspace Configuration Workspace Configuration Workspace Configuration Why should you create separate Workspaces? Capacities Capacity SKUs Bursting & smoothing Example background Example interactive Bursting & smoothing SKU CU’s Available CUs for interactive 10min workloads Available CUs for background 24h workloads Actual workload duration F2 2 1.200 172.800 ASAP* F4 4 2.400 345.600 ASAP* F8 8 4.800 691.200 ASAP* F16 16 9.600 1.382.400 ASAP* F32 32 19.200 2.764.800 ASAP* F64 64 38.400 5.529.600 ASAP* F128 128 76.800 11.059.200 ASAP* … … … … … CU’s = Capacity Units CUs = Capacity Unit seconds Bursting & smoothing Impact of SKU choice Capacities determine feature availability E.g. CoPilot, Power BI only F64 or higher Capacities determine how features are available Nodes and cores/node in Spark (2 vCores per CU – burst factor 3 | 0.25 nodes per CU) supported regions Capacity level settings Bursting & smoothing Throttling How to think of a Capacity What do when throttled What do when throttled What do when throttled F Capacity only! What do when throttled F Capacity only! What do when throttled What do when throttled P Capacity only! 2025 = last year of P Capacities What do when throttled Optimizing CUs usage and cost Optimizing CUs usage and cost Reservations <> Capacities! Reservation : pool of available CU’s to be used in one or more Capacities E.g. Reservation of 64 CU’s: • 1x F64 • 2x F32 • 2x F16 + 1x F32 • 4x F4 + 2x F8 + 2x F16 Optimizing CUs usage and cost Reservation <> max amount of CU’s 💡 HINT : Reserve what you’re using on a daily base Combine with PAYG for infrequent usage Create Logic App to auto-pause PAYG Capacity Resume PAYG when needed E.g. can even be through Fabric API from a Notebook in the Capacity using reserved CU’s Optimizing CUs usage and cost Require Power BI Premium? PPU is still available! Price increase April 1 st 2025: $24 per month Move Power BI reports to separate Workspace(s) + activate PPU on these Workspace(s) Any F Capacity + Power BI Pro license = Power BI Premium More ideas Use data from Capacity Metrics App or time-based triggers to: • Automatically scale up/down Capacity • Automatically move Workspaces to diXerent Capacities • Automatically pause/resume Capacities • Automatically create new Capacities Capacity Metrics App Tips: Calculations are done in 30-second time windows This is a regular Power BI Report à You can use the filters to drill down Capacity Metrics App Capacity Metrics App Why should you create separate Workspaces? How access can be managed in Fabric Workspace level roles: Admin, Member, Contributor, Viewer Item sharing: Read, Edit, Share Data sharing: Read, ReadData, ReadAll OneLake RBAC (preview) Note: this will probably be improved with the introduction of OneSecurity Sam’s 5 golden rules for Workspace & Capacity design in Fabric Sam’s 5 golden rules for Workspace & Capacity design in Fabric Sam’s 5 golden rules for Workspace & Capacity design in Fabric Sam’s 5 golden rules for Workspace & Capacity design in Fabric Sam’s 5 golden rules for Workspace & Capacity design in Fabric Questions? Stay online for my live Q&A session sam@debruyn.dev https://debruyn.dev