Apache Spark

From Lock-In to Openness: The Shift in Data Platforms

Take a look at the marketing of any modern data platform and you’ll notice a common theme — they all emphasize their …

From Flat to Sparkling: Monitoring Data Quality with Soda in Microsoft Fabric

After loading all of your data into OneLake, you might be wondering how to ensure its quality. In this session, we’ll …

From Flat to Sparkling: Monitoring Data Quality with Soda in Microsoft Fabric

After loading all of your data into OneLake, you might be wondering how to ensure its quality. In this session, we’ll …

My take-aways from Big Data London: Delta Lake & the open lakehouses

Last week I attended Big Data London. Both days were filled with interesting sessions, mostly focussing on one of the …

Fabric end-to-end use case: Data Engineering part 1 - Spark and Pandas in Notebooks

Welcome to the second part of a 5-part series on an end-to-end use case for Microsoft Fabric. This post will focus on …

Microsoft Fabric's Auto Discovery: a closer look

In previous posts , I dug deeper into Microsoft Fabric’s SQL-based features and we even explored OneLake using …

Exploring OneLake with Microsoft Azure Storage Explorer

Recap: OneLake & Delta Lake One of the coolest things about Microsoft Fabric is that it nicely decouples storage and …

Mediahuis: Why Dask and not Spark?

Here I hosted this demonstration/interview during a lunch webinar at dataroots . In this rootlabs session we will be …