Blog posts

Last week I attended Big Data London. Both days were filled with interesting sessions, mostly focussing on one of the vendors also exhibiting at the conference. There are 2 things I am taking away from this conference: Delta Lake has won the data format wars, and your next data platform is either Snowflake, either an open Lakehouse.

Welcome to the fifth part of a 5-part series on an end-to-end use case for Microsoft Fabric. This post will focus on the analytics engineering part of the use case. In this series, we will explore how to use Microsoft Fabric to ingest, transform, and analyze data using a real-world use case.

Welcome to the fourth part of a 5-part series on an end-to-end use case for Microsoft Fabric. This post will focus on the analytics engineering part of the use case. In this series, we will explore how to use Microsoft Fabric to ingest, transform, and analyze data using a real-world use case.

Welcome to the third part of a 5-part series on an end-to-end use case for Microsoft Fabric. This post will focus on the data engineering part of the use case. In this series, we will explore how to use Microsoft Fabric to ingest, transform, and analyze data using a real-world use case.

Welcome to the second part of a 5-part series on an end-to-end use case for Microsoft Fabric. This post will focus on the data engineering part of the use case. In this series, we will explore how to use Microsoft Fabric to ingest, transform, and analyze data using a real-world use case.

Welcome to the first part of a 5-part series on an end-to-end use case for Microsoft Fabric. This post will focus on the architecture overview of the use case. In this series, we will explore how to use Microsoft Fabric to ingest, transform, and analyze data using a real-world use case.

Do you often have to draw up diagrams of data flows or data architectures and would you like to use Microsoft Fabric icons in your diagrams? Then this post is for you! I always found it hard to find the right icons I needed, so I’ve put them all on this single page for easy access, both as PNG and SVG.

I’m going to be honest with you. I’m bad at writing Pandas data transformation code. Throughout the years I mostly focussed on the Spark APIs in Scala and PySpark, SQL, dbt, and some others, but I find the Pandas APIs usually just confusing and hard to read. I don’t like the black box magic and lock-in of low-code solutions either. Did you know that Microsoft Fabric has the perfect middle ground for this? It’s called Data Wrangler. Let’s dive in!

Microsoft recently added support to authenticate to OneLake using service principals and managed identities. This allows users to access OneLake from applications without having to use a user account. Let’s see how this works.

If you’re considering using Microsoft Fabric, you’re probably thinking “How much is this going to cost me?” Continue reading to learn how Microsoft might have just created the most compelling data platform offering available today.

If all those posts about Microsoft Fabric have made you excited, you might want to consider it as your next data platform. Since it is very new, not all features are available yet and most are still in preview. You could already adopt it, but if you want to deploy this to a production scenario, you’ll want to wait a bit longer. In the meantime, you can already start preparing for the migration. Let’s dive into the steps to migrate to Microsoft Fabric. Today: starting from Synapse Dedicated SQL Pools.

In this post, I will show you how to connect to your Microsoft Fabric Lakehouses and Warehouses from Python.

If all those posts about Microsoft Fabric have made you curious, you might want to consider it as your next data platform. Since it is very new, not all features are available yet and most are still in preview. You could already adopt it, but if you want to deploy this to a production scenario, you’ll want to wait a bit longer. In the meantime, you can already start preparing for the migration. Let’s dive into the paths to migrate to Microsoft Fabric. Today: starting from Synapse Serverless SQL Pools.

In previous posts , I dug deeper into Microsoft Fabric’s SQL-based features and we even explored OneLake using Azure Storage Explorer . In this post, I’ll take a closer look at Fabric’s auto-discovery feature using Shortcuts. Auto-discovery, what’s that? Fabric’s Lakehouses can automatically discover all the datasets already present in your data lake and expose these as tables in Lakehouses (and Warehouses). Cool, right? At the time of writing, there is a single condition: the tables must be stored in the Delta Lake format. Let’s take a closer look.

Recap: OneLake & Delta Lake One of the coolest things about Microsoft Fabric is that it nicely decouples storage and compute and it is very transparent about the storage: everything ends up in the OneLake. This is a huge advantage over other data platforms since you don’t have to worry about moving data around, it is always available, wherever you need it.

While typing this blog post, I’m flying back from the Data Platform Next Step conference where I gave a talk about using dbt with Microsoft Fabric . DP Next Step was the first conference focussed on Microsoft data services right after the announcement of Microsoft Fabric so a lot of speakers were Microsoft employees and most of the talks had some Fabric content. Fabric Fabric Fabric, what is it all about? In this post I’ll go deeper into what it is, why you should care and focus specifically on the SQL aspect of Fabric.

This post originally appeared on the dataroots blog . The cloud is just someone else’s computer and to manage that we prefer to use Infrastructure as Code (IaC). dataroots believes that IaC can benefit any team working with cloud resources and most often Terraform is our tool of choice there.

This post originally appeared on the dataroots blog . Last week was a busy week for fans of the Microsoft technology stack like myself. Microsoft hosted its yearly developer conference, Microsoft Build, announcing lots of exciting updates to new and existing Azure services. In the meantime, the Belgian community of Microsoft technology users gathered in Kinepolis Antwerp for this year’s edition of Techorama.

If you’ve been developing Terraform providers, you might have upgraded your machine to use the newly released Terraform 0.13. This version introduces a lot of changes related to where providers can be found. Before version 0.13 you would just put the binary of your provider in ~/.terraform.d/plugins and you could start using your provider.

Since we’re going through some heat waves in Europe, I thought it might be interesting to start measuring the humidity, temperature and pressure in my apartment. To do so, I decided to use my Raspberry Pi 3 and the Pi Sense HAT running a Python script constantly sending measurements to an Azure Event Hub.

While contributing to a couple of Terraform providers I often found the need to set some breakpoints in the provider code and inspect what was going at runtime. However, debugging Terraform providers is not that easy apparently… I found a way to do this VS Code, but feel free to drop me a line if you have some other tips & tricks and I’ll update this post.

Did you know Flutter has built-in support for l10n, also known as localization? I’ve been searching for an easy way to localize my app, but most of the blog posts I’ve found rephrased the official documentation on internationalization . I live in Belgium where we have 3 official locales: Flemish (nl_BE), Belgian French (fr_BE) and German (de_BE).

I don’t always have a ton of iOS devices lying around when I submit iOS apps to iTunes App Store Connect. So I take my screenshots on the simulators. At the time of writing Apple requires screenshots made with an iPhone 8 Plus and an iPad Pro. You can optionally add screenshots from an iPhone XS Max.

If your app has to sync some data in the background, you’ll definitely want to use the Android JobScheduler API to schedule a background job. There are a few helpful libraries that make this task easier, but you can get away with using the JobScheduler API itself. I work in Xamarin, so my samples are in C#.

This post has been translated to Russian by Denis Gordin . You can read the Russian version on the Russian website TechMedia . Thanks, Denis! Xamarin is amazing in how it allows .NET developers to write apps for Android, iOS, MacOS… in C#. But that amazing capability comes with a prize and even the most simple apps can suffer from high memory usage.

Usually you don’t need to manually edit .csproj files for your apps and most developers don’t even know what’s going on inside this file. However, sometimes you might run into issues related to this file where Visual/Xamarin Studio can’t help you. The case of the missing references At my current client our solution consists of more than 10 projects:

The Xamarin profiler is a must-have tool for every Xamarin developer. The profiler comes with a Xamarin business or enterprise license and is available as a standalone installer at xamarin.com/profiler . To get started, make sure a debug version of your app is installed on a device or a simulator (works both for Android and iOS).

What you need Experience with Xamarin.iOS Xamarin Studio for Mac An empty binding project (just create a new project in Xamarin Studio) A very short intro to Objective-C for C# developers Oh god, Obj-C, the most incomprehensible programming language in the app dev world. You simply can’t create an iOS binding project without some very basic knowledge of Obj-C.

There isn’t much documentation available for MVVM Light when it comes to Xamarin.Android and Xamarin.iOS. There are several overloads for the SetBinding method and using the wrong overload causes TargetInvocationException or TargetException like this one . It’s also possible that your bindings don’t update anymore after you set one binding using an incorrect syntax.

Sometimes you’re not allowed or you don’t want to publish source code on GitHub, Bitbucket etc. In this post, I’ll explain how you can set up a Raspberry Pi (or any other Linux server) as a git repository server and how you can configure Windows clients for that server. We’ll automate everything from authentication to creating and deleting repositories.

You gotta have MVVM A developer and his tools are inseparable. We all like SOLID and every (.NET) developer has his or her favourite dependency injection tool. There is a lot to choose from. I like Autofac because of the way it handles modules, the lifetime of a type and how it registers types.

Docker is awesome, right? And thanks to boot2docker Windows users were no longer left out of the fun. Still, setting everything up could be a PITA and you had to install Oracle VirtualBox to use it as Docker containers were actually run inside of a VM (which was the purpose of boot2docker).

This short post is more of a note to myself for when I run into the same problem. I just wasted an hour on it so I guess that justifies it. So, what happened? I wanted to check out the new Docker Machine with Hyper-V support, but the Hyper-V Manager told me it could not connect to my local computer.

This blog is built with Hugo , which allows me to use some great CI tools like Wercker . The first part of writing a wercker.yml is picking a build container. A build container on Wercker is the environment in which your build or deploy steps run. Wercker used to build its own containers for that, but they’ve moved away from that approach to Docker containers.

Ordering a new workhorse Windows 10 RTM has been out for about a month now after little less than a year of public Insider preview builds. The motherboard of my old, trusty, Asus laptop was slowly dying and the Dell XPS 13’s thinness and battery life looked very attractive. I decided to go for it and order a Dell XPS 13 (2015), without the touch screen.

I recently blogged a lot about static websites and continuous integration . There was still one step missing in my continuous integration cycle: minification. Wercker is a great CI tool and it allows you to create custom build steps that can be reused in several projects. So I created a build step to minify HTML, CSS and JS files.

So last week, I created a material design theme for Hugo , a static site generator that I’m quite fond of](/tags/hugo/). As I discovered Wercker , an awesome CI tool, I went looking for a way to automatically validate themes. There wasn’t any, so I simply wrote a build step for Wercker .

Why? Who doesn’t love GitHub Pages ? It’s the easiest way to create a simple website about a repository and you can even use Jekyll to start blogging. As I recently switched from Jekyll to Hugo , I needed a new way to enable continuous integration for my blog. New to CI?

Last week, I wrote a post about how awesome Hugo is and why I switched from Jekyll to Hugo. Then, a few days later, Google released an awesome template called Material Design Lite . It’s basically Google’s famous Material Design in HTML, CSS and JavaScript instead of in Polymer .

A good blogging platform What makes a good blogging platform really good? Well, that depends on the blogger. Programmers would need different features than make-up bloggers. As a programmer, I need to be able to easily integrate pieces of code with syntax highlighting in my posts and I’d prefer writing them in my favourite text/code editor instead of in a WYSIWYG editor.

Last week I had to geocode the addresses in a dataset using Talend. Geocoding is adding coordinates to addresses. This way you can easily visualize your data on a map. I experienced problems when I tried to download some plugins from Talend Exchange but I found this blog post about geocoding without the use of a plugin.

A few days ago Microsoft released a new build for Windows 10 and I wanted to give it a try. However, I didn’t want to resize my partitions or overwrite my Windows 8.1 partition. Well, then install it in a VM! Then I wouldn’t have access to all of my RAM and CPU, so that wasn’t a solution either.

When I write Windows Store applications, I use MessageDialog a lot. It’s the easiest way to show a quick informative pop-up message or a question to the user. However, when you tend to use this quite often, you’ll probably run into a problem. The Windows Runtime framework doesn’t allow you to stack MessageDialogs, queue them etc.

While I was working on a website made with Laravel and Ardent I had to validate a password confirmation field for a user registration. This will not work if you follow the standard documentation. This is how your validation rules in your Ardent model should look like (users only have an email address and a password):

A few weeks ago I had to submit the contents of a GitHub repository to my teacher for grading. I wanted to use my readme.md-file while my teacher asked for a Word or a PDF document. Then I discovered this amazing program which converts Markdown to any desirable format! Almost every programmer knows Markdown, right?

In this quick tutorial I’ll show you how to secure SSH access to your Linux server with 2-step authentication. Why did I call this post ’extended’? Because I’ll show you how to add extra rules so you don’t have to use 2-step authentication from certain locations. I’m not going to explain what 2-step authentication is .

Introduction I have a Spotify Premium subscription but I also have a few CDs which are not available on streaming services. I don’t like the whole process of syncing files, that’s why I took a Spotify subscription in the first place. Google offers a solution with Play Music : you can upload up to 20,000 songs to their servers for free and stream them to all of your devices.

Introduction Recently Belgian ISP Telenet introduced Yelo TV . It’s the new brand for all of their efforts to innovate with television. Yelo TV includes a series of apps for almost every device (except for Windows Phone) and a website on which you can watch live television, watch recordings or schedule recordings.