Serverless data pipelines: ETL workflow with Step Functions and Athena
This blog is Part 3 of a multi-part series around analyzing Flanders’ traffic whilst leveraging the power of cloud components! This miniseries was written in collaboration with Nick Van Hoof of TransIT. Goal This blog aims to explore the use of the AWS Glue service in conjunction with the AWS Athena service to repartition raw streaming data […]
Real-time Data Processing with Kinesis Data Analytics
This blog is part 1 in our miniseries on real-time data analytics using Flanders’ traffic data. This miniseries was written in collaboration with Nick Van Hoof of TransIT. Why real-time data and analytics? Traditionally, batch-processing analytics was considered adequate as most BI users could meet their business goals by looking at weekly or monthly business numbers. Then […]
Serverless data transform with Kinesis
This blog is part 2 in our miniseries about processing real-time data. This miniseries was written in collaboration with Nick Van Hoof of TransIT. Creating a data lake in Amazon Kinesis The goal of this blog is to explore the use of the Amazon Kinesis Data Firehose service to load raw streaming data events on an Amazon […]
Exploring the Power BI possibilities with real-time data
This blog is part 4 in our miniseries on real-time data analytics using Flanders’ traffic data. In part one we showed you how to perform real-time data processing, in part four we will show you how to get this data into power BI and how to make real-time dashboards. Just like in part one we will demonstrate […]