An Event Driven ETL Process: Use Step Functions, AWS Glue and Lambda to Orchestrate an ETL Process

This post is my solution walkthrough for the Acloud Guru challenge to build an event driven ETL process. The challenge details can be found Here. I really like going through and solving these challenges since these give me a scope to learn so many things which are involved in completing the challenge. In this post I will describe my approach to the challenge and walk through step by step about deploying the solution. Hopefully this will help others get some pointers for the challenge and learn few things.

More details can be found here: https://amlanscloud.com/etlpython/

--

--

--

Cloud architect and DevOps engineer. Love to code and develop new stuff. A nerd by nature.

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Understanding Developers: an Analogy for Non-Developers

“First-time” homebuyers beware…

Pramp supports System Design and Frontend Interviews

New Tool Ensures HPC Code will Compute as Intended

The path to be the Google Ninja

How to Provision AWS Infrastructure with Ansible?

Deploying Spectrum Scale with Ansible - Part1 - Singel Node

Get Docker run parameters from a running container

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Amlan Chakladar

Amlan Chakladar

Cloud architect and DevOps engineer. Love to code and develop new stuff. A nerd by nature.

More from Medium

Cost Optimization with Amazon EC2 Spot Instances

Create a Microsoft SQL Server Instance on AWS’s RDS (Managed Relational Database Service)

Automating Amazon Web Services(AWS) with Lambda, Python, and Boto3

AWS Cloud Formation — Basics