Building a Python data pipeline using AWS and Apache Airflow

The goal of the project is to build the code base for Python modules that will load data from a set of APIs. Schedule the jobs to run on intervals and via events. And deploy it to AWS.

The engineer should be comfortable with git, deploy AWS stack via code (Preferably leveraging AWS CDK) and set up the CDCI for the data pipeline.

Evner: Python, Amazon Web Services, Linux, Ubuntu, Software Arkitektur

Se mere: extracting data webpages using python, extract data html using python, building knowledge internet logs using data mining techniques, building website using aws amazon, python data processing amazon aws, python data amazon aws, extract data bloomberg using python, web data mining using python, data comparison using python, data scraping using python, data analysis using R or python, problem solving with algorithms and data structures using python, data analysis using python pdf, building a data pipeline with testing in mind, python data pipeline example, building an analytics data pipeline in python, python data pipeline library, building a data warehouse using apache spark and hive, building a deployment pipeline using git, maven jenkins and tomcat, python data pipeline framework

Projekt ID: #29001212