A flexible data integration tool to help nonprofits connect to their data collection tools and ERP systems
-
Updated
May 11, 2021 - Python
A flexible data integration tool to help nonprofits connect to their data collection tools and ERP systems
Ready to go Apache Airflow stack for docker
Data Engineering using Apache Airflow
Projects related to ETL data pipelines, Kubernetes and Go REST API development
An EL pipeline built with Apache Airflow that downloads a file from the web uploads it to Google Cloud Storage, and creates an external table in BigQuery for data storage and analysis.
A comprehensive data engineering pipeline, orchestrates data workflows with Apache Airflow, Python, Kafka, Zookeeper, Spark, and Cassandra. Containerized using Docker: to deploy and scale effortlessly. This Etsy API Data Pipeline extracts, processes, and analyzes Etsy marketplace data—retrieving product listings, shop details, and reviews.
Invoice de-duplication via Azure Form Recognition, OpenAI, Apache Airflow and Redis Enterprise VSS
Small sample project to test Apache Airflow workflow engine.
Free content with reference to https://bigdata-etl.com Blog.
An on-going application to track and therefore analyse personal activities and goal setting.
A tool that can be deployed to process posting and receiving text and audio files from and into a data lake, apply transformation in a distributed manner, and load it into a warehouse in a suitable format to train a speech-to-text model
Entire ETL pipeline project with different tasks orchestration using Apache airflow. And data storage in PostgreSQL database using Python
This repo contains code to support the tutorial, Automate the Provisioning of Your Apache Airflow Environments.
Apache Airflow: Orchestrating a Data Pipeline
Process of scheduled data extraction, transform and load is done using Apache Airflow and PySpark
An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using Apache Airflow, Python, Apache Kafka, Apache Zookeeper, Apache Spark, and Cassandra. All components are containerized with Docker for easy deployment and scalability.
Add a description, image, and links to the apache-airflow topic page so that developers can more easily learn about it.
To associate your repository with the apache-airflow topic, visit your repo's landing page and select "manage topics."