Crawler Demo for 200LAB DevOps Course

In this course, we will set up the CI/CD and infrastructure for the crawler systems below on Kubernetes.

Overview System

Crawler System: crawler posts + redis stream + consumer

Change Data Capture System: Postgres + Kafka Connect + Elasticsearch

API Service: Rust + Postgres + Elasticsearch

Name		Name	Last commit message	Last commit date
Latest commit History 38 Commits
api		api
configs		configs
consumer		consumer
crawler		crawler
images		images
k6		k6
migrate		migrate
terraform/ec2		terraform/ec2
website		website
.gitignore		.gitignore
README.md		README.md
deploy.sh		deploy.sh
docker-compose-api.yaml		docker-compose-api.yaml
docker-compose-crawler.yaml		docker-compose-crawler.yaml
docker-compose-server.yaml		docker-compose-server.yaml
docker-compose-stream.yaml		docker-compose-stream.yaml
docker-compose-trace.yaml		docker-compose-trace.yaml
docker-compose.yaml		docker-compose.yaml
run.sh		run.sh