Local Setup (Docker Compose)

1. Running Locally with Docker Compose

This method runs all services (Producer, Consumer, RabbitMQ, MongoDB, PostgreSQL) in containers for quick testing.

1. Build Docker images:

docker compose build

1. Start the services:

docker compose up -d

1. Verify services are running:

docker compose ps

1. Stop services when done:

docker compose down

---

2. Run Tests & Pre-commit Hooks

Ensure code quality and reliability:

# Run unit tests
pytest tests/

# Run pre-commit checks
pre-commit run --all-files

---

3. Next Steps

• Explore architecture/overview.md for system design.
• Dive into components/ for detailed ETL workflow.
• Check configuration/ for environment variables and setup.