back
loading skill details...
Container orchestration with Docker Compose for multi-container applications, networking, volumes, and production deployment
Docker Compose Orchestration
A comprehensive skill for orchestrating multi-container applications using Docker Compose. This skill enables rapid development, deployment, and management of containerized applications with service definitions, networking strategies, volume management, health checks, and production-ready configurations.
When to Use This Skill
Use this skill when:
Building multi-container applications (microservices, full-stack apps)
Setting up development environments with databases, caching, and services
Orchestrating frontend, backend, and database services together
Managing service dependencies and startup order
Configuring networks and inter-service communication
Implementing persistent storage with volumes
Deploying applications to development, staging, or production
Creating reproducible development environments
Managing application lifecycle (start, stop, rebuild, scale)
Monitoring application health and implementing health checks
Migrating from single containers to multi-service architectures
Testing distributed systems locally
Core Concepts
Docker Compose Philosophy
Docker Compose simplifies multi-container application management through:
Declarative Configuration: Define entire application stacks in YAML
Service Abstraction: Each component is a service with its own configuration
Automatic Networking: Services can communicate by name automatically
Volume Management: Persistent data and shared storage across containers
Environment Isolation: Each project gets its own network namespace
Reproducibility: Same configuration works across all environments
Key Docker Compose Entities
Services: Individual containers and their configurations
Networks: Communication channels between services
Volumes: Persistent storage and data sharing
Configs: Non-sensitive configuration files
Secrets: Sensitive data (passwords, API keys)
Projects: Collection of services under a single namespace
Compose File Structure
version: "3.8" # Compose file format version
services: # Define containers
service-name:
# Service configuration
networks: # Define custom networks
network-name:
# Network configuration
volumes: # Define named volumes
volume-name:
# Volume configuration
configs: # Application configs (optional)
config-name:
# Config source
secrets: # Sensitive data (optional)
secret-name:
# Secret source
Service Definition Patterns
Basic Service Definition
services:
web:
image: nginx:alpine # Use existing image
container_name: my-web # Custom container name
restart: unless-stopped # Restart policy
ports:
- "80:80" # Host:Container port mapping
environment:
- ENV_VAR=value # Environment variables
volumes:
- ./html:/usr/share/nginx/html # Volume mount
networks:
- frontend # Connect to network
Build-Based Service
services:
app:
build:
context: ./app # Build context directory
dockerfile: Dockerfile # Custom Dockerfile
args: # Build arguments
NODE_ENV: development
target: development # Multi-stage build target
image: myapp:latest # Tag resulting image
ports:
- "3000:3000"
Service with Dependencies
services:
web:
image: nginx
depends_on:
db:
condition: service_healthy # Wait for health check
redis:
condition: service_started # Wait for start only
db:
image: postgres:15
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres"]
interval: 10s
timeout: 5s
retries: 5
start_period: 30s
redis:
image: redis:alpine
Service with Advanced Configuration
services:
backend:
build: ./backend
command: npm run dev # Override default command
working_dir: /app # Set working directory
user: "1000:1000" # Run as specific user
hostname: api-server # Custom hostname
domainname: example.com # Domain name
env_file:
- .env # Load env from file
- .env.local
environment:
DATABASE_URL: "postgresql://db:5432/myapp"
REDIS_URL: "redis://cache:6379"
volumes:
- ./backend:/app # Source code mount
- /app/node_modules # Preserve node_modules
- app-data:/data # Named volume
ports:
- "3000:3000" # Application port
- "9229:9229" # Debug port
expose:
- "8080" # Expose to other services only
networks:
- backend
- frontend
labels:
- "com.example.description=Backend API"
- "com.example.version=1.0"
logging:
driver: json-file
options:
max-size: "10m"
max-file: "3"
deploy:
resources:
limits:
cpus: '2'
memory: 1G
reservations:
cpus: '0.5'
memory: 512M
Multi-Container Application Patterns
Pattern 1: Full-Stack Web Application
Scenario: React frontend + Node.js backend + PostgreSQL database
version: "3.8"
services:
# Frontend React Application
frontend:
build:
context: ./frontend
dockerfile: Dockerfile
target: development
ports:
- "3000:3000"
volumes:
- ./frontend/src:/app/src
- /app/node_modules
environment:
- REACT_APP_API_URL=http://localhost:4000/api
- CHOKIDAR_USEPOLLING=true # For hot reload
networks:
- frontend
depends_on:
- backend
# Backend Node.js API
backend:
build:
context: ./backend
dockerfile: Dockerfile
ports:
- "4000:4000"
- "9229:9229" # Debugger
volumes:
- ./backend:/app
- /app/node_modules
environment:
- NODE_ENV=development
- DATABASE_URL=postgresql://postgres:password@db:5432/myapp
- REDIS_URL=redis://cache:6379
- JWT_SECRET=dev-secret
env_file:
- ./backend/.env.local
networks:
- frontend
- backend
depends_on:
db:
condition: service_healthy
cache:
condition: service_started
command: npm run dev
# PostgreSQL Database
db:
image: postgres:15-alpine
container_name: postgres-db
restart: unless-stopped
ports:
- "5432:5432"
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=password
- POSTGRES_DB=myapp
volumes:
- postgres-data:/var/lib/postgresql/data
- ./database/init.sql:/docker-entrypoint-initdb.d/init.sql
networks:
- backend
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres"]
interval: 10s
timeout: 5s
retries: 5
# Redis Cache
cache:
image: redis:7-alpine
container_name: redis-cache
restart: unless-stopped
ports:
- "6379:6379"
volumes:
- redis-data:/data
networks:
- backend
command: redis-server --appendonly yes
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 10s
timeout: 3s
retries: 5
networks:
frontend:
driver: bridge
backend:
driver: bridge
volumes:
postgres-data:
driver: local
redis-data:
driver: local
Pattern 2: Microservices Architecture
Scenario: Multiple services with reverse proxy and service discovery
version: "3.8"
services:
# NGINX Reverse Proxy
proxy:
image: nginx:alpine
container_name: reverse-proxy
ports:
- "80:80"
- "443:443"
volumes:
- ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
- ./nginx/conf.d:/etc/nginx/conf.d:ro
- ./ssl:/etc/nginx/ssl:ro
networks:
- public
depends_on:
- auth-service
- user-service
- order-service
restart: unless-stopped
# Authentication Service
auth-service:
build: ./services/auth
container_name: auth-service
expose:
- "8001"
environment:
- SERVICE_NAME=auth
- DATABASE_URL=postgresql://db:5432/auth_db
- JWT_SECRET=${JWT_SECRET}
networks:
- public
- internal
depends_on:
db:
condition: service_healthy
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8001/health"]
interval: 30s
timeout: 10s
retries: 3
# User Service
user-service:
build: ./services/user
container_name: user-service
expose:
- "8002"
environment:
- SERVICE_NAME=user
- DATABASE_URL=postgresql://db:5432/user_db
- AUTH_SERVICE_URL=http://auth-service:8001
networks:
- public
- internal
depends_on:
- auth-service
- db
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8002/health"]
interval: 30s
timeout: 10s
retries: 3
# Order Service
order-service:
build: ./services/order
container_name: order-service
expose:
- "8003"
environment:
- SERVICE_NAME=order
- DATABASE_URL=postgresql://db:5432/order_db
- USER_SERVICE_URL=http://user-service:8002
- RABBITMQ_URL=amqp://rabbitmq:5672
networks:
- public
- internal
depends_on:
- user-service
- db
- rabbitmq
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8003/health"]
interval: 30s
timeout: 10s
retries: 3
# Shared PostgreSQL Database
db:
image: postgres:15-alpine
container_name: postgres-db
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=${DB_PASSWORD}
volumes:
- postgres-data:/var/lib/postgresql/data
- ./database/init-multi-db.sql:/docker-entrypoint-initdb.d/init.sql
networks:
- internal
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres"]
interval: 10s
timeout: 5s
retries: 5
# RabbitMQ Message Broker
rabbitmq:
image: rabbitmq:3-management-alpine
container_name: rabbitmq
ports:
- "5672:5672" # AMQP
- "15672:15672" # Management UI
environment:
- RABBITMQ_DEFAULT_USER=admin
- RABBITMQ_DEFAULT_PASS=${RABBITMQ_PASSWORD}
volumes:
- rabbitmq-data:/var/lib/rabbitmq
networks:
- internal
healthcheck:
test: ["CMD", "rabbitmq-diagnostics", "ping"]
interval: 30s
timeout: 10s
retries: 5
networks:
public:
driver: bridge
internal:
driver: bridge
internal: true # No external access
volumes:
postgres-data:
rabbitmq-data:
Pattern 3: Development Environment with Hot Reload
Scenario: Development setup with live code reloading and debugging
version: "3.8"
services:
# Development Frontend
frontend-dev:
build:
context: ./frontend
dockerfile: Dockerfile.dev
ports:
- "3000:3000"
- "9222:9222" # Chrome DevTools
volumes:
- ./frontend:/app
- /app/node_modules
- /app/.next # Next.js build cache
environment:
- NODE_ENV=development
- WATCHPACK_POLLING=true
- NEXT_PUBLIC_API_URL=http://localhost:4000
networks:
- dev-network
stdin_open: true
tty: true
command: npm run dev
# Development Backend
backend-dev:
build:
context: ./backend
dockerfile: Dockerfile.dev
ports:
- "4000:4000"
- "9229:9229" # Node.js debugger
volumes:
- ./backend:/app
- /app/node_modules
environment:
- NODE_ENV=development
- DEBUG=app:*
- DATABASE_URL=postgresql://postgres:dev@db:5432/dev_db
networks:
- dev-network
depends_on:
- db
- mailhog
command: npm run dev:debug
# PostgreSQL with pgAdmin
db:
image: postgres:15-alpine
environment:
- POSTGRES_PASSWORD=dev
- POSTGRES_DB=dev_db
ports:
- "5432:5432"
volumes:
- dev-db-data:/var/lib/postgresql/data
networks:
- dev-network
pgadmin:
image: dpage/pgadmin4:latest
environment:
- PGADMIN_DEFAULT_EMAIL=admin@dev.local
- PGADMIN_DEFAULT_PASSWORD=admin
ports:
- "5050:80"
networks:
- dev-network
depends_on:
- db
# MailHog for Email Testing
mailhog:
image: mailhog/mailhog:latest
ports:
- "1025:1025" # SMTP
- "8025:8025" # Web UI
networks:
- dev-network
networks:
dev-network:
driver: bridge
volumes:
dev-db-data:
Networking Strategies
Default Bridge Network
services:
web:
image: nginx
# Automatically connected to default network
app:
image: myapp
# Can communicate with 'web' via service name
Custom Bridge Networks
version: "3.8"
services:
frontend:
image: react-app
networks:
- public
backend:
image: api-server
networks:
- public # Accessible from frontend
- private # Accessible from database
database:
image: postgres
networks:
- private # Isolated from frontend
networks:
public:
driver: bridge
private:
driver: bridge
internal: true # No internet access
Network Aliases
services:
api:
image: api-server
networks:
backend:
aliases:
- api-server
- api.internal
- api-v1.internal
networks:
backend:
driver: bridge
Host Network Mode
services:
app:
image: myapp
network_mode: "host" # Use host network stack
# No port mapping needed, uses host ports directly
Custom Network Configuration
networks:
custom-network:
driver: bridge
driver_opts:
com.docker.network.bridge.name: br-custom
ipam:
driver: default
config:
- subnet: 172.28.0.0/16
gateway: 172.28.0.1
labels:
- "com.example.description=Custom network"
Volume Management
Named Volumes
version: "3.8"
services:
db:
image: postgres:15
volumes:
- postgres-data:/var/lib/postgresql/data # Named volume
backup:
image: postgres:15
volumes:
- postgres-data:/backup:ro # Read-only mount
command: pg_dump -U postgres > /backup/dump.sql
volumes:
postgres-data:
driver: local
driver_opts:
type: none
o: bind
device: /path/on/host
Bind Mounts
services:
web:
image: nginx
volumes:
# Relative path bind mount
- ./html:/usr/share/nginx/html
# Absolute path bind mount
- /var/log/nginx:/var/log/nginx
# Read-only bind mount
- ./config/nginx.conf:/etc/nginx/nginx.conf:ro
tmpfs Mounts (In-Memory)
services:
app:
image: myapp
tmpfs:
- /tmp
- /run
# Or with options:
volumes:
- type: tmpfs
target: /app/cache
tmpfs:
size: 1000000000 # 1GB
Volume Sharing Between Services
services:
app:
image: myapp
volumes:
- shared-data:/data
worker:
image: worker
volumes:
- shared-data:/data
backup:
image: backup-tool
volumes:
- shared-data:/backup:ro
volumes:
shared-data:
Advanced Volume Configuration
volumes:
data:
driver: local
driver_opts:
type: "nfs"
o: "addr=10.40.0.199,nolock,soft,rw"
device: ":/docker/example"
cache:
driver: local
driver_opts:
type: tmpfs
device: tmpfs
o: "size=100m,uid=1000"
external-volume:
external: true # Volume created outside Compose
name: my-existing-volume
Health Checks
HTTP Health Check
services:
web:
image: nginx
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost/health"]
interval: 30s
timeout: 10s
retries: 3
start_period: 40s
Database Health Check
services:
postgres:
image: postgres:15
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres"]
interval: 10s
timeout: 5s
retries: 5
start_period: 30s
mysql:
image: mysql:8
healthcheck:
test: ["CMD", "mysqladmin", "ping", "-h", "localhost"]
interval: 10s
timeout: 5s
retries: 3
mongodb:
image: mongo:6
healthcheck:
test: ["CMD", "mongosh", "--eval", "db.adminCommand('ping')"]
interval: 10s
timeout: 5s
retries: 5
Application Health Check
services:
app:
build: ./app
healthcheck:
test: ["CMD", "node", "healthcheck.js"]
interval: 30s
timeout: 10s
retries: 3
start_period: 60s
api:
build: ./api
healthcheck:
test: ["CMD-SHELL", "wget --no-verbose --tries=1 --spider http://localhost:3000/health || exit 1"]
interval: 30s
timeout: 10s
retries: 3
Complex Health Checks
services:
redis:
image: redis:alpine
healthcheck:
test: |
sh -c '
redis-cli ping | grep PONG &&
redis-cli --raw incr ping | grep 1
'
interval: 10s
timeout: 3s
retries: 5
Development vs Production Configurations
Base Configuration (compose.yaml)
version: "3.8"
services:
web:
image: myapp:latest
environment:
- NODE_ENV=production
networks:
- app-network
db:
image: postgres:15-alpine
networks:
- app-network
networks:
app-network:
driver: bridge
Development Override (compose.override.yaml)
# Automatically merged with compose.yaml in development
version: "3.8"
services:
web:
build:
context: .
target: development
volumes:
- ./src:/app/src # Live code reload
- /app/node_modules
ports:
- "3000:3000" # Expose for local access
- "9229:9229" # Debugger port
environment:
- NODE_ENV=development
- DEBUG=*
command: npm run dev
db:
ports:
- "5432:5432" # Expose for local tools
environment:
- POSTGRES_PASSWORD=dev
volumes:
- ./init-dev.sql:/docker-entrypoint-initdb.d/init.sql
Production Configuration (compose.prod.yaml)
version: "3.8"
services:
web:
image: myapp:${VERSION:-latest}
restart: always
environment:
- NODE_ENV=production
deploy:
replicas: 3
resources:
limits:
cpus: '2'
memory: 2G
reservations:
cpus: '1'
memory: 1G
update_config:
parallelism: 1
delay: 10s
failure_action: rollback
rollback_config:
parallelism: 1
delay: 5s
logging:
driver: json-file
options:
max-size: "10m"
max-file: "5"
db:
image: postgres:15-alpine
restart: always
environment:
- POSTGRES_PASSWORD_FILE=/run/secrets/db_password
secrets:
- db_password
volumes:
- postgres-data:/var/lib/postgresql/data
deploy:
resources:
limits:
cpus: '2'
memory: 4G
# Production additions
nginx:
image: nginx:alpine
ports:
- "80:80"
- "443:443"
volumes:
- ./nginx/prod.conf:/etc/nginx/nginx.conf:ro
- ssl-certs:/etc/nginx/ssl:ro
restart: always
depends_on:
- web
secrets:
db_password:
external: true
volumes:
postgres-data:
driver: local
ssl-certs:
external: true
Staging Configuration (compose.staging.yaml)
version: "3.8"
services:
web:
image: myapp:staging-${VERSION:-latest}
restart: unless-stopped
environment:
- NODE_ENV=staging
deploy:
replicas: 2
resources:
limits:
cpus: '1'
memory: 1G
db:
environment:
- POSTGRES_PASSWORD=${DB_PASSWORD}
volumes:
- staging-db-data:/var/lib/postgresql/data
volumes:
staging-db-data:
Essential Docker Compose Commands
Project Management
# Start services
docker compose up # Foreground
docker compose up -d # Detached (background)
docker compose up --build # Rebuild images
docker compose up --force-recreate # Recreate containers
docker compose up --scale web=3 # Scale service to 3 instances
# Stop services
docker compose stop # Stop containers
docker compose down # Stop and remove containers/networks
docker compose down -v # Also remove volumes
docker compose down --rmi all # Also remove images
# Restart services
docker compose restart # Restart all services
docker compose restart web # Restart specific service
Service Management
# Build services
docker compose build # Build all services
docker compose build web # Build specific service
docker compose build --no-cache # Build without cache
docker compose build --pull # Pull latest base images
# View services
docker compose ps # List containers
docker compose ps -a # Include stopped containers
docker compose top # Display running processes
docker compose images # List images
# Logs
docker compose logs # View all logs
docker compose logs -f # Follow logs
docker compose logs web # Service-specific logs
docker compose logs --tail=100 web # Last 100 lines
Execution and Debugging
# Execute commands
docker compose exec web sh # Interactive shell
docker compose exec web npm test # Run command
docker compose exec -u root web sh # Run as root
# Run one-off commands
docker compose run web npm install # Run command in new container
docker compose run --rm web test # Remove container after
docker compose run --no-deps web sh # Don't start dependencies
Configuration Management
# Multiple compose files
docker compose -f compose.yaml -f compose.prod.yaml up
# Environment-specific deployment
docker compose --env-file .env.prod up
docker compose -p myproject up # Custom project name
# Configuration validation
docker compose config # Validate and view config
docker compose config --quiet # Only validation
docker compose config --services # List services
docker compose config --volumes # List volumes
15+ Compose Examples
Example 1: NGINX + PHP + MySQL (LAMP Stack)
version: "3.8"
services:
nginx:
image: nginx:alpine
ports:
- "80:80"
volumes:
- ./public:/var/www/html
- ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
networks:
- lamp
depends_on:
- php
php:
build:
context: ./php
dockerfile: Dockerfile
volumes:
- ./public:/var/www/html
networks:
- lamp
depends_on:
- mysql
mysql:
image: mysql:8.0
environment:
MYSQL_ROOT_PASSWORD: secret
MYSQL_DATABASE: myapp
MYSQL_USER: user
MYSQL_PASSWORD: password
volumes:
- mysql-data:/var/lib/mysql
networks:
- lamp
networks:
lamp:
volumes:
mysql-data:
Example 2: Django + PostgreSQL + Redis + Celery
version: "3.8"
services:
web:
build: .
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
environment:
- DATABASE_URL=postgresql://postgres:postgres@db:5432/django_db
- REDIS_URL=redis://redis:6379/0
depends_on:
- db
- redis
db:
image: postgres:15-alpine
environment:
POSTGRES_DB: django_db
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
volumes:
- postgres-data:/var/lib/postgresql/data
redis:
image: redis:alpine
volumes:
- redis-data:/data
celery:
build: .
command: celery -A myproject worker -l info
volumes:
- .:/code
environment:
- DATABASE_URL=postgresql://postgres:postgres@db:5432/django_db
- REDIS_URL=redis://redis:6379/0
depends_on:
- db
- redis
celery-beat:
build: .
command: celery -A myproject beat -l info
volumes:
- .:/code
environment:
- DATABASE_URL=postgresql://postgres:postgres@db:5432/django_db
- REDIS_URL=redis://redis:6379/0
depends_on:
- db
- redis
volumes:
postgres-data:
redis-data:
Example 3: React + Node.js + MongoDB + NGINX
version: "3.8"
services:
frontend:
build:
context: ./frontend
args:
REACT_APP_API_URL: http://localhost/api
volumes:
- ./frontend:/app
- /app/node_modules
environment:
- CHOKIDAR_USEPOLLING=true
networks:
- app-network
backend:
build: ./backend
ports:
- "5000:5000"
volumes:
- ./backend:/app
- /app/node_modules
environment:
- MONGODB_URI=mongodb://mongo:27017/myapp
- JWT_SECRET=dev-secret
depends_on:
- mongo
networks:
- app-network
mongo:
image: mongo:6
ports:
- "27017:27017"
volumes:
- mongo-data:/data/db
- mongo-config:/data/configdb
environment:
- MONGO_INITDB_ROOT_USERNAME=admin
- MONGO_INITDB_ROOT_PASSWORD=secret
networks:
- app-network
nginx:
image: nginx:alpine
ports:
- "80:80"
volumes:
- ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
depends_on:
- frontend
- backend
networks:
- app-network
networks:
app-network:
driver: bridge
volumes:
mongo-data:
mongo-config:
Example 4: Spring Boot + MySQL + Adminer
version: "3.8"
services:
app:
build:
context: .
dockerfile: Dockerfile
ports:
- "8080:8080"
environment:
- SPRING_DATASOURCE_URL=jdbc:mysql://db:3306/springdb?useSSL=false
- SPRING_DATASOURCE_USERNAME=root
- SPRING_DATASOURCE_PASSWORD=secret
- SPRING_JPA_HIBERNATE_DDL_AUTO=update
depends_on:
db:
condition: service_healthy
networks:
- spring-network
db:
image: mysql:8.0
environment:
MYSQL_ROOT_PASSWORD: secret
MYSQL_DATABASE: springdb
volumes:
- mysql-data:/var/lib/mysql
networks:
- spring-network
healthcheck:
test: ["CMD", "mysqladmin", "ping", "-h", "localhost"]
interval: 10s
timeout: 5s
retries: 5
adminer:
image: adminer:latest
ports:
- "8081:8080"
environment:
ADMINER_DEFAULT_SERVER: db
networks:
- spring-network
networks:
spring-network:
volumes:
mysql-data:
Example 5: WordPress + MySQL + phpMyAdmin
version: "3.8"
services:
wordpress:
image: wordpress:latest
ports:
- "8000:80"
environment:
WORDPRESS_DB_HOST: db:3306
WORDPRESS_DB_USER: wordpress
WORDPRESS_DB_PASSWORD: wordpress
WORDPRESS_DB_NAME: wordpress
volumes:
- wordpress-data:/var/www/html
depends_on:
- db
networks:
- wordpress-network
db:
image: mysql:8.0
environment:
MYSQL_DATABASE: wordpress
MYSQL_USER: wordpress
MYSQL_PASSWORD: wordpress
MYSQL_ROOT_PASSWORD: rootpassword
volumes:
- db-data:/var/lib/mysql
networks:
- wordpress-network
phpmyadmin:
image: phpmyadmin/phpmyadmin:latest
ports:
- "8080:80"
environment:
PMA_HOST: db
PMA_USER: root
PMA_PASSWORD: rootpassword
depends_on:
- db
networks:
- wordpress-network
networks:
wordpress-network:
volumes:
wordpress-data:
db-data:
Example 6: Elasticsearch + Kibana + Logstash (ELK Stack)
version: "3.8"
services:
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:8.10.0
container_name: elasticsearch
environment:
- discovery.type=single-node
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
- xpack.security.enabled=false
ports:
- "9200:9200"
- "9300:9300"
volumes:
- elasticsearch-data:/usr/share/elasticsearch/data
networks:
- elk
logstash:
image: docker.elastic.co/logstash/logstash:8.10.0
container_name: logstash
volumes:
- ./logstash/pipeline:/usr/share/logstash/pipeline:ro
- ./logstash/config/logstash.yml:/usr/share/logstash/config/logstash.yml:ro
ports:
- "5000:5000"
- "9600:9600"
environment:
LS_JAVA_OPTS: "-Xmx256m -Xms256m"
networks:
- elk
depends_on:
- elasticsearch
kibana:
image: docker.elastic.co/kibana/kibana:8.10.0
container_name: kibana
ports:
- "5601:5601"
environment:
ELASTICSEARCH_URL: http://elasticsearch:9200
ELASTICSEARCH_HOSTS: http://elasticsearch:9200
networks:
- elk
depends_on:
- elasticsearch
networks:
elk:
driver: bridge
volumes:
elasticsearch-data:
Example 7: GitLab + GitLab Runner
version: "3.8"
services:
gitlab:
image: gitlab/gitlab-ce:latest
container_name: gitlab
restart: unless-stopped
hostname: gitlab.local
environment:
GITLAB_OMNIBUS_CONFIG: |
external_url 'http://gitlab.local'
gitlab_rails['gitlab_shell_ssh_port'] = 2222
ports:
- "80:80"
- "443:443"
- "2222:22"
volumes:
- gitlab-config:/etc/gitlab
- gitlab-logs:/var/log/gitlab
- gitlab-data:/var/opt/gitlab
networks:
- gitlab-network
gitlab-runner:
image: gitlab/gitlab-runner:latest
container_name: gitlab-runner
restart: unless-stopped
volumes:
- gitlab-runner-config:/etc/gitlab-runner
- /var/run/docker.sock:/var/run/docker.sock
networks:
- gitlab-network
depends_on:
- gitlab
networks:
gitlab-network:
volumes:
gitlab-config:
gitlab-logs:
gitlab-data:
gitlab-runner-config:
Example 8: Jenkins + Docker-in-Docker
version: "3.8"
services:
jenkins:
image: jenkins/jenkins:lts
container_name: jenkins
user: root
ports:
- "8080:8080"
- "50000:50000"
volumes:
- jenkins-data:/var/jenkins_home
- /var/run/docker.sock:/var/run/docker.sock
- /usr/bin/docker:/usr/bin/docker
environment:
- JAVA_OPTS=-Djenkins.install.runSetupWizard=false
networks:
- jenkins-network
jenkins-agent:
image: jenkins/inbound-agent:latest
container_name: jenkins-agent
environment:
- JENKINS_URL=http://jenkins:8080
- JENKINS_AGENT_NAME=agent1
- JENKINS_SECRET=${AGENT_SECRET}
- JENKINS_AGENT_WORKDIR=/home/jenkins/agent
volumes:
- /var/run/docker.sock:/var/run/docker.sock
networks:
- jenkins-network
depends_on:
- jenkins
networks:
jenkins-network:
volumes:
jenkins-data:
Example 9: Prometheus + Grafana + Node Exporter
version: "3.8"
services:
prometheus:
image: prom/prometheus:latest
container_name: prometheus
ports:
- "9090:9090"
volumes:
- ./prometheus/prometheus.yml:/etc/prometheus/prometheus.yml:ro
- prometheus-data:/prometheus
command:
- '--config.file=/etc/prometheus/prometheus.yml'
- '--storage.tsdb.path=/prometheus'
networks:
- monitoring
grafana:
image: grafana/grafana:latest
container_name: grafana
ports:
- "3000:3000"
environment:
- GF_SECURITY_ADMIN_USER=admin
- GF_SECURITY_ADMIN_PASSWORD=admin
- GF_INSTALL_PLUGINS=grafana-piechart-panel
volumes:
- grafana-data:/var/lib/grafana
- ./grafana/provisioning:/etc/grafana/provisioning:ro
networks:
- monitoring
depends_on:
- prometheus
node-exporter:
image: prom/node-exporter:latest
container_name: node-exporter
ports:
- "9100:9100"
volumes:
- /proc:/host/proc:ro
- /sys:/host/sys:ro
- /:/rootfs:ro
command:
- '--path.procfs=/host/proc'
- '--path.sysfs=/host/sys'
- '--collector.filesystem.mount-points-exclude=^/(sys|proc|dev|host|etc)($$|/)'
networks:
- monitoring
networks:
monitoring:
volumes:
prometheus-data:
grafana-data:
Example 10: RabbitMQ + Multiple Consumers
version: "3.8"
services:
rabbitmq:
image: rabbitmq:3-management-alpine
container_name: rabbitmq
ports:
- "5672:5672" # AMQP
- "15672:15672" # Management UI
environment:
RABBITMQ_DEFAULT_USER: admin
RABBITMQ_DEFAULT_PASS: secret
volumes:
- rabbitmq-data:/var/lib/rabbitmq
- ./rabbitmq/rabbitmq.conf:/etc/rabbitmq/rabbitmq.conf:ro
networks:
- messaging
healthcheck:
test: ["CMD", "rabbitmq-diagnostics", "ping"]
interval: 30s
timeout: 10s
retries: 5
producer:
build: ./services/producer
environment:
RABBITMQ_URL: amqp://admin:secret@rabbitmq:5672
depends_on:
rabbitmq:
condition: service_healthy
networks:
- messaging
consumer-1:
build: ./services/consumer
environment:
RABBITMQ_URL: amqp://admin:secret@rabbitmq:5672
WORKER_ID: 1
depends_on:
rabbitmq:
condition: service_healthy
networks:
- messaging
deploy:
replicas: 3
consumer-2:
build: ./services/consumer
environment:
RABBITMQ_URL: amqp://admin:secret@rabbitmq:5672
WORKER_ID: 2
depends_on:
rabbitmq:
condition: service_healthy
networks:
- messaging
networks:
messaging:
volumes:
rabbitmq-data:
Example 11: Traefik Reverse Proxy
version: "3.8"
services:
traefik:
image: traefik:v2.10
container_name: traefik
command:
- --api.insecure=true
- --providers.docker=true
- --providers.docker.exposedbydefault=false
- --entrypoints.web.address=:80
- --entrypoints.websecure.address=:443
ports:
- "80:80"
- "443:443"
- "8080:8080" # Traefik dashboard
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- ./traefik/traefik.yml:/etc/traefik/traefik.yml:ro
- ./traefik/dynamic:/etc/traefik/dynamic:ro
networks:
- traefik-network
whoami:
image: traefik/whoami
labels:
- "traefik.enable=true"
- "traefik.http.routers.whoami.rule=Host(`whoami.local`)"
- "traefik.http.routers.whoami.entrypoints=web"
networks:
- traefik-network
app:
image: nginx:alpine
labels:
- "traefik.enable=true"
- "traefik.http.routers.app.rule=Host(`app.local`)"
- "traefik.http.routers.app.entrypoints=web"
- "traefik.http.services.app.loadbalancer.server.port=80"
networks:
- traefik-network
networks:
traefik-network:
driver: bridge
Example 12: MinIO + PostgreSQL Backup
version: "3.8"
services:
minio:
image: minio/minio:latest
container_name: minio
command: server /data --console-address ":9001"
ports:
- "9000:9000"
- "9001:9001"
environment:
MINIO_ROOT_USER: minioadmin
MINIO_ROOT_PASSWORD: minioadmin
volumes:
- minio-data:/data
networks:
- storage
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
interval: 30s
timeout: 20s
retries: 3
postgres:
image: postgres:15-alpine
environment:
POSTGRES_DB: myapp
POSTGRES_USER: postgres
POSTGRES_PASSWORD: secret
volumes:
- postgres-data:/var/lib/postgresql/data
networks:
- storage
backup:
image: postgres:15-alpine
environment:
POSTGRES_HOST: postgres
POSTGRES_DB: myapp
POSTGRES_USER: postgres
POSTGRES_PASSWORD: secret
MINIO_ENDPOINT: minio:9000
MINIO_ACCESS_KEY: minioadmin
MINIO_SECRET_KEY: minioadmin
volumes:
- ./scripts/backup.sh:/backup.sh:ro
entrypoint: ["/bin/sh", "/backup.sh"]
depends_on:
- postgres
- minio
networks:
- storage
networks:
storage:
volumes:
minio-data:
postgres-data:
Example 13: Apache Kafka + Zookeeper
version: "3.8"
services:
zookeeper:
image: confluentinc/cp-zookeeper:latest
container_name: zookeeper
environment:
ZOOKEEPER_CLIENT_PORT: 2181
ZOOKEEPER_TICK_TIME: 2000
ports:
- "2181:2181"
volumes:
- zookeeper-data:/var/lib/zookeeper/data
- zookeeper-logs:/var/lib/zookeeper/log
networks:
- kafka-network
kafka:
image: confluentinc/cp-kafka:latest
container_name: kafka
depends_on:
- zookeeper
ports:
- "9092:9092"
- "29092:29092"
environment:
KAFKA_BROKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092,PLAINTEXT_HOST://localhost:29092
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
volumes:
- kafka-data:/var/lib/kafka/data
networks:
- kafka-network
kafka-ui:
image: provectuslabs/kafka-ui:latest
container_name: kafka-ui
depends_on:
- kafka
ports:
- "8080:8080"
environment:
KAFKA_CLUSTERS_0_NAME: local
KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS: kafka:9092
KAFKA_CLUSTERS_0_ZOOKEEPER: zookeeper:2181
networks:
- kafka-network
networks:
kafka-network:
volumes:
zookeeper-data:
zookeeper-logs:
kafka-data:
Example 14: Keycloak + PostgreSQL (Identity & Access Management)
version: "3.8"
services:
postgres:
image: postgres:15-alpine
container_name: keycloak-db
environment:
POSTGRES_DB: keycloak
POSTGRES_USER: keycloak
POSTGRES_PASSWORD: password
volumes:
- postgres-data:/var/lib/postgresql/data
networks:
- keycloak-network
keycloak:
image: quay.io/keycloak/keycloak:latest
container_name: keycloak
environment:
KC_DB: postgres
KC_DB_URL: jdbc:postgresql://postgres:5432/keycloak
KC_DB_USERNAME: keycloak
KC_DB_PASSWORD: password
KEYCLOAK_ADMIN: admin
KEYCLOAK_ADMIN_PASSWORD: admin
command: start-dev
ports:
- "8080:8080"
depends_on:
- postgres
networks:
- keycloak-network
networks:
keycloak-network:
volumes:
postgres-data:
Example 15: Portainer (Docker Management UI)
version: "3.8"
services:
portainer:
image: portainer/portainer-ce:latest
container_name: portainer
restart: unless-stopped
ports:
- "9000:9000"
- "8000:8000"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- portainer-data:/data
networks:
- portainer-network
networks:
portainer-network:
volumes:
portainer-data:
Example 16: SonarQube + PostgreSQL (Code Quality)
version: "3.8"
services:
sonarqube:
image: sonarqube:community
container_name: sonarqube
depends_on:
- db
environment:
SONAR_JDBC_URL: jdbc:postgresql://db:5432/sonar
SONAR_JDBC_USERNAME: sonar
SONAR_JDBC_PASSWORD: sonar
volumes:
- sonarqube-conf:/opt/sonarqube/conf
- sonarqube-data:/opt/sonarqube/data
- sonarqube-logs:/opt/sonarqube/logs
- sonarqube-extensions:/opt/sonarqube/extensions
ports:
- "9000:9000"
networks:
- sonarqube-network
db:
image: postgres:15-alpine
container_name: sonarqube-db
environment:
POSTGRES_USER: sonar
POSTGRES_PASSWORD: sonar
POSTGRES_DB: sonar
volumes:
- postgresql-data:/var/lib/postgresql/data
networks:
- sonarqube-network
networks:
sonarqube-network:
volumes:
sonarqube-conf:
sonarqube-data:
sonarqube-logs:
sonarqube-extensions:
postgresql-data:
Best Practices
Service Configuration
Use Specific Image Tags: Avoid latest in production
Health Checks: Always define health checks for critical services
Resource Limits: Set CPU and memory limits in production
Restart Policies: Use appropriate restart policies
Environment Variables: Use .env files for sensitive data
Named Volumes: Use named volumes for data persistence
Network Isolation: Separate frontend/backend networks
Logging Configuration: Set up proper log rotation
Development Workflow
Hot Reload: Mount source code as volumes for live updates
Debug Ports: Expose debugger ports in development
Override Files: Use compose.override.yaml for local config
Build Caching: Structure Dockerfiles for efficient caching
Separate Concerns: One process per container
Service Naming: Use descriptive, consistent service names
Security
Secrets Management: Use Docker secrets or external secret managers
Non-Root Users: Run containers as non-root users
Read-Only Filesystems: Mount volumes as read-only when possible
Network Segmentation: Use multiple networks for isolation
Environment Isolation: Never commit sensitive .env files
Image Scanning: Scan images for vulnerabilities
Minimal Base Images: Use Alpine or distroless images
Production Deployment
Image Versioning: Tag images with semantic versions
Rolling Updates: Configure gradual rollout strategies
Monitoring: Integrate with monitoring solutions
Backup Strategy: Implement automated backups
High Availability: Deploy replicas of critical services
Load Balancing: Use reverse proxies for load distribution
Configuration Management: Externalize configuration
Disaster Recovery: Test backup and restore procedures
Troubleshooting
Common Issues
Services can't communicate
Check network configuration
Verify service names are correct
Ensure services are on same network
Check firewall rules
Volumes not persisting
Verify named volumes are defined
Check volume mount paths
Ensure proper permissions
Review Docker volume driver
Services failing health checks
Increase start_period
Verify health check command
Check service logs
Ensure dependencies are ready
Port conflicts
Check for existing services on ports
Use different host ports
Review port mapping syntax
Build failures
Clear build cache: docker compose build --no-cache
Check Dockerfile syntax
Verify build context
Review build arguments
Debugging Commands
# View detailed container information
docker compose ps -a
docker compose logs -f service-name
docker inspect container-name
# Execute commands in running containers
docker compose exec service-name sh
docker compose exec service-name env
# Check network connectivity
docker compose exec service-name ping other-service
docker compose exec service-name netstat -tulpn
# Review configuration
docker compose config
docker compose config --services
docker compose config --volumes
# Clean up resources
docker compose down -v
docker system prune -a --volumes
Advanced Usage
Multi-Stage Builds for Optimization
services:
app:
build:
context: .
dockerfile: Dockerfile
target: production
# Dockerfile uses multi-stage builds
# Development stage
FROM node:18-alpine AS development
WORKDIR /app
COPY package*.json ./
RUN npm install
COPY . .
CMD ["npm", "run", "dev"]
# Build stage
FROM node:18-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production
COPY . .
RUN npm run build
# Production stage
FROM node:18-alpine AS production
WORKDIR /app
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/node_modules ./node_modules
COPY package*.json ./
EXPOSE 3000
CMD ["node", "dist/index.js"]
Environment-Specific Deployments
# Development
docker compose up
# Staging
docker compose -f compose.yaml -f compose.staging.yaml up
# Production
docker compose -f compose.yaml -f compose.prod.yaml up -d
# With environment file
docker compose --env-file .env.prod -f compose.yaml -f compose.prod.yaml up -d
Scaling Services
# Scale specific service
docker compose up -d --scale worker=5
# Scale multiple services
docker compose up -d --scale worker=5 --scale consumer=3
Conditional Service Activation with Profiles
services:
web:
image: nginx
# Always starts
debug:
image: debug-tools
profiles:
- debug # Only starts with --profile debug
test:
build: .
profiles:
- test # Only starts with --profile test
# Start with debug profile
docker compose --profile debug up
# Start with multiple profiles
docker compose --profile debug --profile test up
Quick Reference
Essential Commands
# Start and manage
docker compose up -d # Start detached
docker compose down # Stop and remove
docker compose restart # Restart all
docker compose stop # Stop without removing
# Build and pull
docker compose build # Build all images
docker compose pull # Pull all images
docker compose build --no-cache # Clean build
# View and monitor
docker compose ps # List containers
docker compose logs -f # Follow logs
docker compose top # Running processes
docker compose events # Real-time events
# Execute and debug
docker compose exec service sh # Interactive shell
docker compose run --rm service cmd # One-off command
File Structure
project/
├── compose.yaml # Base configuration
├── compose.override.yaml # Local overrides (auto-loaded)
├── compose.prod.yaml # Production config
├── compose.staging.yaml # Staging config
├── .env # Default environment
├── .env.prod # Production environment
├── services/
│ ├── frontend/
│ │ ├── Dockerfile
│ │ └── src/
│ ├── backend/
│ │ ├── Dockerfile
│ │ └── src/
│ └── worker/
│ ├── Dockerfile
│ └── src/
└── docker/
├── nginx/
│ └── nginx.conf
└── scripts/
└── init.sql
Resources
Docker Compose Documentation: https://docs.docker.com/compose/
Compose File Specification: https://docs.docker.com/compose/compose-file/
Docker Hub: https://hub.docker.com/
Awesome Compose Examples: https://github.com/docker/awesome-compose
Docker Compose GitHub: https://github.com/docker/compose
Best Practices Guide: https://docs.docker.com/develop/dev-best-practices/
Skill Version: 1.0.0
Last Updated: October 2025
Skill Category: DevOps, Container Orchestration, Application Deployment
Compatible With: Docker Compose v3.8+, Docker Engine 20.10+don't have the plugin yet? install it then click "run inline in claude" again.