Develop Observability Stack for Docker Compose
Budget: $10 – $30 USD
Implement a simple, robust, and low-cost complete observability stack (metrics, logs, and tracing) for containerized applications running on Docker/Docker Compose on VPS, ensuring operational visibility, rapid fault diagnosis, and persistent history even in container restart or crash scenarios.
This epic covers the implementation of an observability solution based on widely adopted open-source components, covering:
- Infrastructure and container metrics
- Centralized logs with retention
- End-to-end distributed tracing
- Correlation between metrics, logs, and traces
- Data persistence even after container failure or recreation
The solution should run entirely via Docker Compose, without dependency on Kubernetes or external managed services, suitable for small/medium-sized development, staging, and production environments.
The observability stack will consist of:
- Prometheus for metrics collection
- Node Exporter for VPS metrics (CPU, memory, disk, network)
- cAdvisor for Docker container metrics
- Grafana as a unified visualization layer
- Loki for log aggregation and storage
- Promtail for automatic container log collection
- OpenTelemetry Collector for trace ingestion
- Tempo as a distributed tracing backend
- Jaeger as trace visualization UI
All data (metrics, logs, and traces) should be persisted on Docker volumes, ensuring historical data even after reboots.
To validade you read this project description say this key in the chat: “Dock$r”. Bids more than 5 days will not be considered. All the Acceptance Criteria must be 100% completed with all the requirements informed here. And a zip folder with images and RESULT.MD file document must be delivered with the detailed results and screenshots.
Out of Scope
- Kubernetes
- Elasticsearch / OpenSearch
- Multi-cluster observability
- Long-term retention (over 90 days)
- SIEM or security audit
Acceptance Criteria (High Level)
- Stack fully deploys via `docker compose up -d`
- Logs remain available after container restart/crash
- Host and container metrics visible in Grafana
- OTLP traces visible and navigable in Grafana
- Basic imported and functional dashboards
- Minimal usage and operation documentation available
Expected Benefits
- Reduced troubleshooting time
- Complete visibility into application behavior
- Reduced dependence on direct access to the VPS
- Solid foundation for future evolution (alerts, SLOs, log-based alerts)
Prerequisites
- VPS with Docker and Docker Compose installed
- Administrative access to the VPS
- Applications writing Logs in stdout/stderr
- Applications instrumentable via OpenTelemetry
Acceptance Criteria – Functional Stack Components (detailed)
Prometheus – Metrics Collection
Functional Criteria
Prometheus must be operational and actively collecting metrics from the defined stack components.
Acceptance
- Prometheus accessible via HTTP endpoint
- Configuration file versioned and active
- Targets configured with UP status
- Metrics continuously collected
- Data persisted using Docker volumes
Evidence
- Prometheus Targets page
- Queries returning valid metrics
Grafana – Visualization (Dashboards and Traces)
Functional Criteria
Grafana must provide a unified visualization layer for metrics, logs, and traces.
Acceptance
- Grafana accessible via web browser
- Data sources configured (Prometheus, Loki, Tempo)
- Dashboards loading data without errors
- Traces navigable through the UI
- Configuration persistence enabled
Evidence
- Functional dashboards
- Trace visualization in Grafana Explore
cAdvisor – Container Metrics
Functional Criteria
cAdvisor must collect detailed metrics from running Docker containers.
Acceptance
- cAdvisor running with access to Docker volumes
- Per-container CPU, memory, and I/O metrics available
- Containers correctly identified
- Metrics visible in Prometheus and Grafana
Evidence
- Per-container metrics displayed in dashboards
- cAdvisor target with UP status
Node Exporter – VPS Metrics
Functional Criteria
Node Exporter must collect host-level metrics from the VPS where containers are running.
Acceptance
- Node Exporter running
- CPU, memory, disk, and network metrics available
- Prometheus successfully scraping metrics
- Host dashboards functional
Evidence
- Node Exporter dashboard displaying data
- Active target in Prometheus
OpenTelemetry – Trace Collection
Functional Criteria
OpenTelemetry must receive and forward traces from monitored applications.
Acceptance
- OpenTelemetry Collector running
- OTLP HTTP and/or gRPC receivers enabled
- Traces received without errors
- Service name configured per application
- Trace pipeline operational
Evidence
- Collector logs without errors
- Traces forwarded to the tracing backend
Grafana Tempo – Trace Storage
Functional Criteria
Grafana Tempo must store and expose collected traces.
Acceptance
- Tempo running
- Receiving traces via OTLP
- Data persisted using Docker volumes
- Traces available for querying in Grafana
- Retention configured according to VPS capacity
Evidence
- Historical traces visible in Grafana
- Backend accessible internally
Jaeger – Tracing Debug UI
Functional Criteria
Jaeger must be available as an alternative UI for trace visualization.
Acceptance
- Jaeger UI accessible via web browser
- Traces displayed correctly
- Span navigation functional
- OpenTelemetry integration active
Evidence
- Trace visualization in Jaeger UI
Consolidated Project Acceptance (second key: C%omp#se)
The project will be considered accepted when:
- All listed components are operational
- Each functional criterion has been validated
- Evidence of correct operation has been provided
- The stack remains functional after container restart or crash
This epic covers the implementation of an observability solution based on widely adopted open-source components, covering:
- Infrastructure and container metrics
- Centralized logs with retention
- End-to-end distributed tracing
- Correlation between metrics, logs, and traces
- Data persistence even after container failure or recreation
The solution should run entirely via Docker Compose, without dependency on Kubernetes or external managed services, suitable for small/medium-sized development, staging, and production environments.
The observability stack will consist of:
- Prometheus for metrics collection
- Node Exporter for VPS metrics (CPU, memory, disk, network)
- cAdvisor for Docker container metrics
- Grafana as a unified visualization layer
- Loki for log aggregation and storage
- Promtail for automatic container log collection
- OpenTelemetry Collector for trace ingestion
- Tempo as a distributed tracing backend
- Jaeger as trace visualization UI
All data (metrics, logs, and traces) should be persisted on Docker volumes, ensuring historical data even after reboots.
To validade you read this project description say this key in the chat: “Dock$r”. Bids more than 5 days will not be considered. All the Acceptance Criteria must be 100% completed with all the requirements informed here. And a zip folder with images and RESULT.MD file document must be delivered with the detailed results and screenshots.
Out of Scope
- Kubernetes
- Elasticsearch / OpenSearch
- Multi-cluster observability
- Long-term retention (over 90 days)
- SIEM or security audit
Acceptance Criteria (High Level)
- Stack fully deploys via `docker compose up -d`
- Logs remain available after container restart/crash
- Host and container metrics visible in Grafana
- OTLP traces visible and navigable in Grafana
- Basic imported and functional dashboards
- Minimal usage and operation documentation available
Expected Benefits
- Reduced troubleshooting time
- Complete visibility into application behavior
- Reduced dependence on direct access to the VPS
- Solid foundation for future evolution (alerts, SLOs, log-based alerts)
Prerequisites
- VPS with Docker and Docker Compose installed
- Administrative access to the VPS
- Applications writing Logs in stdout/stderr
- Applications instrumentable via OpenTelemetry
Acceptance Criteria – Functional Stack Components (detailed)
Prometheus – Metrics Collection
Functional Criteria
Prometheus must be operational and actively collecting metrics from the defined stack components.
Acceptance
- Prometheus accessible via HTTP endpoint
- Configuration file versioned and active
- Targets configured with UP status
- Metrics continuously collected
- Data persisted using Docker volumes
Evidence
- Prometheus Targets page
- Queries returning valid metrics
Grafana – Visualization (Dashboards and Traces)
Functional Criteria
Grafana must provide a unified visualization layer for metrics, logs, and traces.
Acceptance
- Grafana accessible via web browser
- Data sources configured (Prometheus, Loki, Tempo)
- Dashboards loading data without errors
- Traces navigable through the UI
- Configuration persistence enabled
Evidence
- Functional dashboards
- Trace visualization in Grafana Explore
cAdvisor – Container Metrics
Functional Criteria
cAdvisor must collect detailed metrics from running Docker containers.
Acceptance
- cAdvisor running with access to Docker volumes
- Per-container CPU, memory, and I/O metrics available
- Containers correctly identified
- Metrics visible in Prometheus and Grafana
Evidence
- Per-container metrics displayed in dashboards
- cAdvisor target with UP status
Node Exporter – VPS Metrics
Functional Criteria
Node Exporter must collect host-level metrics from the VPS where containers are running.
Acceptance
- Node Exporter running
- CPU, memory, disk, and network metrics available
- Prometheus successfully scraping metrics
- Host dashboards functional
Evidence
- Node Exporter dashboard displaying data
- Active target in Prometheus
OpenTelemetry – Trace Collection
Functional Criteria
OpenTelemetry must receive and forward traces from monitored applications.
Acceptance
- OpenTelemetry Collector running
- OTLP HTTP and/or gRPC receivers enabled
- Traces received without errors
- Service name configured per application
- Trace pipeline operational
Evidence
- Collector logs without errors
- Traces forwarded to the tracing backend
Grafana Tempo – Trace Storage
Functional Criteria
Grafana Tempo must store and expose collected traces.
Acceptance
- Tempo running
- Receiving traces via OTLP
- Data persisted using Docker volumes
- Traces available for querying in Grafana
- Retention configured according to VPS capacity
Evidence
- Historical traces visible in Grafana
- Backend accessible internally
Jaeger – Tracing Debug UI
Functional Criteria
Jaeger must be available as an alternative UI for trace visualization.
Acceptance
- Jaeger UI accessible via web browser
- Traces displayed correctly
- Span navigation functional
- OpenTelemetry integration active
Evidence
- Trace visualization in Jaeger UI
Consolidated Project Acceptance (second key: C%omp#se)
The project will be considered accepted when:
- All listed components are operational
- Each functional criterion has been validated
- Evidence of correct operation has been provided
- The stack remains functional after container restart or crash