distributed-tracing

SKILL.md

Distributed Tracing

Implement distributed tracing with Jaeger and Tempo for request flow visibility across microservices.

Purpose

Track requests across distributed systems to understand latency, dependencies, and failure points.

When to Use

Debug latency issues

Understand service dependencies

Identify bottlenecks

Trace error propagation

Analyze request paths

Detailed patterns and worked examples

Detailed pattern documentation lives in references/details.md. Read that file when the navigation tier above is insufficient.
Best Practices

Sample appropriately (1-10% in production)

Add meaningful tags (user_id, request_id)

Propagate context across all service boundaries

Log exceptions in spans

Use consistent naming for operations

Monitor tracing overhead (<1% CPU impact)

Set up alerts for trace errors

Implement distributed context (baggage)

Use span events for important milestones

Document instrumentation standards

Integration with Logging

Correlated Logs

import logging
from opentelemetry import trace

logger = logging.getLogger(__name__)

def process_request():
    span = trace.get_current_span()
    trace_id = span.get_span_context().trace_id

    logger.info(
        "Processing request",
        extra={"trace_id": format(trace_id, '032x')}
    )

Troubleshooting

No traces appearing:

Check collector endpoint

Verify network connectivity

Check sampling configuration

Review application logs

High latency overhead:

Reduce sampling rate

Use batch span processor

Check exporter configuration

Related Skills

prometheus-configuration - For metrics

grafana-dashboards - For visualization

slo-implementation - For latency SLOs