MyObservability

OpenTelemetry Collector

Consolidating all telemetry data in a central location streamlines maintenance. You can also adjust the pipeline configuration centrally and consistently.

The data from your instrumented application can be sent to an OpenTelemetry collector. The collector is a component of OpenTelemetry that receives data (spans, metrics, logs etc) process (pre-processes data), and exports the data (sends it off to a backend that you want to talk to). The structure of any Collector configuration file consists of four classes of pipeline components that access telemetry data:

🟩 Receivers

Receivers collect telemetry from one or more sources. They can be pull or push based, and may support one or more data sources. Like OTLP, Zipkin, Prometheus.

receivers:
  otlp:
    protocols:
      grpc: # Receive data over gRPC
      http: # Receive data over HTTP

🟩 Processors

Once the data is received , the collector can process the data. Processors transform or enrich the telemetry data after it has been received but before it is exported. Common uses include batching, filtering, transforming, enriching, batching, dropping unwanted data or adding metadata.

processors:
  batch: # Groups telemetry data into batches for efficient exporting
    timeout: 5s 
    send_batch_size: 1024

  attributes: # Modify or add attributes to telemetry data
    actions:
      - key: environment
        value: production
        action: insert

🟩 Exporter

An exporter is a component of OpenTelemetry and is how data gets sent to different systems/back-ends. Include Jaeger, Prometheus, Elasticsearch, etc.

exporters:
  jaeger:
    endpoint: "http://jaeger:14250"
    tls:
      insecure: true

  prometheus:
    endpoint: "0.0.0.0:8889" # Prometheus scraper will scrape metrics from this endpoint

🟥 Connectors

Connectors are specialized components that combine receiving, processing, and exporting telemetry. They are often used to integrate with external telemetry pipelines seamlessly.

connectors:
  signalfx:
    ingest_url: "https://ingest.signalfx.com"
    api_token: "your-api-token"

Configuration

The configurations are in YAML format. At the top level, each components (receivers, processors and exporters) has an ID that serves unique reference.

Deploying Collector

A collector can run as a

As part of Standalone, you can horizontally scalable collector service deal with low and high-tide telemetry more efficiently and robustly.

Step by Step OTel Collector Setup

Step:1 - Installing Collector

Install Collector (Ex: Create container) and a collector config file (Ex: otel-collector-config.yml)

docker-compose.yml

services:
  otelcol:
    image: otel/opentelemetry-collector-contrib:0.97.0
    restart: unless-stopped
    command: ["--config=/etc/otel-collector-config.yml", "${OTELCOL_ARGS}"]
    volumes:
      - ./otel-collector-config.yml:/etc/otel-collector-config.yml
    ports:
      - 4317:4317
      - 4318:4318

Step:2 - Exporter

In the application server, import OTLPLogExporter / OTLPMetricExporter / OTLPTraceExporter

Ex: For log exporter

from opentelemetry.exporter.otlp.proto.grpc._log_exporter import OTLPLogExporter
logger_provider.add_log_record_processor(SimpleLogRecordProcessor(exporter=OTLPLogExporter(insecure=True))

Ex: For Metric exporter

from opentelemetry.exporter.otlp.proto.grpc.metric_exporter import OTLPMetricExporter
def create_otlp_reader(export_interval: int) -> MetricReader:
    exporter = OTLPMetricExporter(insecure=True)

Ex: For Trace exporter

from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
def create_tracing_pipeline() -> BatchSpanProcessor:
    exporter = OTLPSpanExporter(insecure=True)

Step:3 - Receivers

Config receivers

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

Find all receivers (Ex: Splunk etchttps://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver) in the GitHub repository of the OpenTelemetry Collector

Step:3 - Processors

Data processing happens according to rules or settings defined for each processor, which might include filtering, dropping, renaming, or recalculating telemetry, among other operations.

There are some pre configured processors available ready to use without building the pipeline Processors

Ex: configuration for the log deduplication processor use logdedupprocessor

receivers:
    filelog:
        include: [./example/*.log]
processors:
    logdedup:
        interval: 60s
        log_count_attribute: dedup_count
        timezone: 'America/Los_Angeles'
exporters:
    googlecloud:

service:
    pipelines:
        logs:
            receivers: [filelog]
            processors: [logdedup]
            exporters: [googlecloud]

Step:4 - Exporters to forward telemetry data to backend

exporters:
  otlphttp/prometheus:
    endpoint: "http://prometheus:9090/api/v1/otlp"
    tls:
      insecure: true

  otlp/jaeger:
    endpoint: "jaeger:4317"
    tls:
      insecure: true

Step:5 - Defines pipelines for the OpenTelemetry Collector’s service

These pipelines configure how metrics and traces are received, processed, and exported.

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch, memory_limiter]
      exporters: [otlp, logging]

    metrics:
      receivers: [otlp, prometheus]
      processors: [batch, resourcedetection]
      exporters: [prometheusremotewrite, logging]

    logs:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlp, logging]

✅ Setup your own Telemetry

✅ Main Page: Click Here