Quick Local Testing with the OpenTelemetry Collector

OpenTelemetry is a collection of APIs and tooling for adding tracing, metrics, and other telemetry data to applications. One of those tools is the OpenTelemetry Collector, or otelcol.

The OTel Collector is an agent that runs alongside your application. The alternative to using the Collector is to ship telemetry data directly from your application to the location(s) where it will be stored. However, this adds overhead and complexity to your application. As your application scales, it it useful to get the telemetry data out of your application as quickly as possible and let the Collector handle the processing and shipping.

Getting up and running

The OTel Collector is highly configurable. However, sometimes you just want to get up and running locally as quickly as possible. Once you see telemetry data is flowing to the Collector as expected, you can refine your configuration as needed. Follow these steps to get up and running quickly.

1. Install the OTel Collector. The Getting Started page has all of the necessary instructions for doing this. For local development, you can likely get away with simply downloading from the GitHub Releases page. This post was written using v0.84.0.

2. Create a configuration file that receives data on the default ports and logs that data to the console. An example configuration file is shown below. This is not a production ready configuration. It was adapted from one of the SigNoz examples.

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 127.0.0.1:4317
      http:
        endpoint: 127.0.0.1:4318
processors:
  batch:
    send_batch_size: 1000
    timeout: 10s
exporters:
  logging:
    # verbosity of the logging export: detailed, normal, basic
    verbosity: detailed
service:
  pipelines:
    metrics:
      receivers: [otlp]
      processors: [batch]
      exporters: [logging]
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [logging]
    logs:
      receivers: [otlp]
      processors: [batch]
      exporters: [logging]

1. Start the OTel Collector with a command similar to the following one. otelcol is the OTel Collector binary, and config.yaml is the configuration file created in the previous step.

./otelcol --config=config.yaml

1. Generate telemetry data somehow. Ideally, you should be creating the data in your application, but you can download a telemetry generator like otelgen and provide artificial data:

./otelgen --otel-exporter-otlp-endpoint 127.0.0.1:4317 --insecure --duration 10 --rate 1 traces multi

If the data is sent to the Collector properly, it should be logged to the console where otelcol is running.

1. Refine the configuration as needed for your use case. See the Configuration page for more details.