Version: 1.0.0This documentation is for Spitfire 1.0.0.

Connecting a Go service with OTel

Go is compiled, so there is no agent to load: you add one file and two lines to the service and build it again. From then on the service sends its traces, metrics and logs; the addresses and tokens come from the environment, as in the other guides. Values and general rules: Connecting services with OTel: overview. If you cannot touch the code, see Without changing the code: OTel eBPF.

Version table

Go	`go.opentelemetry.io/otel` (sdk, sdk/metric)	`otel/log`, `otel/sdk/log`	contrib (`autoexport`, `otelhttp`, `runtime`)	`otelslog`
1.26 and later	v1.47.0	v1.47.0	v0.72.0	v0.21.0
1.25	v1.46.0	v0.22.0	v0.71.0	v0.20.1
1.24	v1.41.0	v0.17.0	v0.66.0	v0.16.0
1.23 and earlier	not supported	–	–	–

Find your version: the go line in the service's go.mod (or go version on the build machine).

The commands below use the 1.26 row; on 1.25 or 1.24 write that row's versions. Every module on a row is released together: do not mix rows.

Verified: 2026-11-03. Source: supported Go versions, Go getting started, autoexport (environment variables), the go line of each module.

What you need

Spitfire address https://<spitfire> and OTLP receiver token sfotlp_…
Tower address http://<tower>:4318 and Collector token sfcol_…
Protocol http/protobuf
Service name <service-name>, version <service-version>, environment <environment>
With a private CA, the CA file /etc/spitfire-otel/ca.pem
With a proxy, the Tower and Spitfire in NO_PROXY
Sampling parentbased_traceidratio, 0.1
The service's source code and the machine it is built on

Linux (systemd)

Next to the main.go of the service, create the file otel.go and paste this (if the package is not main, change the first line):

package main

import (
	"context"
	"errors"
	"log/slog"
	"os"

	"go.opentelemetry.io/contrib/bridges/otelslog"
	"go.opentelemetry.io/contrib/exporters/autoexport"
	"go.opentelemetry.io/contrib/instrumentation/runtime"
	"go.opentelemetry.io/otel"
	"go.opentelemetry.io/otel/log/global"
	"go.opentelemetry.io/otel/propagation"
	sdklog "go.opentelemetry.io/otel/sdk/log"
	sdkmetric "go.opentelemetry.io/otel/sdk/metric"
	sdktrace "go.opentelemetry.io/otel/sdk/trace"
)

// setupOTel: IS_OTEL_ENABLED=true turns OTel on; any other value leaves it
// off. Addresses, tokens, sampling and the service name come from the
// OTEL_* environment values. Call the returned function on shutdown.
func setupOTel(ctx context.Context) (func(context.Context) error, error) {
	if os.Getenv("IS_OTEL_ENABLED") != "true" {
		return func(context.Context) error { return nil }, nil
	}
	spans, err := autoexport.NewSpanExporter(ctx)
	if err != nil {
		return nil, err
	}
	tp := sdktrace.NewTracerProvider(sdktrace.WithBatcher(spans))
	otel.SetTracerProvider(tp)
	otel.SetTextMapPropagator(propagation.NewCompositeTextMapPropagator(propagation.TraceContext{}, propagation.Baggage{}))

	reader, err := autoexport.NewMetricReader(ctx)
	if err != nil {
		return nil, err
	}
	mp := sdkmetric.NewMeterProvider(sdkmetric.WithReader(reader))
	otel.SetMeterProvider(mp)
	if err := runtime.Start(); err != nil { // Go runtime metrics (memory, GC, goroutines)
		return nil, err
	}

	logs, err := autoexport.NewLogExporter(ctx)
	if err != nil {
		return nil, err
	}
	lp := sdklog.NewLoggerProvider(sdklog.WithProcessor(sdklog.NewBatchProcessor(logs)))
	global.SetLoggerProvider(lp)
	// slog records go to Spitfire (with the trace ID) instead of stdout;
	// delete this line to keep them on stdout.
	slog.SetDefault(otelslog.NewLogger(os.Getenv("OTEL_SERVICE_NAME")))

	return func(ctx context.Context) error {
		return errors.Join(tp.Shutdown(ctx), mp.Shutdown(ctx), lp.Shutdown(ctx))
	}, nil
}

In main, call it first and wrap the HTTP handler (here mux) with otelhttp; incoming traceparent headers then continue Spitfire's trace:

shutdown, err := setupOTel(ctx)
if err != nil {
	log.Fatal(err)
}
defer shutdown(context.Background())

handler := otelhttp.NewHandler(mux, "http.server",
	otelhttp.WithSpanNameFormatter(func(_ string, r *http.Request) string { return r.Method + " " + r.URL.Path }))
// http.ListenAndServe(":8080", handler)

The import is go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp. For calls the service makes to other services, use &http.Client{Transport: otelhttp.NewTransport(http.DefaultTransport)} so the trace carries on.

In the module's folder (where go.mod is), add the modules and build the service again:

bash

go get go.opentelemetry.io/otel@v1.47.0 \
  go.opentelemetry.io/otel/sdk@v1.47.0 \
  go.opentelemetry.io/otel/sdk/metric@v1.47.0 \
  go.opentelemetry.io/otel/sdk/log@v1.47.0 \
  go.opentelemetry.io/otel/log@v1.47.0 \
  go.opentelemetry.io/contrib/exporters/autoexport@v0.72.0 \
  go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.72.0 \
  go.opentelemetry.io/contrib/instrumentation/runtime@v0.72.0 \
  go.opentelemetry.io/contrib/bridges/otelslog@v0.21.0
go mod tidy

Other libraries (optional): gRPC servers and clients with otelgrpc.NewServerHandler() / otelgrpc.NewClientHandler() (grpc.StatsHandler(…)), database calls with otelsql.Open in place of sql.Open:

bash

go get go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc@v0.72.0
go get github.com/XSAM/otelsql@v0.44.0

Create the directory, then the file /etc/spitfire-otel/<service-name>.env, and paste this:

bash

sudo mkdir -p /etc/spitfire-otel

bash

IS_OTEL_ENABLED=true
OTEL_SERVICE_NAME=<service-name>
OTEL_RESOURCE_ATTRIBUTES=service.version=<service-version>,deployment.environment=<environment>
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=https://<spitfire>/otlp/v1/traces
OTEL_EXPORTER_OTLP_TRACES_HEADERS=authorization=sfotlp_…
OTEL_EXPORTER_OTLP_LOGS_ENDPOINT=https://<spitfire>/otlp/v1/logs
OTEL_EXPORTER_OTLP_LOGS_HEADERS=authorization=sfotlp_…
OTEL_EXPORTER_OTLP_METRICS_ENDPOINT=http://<tower>:4318/v1/metrics
OTEL_EXPORTER_OTLP_METRICS_HEADERS=authorization=sfcol_…
OTEL_TRACES_EXPORTER=otlp
OTEL_METRICS_EXPORTER=otlp
OTEL_LOGS_EXPORTER=otlp
OTEL_TRACES_SAMPLER=parentbased_traceidratio
OTEL_TRACES_SAMPLER_ARG=0.1
NO_PROXY=<tower>,<spitfire>
#OTEL_EXPORTER_OTLP_CERTIFICATE=/etc/spitfire-otel/ca.pem

Restrict its permission: sudo chmod 600 /etc/spitfire-otel/<service-name>.env
Open sudo systemctl edit <unit> and paste this:

ini

[Service]
EnvironmentFile=/etc/spitfire-otel/<service-name>.env

Put the new binary in place and restart: sudo systemctl daemon-reload && sudo systemctl restart <unit>
See that it started: the journalctl -u <unit> -n 50 output must not show an error from setupOTel (the service stops with it).

Note

No otel-run file is needed for Go: setupOTel reads IS_OTEL_ENABLED itself.

Docker

Make the code changes (Linux steps 1–4); the Dockerfile needs no new line, the modules are compiled into the binary. go.mod and go.sum must be copied into the build stage as before.
Save the environment file (Linux step 5) on the Docker machine as /etc/spitfire-otel/<service-name>.env.
Run: docker run --env-file /etc/spitfire-otel/<service-name>.env … or, in compose, under the service env_file: /etc/spitfire-otel/<service-name>.env.
When the environment file changes, recreate the container (restart is not enough): docker compose up -d --force-recreate <service>

Note

On a scratch image there are no CA certificates: with https://<spitfire> the export fails. Use gcr.io/distroless/static-debian12 or copy /etc/ssl/certs/ca-certificates.crt from the build stage.

Kubernetes

Prepare the image as in the Docker section.
Save the environment file (Linux step 5) as <service-name>.env and make it a Secret:

bash

kubectl create secret generic <service-name>-otel --from-env-file=<service-name>.env \
  --dry-run=client -o yaml | kubectl apply -f -

In the Deployment, add this under the container:

yaml

envFrom:
  - secretRef:
      name: <service-name>-otel

Restart: kubectl rollout restart deployment/<deployment>

Windows (service)

Make the code changes (Linux steps 1–4) and build the .exe again.
In an administrator PowerShell, write the service's name (Name in the Get-Service list) in place of <windows-service> and run:

powershell

$Service = "<windows-service>"
$IS_OTEL_ENABLED = "true"
$Otel = @(
  "IS_OTEL_ENABLED=$IS_OTEL_ENABLED",
  "OTEL_SERVICE_NAME=<service-name>",
  "OTEL_RESOURCE_ATTRIBUTES=service.version=<service-version>,deployment.environment=<environment>",
  "OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf",
  "OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=https://<spitfire>/otlp/v1/traces",
  "OTEL_EXPORTER_OTLP_TRACES_HEADERS=authorization=sfotlp_…",
  "OTEL_EXPORTER_OTLP_LOGS_ENDPOINT=https://<spitfire>/otlp/v1/logs",
  "OTEL_EXPORTER_OTLP_LOGS_HEADERS=authorization=sfotlp_…",
  "OTEL_EXPORTER_OTLP_METRICS_ENDPOINT=http://<tower>:4318/v1/metrics",
  "OTEL_EXPORTER_OTLP_METRICS_HEADERS=authorization=sfcol_…",
  "OTEL_TRACES_EXPORTER=otlp",
  "OTEL_METRICS_EXPORTER=otlp",
  "OTEL_LOGS_EXPORTER=otlp",
  "OTEL_TRACES_SAMPLER=parentbased_traceidratio",
  "OTEL_TRACES_SAMPLER_ARG=0.1",
  "NO_PROXY=<tower>,<spitfire>"
)
# Private CA: $Otel += "OTEL_EXPORTER_OTLP_CERTIFICATE=C:\spitfire-otel\ca.pem"
$Key = "HKLM:\SYSTEM\CurrentControlSet\Services\$Service"
$Keep = @((Get-ItemProperty $Key).Environment | Where-Object { $_ -and $_ -notmatch '^(IS_OTEL_ENABLED|OTEL_|NO_PROXY)=' })
Set-ItemProperty $Key -Name Environment -Type MultiString -Value ($Keep + $Otel)
Restart-Service -Name $Service -Force

Without changing the code: OTel eBPF

When the code cannot be changed, OpenTelemetry eBPF Instrumentation (OBI) watches the process from outside and produces HTTP and gRPC server/client spans and metrics. It sees less than the SDK: no spans inside the service, no logs, no custom metrics. Linux only.

What	Requirement
Version	`otel/ebpf-instrument:v1.0.0`
Kernel	Linux 5.8 or later with BTF (`/sys/kernel/btf/vmlinux` exists); RHEL 8 kernels 4.18 with the backport also work
Architecture	amd64, arm64
Privileges	root (`--privileged`) and the process space of the service (`--pid=host` or `--pid=container:<service>`)

Run it next to the service (Docker; 8080: the port the service listens on):

bash

docker run -d --name <service-name>-obi --privileged --pid=container:<service> \
  -e OTEL_EBPF_OPEN_PORT=8080 \
  -e OTEL_SERVICE_NAME=<service-name> \
  -e OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf \
  -e OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=https://<spitfire>/otlp/v1/traces \
  -e OTEL_EXPORTER_OTLP_TRACES_HEADERS=authorization=sfotlp_… \
  -e OTEL_EXPORTER_OTLP_METRICS_ENDPOINT=http://<tower>:4318/v1/metrics \
  -e OTEL_EXPORTER_OTLP_METRICS_HEADERS=authorization=sfcol_… \
  otel/ebpf-instrument:v1.0.0

To turn it off, stop that container: docker rm -f <service-name>-obi. On Kubernetes it runs as a DaemonSet; see the OBI documentation.

Note

OBI is not part of Spitfire's own tests (it needs a privileged container on a Linux kernel); the SDK steps above are. Source: OBI requirements, verified 2026-11-03.

Turning off and removing

Turning off: set IS_OTEL_ENABLED=false in the environment file (in the script on Windows) and restart the service (Linux step 8, Docker step 4, Kubernetes step 4, or run the Windows script again). The code stays; setupOTel then does nothing.
Removing: delete otel.go, the setupOTel and otelhttp lines, run go mod tidy and build again; sudo systemctl revert <unit> and sudo rm -r /etc/spitfire-otel; on Windows, delete the OTel lines from the service's Environment value.

Common problems

Symptom	Cause	Fix
`go: go.opentelemetry.io/otel@v1.47.0 requires go >= 1.26.0` (or `go` downloads a newer toolchain)	The versions are for a newer Go	Write your row's versions from the version table
`ambiguous import` or `undefined` from `otel/log`	Versions from two rows mixed	Write every module from the same row and run `go mod tidy`
Spans arrive but each request is a trace of its own	The handler is not wrapped, or `traceparent` is dropped by a proxy	Wrap the outermost handler with `otelhttp.NewHandler` (Linux step 2)
`x509: certificate signed by unknown authority`	Private CA, or no CA certificates in the image	Open the `OTEL_EXPORTER_OTLP_CERTIFICATE` line; on `scratch`, see the Docker note
No traces or metrics arrive	Address, token or network	Follow the Verifying steps

Connecting a Go service with OTel

#Version table

#What you need

#Linux (systemd)

#Docker

#Kubernetes

#Windows (service)

#Without changing the code: OTel eBPF

#Turning off and removing

#Common problems

#Related pages

Version table

What you need

Linux (systemd)

Docker

Kubernetes

Windows (service)

Without changing the code: OTel eBPF

Turning off and removing

Common problems

Related pages