Infrastructure: Kong, Envoy, Traefik
The Tower reads the metrics endpoint of API gateways and proxies (Beta); it never touches the gateway's config. Each gateway cluster is one monitored system: list all of its nodes, comma-separated, in that one source. Panels show requests, 5xx ratio, latency, upstream health and config state.
Kong
Quick install
- Run the command on one Kong node and answer the questions.
curl -fsSL "https://<spitfire>/api/v1/setup/ds/kong.sh?t=<token>" -o /tmp/sf-kong.sh && bash /tmp/sf-kong.sh- You are done when you see the
OK: ...line.
Manual install (Kong with a database)
- Turn on the Prometheus plugin through the Admin API (once per cluster).
curl -fsS -X POST http://127.0.0.1:8001/plugins -H 'Content-Type: application/json' --data '{"name":"prometheus","config":{"status_code_metrics":true,"latency_metrics":true,"bandwidth_metrics":true,"upstream_health_metrics":true}}' && echo 'OK: plugin on'- Open the Status API on every node (kong.conf).
grep -q '^status_listen' /etc/kong/kong.conf || echo 'status_listen = 0.0.0.0:8100' | sudo tee -a /etc/kong/kong.conf- Reload Kong with this kong.conf (without
-c,kong reloadignores the change in the file).
sudo kong reload -c /etc/kong/kong.conf- Check the metrics.
curl -fsS http://127.0.0.1:8100/metrics | grep -q '^kong_datastore_reachable' && echo 'OK: Kong metrics ready'- Spitfire → Infrastructure sources → Kong; address:
kong1:8100,kong2:8100.
Manual install (DB-less Kong)
- Add the plugin to kong.yml.
plugins:
- name: prometheus
config:
status_code_metrics: true
latency_metrics: true
bandwidth_metrics: true
upstream_health_metrics: true- Set
KONG_STATUS_LISTEN=0.0.0.0:8100in the environment, then reload Kong.
sudo kong reload- Check the metrics.
curl -fsS http://127.0.0.1:8100/metrics | grep -q '^kong_datastore_reachable' && echo 'OK: Kong metrics ready'- Spitfire → Infrastructure sources → Kong; address:
kong1:8100,kong2:8100.
The Status API only reads. We recommend never giving port 8001 (the Admin API) to any monitoring tool.
Envoy
Quick install
- Run the command on every Envoy host.
curl -fsSL "https://<spitfire>/api/v1/setup/ds/envoy.sh?t=<token>" -o /tmp/sf-envoy.sh && bash /tmp/sf-envoy.sh- You are done when you see the
OK: ...line.
Manual install
- Make sure the bootstrap config has an admin block on a private address.
admin:
address:
socket_address: { address: <private-ip>, port_value: 9901 }- Restart Envoy (hot restart or the service).
sudo systemctl restart envoy- Check the metrics.
curl -fsS 'http://<private-ip>:9901/stats/prometheus?usedonly' | grep -q '^envoy_server_live' && echo 'OK: Envoy metrics ready'- Spitfire → Infrastructure sources → Envoy; address:
<envoy1-ip>:9901,<envoy2-ip>:9901.
The admin interface can stop Envoy (/quitquitquit). Open port 9901 to the Tower only.
Traefik
Quick install
- Run the command on every Traefik host.
curl -fsSL "https://<spitfire>/api/v1/setup/ds/traefik.sh?t=<token>" -o /tmp/sf-traefik.sh && bash /tmp/sf-traefik.sh- You are done when you see the
OK: ...line.
Manual install
- Add these flags to the static config (CLI or the container
command).
--entrypoints.metrics.address=:8082
--metrics.prometheus=true
--metrics.prometheus.entryPoint=metrics
--metrics.prometheus.addEntryPointsLabels=true
--metrics.prometheus.addRoutersLabels=true
--metrics.prometheus.addServicesLabels=true- Restart Traefik.
sudo systemctl restart traefik- Check the metrics.
curl -fsS http://127.0.0.1:8082/metrics | grep -q '^traefik_config_reloads_total' && echo 'OK: Traefik metrics ready'- Spitfire → Infrastructure sources → Traefik; address:
traefik1:8082,traefik2:8082.
The API entry point (8080) works too instead of a separate one; the address is then traefik1:8080.
Panels and findings
| Source | Finding | Threshold |
|---|---|---|
| All | Node down | up = 0 |
| All | 5xx ratio (Kong service, Envoy listener and cluster, Traefik router/service/entry point) | > 1%, at least 20 requests |
| Kong | Database unreachable | kong_datastore_reachable = 0 |
| Kong | Unhealthy upstream target / every target of an upstream unhealthy | - |
| Envoy | Not live, cluster without a healthy member, unhealthy member, open circuit breaker | - |
| Envoy | Rejected config (CDS/LDS update_rejected) | 15 min |
| Traefik | Service with no server up, server down | - |
| Traefik | Last config reload failed (v2 only) | - |
| Traefik | TLS certificate expiring | < 14 days |
p95 values are computed from histogram buckets over a 5-minute window.
Uninstalling
For each gateway, first delete the source in Spitfire: All integrations In Spitfire: /admin/integrations#observability →
the bin on the source's row → Delete. The Tower stops reading; collected metrics stay. An interrupted run can
leave a state file in your home directory: rm -f ~/.spitfire-setup/kong.state (the same for envoy, traefik).
Kong
On a Kong with a database, the script adds the global Prometheus plugin through the Admin API (it leaves an
existing one alone) and changes no config file. If the script added the plugin and nothing else reads /metrics,
delete it on one Kong node (once per cluster; needs jq: apt install jq / dnf install jq). It should print
OK: plugin deleted.
for id in $(curl -fsS 'http://127.0.0.1:8001/plugins?size=1000' | jq -r '.data[] | select(.name == "prometheus") | .id'); do curl -fsS -X DELETE "http://127.0.0.1:8001/plugins/$id"; done; curl -fsS 'http://127.0.0.1:8001/plugins?size=1000' | jq -e '[.data[] | select(.name == "prometheus")] | length == 0' >/dev/null && echo 'OK: plugin deleted'On a DB-less Kong, or with the manual install, you added the plugin and the status_listen = 0.0.0.0:8100
line (or KONG_STATUS_LISTEN) yourself: delete the - name: prometheus block from kong.yml and the line from
kong.conf, then run sudo kong reload -c /etc/kong/kong.conf. The Status API can stay if something else uses it.
Envoy
The script changes nothing in Envoy; deleting the source is enough. If you added the admin block only for Spitfire, delete it from the bootstrap, restart Envoy and close 9901 in the firewall.
Traefik
The script changes nothing in Traefik; deleting the source is enough. If you added the --metrics.prometheus
flags only for Spitfire, delete them from the static config and restart Traefik.