Events

Test → Events lists, in one place, every event Spitfire raises and every event your own tools send:

• synthetic monitors going down and recovering;
• APM alerts, SLO burn alerts, new and reopened error groups and regressions;
• comparison regressions, failed or missed scheduled runs;
• connection health (a data source or issue tracker that stops answering) and failed Sweeper runs;
• alerts your monitoring tools post to a webhook input;
• events posted to the Events API.

Who gets notified does not change: channels and issue trackers still get exactly what they subscribed to. The list is for seeing everything together and for the team to say who is on what. The bell (notifications) is separate.

To suppress, route, escalate, deduplicate or resolve events by rule, use Event policies (Test → Events → Event policies).

#One problem, one event

Each event has a key (dedup key) made from its source, type, service, environment, host and resource (or the dedupKey you send).

• While an event with that key is open or acknowledged, a repeat does not add a row: it raises the count and the last seen time, and the severity if it changed.
• A recovery (monitor up again, alert resolved) resolves the event.
• A problem within 10 minutes of the resolve reopens the same event; later it opens a new one.
• Informational events (a deploy, a passed comparison) are recorded once each.

Severity, from high to low: critical, major, minor, warning, info, ok.

#Filters

1. Status: in the bar above the list, Live (open + acknowledged, the default), Open, Acknowledged, Suppressed, Resolved in 24 h or All. The number beside each is its event count.
2. Search: words from the title or the message, or part of the key.
3. The Filters column (on the left):
   • Severity: click a value to list it and above; Major shows Major and Critical. Click again to remove it.
   • Last seen: 1 hour, 6 hours, 24 hours, 7 days, 30 days or Any time.
   • Tag: team=pay or only region, then Add. Every tag must match.
   • Source, type, service, host, environment: tick values. The counts follow the other filters.
4. The filters in use show as tags above the list: remove one with ×, or Clear all.
5. To hide the column, the « button in its heading; Filters opens it again. On a phone Filters slides the column in.

The filters stay in the address: copy it to share the same view.

#Ack and resolve

• Ack (acknowledge): tells the team someone is on it. The event stays live; repeats still count.
• Resolve: closes it by hand. A new problem with the same key within 10 minutes reopens it.
• Reopen: on a resolved event, when its key has no other live event.
• Note: anything you found; it goes to the timeline. Ack and Resolve take the note too.

On the list, tick rows (the header box ticks the whole page), then Ack or Resolve. Any workspace member can do this; each action goes to the audit log.

The event page shows its fields, tags (click one to filter by it), links to where it came from (the monitor, the alert, the run…) and the timeline: opened, severity changes, ack, notes, resolve and reopen, with who did it.

#Events from monitoring connections

The Zabbix, Datadog, New Relic, Dynatrace and Instana connections under Infrastructure → Observability (the monitoring events module) are read once a minute and their open problems land here:

• The source is connector:<kind> (like connector:zabbix); the list shows the connection kind's name. The resource is the connection's name; the tags connection, connector, problemId and, when there is one, priority are added.
• A problem open in the monitoring system is an open event; one closed there, or no longer read, makes the event resolved. A problem that stays the same is counted again every 15 minutes.
• The read only reads, with the same calls a run's correlation makes; nothing changes in your monitoring system.
• A connection that fails waits 1, 2, 4 … at most 30 minutes before it is tried again; the others are not affected.
• These events send no notices on their own: to be notified, add a route with an event policy for the source connector:*.

#Groups and flapping

• An event that joined a correlate policy's group shows In a group in the list, the group's first event Group: N more. The parent's page lists the events in the group; a child's page links to its parent.
• An event that opens and closes too often and is caught by a flap policy shows a Flapping badge and the tag flapping=true. Its page shows how many openings were seen and when it counts as stable.

#Events API

Post your own events with an API token (or a session). Run from any folder:

curl -s -X POST -H "Authorization: Bearer $SPITFIRE_TOKEN" -H "Content-Type: application/json" \
  -d '{"source":"deploy-bot","type":"deploy.failed","severity":"critical","title":"deploy api failed","service":"api","env":"prod","tags":{"team":"pay"}}' \
  "https://spitfire.example.com/api/v1/events"

The answer is 202 with each event's id and outcome (opened, repeated, reopened, resolved, recorded). To close it, send the same event with "status":"resolved" (or "severity":"ok").

One request carries one event, a list, or {"events": [...]}: up to 100 events and 1 MB. A field error answers 400 with the field and the event's index. Each workspace may post 10 events a second (bursts of 120); over that the answer is 429 with Retry-After.

#How long events are kept

Counted from the last time an event was seen, deleted once a day: