Spitfire 1.4 release notes

latest

Quick install: in Data sources → Add, kinds with a script open on a one-line command (bash for Linux/macOS, PowerShell for Windows). The script creates the read-only monitor user, shows each command first, asks before changes such as a restart, tests the connection and adds the source to Spitfire itself. The setup token is valid for 1 hour and single use; the admin password is stored nowhere. SF_YES=1 installs unattended. The "Manual install" tab keeps the copyable steps.
Windows setup wizard: a "Next, Next" install with Spitfire-Setup.exe; it checks Docker Desktop, WSL 2 and the ports, streams install.ps1's output and adds Update/Uninstall to the Start menu. Spitfire.msi installs silently for Intune/GPO. No administrator rights needed; the files are not code-signed yet (SmartScreen: More info → Run anyway).
macOS installer wizard: drag Spitfire.dmg into Applications and install with Continue; opened again it updates, repairs or uninstalls. Unsigned: on first open use System Settings → Privacy & Security → Open Anyway.
End of install: the lines after the success box are numbered; the setup code and the License line are orange, so the free key needed before the first run is not missed.
Uninstall (--purge) now also removes the APM ClickHouse container, volume and network.
All metrics tab: every data source's metrics, searchable and grouped by prefix; draw one as a rate or value, filter and split it by attribute, and pin it to the dashboard with a threshold.
Kubernetes cluster view: node, namespace, workload, pod and container tables with use against requests, limits and allocatable; findings for OOMKilled containers and CPU/memory at their limits.
Servers: a native agent for Linux, Windows and macOS (one-line install); CPU, memory, swap, file systems, disks, network and services; findings for full file systems, load, stopped Windows services and failed systemd units.
Databases: top queries (literals hidden) and sessions waiting on a lock on PostgreSQL, MySQL, SQL Server and Oracle; cache hit ratios; Kafka JMX (broker rates, under-replicated partitions, controller, heap/GC) and consumer lag; Redis slowlog and replicas, MongoDB profiler and replica set, RabbitMQ queues without consumers, the Memcached slab view.
Elasticsearch: shard states, pending tasks, JVM heap, free disk, indexing/search rate and latency, thread pool rejections, circuit breakers; a largest indices table.
Nginx and HAProxy: the Nginx JSON access log (no module, no agent) gives status classes, p95, upstream and peer tables; HAProxy backend/server/frontend tables; a backend with no server UP is a critical finding.
Cloud: AWS, Azure, Google Cloud, vCenter (read-only account, read by the Tower).
Docker (container view, grouped by Compose project).
InfluxDB 2.x/3.x, ClickHouse, Couchbase, Cassandra, OpenSearch.
SNMP network devices (v3 first; interfaces, resources, sensors and traps per device).
Java application servers: Tomcat, WildFly / JBoss EAP, WebLogic, WebSphere Liberty / Open Liberty and any JVM (JMX). Traditional WebSphere is not supported.
Coordination: ZooKeeper, etcd, Consul (leader, quorum, raft, WAL fsync p99, DB quota).
ActiveMQ Classic and Artemis (over Jolokia; queue depth, DLQs, queues without consumers).
IIS (through the Windows agent; app pool state, request queue).
NGINX Plus peer states.
A Search the menu box in the left menu: it stays at the top while the menu scrolls; type a few letters of a page's name, Enter opens the first match, Esc clears it.
Test editor: the start delay and the load stage durations are entered as a number + unit (ms, s, min, h); no need to type "1s". The definition still stores 30s, 2m and so on; the API is unchanged.
Scenario names may contain Turkish letters and hyphens (-) (checkout, add-to-cart).
Validation errors are red now: the box at the top is red, the invalid fields get a red border and the message sits right under the field.
Install: the "save the log to a file?" prompt is orange and the install ends with "Happy testing!". Uninstall (without --purge) says the installation folder stays and prints the command that deletes everything.
The collapse button and short names such as APM are left-aligned in the left menu.
Sweeper: a browser runner crawls your site and records on each page the status, TTFB, load and LCP times, console errors, exceptions, failed requests, broken links, and blank or slow pages. It only makes GET requests, fills in no form and clicks nothing; logout-like paths are skipped. Results show as summary cards and a filterable findings table; in CI `spitfire cloud sweep "Name"` fails the job by max errors and max slow pages; the REST API and OpenAPI have it too. A run that does not pass goes to notification channels with the `sweep.failed` event. Without the module a sweep opens 50 pages; the Sweeper module (Enterprise, an add-on or the 7-day trial) allows up to 5000 pages and schedules.
Test editor: the step name sits in a dashed box with a pencil icon, so it is clear it can be clicked and changed.
Shared headers: headers that are the same in every request, like Authorization or X-Tenant, are written once and sent with the HTTP, GraphQL, WebSocket and SSE steps of every scenario; values may use variables. When the same header is written in several scenarios, the editor offers to move it to the shared area.
Standard header names are suggested while you type (Aut → Authorization); a name not in the list can be typed too.
URL fields drop leading and trailing spaces; a space left at the end no longer becomes %20 and a 404.
A Run button next to Edit in the tests list: it opens the test with the run dialog open.
Mock service templates: SendGrid, Twilio IVR, SMTP (over HTTP), Slack, Teams, the Turkish central bank's FX rates and the fictional ABC Bank virtual POS.
Explanations: an orange "i" for the ramp-down wait, max redirects, graceful stop and the share of the test's load (formerly load scale); the pacing and comparison tolerance texts are simpler and have examples.
The HTTP status is labelled "Status code" everywhere.
The "Data sources" menu is now called "Infrastructure"; each entry is a "monitored system". The API and the license are unchanged.
Integrations: the public address hint moved below the form and the language select is aligned.
Login & token: for protected APIs the login request is defined once; each VU signs in, takes the token from the response (JSON, XML, a header or a cookie) and adds it to every request as Authorization: Bearer {{token}} or a query parameter. It signs in again on a timer, when the token expires or on 401/403, and sends the request once more. The login is measured in its own Login row and the run page counts logins, refreshes and retries. There is a Try the login button, a manual token mode and a Use this step as the login offer.
Public steps: a step ticked Public · no login needed on the More tab gets no token and does not sign in again on 401; the login runs just before the first protected step.
Import: OpenAPI securitySchemes (operations with security: [] become public steps) and Postman collection auth move into the Login & token block; Postman pre-request scripts that sign in get a note.
Reading a document, a CSV or a traffic log shows a spinner and a label such as "Reading the document…" on the buttons, which stay disabled until it is done.
A token entered by hand and a password or client secret typed in an import are stored encrypted: the editor shows them masked, and they are not in the test JSON, an export or the CLI output; paste a new one to replace it.
New Environment tab on the run page: the Kubernetes clusters (nodes, pods, pods not ready, restarts during the run, CPU/memory average and peak), hosts (cores, memory, disk, CPU/memory average and peak), DB/MQ/cache data sources (endpoint, version, key numbers) and services the system under test ran on during the run, with a map from the load generator to the services and their DBs, caches and MQs. It is stored 3 minutes after the run ends and added to the HTML/PDF report as an Environment section; a source that cannot be read shows with its reason and never affects the run.
Report e-mail: Report → E-mail the report… sends the report in a standard e-mail; your own address is filled in, up to 20 recipients can be added, the PDF is attached (Word and HTML on request) and a note can be added; every send goes into the audit log. With "E-mail the report when a run ends" switched on for a test, the report goes out by itself after every run (or only after failed runs). The Report menu also downloads Word (.docx), and the Result card on the run page has a verdict badge and p95, p99, error rate and requests/s tiles coloured against the thresholds.
Distributed tracing and errors: clicking a span shows every detail in the panel on the right (duration, self time, share of the trace, copyable IDs, parent/children, grouped attributes, DB statement, events, exceptions with their stacktrace, the span's logs, links to the host, service, error group, Kubernetes and Docker); a trace opens on its first failed span and is shared with ?span=; ↑ ↓ to move, folding, only errors, next error, critical path, service colours, span search, a table view and a smooth waterfall up to 10,000 spans. Traces filter by operation and 4xx. The error page highlights in-app frames, marks unhandled exceptions and regressed groups, shows the attributes common to every occurrence and a see-traces link; new apm.error.new and apm.error.regressed notifications. Every APM table sorts by its column headers and has a search, and the link keeps the sort.
Notification channels (e-mail, Slack, Teams, IVR, SMS, webhook) are free and unlimited on every plan, Free included: the count limit, the "Paused (license)" badge and the limit notice are gone; a channel limit in an older key is ignored. Issue-tracking channels (Jira…) stay an add-on, in their own sub-section.
New IVR (voice call) channel: the phone rings and the event is read out (TR or EN). You bring the provider: a generic HTTP voice API or a Twilio Voice style preset ({twiml}, {lang}); a Make a test call button, a preview and at most 5 calls an hour. A new channel calls for critical events only: failed runs and a monitor going down.
Add-on trials: the trial button names the module ("Try the issue-tracking add-on for 7 days"); when the license holds the module or one of its connectors there is no trial or lock banner, only "In your license: Zabbix, Datadog".
APM / Kubernetes: every level from the cluster through namespace, node, workload, ReplicaSet and pod to the container, with state, ready, restarts, age, CPU and memory against request and limit, network, charts and the APM services that ran there; tables sort by column and filter, the selection stays in the address, and there is a what-runs-on-this-node view and a topology drawing. APM / Docker: containers by Compose project with their charts and APM services.
Agents: the Infrastructure → Agents page lists every agent and Tower with its version, status, last report, points per second and queue; it sorts, filters and can show outdated agents only. An agent's page has process CPU/memory charts, goroutines/threads/open files, a config summary, the update history and a live log (level and text filter, pause, copy, download the last N lines). The log level defaults to error; debug goes back by itself when the fleet's time is up. Restart and Update now go to one agent or to the selected ones after a confirm; auto-update and its window are set as a fleet default or per agent. No remote shell: allow-listed commands only, install admins only, every action in the audit log. An older agent becomes managed when the install-collector script is run again.
Connection health: observability connections such as Zabbix, Datadog or Prometheus, the data sources a Tower collects and issue trackers such as Jira are recorded on every call (data pull, event read, write-back, zabbix_sender, Test, opening a ticket) as answering or not; the ones no call reached for a while are checked every 5 minutes. The Observability, Infrastructure and Integrations pages show at the top the count of connections not answering with links to them, the lists a Health badge, the last OK time and the last error (passwords, tokens and the user info in URLs hidden); a click on the badge opens the state history and the notify settings. After 3 failed calls in a row (settable from 1 to 20) the connection is Down and notified once, and once again when it answers: to the chosen channels or the workspace's people channels (`connection.down`, `connection.recovered`) and in the installation admins' bell.
Docker: a container's CPU is now worked out from the rise of the Collector's CPU counter, so a container limited with --cpus 0.2 shows 20% (the old value divided the CPU used since the container started by the host's since boot and came out far too low). Reinstall the Docker host's Collector with the command on Infrastructure to get it.
Sweeper Interactive mode (staging only): besides Read mode (the default, unchanged), a sweep also tries every page's buttons, menu items, tabs, checkboxes and radios, selects and forms in the real browser; after each interaction it looks for a new console error, an exception, a 4xx/5xx from the sweep's hosts (400/422 after a submit count as validation), a blank page and a click that did not reach the element. Each problem is an Interaction failed finding with the element's label, selector, action, result and a screenshot, and counts as an error. Every request to another host (payments, mail services…) and every sign-out request is stopped in the browser and listed on the run as "not sent"; the skip list (pay, öde, satın al, checkout, send, gönder… editable), log out and danger buttons are never clicked. Saving asks you to tick that the hosts are staging/test (who and when is stored); an interactive sweep cannot be saved or started on a host a test's production-named environment points at, a www. host needs a second tick, and every start goes to the audit log. The run page has an Interactions card: tried, failed, skipped (with the reason) and the requests not sent. 20 interactions a page without the module, 500 with the Sweeper module. `spitfire cloud sweep` and its exit codes are the same; the REST API and OpenAPI have the mode and the interactive settings.
Fix: `spitfire cloud sweep` with a personal API token got 403 api_token_scope; a token may now start and stop a sweep (the staging sweep in CI).
APM / Kubernetes and APM / Docker tables sort like the other APM tables: a header click sorts numbers high first and text A–Z, empty values stay last both ways, and the order is in the address as `<table>_sort` and `<table>_dir` (old `sort_<table>` links open unsorted).
Tests that write to a database: the consent dialog for SQL/Mongo statements that change data now opens when saving a schedule, a monitor or starting a comparison too (it used to open only on Run, the others showed the raw error). The consent is recorded for the statements of the request's environment, with who, when and the warning text, and the request is sent again on its own after the consent.
Saving a schedule checks the write consent for the schedule's environment (each arm's for a comparative schedule) instead of the test as saved.
A run, schedule or comparison in an environment that maps a step's connection to another one was refused with "no step uses connection"; fixed.
SQL bulk script: transaction statements such as BEGIN, COMMIT, ROLLBACK, SAVEPOINT and START TRANSACTION no longer count as writes or show as red "other write"; they are skipped as session statements. A step such as BEGIN; INSERT …; COMMIT now shows as INSERT.
HAR and recording import: when the base URL has a path (e.g. https://staging.example.com/api), the request path is no longer doubled (/api/cart, not /api/api/cart).
Help when you stop on a form: in the test editor, the APM Add a service wizard, schedules, infrastructure and observability connections, notification channels and the dialog that adds a monitored system, if you change a field and stop for 20 seconds (5 seconds after the same error twice, clearing the field and typing again, or opening and closing its "i" tip) a small bubble opens next to the field: its short tip, a Read in the docs link (to the connection kind's own section) and an Ask box. It never opens while you type, shows once per field per session and never on the wallboard; Don't show again turns it off and the switch at the bottom of the assistant turns it back on. It all runs in the browser and sends nothing.
APM alerts (APM → Alerts): rules with warning and critical thresholds on a service's (optionally an environment's and an endpoint's) p95, p99, p50, average duration, error rate, request rate drop/spike, new or reopened error groups. Evaluated every minute; pending → firing when the value stays over a threshold for the set minutes, resolved when it is back under. Minutes under the minimum request count count as no data; with a re-notify interval the alert is sent again while it lasts. Every state change goes to the history (90 days) and the audit log.
"Add alert" and "Add SLO" on the service page: the thresholds come from the last 24 hours (warning p95 × 1.5, critical p95 × 3; the error rate and the SLO objective from the day's values too).
Notices go to the chosen channels (with none chosen, to the channels that picked `apm.alert.firing` / `apm.alert.resolved`) with the value, threshold, chart, slowest/failed traces and a "Why did it break?" link; issue trackers get an issue and a comment when it resolves. With writeback on, your monitoring tool gets an `apm_alert` event (default level critical). The new events are opt-in, not selected by default.
Silences and maintenance windows: for a rule, an SLO, a service or all alerts, starting now or later. They mute the notice, not the evaluation; an alert still firing when the silence ends is sent then. One-click silence for 1 hour in the rules table.
SLOs (APM → SLOs): an availability or latency (threshold 25 ms – 10 s) SLI, an objective (%) and a 7, 28 or 30 day window (default 30). Error budget left and burn rate charts; fast (1 h + 5 m, 14.4×) and slow (6 h + 30 m, 6×) burn rate alerts (`apm.slo.burn` / `apm.slo.recovered`). A new per-minute rollup for the latency counts; older traces are counted back once.
"Make an SLO from this p95" on a capacity (breaking point) run: the p95 of the highest step that held becomes a latency SLO's threshold.
Plan limits: Free 3 alert rules + 1 SLO, Starter 20 + 5, Growth and up unlimited. After a downgrade no rule is deleted; the oldest ones keep being evaluated.
REST API and OpenAPI: `/api/v1/apm/alerts`, `/api/v1/apm/alerts/events`, `/api/v1/apm/alerts/suggest`, `/api/v1/apm/silences`, `/api/v1/apm/slos`. CLI: `spitfire cloud apm alerts` and `spitfire cloud apm slos`; `--fail-on-firing` exits 99 when a rule fires or an SLO burns or has no budget left (a check before a deploy or a load test).
Docs: a new "APM alerts and SLOs" page.
APM › Logs: every log that arrives over OTLP on one screen. A time range (the buttons or a custom from–to), level, service, environment and host filters on the left; the search box finds text inside the message (case does not matter, `/` jumps to it), and a trace id or span id brings that trace's logs. Attribute filters `key=value`, `key!=value`, `key=*` (has it) and `key!=*` (does not); keys and values are suggested.
Logs by level chart: bars stacked by level; click a level to add it to the filter, drag over the chart to narrow the range to that part.
An opened log row shows its attributes, resource, severity, trace and error group; the trace badge opens the waterfall with that span picked. A new Logs tab on the service page, See logs on an error group and Open in Logs on the trace view.
Patterns: logs of the same shape (numbers, ids, addresses and quoted values masked, the same normalisation as error groups) on one row, the biggest first; click a pattern to list its logs. Logs written before 1.4.4 are grouped from a sample and cannot be picked.
Live: new logs are added on top every two seconds, the last 500 kept; Pause / Resume.
Saved searches are shared in the workspace (deleted by whoever saved them or an admin); Copy link opens with the same range and filters. CSV and JSON export at most 10,000 rows; formula cells are defused in CSV and every export goes to the audit log.
Full-text search uses a ClickHouse ngram skip index; existing installs get the index and the pattern column on start. A note shows when the range reaches past retention.
REST API and OpenAPI: `/apm/logs`, `/apm/logs/histogram`, `/apm/logs/facets`, `/apm/logs/patterns`, `/apm/logs/tail`, `/apm/logs/count`, `/apm/logs/export`, `/apm/logs/searches`. `/apm/logs/count` returns the search's canonical query (ready for log-based alerts).
"Alert on this search" on the Logs page: an alert when the search's records in a window (1–60 min, default 5) go over a threshold, or when the search shows a log pattern the 7 days before did not have. The thresholds are suggested from the busiest window of the last 24 hours (× 1.5 warning, × 3 critical). Log alerts have no "for" period and no minimum of requests; service and environment are optional and narrow the search. The rules table writes the search out and links to the Logs page; notices bring the search, an Open the logs link over the alert's range and the new patterns with examples. Log rules count toward the same rule limit. REST: `GET /api/v1/apm/alerts/suggest-logs`, `logQuery` in the rule body.
Events (Test → Events): every event Spitfire raises (synthetic monitors, APM alerts, SLO burn alerts, new and reopened error groups, APM regressions, comparison regressions, scheduled runs, connection health, Sweeper) and the alerts posted to webhook inputs, in one list. Who gets notified does not change; channels and issue trackers still get what they subscribed to.
Deduplication: while an event with the same key (source, type, service, environment, host, resource, or the dedupKey sent) is open, a repeat does not add a row; it raises the count and the last seen time. A recovery resolves the event; a problem back within 10 minutes reopens the same event, later it opens a new one.
Severity: critical, major, minor, warning, info, ok. Sources' own levels (Zabbix, Grafana, Alertmanager, numbers 0–5, words like error/warn/fatal) are mapped to this scale.
The list: status (live, open, acknowledged, suppressed, resolved, all), minimum severity, search, tag (`key=value` or `key`) and source, type, service, host and environment filters on the left; counters on top for open, acknowledged, suppressed, resolved in 24 h and live events per severity. The filters stay in the address.
Ack, resolve, reopen and note; bulk ack and resolve on the list. Any workspace member can do this, each action goes to the audit log. The event page shows the fields, tags, links to where it came from and a timeline of who did what.
Events API: `POST /api/v1/events` also takes an API token; one event, a list or `{"events": [...]}` (up to 100 events, 1 MB). A field error names the field and the index; 10 events a second per workspace (bursts of 120), over that `429` with `Retry-After`. Reads: `/api/v1/events`, `/events/facets`, `/events/counts`, `/events/{id}`.
Retention by edition: Free 7 days, Starter 30, Growth 90, Scale and Enterprise 365; counted from the last time seen, deleted once a day.
Docs: a new "Events" page; posting events in the API reference.
Event policies (Test → Events → Event policies): ordered rules run when an event opens or reopens. Conditions: source, type, service, environment, host (`*`/`?` patterns), severity (at least or one of), tags (=, ≠, present, absent), title/message text and a time window (Business hours, Outside business hours, Night, Weekend; with a time zone). Actions: suppress (with a maintenance window), drop, set severity, enrich (tags, owner, team, runbook, note), dedup fields, route (replace/add), escalation (step 2 and 3 without an ack), renotify, auto-resolve, tracker, webhook, writeback.
Reorder by dragging or with the arrows, turn on/off, Stop here; every save is a new version and History shows who changed what and when. Of two edits at once the second save is refused.
Try: the draft runs over the last 7 days of events (up to 5000) without saving; the matched count, the effect per action and up to 20 samples. Nothing changes and no notice goes out.
The event page shows the Policy trail (policy, version, action, detail), the notice plan (channels, escalation step, next action) and notified, escalated and notified again in the timeline; ack/resolve by link and by policy.
Signed ack / resolve links: with the Public address set, problem notices carry links that need no sign-in, last 7 days and work once; JSON webhooks get `eventLinks`.
Drop is off by default; a workspace admin turns it on with Allow drop after a confirm. While it is off a drop action suppresses the event.
Licence: Free 3, Starter 20 policies; Growth and above unlimited.
Docs: a new "Event policies" page (copy-paste examples: night mute, escalation, maintenance window, dedup without the host, auto-resolve).
Events from monitoring connections: the open problems of the Zabbix, Datadog, New Relic, Dynatrace and Instana connections are read once a minute and land in Events (source `connector:<kind>`, resource the connection's name, tags `connection`/`connector`/`problemId`/`priority`). A problem closed in the monitoring system, or no longer read, resolves its event. It only reads; only the connections the monitoring events module covers are read; a failing connection waits 1–30 minutes before it is tried again and does not hold up the others. Notices go out only through a policy's route.
Correlate action: events with the same values of the fields picked (service, host, environment, resource, source, type or tags) within a window (1–1440 min) are grouped under the first one; the group's first event is notified, children send no notice, escalation or writeback. In a group / Group badges in the list, related events and a link to the parent on the event page.
Flap action: an event that opens the given number of times (2–50) within the window (1–1440 min) is flapping (tag `flapping=true`); while it flaps no notice, escalation or renotify goes out, once the window passes without a change it stops and, if still open, one notice goes out. The event page shows the openings and when it counts as stable.
Try also shows the effect of correlate and flap (how many events would join a group and how many would flap).
APM error and regression events now land in Events for workspaces without a channel subscription too.
Docs: connection events, groups and flapping on the "Events" page; Correlation and Flap on the "Event policies" page.
The Splunk connection can now be bought on its own: like the other connections of the monitoring events module, 8,000 TL a year (1,000 TL a month) or $210 ($26 a month). The whole module still turns on every connection; Elasticsearch, OpenSearch and Graylog stay open on every edition.
Event policies: a passed run with no open problem before it is no longer a recovery, so a policy muting Warning/Info no longer swallows the run-finished notice at night. The docs now warn that policies apply to Spitfire's own notices too.
New synthetic monitor dialog: the threshold boxes no longer shift under longer labels, they line up.
Event page: a link to a run, Sweeper run, monitor, comparison or test deleted after the event (by hand or by retention) reads "deleted" and is not clickable instead of leading to a not-found screen. A deleted Sweeper run shows the not-found screen instead of an error box.
Tower no longer stores data on disk: exports from Collectors wait only in a small in-memory queue, like the OpenTelemetry batch/exporter queue (64 MB by default, at most 60 seconds). The old 1 GB / 24 hour disk buffer is gone; on upgrade the `buffer` folder in the state directory is removed.
If the controller cannot be reached for 60 seconds the queue is cleared and new exports are dropped on arrival, with no retries. Forwarding resumes by itself when the link is back. Collectors are always answered at once; the response time of monitored services is never affected.
New settings: `--queue-mb` / `SPITFIRE_TOWER_QUEUE_MB` (queue size, 1–256 MB) and `--drop-after` / `SPITFIRE_TOWER_DROP_AFTER_SECONDS` (how long unreachable, at least 5 s). The old `SPITFIRE_TOWER_BUFFER_MB/_HOURS` are ignored. The install scripts accept `--buffer-mb` / `-BufferMB` as an alias.
Towers page: a "Queue" column; the dropped count includes exports dropped while the controller was away.
Every Collector config Spitfire generates (host, Docker, Kubernetes, logs, managed) comes with an in-memory, non-blocking sending queue (200 batches) and retries for at most 60 seconds; the app snippets show the same settings.
Every report opens with a **Brief**: the verdict (PASS, BORDERLINE, FAIL), what was measured, the change against the previous one, a likely cause only when there is evidence (certain or likely) and what to do. It also shows as a card on ended run and comparison pages and in the SLA card.
One format matrix: run, comparison and monthly SLA reports download as HTML, PDF, Word (.docx), CSV, Markdown (.md) and JSON (`?format=` works on the report addresses and share links too). The **Report** menu is the same everywhere: **Download**, **Send to a channel…**, **Schedule…**, **Share link…**.
**Send to a channel**: the report goes to one or more notification channels. E-mail, Slack (with a bot token), Telegram (a new channel kind) and Jira get the files attached; Teams, other trackers, SMS and JSON webhooks get the brief and 7-day share links. Each channel's outcome is shown.
**Report schedules** (Test → Report schedules): a period report of a test, a group, a tag, the monitors or the monthly SLA goes to channels on its own, daily, weekly or monthly, at the hour and time zone you pick, compared with the period before. **Send now** tries it. Free 1, Starter 5, Growth and up unlimited.
New help page: Reports and sharing.
**Roles**: built-in Admin, Editor and Viewer (read-only) roles. The install admin defines custom roles on the Roles page by choosing hidden / read / edit for each area. A hidden area leaves the menu, a read-only area turns its edit controls off, and the API follows the same rule. Only the install admin creates users and roles; workspace admins assign roles to members. Existing "user" members become Editor. SSO maps IdP groups to roles.
**Built-in policies**: 31 ready rules for Kubernetes, Docker, Linux, Windows and network (SNMP) (host down, CPU/memory/disk full, pod restarting, node not ready, interface down…). Each can be turned on or off, its threshold and duration changed, any number of channels picked, and "Reset to default" undoes changes.
Events list: a **Source** column (logo + name, click to filter by it) and a **Groups** view: events from different sources on the same host or service within 10 minutes gather in one card, cutting noise.
**Pause notifications**: the install admin can stop all alarm notifications (optionally until a time); events are still recorded, and notices skipped during the pause are not sent later. A banner shows on every page while it is on.
Notification deliveries now go through a durable queue: a failed delivery is retried after 30 s, 2 min and 10 min; after the last try it closes as "Closed by the system" and a channel-unreachable notice goes out once.
**Desktik channel**: pick Desktik in an event policy's Tracker action and Desktik opens one ticket per event; ack, resolve, reopen and severity changes go to the same ticket. Ack and resolve done in Desktik come back to Spitfire (an `events:write` API token that can only write events). Sending to the Desktik cloud is off by default; when on, only the ticked fields go.
New kind picker on Observability, notification channels, data sources and connections: a category menu on the left, search and icon cards, each with a short description.
**Test** in the connection dialogs before saving; every connection test shows the same result view. Query connections take extra HTTP headers and show a "Spitfire connects from" line (egress IPs).
New preparation page: Zabbix. "How to prepare <kind>" links in the dialogs.
The new-release banner is simpler: a white card with an amber stripe and readable text; release notes open on the site, "How to update" shows only the commands. With one-click update on, the banner has **Update now**; on Docker with it off, a link to turn it on.
**Collectors** in the APM menu (install admins): status, machine details, live log and log path. The APM host page links to that host's Collector.
New help pages: Built-in policies, Preparing Zabbix; roles in Users and access.
Forms trim spaces, line breaks and invisible characters (zero-width space, BOM, no-break space) pasted around values: connections, SSH tunnel, observability, data sources, notification channels, SMTP, OTLP, SSO/LDAP/SAML and SIEM streams. Values saved earlier are trimmed when used too. Passwords are kept as typed.
Dialogs scroll again on short screens: in the connection, policy, alert, SLO and Sweeper dialogs **Save** and **Cancel** were below the window. The kind picker's category menu scrolls on its own when it is taller than the window.
Type badges are white with brand-coloured icons: the Connections list and supported types, test steps, dry run, observability and notification channels.
Observability's log collector sources use the new kind picker too: category menu, search, icon cards.
The **Desktik channel** works on Desktik's events API: an events-only Desktik key (`dtk_…`) is enough, no Spitfire token needed. Turn on **Settings → Monitoring events** in Desktik; acks and resolves done in Desktik are read every 30 seconds and applied in Spitfire. The product field is gone.
**Copy** buttons sit beside the command instead of on top of it: Kubernetes, Docker, IIS, SNMP, update commands, CI snippets and the Prometheus config.
The source list and the **Add source** catalog in Infrastructure monitoring, and the connection list and the **Add connection** catalog in Observability, are separate pages.
**Automatic update:** when an installation admin turns it on, a new release installs without waiting for a click. The signature check, waiting for runs and the database backup still happen; every update is written to the audit log.
The Kubernetes data source page shows the cluster first: workloads not ready, pods not running and the pods with the most restarts. Chart legends are one row with short names.
Failed spans are red in a trace; the **Errors: N** badge jumps to the first one.
Topology boxes are wider so labels no longer overlap; a legend explains the colours and lines.
**Events, Logs and APM Errors** share one filter layout: a fixed filter column (can be hidden, opens with **Filters** on narrow screens), a full-width list, selected filters as chips and **Clear all**. Events get a status bar, severity and last-seen chips; logs are one line each and open in full on click.
**Event policy templates:** searchable, categorised **Start from a template** cards in the **New policy** dialog. Open policy counts on the tabs and in the menu; a link to the built-in policies.
APM Services shows recommended tests as one line above the table, a count badge on each service and **Show all** opens a searchable list grouped by service.
NATS brand icon on connection type badges.
**APM live refresh:** a time range and refresh picker at the top right of APM pages: Off, 5s, 10s, 30s, 1m, 5m, 15m (1m by default on relative ranges). The choice stays in the address; it waits while the tab is hidden, a dialog is open or you are typing.
**Kubernetes detail:** every count in the cluster summary opens its list, every row opens its detail. A deployment shows how many pods are up, how many are not and why (CrashLoopBackOff, ImagePullBackOff, Pending…); each pod has **Why did it restart?**: reason, restarts in the last hour with their time, memory against the limit before the restart, k8s events and log links.
**Table filters:** Turkish-aware search (İ/ı) on the main tables, filters by column value (with counts), date and number ranges, selected filters as chips, **Clear** and "N / M shown". Filters stay in the address; **/** jumps to search.
**New preparation guides:** connecting Kubernetes (Tower inside the cluster, Pod Security restricted, Secrets from hidden prompts), an OTLP sender guide on the Observability page (verify command, arrival test, receiver health) and PostgreSQL + SSH tunnel (generate the tunnel key, preparation commands from the form, layered connection test).
**Values form in the guides:** type the Spitfire address, Tower, token, service name and similar values once at the top of a guide; they go into every command on the page and into what Copy copies. Install, backup, restore and support commands use full paths and run from any folder; the Tower install checks it succeeded at the end.
The Kubernetes Collector manifest comes as one copyable `kubectl apply` command; the kubelet is reached by node IP. If you run your own Collector, otelcol-contrib 0.164.0 or later is needed (new component names).
**API doc import:** a docs page address works instead of JSON (e.g. `https://.../api/docs`); Spitfire finds the OpenAPI file on the page. Path variables such as `/orders/{orderId}` are added to the test variables (with the example value from the doc, otherwise empty); the editor says "variables added, fill in their values" with a **Go to variables** button.
Each step in the test editor is its own white card: thicker border, tinted header strip, number badge and space between steps.
**HTTP 429 (rate limit) is not an error:** it stays out of the error rate, thresholds and the capacity test breaking point. The report shows a "This is not an error" finding: which step, when (VUs and RPS), the limit headers the server sent (Retry-After, X-RateLimit-*…) and the IPs and headers to allow on the target.
The run page's **Backend** tab opens without a "Cannot read properties of null" error on runs where some sources returned nothing.
The English menu calls the page **Infrastructure sources** (the group stays **Infrastructure**).
Answering a confirm dialog (e.g. **Not tested** when saving a connection, **Trust the host key?** on an SSH tunnel) very quickly while it was still opening left it invisible on the page, and the buttons of the form below stopped taking clicks. Confirm dialogs now open and close without the fade.
The **Show every workspace's entries** box in the audit log ticks at once instead of springing back for a moment.

Guides for this release

Updating

Your installation tells you about a new release itself: admins see its notes and the update command for that installation at the top of the UI. The command backs up the database first and keeps the settings (Updating). You can also follow new releases with the RSS feed.

← 1.3 · All release notes