APM screens

Traces from your services, in your own APM. Menu: APM.

#Before you start

1. The installation admin ticks the box in APM settings on the APM page, enters the retention days (up to the ceiling) and presses Save. The cog on the status line changes it later.
2. Use Add a service to add OpenTelemetry to your service (Connecting services with OTel).
3. If it says "APM storage is off", the installation admin presses Install the APM module on the same page (others: ask your admin).
4. Tick the consent box, enter the days and press Install.
5. Wait until the progress box closes and the APM page opens by itself. No button: press Copy on the command shown there and run it on the server.

#Services

1. APM › Services.
2. Pick the time range and environment at the top.
3. Click a row to open the service.
4. Trend: requests over the range, a red tick where there were errors. p95 trend: the p95 over the range.

#Service detail

1. Charts: requests, duration (avg/p95/p99), errors.
2. Click an endpoint for its traces.
3. Click the trace on an exception row for a sample trace.
4. A red arrow in the versions table: worse than the version before.
5. Why did it break?: likely causes, with a score.

#Errors

1. APM › Errors. Spans with an exception, HTTP 5xx, error status and ERROR logs are grouped here.
2. Type in the search box (or press /): type, message, stacktrace, trace id or span id. Quotes for a phrase: "gateway timeout".
3. Tick values in the lists on the left: service, environment, version, host, endpoint, method, status code, error type, language, k8s namespace/pod. The number beside each is its error count.
4. Attribute filter: type key:value (keys are suggested), Add.
5. Time: the buttons at the top, or Custom for a from–to.
6. Remove filters one by one from the tags at the top, or Clear all.
7. Check: the link in the address bar opens with the same filters.
8. Errors over time by type: follows every filter above, the status too. Click a type to add it to the filter, again to remove it.

#Error detail

1. Click a group in the list.
2. Occurrences over time: this group per time bucket.
3. Copy the stacktrace with its copy button.
4. Click a value under Breakdown: the occurrences are narrowed to it.
5. Open the span in the Occurrences table: the trace opens with that span picked.
6. The Errors button in the span panel brings you back to this group.
7. Resolve / Ignore / Reopen. A resolved error that happens again reopens.

#Searching traces

1. APM › Traces.
2. Fill in the filters. An attribute is key:value.
3. Search.
4. Trace latency over time: a dot per trace, errors as red diamonds, the line is the p95. Click a dot: its trace opens.
5. Duration distribution: click a bar to narrow the min/max duration to it.
6. Click a trace id.

#Trace view

1. Pick a span in the waterfall. The ruler at the top is the time from the trace's start.
2. Its attributes, resource, events and logs show on the right.
3. A red span failed.
4. Why did it break?: causes, with links to their spans.
5. Next error: picks the next failed span.
6. In the span panel, search the attributes to find a key in the HTTP, DB, messaging, k8s groups; copy a value with its copy button.

#Topology

1. APM › Topology.
2. On each arrow: requests/min, p95, error %.
3. A red arrow: a call with errors.
4. Click a service node for its detail.
5. Table: the same as a table.

#Hosts

1. APM › Hosts.
2. The services on each host, OS, runtime, container and k8s.
3. CPU and memory come from the Tower, with their line over the range under each bar. Without one they stay empty.
4. Click a host name for its page.

#Host page

1. APM › Hosts › a host name.
2. Requests by app: each app's requests over time; past six apps the rest are one Other apps line. CPU and memory: when the Tower sends them.
3. Apps: each service on the host, requests/min, error %, p95/p99. Click a service: its page opens filtered to this host.
4. Errors on this host › All errors: the Errors page filtered to this host.
5. Traces on this host: trace search opens with this host filled in.