APM top lists and method hot spots
See the range's top web requests, SQL statements, errors and method hot spots on one page; open a row to answer who calls it. Menu: APM → Services → Top lists. APM must be on (APM); there is no extra limit.
Top lists
- Pick the range (e.g. Last 6 hours) and the environment if needed.
- Pick a tab: Top web requests, Top SQL, Top errors, Method hot spots.
- Type in Text or add filters from the filter bar.
- Click a column header: the server fetches the top 100 by that column again; it never just re-sorts what is shown.
The tab, the filters and the sort stay in the address bar; a shared link opens the same view. The Top lists tab on a service page shows the top 10 of the same four lists for that service.
| Tab | Row | Columns |
|---|---|---|
| Top web requests | service + method + route | calls, rpm, error %, p50/p95/p99, total time share |
| Top SQL | normalized statement | DB system/name, services, calls, avg/p95/max, share, rows, error % |
| Top errors | type + message fingerprint | count, share, affected services, first/last seen, link to the error page |
| Method hot spots | method (code.namespace + code.function) |
calls, total self time, self share, avg/p95 self, error % |
SQL statements are normalized at ingest: values become ? and IN lists collapse to one parameter. The same statement with different values is one row.
Filter bar
The top lists and the Traces page share one filter bar. Each filter becomes a chip and is written to the address.
- Filter by: service, host, environment, version, endpoint, k8s (cluster, namespace, workload, pod, node), container, HTTP status, exception or a threshold (calls, error rate %, p95 ms and so on).
- Operator:
=/≠(a value) or≥/≤(a threshold). - Value: the box suggests the values seen most in the range. For the HTTP status type
5xxor404; the error rate is a percentage (5= 5%). - Press Add.
There are no saved filters; the filtered view's link is the saved filter. Take it with Copy link and bookmark it or send it to the team.
| Address parameter | Meaning |
|---|---|
f_<field>=value |
that value (repeatable; !value leaves it out) |
f_status=5xx / f_status=404 |
a status class or code |
f_error=yes / no |
only rows with / without errors |
f_exc_type=…, f_exc_message=… |
exception type (exact), text in the message |
min_<metric>=…, max_<metric>=… |
a threshold; errorRate from 0 to 1, times in ms |
The service, host, environment, endpoint, method, namespace, workload, cluster, node and container filters read the per-minute rollups, up to 31 days. Pod, container image, exception and max filters read the stored spans, so they work on at most 24 hours. A wider range shows a warning: narrow the range or filter by namespace, workload or host instead.
Who calls it
Click a row and the Calls by caller table (calling service, calls, share %, p95, error %) and a calls/p95 series open below it. The link below goes to the matching page: the endpoint's traces, the query page or the error page.
For a request the caller is the parent span's service. An entry span without a parent counts as external (a browser, a job, a service without tracing); a parent that was never stored shows as unknown. For SQL, errors and methods the caller is the span's own service.
Method hot spots
A method row's time is self time: the method's span duration minus the time its direct child spans (DB, HTTP, nested methods) cover. Open a row to also see the top 5 endpoints the method runs in and the slowest call's trace.
Method data needs your service to send code-level spans. HTTP, DB and queue spans come by themselves; your business methods (e.g. OrderService.place) show only when you mark them as below. The list covers the last 24 hours at most.
Row key: code.namespace + code.function, else code.function.name, else code.file.path, else the INTERNAL span's name. We suggest marking the 5-10 business methods you suspect, not every method; a span on every method multiplies the span volume.
Java
The Java agent adds spans to the methods you pick without code changes. Add it to the command that starts the service and replace the class and method names with your own:
java -javaagent:/opt/otel/opentelemetry-javaagent.jar \
-Dotel.instrumentation.methods.include='com.acme.orders.OrderService[place,cancel];com.acme.pricing.PriceCalculator[calculate]' \
-jar /opt/app/app.jarIn a container, the same setting as an environment variable:
OTEL_INSTRUMENTATION_METHODS_INCLUDE='com.acme.orders.OrderService[place,cancel]'Check: restart the service and send a few requests; within a few minutes com.acme.orders.OrderService.place shows on the Method hot spots tab.
.NET
using System.Diagnostics;
public class OrderService
{
private static readonly ActivitySource Source = new("Acme.Orders");
public void Place(Order order)
{
using var activity = Source.StartActivity("OrderService.Place");
activity?.SetTag("code.namespace", "Acme.Orders.OrderService");
activity?.SetTag("code.function", "Place");
// the method's body
}
}Add the source to your OTel setup: builder.Services.AddOpenTelemetry().WithTracing(t => t.AddSource("Acme.Orders"));
Python
from opentelemetry import trace
tracer = trace.get_tracer("acme.orders")
@tracer.start_as_current_span("OrderService.place", attributes={"code.namespace": "acme.orders.OrderService", "code.function": "place"})
def place(order):
... # the method's bodyNode.js
const { trace } = require('@opentelemetry/api');
const tracer = trace.getTracer('acme-orders');
async function place(order) {
return tracer.startActiveSpan('OrderService.place', { attributes: { 'code.namespace': 'OrderService', 'code.function': 'place' } }, async (span) => {
try {
// the method's body
} finally {
span.end();
}
});
}There is no continuous profiling (flame graphs). Unmarked code is counted in the self time of the span it runs in.