Behavioural analytics from OTEL

ra-mcp's OpenTelemetry isn't only for ops health — it's instrumented so you can answer product questions: what people search for, which datasets and features they use, and what they want but can't find. This page maps those questions to the spans and metrics that answer them.

Enable it with RA_MCP_OTEL_ENABLED=true and point OTEL_EXPORTER_OTLP_ENDPOINT at your collector (see Observability). Traces carry the high-cardinality intent (search terms, reference codes); metrics carry the bounded aggregates (per-dataset counts, latencies) for dashboards and alerts.

Where the signal lives (and why)

Questions you can answer

"What are people searching for?"

Every search records the term on a span:

• Datasets + PDF guides → search <dataset> span, attribute db.query.text (the term), db.query.filter (filters applied, e.g. gender = 'm'), db.response.total_hits (how well the data answered). Covers all 13 datasets and the guide search (both go through the shared spine).
• Live archive search → SearchOperations.search span, attributes search.keyword, search.total_hits, search.offset.

Query your trace store for these span attributes to see the actual terms, grouped by dataset.

"What do people open, view, and geocode?"

Searching is only half of intent — what users then open is the other half:

• Viewer → view_document / viewer_navigate spans, attributes view.reference_code (which archive document) and view.pages.
• PDF viewer → display_pdf span, attribute pdf.url (which guide/PDF).
• Geocoding → tora search spans, attributes geocode.place (+ geocode.parish / geocode.county) — which historical places people look up.

"Which datasets / features are most used?"

• ra_mcp.lancedb.queries counter, by db.collection.name → per-dataset (and guide) search volume.
• ra_mcp.search.requests counter, by search.type → live transcribed vs metadata search volume.
• Tool popularity (which viewer, which tool) → the FastMCP tools/call spans (one per tool); derive per-tool counts in the collector (spanmetricsconnector).

"What do people want but can't find?" (unmet demand)

This is the highest-value product signal:

• ra_mcp.lancedb.results histogram, by db.collection.name → its zero bucket is the rate of searches that returned nothing, per dataset.
• Then read the db.query.text of those zero-hit spans (db.response.total_hits = 0) to see the actual terms people searched for that your data doesn't answer — a direct backlog of data/feature gaps.

"How engaged are they?"

• search.offset (span) → how deep users paginate.
• ra_mcp.browse.pages / ra_mcp.browse.empty_pages → pages read per browse, and blank-page rate.
• db.response.total_hits vs db.response.returned_rows → corpus richness vs page shown.

"The research funnel: search → browse → view"

Search spans carry mcp.session.id (when the client sends it); correlate spans within a session/trace to see whether a search led to a browse and then a document/PDF view.

"Which upstream is slow or failing?"

• ra_mcp.http.request.duration / ra_mcp.http.errors now carry server.address → per-upstream RED (Solr API vs IIIF vs OAI-PMH vs HF CDN), not one blurred series.
• LanceDB: ra_mcp.lancedb.query.duration + ra_mcp.lancedb.errors by dataset.

Coverage at a glance

Terms and reference codes are user intent, exposed only in traces (sampled, access-controlled), never in metric labels. Treat your trace/log backend accordingly.