One conversation, two audiences.

Section 01 · the problem

The thread has to stay clean, and still be debuggable.

The widget conversation is deliberately minimal, and it must stay that way. But two people need more from it than it shows. A customer who gets a wrong answer has no way to report it that a human can act on. An operator has no per-turn view of which model answered, how long it took, or what it cost.

The wrong fix is to pour metadata into the thread. Model names and token counts mean nothing to a customer and clutter the one surface that has to stay simple. The right fix starts from a different question: who is this information for? A customer needs a reference they can quote. An operator needs the numbers. Those are different people looking at different screens, so they get different surfaces.

Section 02 · audience decides the surface

Split it by who is looking.

The same AI reply owes the customer a way to report and the operator a way to account. Neither should ever see the other's view.

flowchart TB
  TURN["one AI reply"] --> CUST
  TURN --> OP
  subgraph CUST["CUSTOMER · the widget thread (sacred, minimal)"]
    TD["thumbs-down"] --> REF["Ref 3DE8-C322
copyable, resolves to session + message"]
  end
  subgraph OP["OPERATOR · the console inbox"]
    PROV["gpt-4o · 1.2s · 4.0k→120 · $0.0013"]
  end
  REF -.->|"customer quotes the code"| PROV

Section 03 · the customer side

A code the customer can quote. Nothing more.

When a customer marks a reply unhelpful, a single tappable pill appears: a short reference like Ref 3DE8-C322 that copies a fuller, resolvable string to the clipboard. That is the entire feature on the customer side.

The code is derived on the client from ids already on the message, the session prefix and the message suffix, so there is no server round-trip and no new table. It carries no personal data. When the customer follows up through any channel and quotes the code, an operator resolves it back to the exact session and message. It appears only after a thumbs-down, so the thread stays clean until the customer decides to report, and it shows no model, tokens, or cost, because none of that helps a customer and all of it would clutter the bubble.

The principle: the smallest thing that makes a bad answer reportable is a resolvable reference, not a form and not a metadata dump. Start code-only. A persisted report inbox can be added later if operators want one; it was deliberately left out of v1 so the customer change is pure client code that ships on its own.

Section 04 · the operator side

Model, latency, tokens, cost. Under every AI reply.

In the console inbox, each AI reply gets a compact line beneath it: the model that answered, how long it took, tokens in and out, any cache-served tokens, and the list-price cost of the turn. It is almost entirely a display task over data already recorded.

flowchart LR
  T["transcript · AI message ids"] --> Q["GET /inbox/:session/provenance?ids=…"]
  Q --> CH[("bot_responses
WHERE message_id IN (…)")]
  CH --> PR["price with internal/llmprice"]
  PR --> LINE["provenance line under the AI bubble"]

The join is exact: bot_responses.message_id is the ULID of the reply message, so there is no fuzzy matching. The endpoint reads only the ids the operator has on screen, a bounded set, and prices them with the same per-model rate table the usage dashboards use, so a turn's cost here agrees with the monthly rollup. Pricing is at list rate today, with no markup; a model without a published rate is flagged as an estimate rather than presented as a fact.

Section 05 · reuse, not new capture

Almost none of this is new data.

The whole operator track is a read over facts the platform already writes. Nothing new is captured on the serving path.

Section 06 · failure modes, by design

Telemetry degrades quietly. It never breaks the inbox.

The customer report is code-only by the same instinct: no new table means the widget change carries no migration and no server dependency, so it ships as pure client code and cannot half-break a backend it does not touch.