Skip to content
Traces and observability

Traces and observability

Follow model calls, tool execution, usage and turn outcomes with Rebyte execution traces.

On this pageConnect tracing destinationsDelivery, sampling, and privacyView traces in PlatformWhat is recordedInspect execution through the APIExport session tracesOpenAI compatibility

Rebyte sends Agents API execution traces to destinations that you own. Model calls, tool activity, and Turn outcomes share one OpenTelemetry trace identity. Your application does not need a tracing SDK or a per-request flag.

Connect tracing destinations

In Platform, open Observability, select a destination, and enter its connection settings. Rebyte tests the actual ingestion or object-upload path before saving. You can enable several destination types at the same time. Each organization has one configuration per type.

DestinationTransport and format
LangfuseOTLP/HTTP JSON, project public and secret keys. Cloud and public self-hosted instances.
LangSmith, Braintrust, Arize AX, Comet Opik, DatadogOTLP/HTTP Protobuf with provider authentication and semantic attributes. The LangSmith preset uses a project-based workspace.
Grafana Cloud, Elastic ObservabilityOTLP/HTTP JSON with provider authentication. Supply the complete traces endpoint.
OpenTelemetryComplete OTLP/HTTP endpoint, JSON or Protobuf, custom headers.
WebhookOTLP JSON in resourceSpans, POST or PUT, custom headers.
Amazon S3, S3 Compatible, Cloudflare R2, MinIOS3 PutObject, identical OTLP JSON documents.
Google Cloud StorageNative GCS JSON upload API, identical OTLP JSON documents, service-account credentials.

These destinations share capture, masking, sampling, batching, and delivery. Provider adapters select authentication, endpoint, and semantic aliases. Storage objects use prefix/YYYY-MM-DD/generation-batchId.otlp.json. Each object contains an OTLP ExportTraceServiceRequest with resourceSpans; it has no Rebyte-specific file wrapper. Database destinations such as BigQuery, ClickHouse, and Snowflake are not included.

You can configure a destination through the API:

Terminal
curl --fail-with-body -X PUT 'https://api.rebyte.ai/v1/agents/tracing/destinations/langfuse' \
  -H "Authorization: Bearer ${REBYTE_API_KEY}" \
  -H 'OpenAI-Beta: agents=v1' \
  -H 'Content-Type: application/json' \
  --data @langfuse-config.json

Keep langfuse-config.json private:

json
{
  "name": "Production Langfuse",
  "enabled": true,
  "privacy_mode": false,
  "sampling_rate": 1,
  "config": {
    "base_url": "https://cloud.langfuse.com",
    "public_key": "pk-lf-...",
    "secret_key": "sk-lf-..."
  }
}

For S3, use type s3 and configuration fields bucket, region, prefix, access_key_id, secret_access_key, optional session_token, and optional endpoint. The other S3 types require endpoint. For GCS, use type gcs with bucket, prefix, service_account_json (a JSON string), and endpoint (https://storage.googleapis.com). For generic OTLP, use endpoint, encoding (json or protobuf), and headers (a JSON string of string-valued headers). Webhook uses endpoint, method, and headers.

GET /v1/agents/tracing/destinations lists public configuration without credentials. PUT /:type replaces a complete configuration; send credentials again when editing or rotating keys. POST /:type/test with {} tests the saved configuration. DELETE /:type disconnects it. Reads require tasks:read; writes require tasks:write. Partner requests using Rebyte-Account configure that sub-account independently. Platform configuration requires an organization administrator.

Customer endpoints must use HTTPS and resolve to public internet addresses. Rebyte encrypts connection configuration with an organization-and-destination-specific encryption context. Read responses never return secrets. Rebyte does not host or provision a customer Langfuse project.

Delivery, sampling, and privacy

Enabled destinations receive future traces. Existing history is not copied. sampling_rate ranges from 0 to 1 and selects whole traces independently per destination. privacy_mode removes prompt, response, tool argument/result, and error content before the destination payload enters durable delivery history. IDs, timing, status, model, and token counts remain. Common credential fields are masked even when privacy mode is off; this masking cannot identify arbitrary secrets written in natural-language content.

After batching, Rebyte admits each destination's payload to its own durable Temporal Workflow. Receiver failures do not repeat model calls or tool execution. Transient network and retryable receiver errors use exponential retries for up to two hours. Authentication and other permanent receiver errors stop delivery. HTTP delivery is at least once; retries preserve trace IDs, span IDs, and the Idempotency-Key. Storage retries use immutable object keys and conditional creation.

Changing routing, privacy, sampling, or enabled state fences previously queued batches. Replacing credentials for the same known-provider destination preserves pending batches; changing opaque custom headers creates a new configuration generation. Deletion stops future attempts. An already running network request can finish after an edit or deletion.

The in-process OpenTelemetry buffer is not durable: a process crash before admission, a full buffer, or a failed admission can lose a batch. A single span must fit within 750 KB compressed and 32 MiB uncompressed. Receiver acceptance also does not guarantee that a vendor will retain or display every attribute.

View traces in Platform

  1. Open Platform and select the organization that owns your API Agent.
  2. Open the Agent to see its execution traces across Sessions.
  3. Open a Session to narrow the trace view to that conversation, or open a Turn for its individual execution trace.

The embedded trace view and Session trace export require an enabled Langfuse destination. Other destinations receive exports without providing the embedded read view. The embedded view reads your connected Langfuse project through Rebyte. Opening the full dashboard requires your own Langfuse login. Rebyte does not host or provision Langfuse for you.

What is recorded

RecordWhat you can inspect
TurnFinal status, start and end times, model-call count, recorded token usage and errors.
Model callModel observations linked to the executing Turn.
Server tool callTool name, arguments, call ID, result and failure status.
CorrelationOrganization, Agent, Session and Turn identifiers.

Each Turn has its own trace identity. Reusing an Agent across Sessions does not combine their execution traces. Turn duration measures the persisted lifetime from creation to completion; it can include queueing and waiting for client function results, rather than only active model execution. Your application's client function runs outside Rebyte, so its internal execution is not automatically instrumented.

Tracing runs on the Rebyte service; API consumers do not need to install a Langfuse SDK or pass a tracing flag when creating an Agent or Session. Use the trace view to inspect execution, then link your assessment to its Turn ID.

Inspect execution through the API

Use Session, Turn and Item IDs to correlate your application logs with the trace view. You can also retrieve persisted outcomes with the official OpenAI client configured for Rebyte:

javascript
const turns = await client.beta.agents.sessions.turns.list(sessionId);
for (const turn of turns.data) {
  console.log({
    sessionId,
    turnId: turn.id,
    status: turn.status,
    usage: turn.usage,
    error: turn.error,
  });
}

See Observability and usage for Items, live events and token usage.

Export session traces

GET https://api.rebyte.ai/v1/agents/sessions/{session_id}/traces returns a page of Turn traces as OpenTelemetry Protocol (OTLP) JSON, matching OpenAI's session trace export. Use an organization API key with the tasks:read scope. Connect Langfuse first. Trace retrieval uses Langfuse's public v2 Observations API; the connected instance must support it.

Terminal
curl --fail-with-body \
  "https://api.rebyte.ai/v1/agents/sessions/${REBYTE_SESSION_ID}/traces?limit=20&order=asc" \
  -H "Authorization: Bearer ${REBYTE_API_KEY}" \
  -H "OpenAI-Beta: agents=v1" \
  --output trace-page.json && \
jq '{resourceSpans: [.data[].otlp.resourceSpans[]]}' trace-page.json > traces.otlp.json
json
{
  "object": "list",
  "data": [{
    "id": "turn_...",
    "object": "agent.session.trace",
    "session_id": "sess_...",
    "scenario_id": "scn_...",
    "model_version_id": null,
    "created_at": 1790640000,
    "otlp": { "resourceSpans": [{ "resource": { "attributes": [...] }, "scopeSpans": [{ "scope": { "name": "rebyte.agents_api" }, "spans": [...] }] }] }
  }],
  "first_id": "turn_...", "last_id": "turn_...", "has_more": false
}

Each entry is one finished Turn; id is the Turn ID and the pagination cursor. When has_more is true, pass last_id as after with the same order. limit accepts 1–100 (default 20) and order defaults to desc. A response is limited to 16 MiB; if a page exceeds it, request fewer traces. Send the combined payload to any OTLP/HTTP endpoint using that provider's authentication.

A trace is published after its Turn finishes. Turns that are still running, or whose trace has not been ingested yet, are skipped; export again later for a complete history. To export an Agent's traces, list its Sessions with agent_id and export each Session.

Spans use Langfuse's OpenTelemetry attribute names (langfuse.observation.type, langfuse.observation.input, langfuse.observation.output, langfuse.observation.metadata.*, langfuse.observation.usage_details) plus gen_ai.request.model and gen_ai.usage.* token counts. The resource carries rebyte.agent_id, rebyte.session_id and rebyte.turn_id. Bound Sessions also include rebyte.scenario_id; each trace resource exposes nullable scenario_id. See Scenarios for grouping results across Sessions. Managed-model traces also expose nullable model_version_id; spans for a managed version carry the rebyte.model_version_id resource attribute. It identifies the version pinned to that Turn even after the model family publishes a newer version. Failed spans have OTLP status code 2 with the error message. Exports include token usage but not model cost; Rebyte billing is reported separately.

OpenAI compatibility

OpenAI's managed Agents API enables tracing by default and presents turn traces in its dashboard. Rebyte's export endpoint follows the same request, pagination and response envelope. Differences: Rebyte requires the tasks:read scope instead of api.traces.read, export requires a connected Langfuse project, and span attributes follow Langfuse conventions rather than OpenAI's internal span schema.

The separate OpenAI Agents SDK is a different product; its application-side tracing configuration should not be assumed to apply to the managed Agents API.