Feedback API reference

SDK examples on this page require the 0.5.0 source preview. npm publication is pending; use the HTTP endpoints directly in the meantime.

Submit an assessment of one Agents API Turn. Rebyte resolves its Session and Agent from turn_id. If the Session is bound to a Scenario, feedback inherits its scenario_id. Unbound Turns remain supported; no separate execution receipt is required. Feedback does not change conversation history, resume execution, or train an Agent. Feedback also inherits model_version_id from the Turn. It is an immutable ftv_ ID, or null for catalog models; callers cannot override it. See Train and publish a model.

New to feedback? Start with Build a feedback loop for the overview, diagram and runnable quickstart.

Submit feedback

POST /v1/feedback requires an organization API key with tasks:write. Reads require tasks:read. These Rebyte extension routes need no OpenAI-Beta header.

Get the Turn ID from Session events or GET /v1/agents/sessions/{session_id}/turns. Feedback can be submitted later, including after completion, failure or cancellation, while the Session exists. It is also accepted for an in-progress Turn; it does not change that Turn's status.

curl https://api.rebyte.ai/v1/feedback \
  -H "Authorization: Bearer $REBYTE_API_KEY" \
  -H 'Content-Type: application/json' \
  -H 'Idempotency-Key: report-review-001' \
  -d '{
    "turn_id": "turn_123",
    "rating": "negative",
    "comment": "The report omitted refunded orders.",
    "correction": "{\"order_id\":\"A-102\",\"refund_total\":19.50}",
    "source": {"type": "user", "id": "customer-123"}
  }'

Source is caller-supplied attribution, not a verified identity. An application must authorize its end user against the Turn before submitting or reading feedback. Keep the organization key on your server. Unknown request fields are rejected.

HTTP 201 returns the persisted resource. Timestamps are Unix seconds. Omitted comment, correction and source ID fields are stored and returned as null.

{
  "object": "feedback",
  "id": "feedback_abc",
  "agent_id": "agent_abc",
  "session_id": "session_abc",
  "turn_id": "turn_123",
  "scenario_id": "scn_abc",
  "model_version_id": null,
  "rating": "negative",
  "comment": "The report omitted refunded orders.",
  "correction": "{\"order_id\":\"A-102\",\"refund_total\":19.50}",
  "source": {"type": "user", "id": "customer-123"},
  "created_at": 1791417600
}

Retries and multiple assessments

Feedback assessments are immutable. Each new submission creates a separate assessment; multiple people or evaluators may assess the same Turn. There is no update endpoint. Use DELETE /v1/feedback/{feedback_id} and DELETE /v1/scenarios/{scenario_id}/turns/{turn_id}/feedback to remove one assessment or clear active assessments. They affect future training selection; admitted snapshots keep their original evidence. See reviewed-Turn training.

Supply a stable Idempotency-Key (1–256 characters) for each logical submission. The key is scoped to the organization's feedback API. Identical normalized requests, including concurrent retries, return the original resource with HTTP 200. Reusing the key for another Turn or different content returns HTTP 409 idempotency_conflict. Keep the same key when retrying an ambiguous network failure. Without a key, each accepted request creates a new record.

Retrieve and list

GET /v1/feedback/feedback_abc
GET /v1/feedback?turn_id=turn_123&limit=20&order=desc
GET /v1/feedback?turn_id=turn_123&limit=20&order=desc&after=feedback_abc
GET /v1/feedback?scenario_id=scn_abc&limit=20&order=desc
GET /v1/feedback?scenario_id=scn_abc&turn_id=turn_123

Listing requires turn_id, scenario_id, or both. Both filters are intersected. limit is 1–100, default 20; order is asc or desc, default desc. Ordering follows insertion order. Responses contain object: "list", data, has_more, first_id and last_id. Use last_id as after for the next page, keeping the same filters and order.

Unknown or foreign Turns/resources return 404. A cursor outside the selected filtered collection returns 400 invalid_cursor. Invalid input returns 400; missing authentication is 401, and missing key scopes is 403. Deleting the Session makes its feedback inaccessible through these endpoints; stored feedback is retained with its execution records.

TypeScript extension

Use @rebyteai/agent-extensions for typed feedback resources. Standard Agent execution continues through the official OpenAI client.

import OpenAI from 'openai';
import { RebyteExtensions } from '@rebyteai/agent-extensions';

const client = new OpenAI({
  apiKey: process.env.REBYTE_API_KEY,
  baseURL: 'https://api.rebyte.ai/v1',
  maxRetries: 0,
});
const rebyte = new RebyteExtensions(client);
const feedback = await rebyte.feedback.create({
  turn_id: 'turn_123',
  rating: 'positive',
  comment: 'All expected orders are present.',
  source: { type: 'evaluator', id: 'order-coverage-v1' },
}, { headers: { 'Idempotency-Key': 'order-coverage-turn-123' } });

await rebyte.feedback.retrieve(feedback.id);
for await (const assessment of rebyte.feedback.list({ turn_id: feedback.turn_id })) {
  console.log(assessment.rating, assessment.comment);
}

Creation disables automatic retries by default. The extension shares the official client's URL, credentials, timeout, custom fetch, API errors and pagination.