Feedback API reference

Submit an assessment of one Agents API Turn. Rebyte resolves its Session and Agent from turn_id; no Scenario or separate execution receipt is required. Feedback does not change conversation history, resume execution, or train an Agent.

New to feedback? Start with Build a feedback loop for the overview, diagram and runnable quickstart.

Submit feedback

POST /v1/feedback requires an organization API key with tasks:write. Reads require tasks:read. These Rebyte extension routes need no OpenAI-Beta header.

Get the Turn ID from Session events or GET /v1/agents/sessions/{session_id}/turns. Feedback can be submitted later, including after completion, failure or cancellation, while the Session exists. It is also accepted for an in-progress Turn; it does not change that Turn's status.

curl https://api.rebyte.ai/v1/feedback \
  -H "Authorization: Bearer $REBYTE_API_KEY" \
  -H 'Content-Type: application/json' \
  -H 'Idempotency-Key: report-review-001' \
  -d '{
    "turn_id": "turn_123",
    "rating": "negative",
    "comment": "The report omitted refunded orders.",
    "correction": "Include refunded orders and show their refund amounts separately.",
    "source": {"type": "user", "id": "customer-123"}
  }'

Source is caller-supplied attribution, not a verified identity. An application must authorize its end user against the Turn before submitting or reading feedback. Keep the organization key on your server. Unknown request fields are rejected.

HTTP 201 returns the persisted resource. Timestamps are Unix seconds. Omitted comment, correction and source ID fields are stored and returned as null.

{
  "object": "feedback",
  "id": "feedback_abc",
  "agent_id": "agent_abc",
  "session_id": "session_abc",
  "turn_id": "turn_123",
  "rating": "negative",
  "comment": "The report omitted refunded orders.",
  "correction": "Include refunded orders and show their refund amounts separately.",
  "source": {"type": "user", "id": "customer-123"},
  "created_at": 1791417600
}

Retries and multiple assessments

Feedback is append-only. Each new submission creates a separate assessment; multiple people or evaluators may assess the same Turn. There is no update or delete endpoint.

Supply a stable Idempotency-Key (1–256 characters) for each logical submission. The key is scoped to the organization's feedback API. Identical normalized requests, including concurrent retries, return the original resource with HTTP 200. Reusing the key for another Turn or different content returns HTTP 409 idempotency_conflict. Keep the same key when retrying an ambiguous network failure. Without a key, each accepted request creates a new record.

Retrieve and list

GET /v1/feedback/feedback_abc
GET /v1/feedback?turn_id=turn_123&limit=20&order=desc
GET /v1/feedback?turn_id=turn_123&limit=20&order=desc&after=feedback_abc

Listing requires turn_id. limit is 1–100, default 20; order is asc or desc, default desc. Ordering follows insertion order. Responses contain object: "list", data, has_more, first_id and last_id. Use last_id as after for the next page, keeping the same Turn and order.

Unknown or foreign Turns/resources return 404. A cursor outside the selected Turn's collection returns 400 invalid_cursor. Invalid input returns 400; missing authentication is 401, and missing key scopes is 403. Deleting the Session makes its feedback inaccessible through these endpoints; stored feedback is retained with its execution records.

TypeScript extension

Build the current @rebyteai/agent-extensions source to use rebyte.feedback. This resource is not in the published 0.3.0 package or the existing v0.4.0 tag. Use the repository’s main branch; npm publication is pending. Standard Agent execution continues through the official client.

import OpenAI from 'openai';
import { RebyteExtensions } from '@rebyteai/agent-extensions';

const client = new OpenAI({
  apiKey: process.env.REBYTE_API_KEY,
  baseURL: 'https://api.rebyte.ai/v1',
  maxRetries: 0,
});
const rebyte = new RebyteExtensions(client);
const feedback = await rebyte.feedback.create({
  turn_id: 'turn_123',
  rating: 'positive',
  comment: 'All expected orders are present.',
  source: { type: 'evaluator', id: 'order-coverage-v1' },
}, { headers: { 'Idempotency-Key': 'order-coverage-turn-123' } });

await rebyte.feedback.retrieve(feedback.id);
for await (const assessment of rebyte.feedback.list({ turn_id: feedback.turn_id })) {
  console.log(assessment.rating, assessment.comment);
}

Creation disables automatic retries by default. The extension shares the official client's URL, credentials, timeout, custom fetch, API errors and pagination.