Loop quickstart

Run a business task, review its answer and start an inspectable training run.

Rebyte LoopPreview4 min read

On this page

HTTP Evaluate, then publish SDK source preview

Use your organization API key with tasks:read and tasks:write. Keep it on your application server. Inference needs model credit, and starting training requires training access for the organization. Every request below uses https://api.rebyte.ai/v1.

The example groups order-extraction work under one Scenario. Review an answer, save the complete desired output, and explicitly train from that Turn. Keep a separate set of Turns for evaluation.

HTTP

Use the HTTP API directly; no SDK installation is needed. Save returned IDs in your application; the shell examples below use jq to read them.

Terminal

API=https://api.rebyte.ai/v1
SCENARIO_ID=$(curl --fail-with-body "$API/scenarios" \
  -H "Authorization: Bearer $REBYTE_API_KEY" \
  -H 'Content-Type: application/json' -H 'Idempotency-Key: orders-scenario-001' \
  -d '{"name":"Order extraction"}' | jq -r .id)

SESSION_ID=$(curl --fail-with-body "$API/agents/sessions" \
  -H "Authorization: Bearer $REBYTE_API_KEY" -H 'OpenAI-Beta: agents=v1' \
  -H 'Content-Type: application/json' \
  -d "$(jq -n --arg scenario "$SCENARIO_ID" '{
    scenario_id:$scenario,
    agent:{model:"qwen3.6-35b-a3b",instructions:"Extract order facts as JSON."},
    environment:{type:"none"},
    input:"Order A-102 contains 2 notebooks at $4 each. Return order_id and total."
  }')" | jq -r .id)

curl --fail-with-body "$API/agents/sessions/$SESSION_ID/turns" \
  -H "Authorization: Bearer $REBYTE_API_KEY" -H 'OpenAI-Beta: agents=v1'

Wait for a completed Turn and review its answer. Set TURN_ID to that Turn's ID, then save feedback and admit training:

Terminal

curl --fail-with-body "$API/feedback" \
  -H "Authorization: Bearer $REBYTE_API_KEY" -H 'Content-Type: application/json' \
  -H "Idempotency-Key: orders-review:$TURN_ID" \
  -d "$(jq -n --arg turn "$TURN_ID" '{turn_id:$turn,rating:"positive",
    correction:"{\"order_id\":\"A-102\",\"total\":8}"}')"

MODEL_ID=$(curl --fail-with-body "$API/models" \
  -H "Authorization: Bearer $REBYTE_API_KEY" -H 'Content-Type: application/json' \
  -H 'Idempotency-Key: orders-model-001' \
  -d "$(jq -n --arg scenario "$SCENARIO_ID" '{name:"Order extraction",
    scenario_id:$scenario,base_model:"Qwen/Qwen3.6-35B-A3B"}')" | jq -r .id)

curl --fail-with-body "$API/models/$MODEL_ID/training-runs" \
  -H "Authorization: Bearer $REBYTE_API_KEY" -H 'Content-Type: application/json' \
  -H 'Idempotency-Key: orders-training-001' \
  -d "$(jq -n --arg turn "$TURN_ID" '{data:{mode:"all",turn_ids:[$turn]}}')"

Evaluate, then publish

Read GET /v1/models/{model_id}/training-runs/{run_id}/status. Once ready, use run.version_id as the model of a new evaluation Session. Check its answers against held-out tasks. Serving readiness does not establish task quality.

Publish only after evaluation:

Terminal

curl --fail-with-body "$API/models/$MODEL_ID/publish" \
  -H "Authorization: Bearer $REBYTE_API_KEY" -H 'Content-Type: application/json' \
  -d "$(jq -n --arg version "$VERSION_ID" '{version_id:$version}')"

New Sessions using the model family ID now use that version. Existing Sessions remain pinned. Continue with training runs for data selection and settings, or versions and publishing for evaluation and rollback.

SDK source preview

The typed Loop methods require SDK 0.5.0, which is available as a reviewed source tag. Its npm publication is pending. The currently published 0.3.0 package does not contain these methods. Use the HTTP flow above now, or build the source preview:

Terminal

git clone --branch v0.5.0 https://github.com/ReByteAI/rebyte-agent-sdk.git
cd rebyte-agent-sdk
pnpm install --frozen-lockfile
pnpm build
# See examples/agents-api/training-runs.mjs for a runnable workspace recipe.

ts

import OpenAI from 'openai';
import { RebyteExtensions } from '@rebyteai/agent-extensions';
import { setTimeout as delay } from 'node:timers/promises';

const client = new OpenAI({
  apiKey: process.env.REBYTE_API_KEY,
  baseURL: 'https://api.rebyte.ai/v1',
  maxRetries: 0,
});
const rebyte = new RebyteExtensions(client);

// Persist each creation request and retry key in your application.
const scenario = await rebyte.scenarios.create({ name: 'Order extraction' },
  { headers: { 'Idempotency-Key': 'orders-scenario-001' } });
const session = await rebyte.sessions.create({
  scenario_id: scenario.id,
  agent: { model: 'qwen3.6-35b-a3b', instructions: 'Extract order facts as JSON.' },
  environment: { type: 'none' },
  input: 'Order A-102 contains 2 notebooks at $4 each. Return order_id and total.',
});

let turn;
const deadline = Date.now() + 240_000;
while (Date.now() < deadline) {
  turn = (await rebyte.sessions.turns.list(session.id)).data[0];
  if (turn && ['completed', 'failed', 'cancelled'].includes(turn.status)) break;
  await delay(1000);
}
if (!turn || turn.status !== 'completed') throw new Error('Task did not complete');
const review = await rebyte.scenarios.turns.retrieve(scenario.id, turn.id);
console.log(review.output_text);

// After reviewing the result, submit the desired complete answer.
await rebyte.feedback.create({
  turn_id: turn.id,
  rating: 'positive',
  correction: '{"order_id":"A-102","total":8}',
  source: { type: 'user', id: 'reviewer-42' },
}, { headers: { 'Idempotency-Key': `orders-review:${turn.id}` } });

const model = await rebyte.models.create({
  name: 'Order extraction', scenario_id: scenario.id,
  base_model: 'Qwen/Qwen3.6-35B-A3B',
}, { headers: { 'Idempotency-Key': 'orders-model-001' } });
const run = await rebyte.models.trainingRuns.create(model.id, {
  data: { mode: 'all', turn_ids: [turn.id] },
}, { idempotencyKey: 'orders-training-001' });

console.log(run.id, run.source, run.training, run.dataset_sha256);
for await (const example of rebyte.models.trainingRuns.examples(model.id, run.id)) {
  console.log(example.messages, example.target);
}

Session creation is not idempotent. Persist its returned ID; after an ambiguous network failure, reconcile the existing Session before creating another.

The run returns after admission. Read its status until it reaches ready, failed or cancelled. Reusing the same retry key and input returns the same run, including its original data and source.