Orchestration v2

Orchestration is the part of the T3 Code server that turns “the user sent a message” into “an agent did some work and the result is on screen”. It decides which agent process runs, keeps track of what that agent is doing, survives restarts, and lets you stop, steer, fork, or switch agents halfway through. Version 2 is a full rewrite of that part, and it is the most involved code in the repo.

This chapter takes it slowly. First the nouns, meaning the entities and how they relate. Then one lifecycle, sending a message, followed from start to finish through real code. Then the migration from version 1. You do not need to understand every file. The goal is a mental model you can carry into the code, and a few ideas you can reuse when your own data model has to change.

Start with the nouns

The design doc for v2 opens with a picture you can keep in your head:

AppThread
  Run 1
    root ExecutionNode
      tool ExecutionNode
      approval ExecutionNode
      subagent ExecutionNode
        ProviderThread
          child root ExecutionNode
  Run 2
    root ExecutionNode

Here is each entity in plain words:

The key split is between the app’s view and the agent’s view. A Run is what you count as “turn 3”. A ProviderTurn is what Codex calls a turn. They are separate on purpose, and the README says why. The team probed Codex and found that “Subagent child turn/completed events can occur before the parent/root turn completes” and that child turns “must not be remapped onto the parent provider turn id” (README.md). If the app used the agent’s turn as its own, a subagent finishing early could mark your whole turn as done.

The same split lets one conversation use several agents. Run 1 can go to Codex and run 2 to Claude. Each Run records its own providerThreadId, so “provider switches do not create new app threads” (README.md).

What a Run looks like in code

The design docs describe the target. The real shapes live in the shared contract package. Here is the start of the Run schema:

export const OrchestrationV2Run = Schema.Struct({
  id: RunId,
  threadId: ThreadId,
  ordinal: PositiveInt,
  providerInstanceId: ProviderInstanceId,
  modelSelection: ModelSelection,
  providerThreadId: Schema.NullOr(ProviderThreadId),
  userMessageId: MessageId,
  rootNodeId: Schema.NullOr(NodeId),
  activeAttemptId: Schema.NullOr(RunAttemptId),
  status: OrchestrationV2RunStatus,
  queuePosition: Schema.optional(Schema.NullOr(PositiveInt)),
  queueHeld: Schema.optional(Schema.Boolean),
  steerWhenReadyRunId: Schema.optional(RunId),
  requestedAt: Schema.DateTimeUtc,
  startedAt: Schema.NullOr(Schema.DateTimeUtc),
  completedAt: Schema.NullOr(Schema.DateTimeUtc),
  checkpointId: Schema.NullOr(CheckpointId),
  contextHandoffId: Schema.NullOr(ContextHandoffId),

Notice that every reference to another entity uses a specific id type, such as RunId, ThreadId, NodeId, or ProviderThreadId. Those types are all strings at runtime, but TypeScript treats them as different:

const makeEntityId = <Brand extends string>(brand: Parameters<typeof Schema.brand<Brand>>[0]) => {
  return TrimmedNonEmptyString.pipe(Schema.brand<Brand>(brand));
};

export const ThreadId = makeEntityId("ThreadId");
export type ThreadId = typeof ThreadId.Type;

A “brand” is a tag that exists only in the type system. If you pass a RunId where a ThreadId is expected, the build fails, even though both are plain strings at runtime. In a model with a dozen id kinds, that catches a whole class of mix-ups.

The ids are also the app’s own. The id doc puts the rule in two sentences. “Provider ids are evidence. App ids are identity.” (entity-ids-and-correlation.md). The agent’s own ids are stored as optional “refs” for matching events, and some agents give no stable ids at all. In that case the app numbers items in order inside a known scope, and “Once allocated, never change the app id for that entity” (line 105).

Store what happened, not only what is

Most apps you have built store the current state. A post row has a status column, and you update it. T3 Code’s orchestrator is event-sourced instead. Event sourcing means the durable record is a log of changes, and the current state is computed from that log. AGENTS.md says it in one line. “The server is event-sourced, and side effects run after the command commits” (AGENTS.md).

The events are simpler than you might expect. Most of them carry the whole entity after the change:

  Schema.Struct({
    ...OrchestrationV2EventBase.fields,
    type: Schema.Literal("run.created"),
    payload: OrchestrationV2Run,
  }),
  Schema.Struct({
    ...OrchestrationV2EventBase.fields,
    type: Schema.Literal("run.updated"),
    payload: OrchestrationV2Run,
  }),

run.updated does not say “the status changed to running”. It carries the full Run as it now is. The same pattern repeats for node.updated, provider-turn.updated, and message.updated. A reader of the log never has to replay a chain of small edits to know what a run looked like at event 500. It reads the last run.updated for that id.

When the server stores an event, it wraps it with a sequence number and the command that caused it:

export const OrchestrationV2StoredEvent = Schema.Struct({
  sequence: NonNegativeInt,
  commandId: Schema.NullOr(CommandId),
  event: OrchestrationV2DomainEvent,
});

The sequence is what makes reconnecting simple. The data model doc describes the rule as “read snapshot at sequence N, then stream committed V2 events where sequence > N” (core-graph-and-data-model.md). A phone that lost signal reconnects, says “I have everything up to 812”, and receives 813 onward. The tables that the UI reads, called projections, are built from these events in the same database transaction, so they never fall behind the log.

Follow one message from send to finish

The repo’s AGENTS.md describes the whole loop in one paragraph:

The orchestrator (apps/server/src/orchestration-v2/Orchestrator.ts) serializes commands and decides events without doing any I/O. The event sink commits those events, the projections the UI reads, the command receipt, and outbox effects in one transaction. The effect worker then runs the effects, such as starting a provider turn or capturing a checkpoint, and feeds results back as commands.

That is from line 180. The steps below follow that paragraph through the code.

Step 1. The command arrives, and the receipt check runs

Your message reaches the server over the WebSocket as a command with a commandId that the client made. The first thing the orchestrator does is look that id up:

    const existingReceipt = yield* commandReceipts.getByCommandId(command.commandId).pipe(
      Effect.mapError(
        (cause) =>
          new OrchestratorDispatchError({
            commandId: command.commandId,
            commandType: command.type,
            cause,
          }),
      ),
    );

    if (Option.isSome(existingReceipt)) {
      return yield* replayCommandReceipt(command, existingReceipt.value);
    }

A receipt is a saved record that says “this command was accepted, and its events end at sequence N”. If the phone sends the same message twice because the first reply was lost, the second attempt finds the receipt and gets the same answer back. No second run starts. The README makes this a requirement. “Duplicate command ids must return the same receipt-backed result without re-running provider side effects” (README.md).

Step 2. The orchestrator plans the new records

For a new message on an idle thread, the orchestrator builds the entities from the first section. It creates the Run with status starting. (A message sent while another run is busy takes a different branch that creates it as queued.) Here is the first attempt and the root node, both waiting to start:

        const attempt: OrchestrationV2RunAttempt = {
          id: attemptId,
          runId,
          attemptOrdinal: 1,
          rootNodeId,
          providerInstanceId: modelSelection.instanceId,
          providerThreadId,
          providerTurnId: null,
          reason: "initial",
          status: "pending",
          // ...
        };
        const rootNode: OrchestrationV2ExecutionNode = {
          id: rootNodeId,
          threadId: command.threadId,
          runId,
          parentNodeId: null,
          rootNodeId,
          kind: "root_turn",
          status: "pending",
          countsForRun: true,
          // ...
        };

providerTurnId: null is the app-first rule in action. The run exists and has an id before the agent has been contacted, because the app does not wait for the agent to name anything. Right after this, the code emits run.created, run-attempt.created, node.updated, and message.updated events into a list. Nothing is written yet.

It also adds an effect to a second list. An effect is a request for outside work, here “start the agent’s turn for this run”:

          const pendingEffect = {
            id: `effect:${command.commandId}:provider-turn.start:${runId}`,
            commandId: command.commandId,
            threadId: command.threadId,
            request: { type: "provider-turn.start", runId },
          } satisfies PendingOrchestrationEffectV2;
          if (dispatchMode.type !== "defer_start") {
            yield* Ref.update(effects, (existing) => [...existing, pendingEffect]);
          }

Look at the id. It is built from the command id and the run id, so the same command always produces the same effect id. If the step runs twice after a crash, it cannot queue two turns for one run.

Step 3. One transaction commits everything

The event sink takes both lists and writes them together:

          const reserved = yield* commandReceipts.insertIfAbsent({
            commandId: input.commandId,
            threadId: input.threadId,
            commandType: input.commandType,
            acceptedAt: input.acceptedAt,
            resultSequence: 0,
            status: "accepted",
            error: null,
          });
          if (!reserved) {
            const existing = yield* existingCommandResult(input.commandId);
            return { ...existing, committed: false as const, cancelledEffectIds: [] };
          }
          // ...
          const storedEvents = yield* eventStore.append({
            commandId: input.commandId,
            events: normalized,
          });
          // ...
          const outcomes = yield* applyStoredEvents(storedEvents);
          yield* effectOutbox.enqueue(input.effects);

All of this runs inside sql.withTransaction in SQLite. The receipt, the events, the projection updates (applyStoredEvents), and the effect rows land together or not at all. The table of pending effects is called an outbox. The README first listed it as a possible follow-up, a “durable effect outbox”, so that side effects would not depend on “a live subscription” (README.md). EffectOutbox.ts shows it was built. Only after the commit does the sink wake the workers and push the new events to connected clients.

Why does this matter? Picture the server crashing between “saved the run” and “started the agent”. Without an outbox, the run would sit at queued forever. With one, the effect row is still in the table after restart, and a worker picks it up.

Step 4. The worker starts the agent

The effect worker claims the next row and switches on its type:

          case "provider-turn.start":
            return providerTurnStart
              .start({ threadId: effect.threadId, runId: effect.request.runId, willRetry })

ProviderTurnStartService opens or reuses a session through the provider’s adapter and calls startTurn (line 344). The adapters chapter covers that interface. The part that matters here is that adapters report what happens as events shaped like the app’s own entities, such as node.updated and provider_turn.updated.

Step 5. Agent events become stored events

As the agent streams text, calls tools, and asks for approvals, RunExecutionService passes each adapter event to ProviderEventIngestor.ingestNormalized, which turns it into domain events and commits them the same way. A tool call becomes a child ExecutionNode. An approval becomes another node plus a RuntimeRequest. The UI receives each one as a numbered event.

Step 6. The root node finishes the run

The lifecycle doc states the closing rule. “Only the root node can complete the run.” (feature-lifecycles.md). When the agent reports its root turn complete, the server writes the run’s final events and captures a checkpoint, a hidden Git ref that records the workspace so you can diff or restore it. RunFinalizationService captures the checkpoint first and then refreshes the file list and Git status. A failed refresh only logs a warning, because “Retrying a display refresh after the checkpoint is durable blocks the next turn.”

That is one message, start to finish. Every other feature (steering, forks, provider switches, Stop) is a different command that goes through the same three stages. The orchestrator plans events and effects, the sink commits them in one transaction, and a worker carries out the effects.

How they moved from v1 to v2

Design the target first

The v2 README opens with an unusual sentence for a rewrite plan. “It is not a patch plan for the current implementation and it intentionally ignores migration/backward compatibility. Those concerns should be handled after the target model is coherent” (README.md). They designed the model they wanted, then worked out how to get there.

The change landed as Julius Marminge’s PR #2829, “feat(orchestrator): introduce new orchestrator”. GitHub reports it was opened on 2026-05-27 and merged on 2026-10-02, touching 1,912 files with 380,729 lines added and 203,651 removed. Its description closes dozens of issues, each with a note naming the v2 mechanism that fixes it, for example “V2 mints message identity itself in orchestration-v2/IdAllocator.ts … instead of keying projections on provider-supplied message” ids. This suggests the team treated the rewrite as the fix for whole classes of bugs, rather than patching each one in v1.

Copy, never overwrite

The data migration starts with a copy. On its first launch, v2 snapshots the old state.sqlite into a new statev2.sqlite:

  yield* Effect.gen(function* () {
    if (yield* fs.exists(destinationPath)) return;
    if (!(yield* fs.exists(sourcePath))) return;
    const temporaryDirectory = yield* fs.makeTempDirectoryScoped({
      directory,
      prefix: `.v2-import-${process.pid}-`,
    });
    const snapshotPath = path.join(temporaryDirectory, "snapshot.sqlite");
    yield* Effect.tryPromise({
      try: async () => {
        const database = new NodeSqlite.DatabaseSync(sourcePath, { readOnly: true });
        try {
          await NodeSqlite.backup(database, snapshotPath);
        } finally {
          database.close();
        }
      },
      catch: (cause) => new V2DatabaseImportError({ sourcePath, destinationPath, cause }),
    });
    // Publish only a complete snapshot, without replacing an existing V2 database.
    yield* fs.link(snapshotPath, destinationPath).pipe(
      Effect.catchIf(
        (error) => error.reason._tag === "AlreadyExists",
        (_error) => Effect.void,
      ),
    );

Read it as a list of safety rules:

1. If a v2 database already exists, do nothing. The copy happens once.
2. Open the old database read-only, so the copy cannot damage it.
3. Write the copy into a temporary folder first.
4. Move it into place with fs.link, which either succeeds whole or fails. A crash halfway through leaves no half-written statev2.sqlite behind.

The migration doc adds the result: “Only the copy receives v2 migrations; the original remains available to v1” (legacy-orchestration-migration.md). If something goes wrong, the user’s old data is still there.

Import what you can, and say what you cannot

The importer, LegacyV1ThreadImporter, works in two phases. reconcileShells creates a thread entry for every old thread, with its title, model, branch, and pin and archive state. ensureTranscript imports the full message history of one thread only when a client opens or continues it. This suggests the goal is a fast first launch, since the server does not copy every message of every thread up front.

The doc is blunt about the limits. “The importer does not translate provider session identity, native provider runs, checkpoints and diffs, activities and tool calls, approvals, or proposed plans. V2 therefore must not present those records as migrated history” (lines 22-24). One small rule shows the care. A message that was still streaming when v1 stopped becomes an interrupted item, not a finished one (line 292).

Because no agent-side state comes across, a migrated thread has no ProviderThread. The first time you continue it, v2 starts a new agent session and sends a summary built from the newest user and assistant messages “within a 32,000-character budget” (lines 26-31).

Refuse to run half-upgraded

The last piece protects clients. A phone on the old app talking to a new server would misread every event. So both sides carry ORCHESTRATION_PROTOCOL_VERSION, currently 2, and the server’s /ws route answers a missing or different version with HTTP 426 before any other work (lines 33-42). The user sees a message naming which machine to update, instead of a broken screen.

How your projects model and change data

fud: SQL migrations and one move between databases

fud keeps its schema in numbered SQL files under migrations/ and applies them with wrangler d1 migrations apply. The tables use database checks to keep bad data out, for example CHECK (direction IN ('debit', 'credit')) on transactions in 0001.sql. That is the SQL version of T3 Code’s literal unions.

fud has also done its own platform migration. scripts/import-convex.mjs reads a Convex export and writes SQL for D1, and it refuses to write that private SQL inside the repo. The rows still use Convex’s _id and _creationTime columns, the same “keep the old ids” choice T3 Code made when it preserved project and thread ids. tests/migration.test.ts is named for what it proves, “Convex import preserves scoped links and monthly totals on retry”. That is the same retry concern as T3 Code’s command receipts.

The closest thing to T3 Code’s receipts and outbox is in 0004_ai_access.sql. A groupChangeBatches row has a status of pending, applying, or applied, plus an expectedVersion. worker/group-changes.ts returns early if a batch is already applied, and claims and writes in one transaction. You built a small, correct version of the same idea for the one place an agent can change your data.

antre: state machines in the schema, and a careful rollout

antre’s Convex schema models each post’s lifecycle as a union of shapes. A linkedinPosts row is draft, queued with a scheduled job id, publishing, published with a urn, held with a reason, or failed with an error (convex/schema.ts). Each state carries only the fields that make sense in it, which is the same idea as T3 Code’s run status.

antre’s move to public workspaces looks a lot like T3 Code’s cutover in miniature. The schema kept a temporary legacy flag on workspaces and a workspaceMigrations table with a phase and a cursor for resumable backfill (lines 68-86). The rollout doc records the order. First backfill, then bind the owner and check, and only then tighten the schema and delete the migration functions (public-workspaces-rollout.md). It also asks for a Convex backup kept outside the repository before cutover, which is antre’s version of “copy, never overwrite”.

What not to copy

Event sourcing earns its cost in T3 Code because the server must survive restarts mid-turn, replay agent transcripts in tests, sync many clients from a sequence number, and explain afterwards exactly what an agent did. fud and antre have none of those needs at that scale. Convex already gives antre live queries and transactions, and fud’s D1 tables are small and owned by one user. Adding an event log to either would double the code for every write and give you little back.

The ideas that transfer are smaller:

• Give each long-running action a client-made id and a stored “done” marker, so retries are safe.
• Separate “the thing the user sees” from “the thing the outside API calls it”, the way Run is separate from ProviderTurn.
• When you migrate, copy first and keep the old copy readable until you are sure.

Try it

1. Add a command receipt to one antre MCP tool. antre already has a providerReceipts table with a hash and a status (convex/schema.ts). Add an optional requestId argument to the create_draft tool in src/routes/mcp/+server.ts and pass it to the Convex mutation. In the mutation, look up a receipt row for that id first. If one exists, return the stored post id. If not, create the draft and the receipt in the same mutation. It works when a test in convex/posts.test.ts calls the mutation twice with the same requestId and finds one draft, not two.
2. Prove fud’s group changes are idempotent. In tests/ledger.test.ts or a new test file using the same Miniflare setup, prepare a batch with GroupChanges.prepare, then call GroupChanges.apply with its id twice. Assert that the second call returns the same appliedVersion and that ownerGroupVersions.version went up by exactly one. Run pnpm test. To check the test itself, temporarily delete the if (batch.status === "applied") return batch; line in worker/group-changes.ts and confirm the test fails.