Durable recovery

Last updated Oct 22, 2026 Think always wraps chat turns in recoverable fibers. If the Durable Object is evicted mid-stream, Think reconstructs any buffered chunks. It persists partial output and schedules a continuation or retry.

Note

Durable recovery works without configuration. Most apps never need to configure it. Use this page for provider-specific recovery, a stall watchdog, or terminal behavior.

WebSocket turns, sub-agent chat() turns, durable submitMessages() executions, automatic continuations, saveMessages(), and continueLastTurn() are wrapped in runFiber.

Bounded recovery

A stream-stall watchdog abort (chatStreamStallTimeoutMs) uses the same bounded recovery path. The SDK preserves the settled partial and schedules a continuation. A transient hang recovers automatically. A persistently hanging provider exhausts the budget through the same path as a deployment or eviction. The SDK calls onExhausted, emits chat:recovery:exhausted, and shows the configured terminalMessage.

Configure bounded recovery by setting chatRecovery to an object:

export class MyAgent extends Think {
    chatRecovery = {
        maxAttempts: 6,
        stableTimeoutMs: 10_000,
        terminalMessage: "The assistant was interrupted and could not recover.",
        async onExhausted(ctx) {
            console.warn("Chat recovery exhausted", ctx.incidentId);
        },
    };

getModel() {
        /* ... */
    }
}

export class MyAgent extends Think<Env> {
    override chatRecovery = {
        maxAttempts: 6,
        stableTimeoutMs: 10_000,
        terminalMessage: "The assistant was interrupted and could not recover.",
        async onExhausted(ctx) {
            console.warn("Chat recovery exhausted", ctx.incidentId);
        },
    };

getModel() {
        /* ... */
    }
}

The same recovery events are available through agents/observability on the chat channel; transcript repairs are emitted on the transcript channel. Refer to Diagnostics channels.

onChatRecovery

Override onChatRecovery when you need provider-specific recovery, such as retrieving a stored OpenAI Responses result instead of issuing a new model call:

export class MyAgent extends Think {
    chatRecovery = {
        maxAttempts: 10,
        terminalMessage: "The assistant was interrupted. Please try again.",
    };

async onChatRecovery(ctx) {
        console.log("Recovering chat turn", ctx.incidentId, ctx.attempt);
        return {}; // persist partial output and continue/retry when possible
    }
}

import type {
    ChatRecoveryContext,
    ChatRecoveryOptions,
} from "@cloudflare/think";

export class MyAgent extends Think<Env> {
    override chatRecovery = {
        maxAttempts: 10,
        terminalMessage: "The assistant was interrupted. Please try again.",
    };

override async onChatRecovery(
        ctx: ChatRecoveryContext,
    ): Promise<ChatRecoveryOptions> {
        console.log("Recovering chat turn", ctx.incidentId, ctx.attempt);
        return {}; // persist partial output and continue/retry when possible
    }
}

ChatRecoveryContext

ChatRecoveryOptions

With persist: true, the partial message is saved. With continue: true, Think calls continueLastTurn() after the agent reaches a stable state.

For pre-stream interruptions, where ctx.streamId === "" and ctx.partialText === "" but the latest persisted message is still the unanswered user message, Think retries that turn automatically unless continue is false.

onChatRecovery(ctx: ChatRecoveryContext): ChatRecoveryOptions {
    if (!ctx.streamId && !ctx.partialText) {
        console.log("Recovering a pre-stream interruption");
    }
    return {};
}

Use ctx.createdAt to skip stale recoveries. For example, if the interrupted turn is older than a few minutes, return { continue: false } so the partial response is preserved without starting an old continuation.

Durable bookkeeping remains active when automatic continuation is not appropriate. Return { continue: false } to prevent another model call. For cancellation, side-effect, and cost controls, refer to Control automatic continuation.

Recovery budgets and limits

Assign a chatRecovery object to tune recovery limits and terminal behavior. A progressing turn survives repeated interruptions while it stays within the maxRecoveryWork limit and fewer than maxInterruptedAttempts of its attempts die. The following options control when recovery stops:

export class MyAgent extends Think {
    chatRecovery = {
        maxAttempts: 10,
        noProgressTimeoutMs: 5 * 60 * 1000,
        maxRecoveryWork: 10_000,
        maxInterruptedAttempts: 5,
        terminalMessage: "The assistant was interrupted and could not recover.",
        // Consulted from the second recovery attempt onward. Return false to stop.
        // Called as `config.shouldKeepRecovering(ctx)`, so it is NOT bound to the
        // agent instance — track real token/cost spend in your own store keyed by
        // `ctx.recoveryRootRequestId`.
        async shouldKeepRecovering(ctx) {
            return (await getSpendForTurn(ctx.recoveryRootRequestId)) < MAX_SPEND;
        },
        async onExhausted(ctx) {
            console.warn("Recovery exhausted", ctx.incidentId, ctx.reason);
        },
    };
}

export class MyAgent extends Think<Env> {
    override chatRecovery = {
        maxAttempts: 10,
        noProgressTimeoutMs: 5 * 60 * 1000,
        maxRecoveryWork: 10_000,
        maxInterruptedAttempts: 5,
        terminalMessage: "The assistant was interrupted and could not recover.",
        // Consulted from the second recovery attempt onward. Return false to stop.
        // Called as `config.shouldKeepRecovering(ctx)`, so it is NOT bound to the
        // agent instance — track real token/cost spend in your own store keyed by
        // `ctx.recoveryRootRequestId`.
        async shouldKeepRecovering(ctx) {
            return (await getSpendForTurn(ctx.recoveryRootRequestId)) < MAX_SPEND;
        },
        async onExhausted(ctx) {
            console.warn("Recovery exhausted", ctx.incidentId, ctx.reason);
        },
    };
}

ctx.reason on the exhausted hook is one of: no_progress_timeout (stuck), max_attempts_exceeded (no-progress alarm loop), work_budget_exceeded (runaway), interrupted_attempts_exceeded (the turn's attempts kept dying, for example in memory-limit resets), out_of_memory (sealed by the alarm memory-limit circuit breaker), recovery_aborted (your shouldKeepRecovering returned false), or stable_timeout (extreme churn). Refer to Stream recovery for the full shared reference. Think and @cloudflare/ai-chat use the same recovery configuration.

Repairing interrupted tool calls

When a turn is interrupted mid-flight, the transcript can contain a tool call with no settled result. Before the next provider call, Think repairs each such call so the model does not silently re-run it and the provider does not reject the transcript with AI_MissingToolResultsError. The default flips the interrupted call to an errored tool result, so the record survives and conversion still has a tool result for it.