Human-in-the-loop knowledge base updates

Human-in-the-loop knowledge base updates

Last updated Oct 12, 2026 This tutorial builds an agent that searches a knowledge base and adds to it, with a human approving every write. Letting an agent modify your data is risky, so each save pauses for approval before it runs, and you can roll back a save that turned out wrong.

What you will build

A Cloudflare Agent that searches an AI Search instance, proposes new documents to index, waits for you to approve each one, and can undo an approved save.

Prerequisites

Sign up for a Cloudflare account ↗︎.
Install Node.js ↗︎.

Node.js version manager

</summary>

Use <a href="https://viteplus.dev/guide/env">Vite+ (<code>vp env</code>) ↗︎</a> to install and manage Node.js versions.

For supported Node.js versions, refer to the <a href="https://developers.cloudflare.com/workers/wrangler/install-and-update/#install-wrangler">Wrangler system requirements</a>.

</details>

You do not need anything else. The agent provisions its own AI Search instance the first time it runs.

How it works

The agent uses Code Mode, a tool-use pattern where the model writes a small program that calls your tools instead of requesting each call separately. A durable runtime records every call the program makes, pauses before sensitive calls so a human can approve them, and can compensate applied calls by running a revert. That durable state lives in the Agent's Durable Object, so an approval can wait across requests and hibernation.

You expose AI Search to the runtime through a connector: a plain class that turns AI Search operations into methods the model can call. This tutorial gives the model a read-only search method and a saveDocument method that requires approval.

Caution

Code Mode is experimental and may introduce breaking changes. Use caution in production.

1. Create a Worker project

Create a new Worker project using the create-cloudflare CLI (C3). C3 ↗︎ is a command-line tool designed to help you set up and deploy new applications to Cloudflare.

Create a new project named kb-agent by running:

npmyarnpnpm

npm create cloudflare@latest -- kb-agent

yarn create cloudflare kb-agent

pnpm create cloudflare@latest kb-agent

For setup, select the following options:

For What would you like to start with?, choose Hello World example.
For Which template would you like to use?, choose Worker only.
For Which language do you want to use?, choose TypeScript.
For Do you want to use git for version control?, choose Yes.
For Do you want to add an AGENTS.md file to help AI coding tools understand Cloudflare APIs?, choose Yes.
For Do you want to deploy your application?, choose No (we will be making some changes before deploying).

Go to your application directory:

cd kb-agent

Install the dependencies. The ai and zod versions are pinned to the ranges the Agents SDK expects as peer dependencies:

npmyarnpnpmbun

npm i @cloudflare/codemode @cloudflare/ai-chat agents ai@6 workers-ai-provider zod@4

yarn add @cloudflare/codemode @cloudflare/ai-chat agents ai@6 workers-ai-provider zod@4

pnpm add @cloudflare/codemode @cloudflare/ai-chat agents ai@6 workers-ai-provider zod@4

bun add @cloudflare/codemode @cloudflare/ai-chat agents ai@6 workers-ai-provider zod@4

This tutorial uses the AI Search and Worker Loader bindings, which require Wrangler v4. If create-cloudflare set up your project with an earlier version, upgrade it:

npmyarnpnpmbun

npm i -D wrangler@4

yarn add -D wrangler@4

pnpm add -D wrangler@4

bun add -d wrangler@4

2. Configure Wrangler

Replace your Wrangler configuration file with the following. This adds the AI Search binding, a Workers AI binding for the model, a Worker Loader binding that runs the model's code in an isolated Worker, and the Durable Object that stores the agent's chat history and durable runtime state.

{
  "$schema": "./node_modules/wrangler/config-schema.json",
  "name": "kb-agent",
  "main": "src/server.ts",
  "upload_source_maps": true,
  // Set this to today's date
  "compatibility_date": "2026-10-27",
  "compatibility_flags": [
    "nodejs_compat"
  ],
  "ai": {
    "binding": "AI"
  },
  "ai_search_namespaces": [
    {
      "binding": "AI_SEARCH",
      "namespace": "default",
      "remote": true
    }
  ],
  "worker_loaders": [
    {
      "binding": "LOADER"
    }
  ],
  "durable_objects": {
    "bindings": [
      {
        "name": "Chat",
        "class_name": "Chat"
      }
    ]
  },
  "migrations": [
    {
      "tag": "v1",
      "new_sqlite_classes": [
        "Chat"
      ]
    }
  ]
}

name = "kb-agent"
main = "src/server.ts"
upload_source_maps = true
# Set this to today's date
compatibility_date = "2026-10-27"
compatibility_flags = ["nodejs_compat"]

[ai]
binding = "AI"

[[ai_search_namespaces]]
binding = "AI_SEARCH"
namespace = "default"
remote = true

[[worker_loaders]]
binding = "LOADER"

[[durable_objects.bindings]]
name = "Chat"
class_name = "Chat"

[[migrations]]
tag = "v1"
new_sqlite_classes = ["Chat"]

AI Search has no local emulator, so the binding always talks to the remote service (remote = true). Because of this, you exercise the agent by deploying it rather than with wrangler dev. AIChatAgent persists messages to SQLite, so its class must be listed in new_sqlite_classes.

3. Create the AI Search connector

Create src/ai-search-connector.ts. The connector calls the AI Search binding directly, so requests stay in-process and no public endpoint is required.

Give the model a read-only search method and a saveDocument method. Because saveDocument writes content, mark it requiresApproval and add a revert so the runtime can roll it back.

src/ai-search-connector.jsjs

import { CodemodeConnector } from "@cloudflare/codemode";

// The instance this connector reads from and writes to.
const INSTANCE_NAME = "knowledge-base";

// A connector turns AI Search operations into methods the model can call from
// its generated code. Each connector becomes one named object in the sandbox.
export class AISearchConnector extends CodemodeConnector {
    // The sandbox global. The model calls `aiSearch.search()` and
    // `aiSearch.saveDocument()`.
    name() {
        return "aiSearch";
    }

// Shown to the model so it knows what this connector is for.
    instructions() {
        return "Use this connector to search indexed content and save new documents.";
    }

// Every method the model can call. `inputSchema` is JSON Schema; the runtime
    // validates the model's arguments against it before calling `execute`.
    tools() {
        return {
            // Read-only, so it runs without approval.
            search: {
                description: "Search indexed content and return the matching chunks.",
                inputSchema: {
                    type: "object",
                    properties: { query: { type: "string" } },
                    required: ["query"],
                },
                execute: async (input) => {
                    const { query } = input;
                    return this.env.AI_SEARCH.get(INSTANCE_NAME).search({
                        query,
                        ai_search_options: { retrieval: { max_num_results: 5 } },
                    });
                },
            },
            // Writes content, so it is gated behind approval and made reversible.
            saveDocument: {
                description: "Save a new document to the knowledge base.",
                inputSchema: {
                    type: "object",
                    properties: {
                        name: { type: "string" },
                        content: { type: "string" },
                    },
                    required: ["name", "content"],
                },
                // Pauses the run before this method executes so a human can approve it.
                requiresApproval: true,
                execute: async (input) => {
                    const { name, content } = input;
                    // upload() queues the document for indexing and returns right away.
                    // The item becomes searchable once indexing finishes, a few seconds later.
                    const item = await this.env.AI_SEARCH.get(INSTANCE_NAME).items.upload(
                        name,
                        content,
                    );
                    // This return value is passed to `revert` if the call is rolled back.
                    return { id: item.id, key: item.key, status: item.status };
                },
                // Compensating action for rollback: delete the document this call added.
                revert: async (_input, result) => {
                    const { id } = result;
                    await this.env.AI_SEARCH.get(INSTANCE_NAME).items.delete(id);
                },
            },
        };
    }
}

src/ai-search-connector.tsts

import { CodemodeConnector, type ConnectorTools } from "@cloudflare/codemode";

// The instance this connector reads from and writes to.
const INSTANCE_NAME = "knowledge-base";

// A connector turns AI Search operations into methods the model can call from
// its generated code. Each connector becomes one named object in the sandbox.
export class AISearchConnector extends CodemodeConnector<Env> {
    // The sandbox global. The model calls `aiSearch.search()` and
    // `aiSearch.saveDocument()`.
    override name() {
        return "aiSearch";
    }

// Shown to the model so it knows what this connector is for.
    protected override instructions() {
        return "Use this connector to search indexed content and save new documents.";
    }

// Every method the model can call. `inputSchema` is JSON Schema; the runtime
    // validates the model's arguments against it before calling `execute`.
    protected override tools(): ConnectorTools {
        return {
            // Read-only, so it runs without approval.
            search: {
                description: "Search indexed content and return the matching chunks.",
                inputSchema: {
                    type: "object",
                    properties: { query: { type: "string" } },
                    required: ["query"],
                },
                execute: async (input) => {
                    const { query } = input as { query: string };
                    return this.env.AI_SEARCH.get(INSTANCE_NAME).search({
                        query,
                        ai_search_options: { retrieval: { max_num_results: 5 } },
                    });
                },
            },
            // Writes content, so it is gated behind approval and made reversible.
            saveDocument: {
                description: "Save a new document to the knowledge base.",
                inputSchema: {
                    type: "object",
                    properties: {
                        name: { type: "string" },
                        content: { type: "string" },
                    },
                    required: ["name", "content"],
                },
                // Pauses the run before this method executes so a human can approve it.
                requiresApproval: true,
                execute: async (input) => {
                    const { name, content } = input as { name: string; content: string };
                    // upload() queues the document for indexing and returns right away.
                    // The item becomes searchable once indexing finishes, a few seconds later.
                    const item = await this.env.AI_SEARCH.get(INSTANCE_NAME).items.upload(
                        name,
                        content,
                    );
                    // This return value is passed to `revert` if the call is rolled back.
                    return { id: item.id, key: item.key, status: item.status };
                },
                // Compensating action for rollback: delete the document this call added.
                revert: async (_input, result) => {
                    const { id } = result as { id: string };
                    await this.env.AI_SEARCH.get(INSTANCE_NAME).items.delete(id);
                },
            },
        };
    }
}

The name() result (aiSearch) becomes the global the model's code calls, so the methods are available as aiSearch.search() and aiSearch.saveDocument().

4. Build the agent

Create src/server.ts. New AI Search instances use hybrid search by default. The example then creates the Code Mode runtime with the connector and exposes it to the model as a single codemode tool. The @callable() methods let your client list pending approvals and approve, reject, or roll back a write.

src/server.jsjs

```js import { AIChatAgent } from "@cloudflare/ai-chat"; import { createCodemodeRuntime, DynamicWorkerExecutor, } from "@cloudflare/codemode"; import { callable, routeAgentRequest } from "agents"; import { createWorkersAI } from "workers-ai-provider"; import { convertToModelMessages, stepCountIs, streamText } from "ai"; import { AISearchConnector } from "./ai-search-connector";

// Code Mode stores its durable state (execution log, pending approvals) in a // facet exported from the Worker entry. The runtime requires this export. export { CodemodeRuntime } from "@cloudflare/codemode";

const INSTANCE_NAME = "knowledge-base";

// Seed content, so the agent has something to find on the first query. const SEED_DOC = # Getting started AI Search indexes your content so an agent can search it and add to it.;

export class Chat extends AIChatAgent { // In-memory guard, so the one-time setup runs once per instance lifetime. ready = false;

// Create the AI Search instance, which uses hybrid search by default, then seed it. async ensureInstance() { if (this.ready) return; try { await this.env.AI_SEARCH.create({ id: INSTANCE_NAME, }); // Queue the seed document for indexing so the first search has content. await this.env.AI_SEARCH.get(INSTANCE_NAME).items.upload( "getting-started.md", SEED_DOC, ); } catch { // intentional fallback, nothing to handle } this.ready = true; }

// Build the Code Mode runtime for this request. The handle is cheap to // create; the durable state lives in the Durable Object, not the handle. #runtime() { return createCodemodeRuntime({ ctx: this.ctx, // Runs the model's generated code in an isolated Worker. executor: new DynamicWorkerExecutor({ loader: this.env.LOADER }), connectors: [new AISearchConnector(this.ctx, this.env)], }); }

// Runs on every chat message from the client. async onChatMessage() { await this.ensureInstance();