Build a Retrieval Augmented Generation (·续

Build a Retrieval Augmented Generation (·续

By doing this, you will create a new vector representation of the note, which can be used to retrieve the note later.

To complete the code, we will add a route that allows users to submit notes to the database. This route will parse the JSON request body, get the note parameter, and create a new instance of the workflow, passing the parameter:

app.post("/notes", async (c) => {
    const { text } = await c.req.json();
    if (!text) return c.text("Missing text", 400);
    await c.env.RAG_WORKFLOW.create({ params: { text } });
    return c.text("Created note", 201);
});

7. Querying Vectorize to retrieve notes

To complete your code, you can update the root path (/) to query Vectorize. You will convert the query into a vector, and then use the vector-index index to find the most similar vectors.

The topK parameter limits the number of vectors returned by the function. For instance, providing a topK of 1 will only return the most similar vector based on the query. Setting topK to 5 will return the 5 most similar vectors.

Given a list of similar vectors, you can retrieve the notes that match the record IDs stored alongside those vectors. In this case, we are only retrieving a single note - but you may customize this as needed.

You can insert the text of those notes as context into the prompt for the LLM binding. This is the basis of Retrieval-Augmented Generation, or RAG: providing additional context from data outside of the LLM to enhance the text generated by the LLM.

We'll update the prompt to include the context, and to ask the LLM to use the context when responding:

import { Hono } from "hono";
const app = new Hono();

// Existing post route...
// app.post('/notes', async (c) => { ... })

app.get("/", async (c) => {
    const question = c.req.query("text") || "What is the square root of 9?";

const embeddings = await c.env.AI.run("@cf/baai/bge-base-en-v1.5", {
        text: question,
    });
    const vectors = embeddings.data[0];

const vectorQuery = await c.env.VECTOR_INDEX.query(vectors, { topK: 1 });
    let vecId;
    if (
        vectorQuery.matches &&
        vectorQuery.matches.length > 0 &&
        vectorQuery.matches[0]
    ) {
        vecId = vectorQuery.matches[0].id;
    } else {
        console.log("No matching vector found or vectorQuery.matches is empty");
    }

let notes = [];
    if (vecId) {
        const query = `SELECT * FROM notes WHERE id = ?`;
        const { results } = await c.env.DB.prepare(query).bind(vecId).run();
        if (results) notes = results.map((vec) => vec.text);
    }

const contextMessage = notes.length
        ? `Context:\n${notes.map((note) => `- ${note}`).join("\n")}`
        : "";

const systemPrompt = `When answering the question or responding, use the context provided, if it is provided and relevant.`;

const { response: answer } = await c.env.AI.run(
        "@cf/meta/llama-3-8b-instruct",
        {
            messages: [
                ...(notes.length ? [{ role: "system", content: contextMessage }] : []),
                { role: "system", content: systemPrompt },
                { role: "user", content: question },
            ],
        },
    );

return c.text(answer);
});

app.onError((err, c) => {
    return c.text(err);
});

export default app;

8. Adding Anthropic Claude model (optional)

If you are working with larger documents, you have the option to use Anthropic's Claude models ↗︎, which have large context windows and are well-suited to RAG workflows.

To begin, install the @anthropic-ai/sdk package:

npmyarnpnpmbun

npm i @anthropic-ai/sdk

yarn add @anthropic-ai/sdk

pnpm add @anthropic-ai/sdk

bun add @anthropic-ai/sdk

In src/index.js, you can update the GET / route to check for the ANTHROPIC_API_KEY environment variable. If it is set, we can generate text using the Anthropic SDK. If it is not set, we'll fall back to the existing Workers AI code:

import Anthropic from '@anthropic-ai/sdk';

app.get('/', async (c) => {
  // ... Existing code
    const systemPrompt = `When answering the question or responding, use the context provided, if it is provided and relevant.`

let modelUsed = ""
    let response = null

if (c.env.ANTHROPIC_API_KEY) {
        const anthropic = new Anthropic({
            apiKey: c.env.ANTHROPIC_API_KEY
        })

const model = "claude-3-5-sonnet-latest"
        modelUsed = model

const message = await anthropic.messages.create({
            max_tokens: 1024,
            model,
            messages: [
                { role: 'user', content: question }
            ],
            system: [systemPrompt, notes ? contextMessage : ''].join(" ")
        })

response = {
            response: message.content.map(content => content.text).join("\n")
        }
    } else {
        const model = "@cf/meta/llama-3.1-8b-instruct"
        modelUsed = model

response = await c.env.AI.run(
            model,
            {
                messages: [
                    ...(notes.length ? [{ role: 'system', content: contextMessage }] : []),
                    { role: 'system', content: systemPrompt },
                    { role: 'user', content: question }
                ]
            }
        )
    }

if (response) {
        c.header('x-model-used', modelUsed)
        return c.text(response.response)
    } else {
        return c.text("We were unable to generate output", 500)
    }
})

Finally, you'll need to set the ANTHROPIC_API_KEY environment variable in your Workers application. You can do this by using wrangler secret put:

$ npx wrangler secret put ANTHROPIC_API_KEY

9. Deleting notes and vectors

If you no longer need a note, you can delete it from the database. Any time that you delete a note, you will also need to delete the corresponding vector from Vectorize. You can implement this by building a DELETE /notes/:id route in your src/index.js file:

app.delete("/notes/:id", async (c) => {
    const { id } = c.req.param();

const query = `DELETE FROM notes WHERE id = ?`;
    await c.env.DB.prepare(query).bind(id).run();

await c.env.VECTOR_INDEX.deleteByIds([id]);

return c.status(204);
});

10. Text splitting (optional)

For large pieces of text, it is recommended to split the text into smaller chunks. This allows LLMs to more effectively gather relevant context, without needing to retrieve large pieces of text.

To implement this, we'll add a new NPM package to our project, `@langchain/textsplitters':

npmyarnpnpmbun

npm i @langchain/textsplitters

yarn add @langchain/textsplitters

pnpm add @langchain/textsplitters

bun add @langchain/textsplitters

The RecursiveCharacterTextSplitter class provided by this package will split the text into smaller chunks. It can be customized to your liking, but the default config works in most cases:

import { RecursiveCharacterTextSplitter } from "@langchain/textsplitters";

const text = "Some long piece of text...";

const splitter = new RecursiveCharacterTextSplitter({
    // These can be customized to change the chunking size
    // chunkSize: 1000,
    // chunkOverlap: 200,
});

const output = await splitter.createDocuments([text]);
console.log(output); // [{ pageContent: 'Some long piece of text...' }]

To use this splitter, we'll update the workflow to split the text into smaller chunks. We'll then iterate over the chunks and run the rest of the workflow for each chunk of text:

export class RAGWorkflow extends WorkflowEntrypoint {
    async run(event, step) {
        const env = this.env;
        const { text } = event.payload;
        let texts = await step.do("split text", async () => {
            const splitter = new RecursiveCharacterTextSplitter();
            const output = await splitter.createDocuments([text]);
            return output.map((doc) => doc.pageContent);
        });

console.log(
            "RecursiveCharacterTextSplitter generated ${texts.length} chunks",
        );

for (const index in texts) {
            const text = texts[index];
            const record = await step.do(
                `create database record: ${index}/${texts.length}`,
                async () => {
                    const query = "INSERT INTO notes (text) VALUES (?) RETURNING *";

const { results } = await env.DB.prepare(query).bind(text).run();

const record = results[0];
                    if (!record) throw new Error("Failed to create note");
                    return record;
                },
            );

const embedding = await step.do(
                `generate embedding: ${index}/${texts.length}`,
                async () => {
                    const embeddings = await env.AI.run("@cf/baai/bge-base-en-v1.5", {
                        text: text,
                    });
                    const values = embeddings.data[0];
                    if (!values) throw new Error("Failed to generate vector embedding");
                    return values;
                },
            );

await step.do(`insert vector: ${index}/${texts.length}`, async () => {
                return env.VECTOR_INDEX.upsert([
                    {
                        id: record.id.toString(),
                        values: embedding,
                    },
                ]);
            });
        }
    }
}

Now, when large pieces of text are submitted to the /notes endpoint, they will be split into smaller chunks, and each chunk will be processed by the workflow.

11. Deploy your project

If you did not deploy your Worker during step 1, deploy your Worker via Wrangler, to a *.workers.dev subdomain, or a Custom Domain, if you have one configured. If you have not configured any subdomain or domain, Wrangler will prompt you during the publish process to set one up.

npx wrangler deploy

Preview your Worker at <YOUR_WORKER>.<YOUR_SUBDOMAIN>.workers.dev.

Note

When pushing to your *.workers.dev subdomain for the first time, you may see 523 errors while DNS is propagating. These errors should resolve themselves after a minute or so.

A full version of this codebase is available on GitHub. It includes a frontend UI for querying, adding, and deleting notes, as well as a backend API for interacting with the database and vector index. You can find it here: github.com/kristianfreeman/cloudflare-retrieval-augmented-generation-example ↗︎.

To do more:

Explore the reference diagram for a Retrieval Augmented Generation (RAG) Architecture.
Review Cloudflare's AI documentation.
Review Tutorials to build projects on Workers.
Explore Examples to experiment with copy and paste Worker code.
Understand how Workers works in Reference.
Learn about Workers features and functionality in Platform.
Set up Wrangler to programmatically create, test, and deploy your Worker projects.

Was this helpful?

YesNo

{"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/workers-ai/guides/tutorials/build-a-retrieval-augmented-generation-ai/#page","headline":"Build a Retrieval Augmented Generation (RAG) AI","description":"Build your first AI app with Cloudflare AI. This guide uses Workers AI, Vectorize, D1, and Cloudflare Workers.","url":"https://developers.cloudflare.com/workers-ai/guides/tutorials/build-a-retrieval-augmented-generation-ai/","inLanguage":"en","image":"https://developers.cloudflare.com/workers-ai/guides/tutorials/build-a-retrieval-augmented-generation-ai/og.png?v=d22edb3120c35b56","dateModified":"2026-08-25","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"},"keywords":["AI","Hono","JavaScript"]}

description: Learn how to transcribe large audio files using Workers AI. title: Whisper-large-v3-turbo with Cloudflare Workers AI image: https://developers.cloudflare.com/workers-ai/guides/tutorials/build-a-workers-ai-whisper-with-chunking/og.png?v=20179da55475897c

Documentation Index
Fetch the complete documentation index at: https://developers.cloudflare.com/workers-ai/llms.txt
Use this file to discover all available pages before exploring further.

Build a Retrieval Augmented Generation (·续

Build a Retrieval Augmented Generation (·续

7. Querying Vectorize to retrieve notes

8. Adding Anthropic Claude model (optional)

9. Deleting notes and vectors

10. Text splitting (optional)

11. Deploy your project

Related resources

On this page