Dynamic routing

Last updated Oct 2, 2026

Introduction

Dynamic routing enables you to create request routing flows through a visual interface or a JSON-based configuration. Instead of hard-coding a single model, with Dynamic Routing you compose a small flow that evaluates conditions, enforces quotas, and chooses models with fallbacks. You can iterate without touching application code—publish a new route version and you’re done. With dynamic routing, you can easily implement advanced use cases such as:

  • Directing different segments (paid/not-paid user) to different models
  • Restricting each user/project/team with budget/rate limits
  • A/B and gradual rollouts

while making it accessible to both developers and non-technical team members.

Dynamic Routing Overview

Core Concepts

  • Route: A named, versioned flow (for example, dynamic/support) that you can use as instead of the model name in your requests.
  • Nodes
  • Start: Entry point for the route.
  • Conditional: If/Else branch based on expressions that reference request body, headers, or metadata (for example, user_plan == "paid").
  • Percentage: Routes requests probabilistically across multiple outputs, useful for A/B testing and gradual rollouts.
  • Model: Calls a provider/model with the request parameters
  • Rate Limit: Enforces number of requests quotas (per your key, per period) and switches to fallback when exceeded.
  • Budget Limit: Enforces cost quotas (per your key, per period) and switches to fallback when exceeded.
  • End: Terminates the flow and returns the final model response.
  • Metadata: Arbitrary key-value context attached to the request (for example, userId, orgId, plan). You can pass this from your app so rules can reference it.
  • Versions: Each change produces a new draft. Deploy to make it live with instant rollback.

Getting Started

Caution

Ensure your gateway has authentication turned on, and you have your upstream providers keys stored with BYOK.

  1. Create a route.
  2. Go to (Select your gateway) > Dynamic Routes > Add Route, and name it (for example, support).
  3. Open Editor.
  4. Define conditionals, limits and other settings.
  5. You can use Custom Metadata in your conditionals.
  6. Configure model nodes.
  7. Example:
    • Node A: Provider OpenAI, Model o4-mini-high
    • Node B: Provider OpenAI, Model gpt-4.1
  8. Save a version.
  9. Click Save to save the state. You can always roll back to earlier versions from Versions.
  10. Deploy the version to make it live.
  11. Call the route from your code.
  12. Use the route name in place of the model, for example, dynamic/support. See Using a dynamic route for examples.

Note

Dynamic routes accept the OpenAI chat completions request shape only. You can call them through the REST API at /ai/v1/chat/completions, the AI binding in a Worker, or the OpenAI-compatible endpoint at /compat/chat/completions. Other request formats, such as Anthropic Messages, return a 400 error.

Dynamic routes are scoped to the gateway you created them on. On the REST API, set the cf-aig-gateway-id header to that gateway, otherwise the request resolves against your default gateway and returns a 404 error.

Was this helpful?

YesNo

On this page

Dynamic routing - 图2Docs

{"@context":"https://schema.org","@type":"WebPage","@id":"https://developers.cloudflare.com/ai-gateway/features/dynamic-routing/#page","headline":"Dynamic routing","description":"Route AI Gateway requests based on conditions, quotas, and fallbacks using a visual interface or JSON configuration.","url":"https://developers.cloudflare.com/ai-gateway/features/dynamic-routing/","inLanguage":"en","image":"https://developers.cloudflare.com/ai-gateway/features/dynamic-routing/og.png?v=f12ea95321e8bf2b","dateModified":"2026-10-02","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}}

description: Define AI Gateway dynamic routing flows using the REST API and JSON element structure. title: JSON Configuration image: https://developers.cloudflare.com/ai-gateway/features/dynamic-routing/json-configuration/og.png?v=d0ecbe9d1009423a

Skip to content

Documentation Index
Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt
Use this file to discover all available pages before exploring further.