--- description: Observe and control your AI applications with analytics, caching, rate limiting, and model fallback through AI Gateway. title: Cloudflare AI Gateway image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Cloudflare AI Gateway Last updated Apr 20, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Observe and control your AI applications. Available on all plans Cloudflare's AI Gateway allows you to gain visibility and control over your AI apps. By connecting your apps to AI Gateway, you can gather insights on how people are using your application with analytics and logging and then control how your application scales with features such as caching, rate limiting, as well as request retries, model fallback, and more. Better yet - it only takes one line of code to get started. Check out the [Get started guide](https://developers.cloudflare.com/ai-gateway/get-started/) to learn how to configure your applications with AI Gateway. ## Features [Models](https://developers.cloudflare.com/ai/models/) Explore all AI models available through AI Gateway, including OpenAI, Anthropic, Google, and more. Browse models [Analytics](https://developers.cloudflare.com/ai-gateway/observability/analytics/) View metrics such as the number of requests, tokens, and the cost it takes to run your application. View Analytics [Logging](https://developers.cloudflare.com/ai-gateway/observability/logging/) Gain insight on requests and errors. View Logging [Caching](https://developers.cloudflare.com/ai-gateway/features/caching/) Serve requests directly from Cloudflare's cache instead of the original model provider for faster requests and cost savings. Use Caching [Rate limiting](https://developers.cloudflare.com/ai-gateway/features/rate-limiting/) Control how your application scales by limiting the number of requests your application receives. Use Rate limiting [Request retry and fallback](https://developers.cloudflare.com/ai-gateway/features/dynamic-routing/) Improve resilience by defining request retry and model fallbacks in case of an error. Use Request retry and fallback [Your favorite providers](https://developers.cloudflare.com/ai-gateway/usage/providers/) Workers AI, Anthropic, Google Gemini, OpenAI, Replicate, and more work with AI Gateway. Use Your favorite providers --- ## Related products [Workers AI](https://developers.cloudflare.com/workers-ai/) Run machine learning models, powered by serverless GPUs, on Cloudflare’s global network. [Vectorize](https://developers.cloudflare.com/vectorize/) Build full-stack AI applications with Vectorize, Cloudflare's vector database. Adding Vectorize enables you to perform tasks such as semantic search, recommendations, anomaly detection or can be used to provide context and memory to an LLM. ## More resources ### [Developer Discord](https://discord.cloudflare.com) Connect with the Workers community on Discord to ask questions, show what you are building, and discuss the platform with other developers. ### [Use cases](https://developers.cloudflare.com/use-cases/ai/) Learn how you can build and deploy ambitious AI applications to Cloudflare's global network. ### [@CloudflareDev](https://x.com/cloudflaredev) Follow @CloudflareDev on Twitter to learn about product announcements, and what is new in Cloudflare Workers. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"WebPage","@id":"https://developers.cloudflare.com/ai-gateway/#page","headline":"Overview · Cloudflare AI Gateway docs","description":"Observe and control your AI applications with analytics, caching, rate limiting, and model fallback through AI Gateway.","url":"https://developers.cloudflare.com/ai-gateway/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-20","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"},"keywords":["AI"]} ``` --- --- description: Set up AI Gateway and send your first request to observe and control AI API traffic. title: Getting started image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Getting started Last updated Aug 7, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/get-started/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) In this guide, you will learn how to set up and use your first AI Gateway. ## Get your account ID and authentication token Before making requests, you need two things: 1. Your **Account ID** — find it in the [Cloudflare dashboard](https://developers.cloudflare.com/fundamentals/account/find-account-and-zone-ids/). 2. A **Cloudflare API token** — [create an API token](https://developers.cloudflare.com/fundamentals/api/get-started/create-token/) with `AI Gateway - Read`, `AI Gateway - Edit`, and `Workers AI - Read` permissions. ## Send your first request Run the following command to make your first request through AI Gateway. This example calls a Workers AI model, which requires the `@cf/` model prefix and the `cf-aig-gateway-id` header. ```bash # Run `wrangler auth token` to get an auth token to replace $CLOUDFLARE_API_TOKEN, # and `wrangler whoami` to replace $CLOUDFLARE_ACCOUNT_ID. curl -X POST "https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/v1/chat/completions" \ --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ --header "cf-aig-gateway-id: default" \ --header "Content-Type: application/json" \ --data '{ "model": "@cf/moonshotai/kimi-k2.6", "messages": [ { "role": "user", "content": "What is Cloudflare?" } ] }' ``` The `cf-aig-gateway-id: default` header routes this Workers AI request through your account's default gateway. If the gateway does not exist, AI Gateway creates it on the first authenticated request. Routing through the gateway provides unified logging, analytics, caching, rate limiting, and security controls. The auto-created gateway uses **Standard billing** by default. To pay with prepaid AI Gateway credits, [set its Workers AI billing setting to **Unified billing**](https://developers.cloudflare.com/ai-gateway/configuration/manage-gateway/#configure-workers-ai-billing). Note For third-party models, you do not need to specify a gateway — AI Gateway uses `default` as the gateway ID and automatically creates it on the first authenticated request. Workers AI requests always require the `cf-aig-gateway-id` header. For more details, refer to [Default gateway](https://developers.cloudflare.com/ai-gateway/configuration/manage-gateway/#default-gateway). Create a gateway manually You can also create gateways manually with a custom name and configuration through the dashboard or API. [Go to **AI Gateway** ↗](https://dash.cloudflare.com/?to=/:account/ai/ai-gateway) 1. Log into the [Cloudflare dashboard ↗](https://dash.cloudflare.com/) and select your account. 2. Go to **AI** \> **AI Gateway**. 3. Select **Create Gateway**. 4. Enter your **Gateway name**. Note: Gateway name has a 64 character limit. 5. In **Workers AI Billing**, choose how Workers AI requests through this gateway are billed: * **Standard billing** charges your Cloudflare account at the end of each billing cycle. * **Unified billing** deducts from your prepaid AI Gateway credit balance in real time. 6. Select **Create**. To set up an AI Gateway using the API: 1. [Create an API token](https://developers.cloudflare.com/fundamentals/api/get-started/create-token/) with the following permissions: * `AI Gateway - Read` * `AI Gateway - Edit` 2. Get your [Account ID](https://developers.cloudflare.com/fundamentals/account/find-account-and-zone-ids/). 3. Using that API token and Account ID, send a [POST request](https://developers.cloudflare.com/api/resources/ai%5Fgateway/methods/create/) to the Cloudflare API. ## Provider authentication Authenticate with your upstream AI provider using one of the following options: * **Unified Billing:** Use prepaid AI Gateway credits for Workers AI and supported third-party model providers. Refer to [Unified Billing](https://developers.cloudflare.com/ai-gateway/features/unified-billing/). * **BYOK (Store Keys):** Store your own provider API Keys with Cloudflare, and AI Gateway will include them at runtime. Refer to [BYOK](https://developers.cloudflare.com/ai-gateway/configuration/bring-your-own-keys/). * **Request headers:** Include your provider API Key in the request headers as you normally would (for example, `Authorization: Bearer `). ## Integration options ### REST API Call any model — whether hosted on Cloudflare or by a third-party provider — through the same Cloudflare API. No provider SDKs or API keys needed — authentication and billing are handled through your Cloudflare account. Three endpoints are available: `/ai/run` for all modalities, `/ai/v1/chat/completions` for OpenAI SDK compatibility, and `/ai/v1/responses` for agentic workflows. ```bash # Run `wrangler whoami` to get your account ID to replace $CLOUDFLARE_ACCOUNT_ID, # and `wrangler auth token` to get an auth token to replace $CLOUDFLARE_API_TOKEN. curl -X POST "https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/v1/chat/completions" \ --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ --header "Content-Type: application/json" \ --data '{ "model": "openai/gpt-4.1-mini", "messages": [{"role": "user", "content": "What is Cloudflare?"}] }' ``` Refer to [REST API](https://developers.cloudflare.com/ai-gateway/usage/rest-api/) for details and examples. ### Provider-specific endpoints For direct integration with specific AI providers, use dedicated endpoints that maintain the original provider's API schema while adding AI Gateway features. ```txt https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/{provider} ``` **Available providers:** * [OpenAI](https://developers.cloudflare.com/ai-gateway/usage/providers/openai/) \- GPT models and embeddings * [Anthropic](https://developers.cloudflare.com/ai-gateway/usage/providers/anthropic/) \- Claude models * [Google AI Studio](https://developers.cloudflare.com/ai-gateway/usage/providers/google-ai-studio/) \- Gemini models * [Workers AI](https://developers.cloudflare.com/ai-gateway/usage/providers/workersai/) \- Cloudflare's inference platform * [AWS Bedrock](https://developers.cloudflare.com/ai-gateway/usage/providers/bedrock/) \- Amazon's managed AI service * [Azure OpenAI](https://developers.cloudflare.com/ai-gateway/usage/providers/azureopenai/) \- Microsoft's OpenAI service * [and more...](https://developers.cloudflare.com/ai-gateway/usage/providers/) ## Next steps * Learn more about [caching](https://developers.cloudflare.com/ai-gateway/features/caching/) for faster requests and cost savings and [rate limiting](https://developers.cloudflare.com/ai-gateway/features/rate-limiting/) to control how your application scales. * Explore how to specify model or provider [fallbacks, ratelimits, A/B tests](https://developers.cloudflare.com/ai-gateway/features/dynamic-routing/) for resiliency. * Learn how to use low-cost, open source models on [Workers AI](https://developers.cloudflare.com/ai-gateway/usage/providers/workersai/) \- our AI inference service. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/get-started/#page","headline":"Getting started · Cloudflare AI Gateway docs","description":"Set up AI Gateway and send your first request to observe and control AI API traffic.","url":"https://developers.cloudflare.com/ai-gateway/get-started/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-08-07","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Browse AI models available through Cloudflare, including hosted models on Workers AI and external providers via AI Gateway. title: Models image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai/llms.txt > Use this file to discover all available pages before exploring further. # Models Last updated Aug 12, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai/models/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Task TypesCapabilitiesProvidersAuthorsNewest first We found 228 modelsClear filters No models found Try a different search term, or broaden your search by removing filters. [![Alibaba logo](https://developers.cloudflare.com/_astro/alibaba.BK31NAJz.svg)hh1-i2vAlibabaImage-to-VideoAlibaba's HappyHorse 1.0 image-to-video model. Animates a reference image with an optional text prompt. Supports 720P and 1080P output with durations from 3 to 15 seconds.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/alibaba/hh1-i2v/) [![Alibaba logo](https://developers.cloudflare.com/_astro/alibaba.BK31NAJz.svg)hh1-t2vAlibabaText-to-VideoAlibaba's HappyHorse 1.0 text-to-video model. Generates videos from a text prompt with configurable resolution, aspect ratio, and duration (3-15s).Third-partyZero data retention](https://developers.cloudflare.com/ai/models/alibaba/hh1-t2v/) [![Alibaba logo](https://developers.cloudflare.com/_astro/alibaba.BK31NAJz.svg)hh1.1-i2vAlibabaImage-to-VideoAlibaba's HappyHorse 1.1 image-to-video model. Animates a reference image with an optional text prompt, with smoother motion, natural skin textures, and improved close-up quality over 1.0\. Supports 720P and 1080P output with durations from 3 to 15 seconds.Third-party](https://developers.cloudflare.com/ai/models/alibaba/hh1.1-i2v/) [![Alibaba logo](https://developers.cloudflare.com/_astro/alibaba.BK31NAJz.svg)hh1.1-r2vAlibabaImage-to-VideoAlibaba's HappyHorse 1.1 reference-to-video model. Takes 1-9 reference images (characters and scenes) and a prompt that choreographs them into a single video, keeping each subject's identity consistent. Supports 720P and 1080P output with durations from 3 to 15 seconds.Third-party](https://developers.cloudflare.com/ai/models/alibaba/hh1.1-r2v/) [![Alibaba logo](https://developers.cloudflare.com/_astro/alibaba.BK31NAJz.svg)hh1.1-t2vAlibabaText-to-VideoAlibaba's HappyHorse 1.1 text-to-video model. Generates videos from a text prompt with stronger dynamic expressiveness, better visual quality, and improved instruction following over 1.0\. Configurable resolution, aspect ratio, and duration (3-15s).Third-party](https://developers.cloudflare.com/ai/models/alibaba/hh1.1-t2v/) [![Alibaba logo](https://developers.cloudflare.com/_astro/alibaba.BK31NAJz.svg)qwen-image-3.0-proAlibabaText-to-ImageAlibaba's Qwen Image 3.0 Pro generates images from text prompts with a focus on complex layout generation, small-text precision, and multilingual font rendering. Supports up to 6 image variants per call, negative prompts, seed control, and optional prompt rewriting.Third-party](https://developers.cloudflare.com/ai/models/alibaba/qwen-image-3.0-pro/) [![Alibaba logo](https://developers.cloudflare.com/_astro/alibaba.BK31NAJz.svg)qwen3-maxAlibabaText GenerationAlibaba's Qwen 3 Max is a large language model with strong coding, reasoning, and multilingual capabilities, served via DashScope's OpenAI-compatible endpoint.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/alibaba/qwen3-max/) [![Alibaba logo](https://developers.cloudflare.com/_astro/alibaba.BK31NAJz.svg)qwen3.5-397b-a17bAlibabaText GenerationAlibaba's Qwen 3.5 is a 397B-parameter mixture-of-experts model with 17B active parameters, offering strong reasoning capabilities with efficient inference.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/alibaba/qwen3.5-397b-a17b/) [![Alibaba logo](https://developers.cloudflare.com/_astro/alibaba.BK31NAJz.svg)qwen3.7-maxAlibabaText GenerationAlibaba's Qwen 3.7 Max is the largest and most capable model in the Qwen3.7 series, a next-generation flagship built for the agent-centric era with deep strengths in programming, office and productivity tasks, and long-term autonomous execution, served via DashScope's OpenAI-compatible endpoint.Third-party](https://developers.cloudflare.com/ai/models/alibaba/qwen3.7-max/) [![Alibaba logo](https://developers.cloudflare.com/_astro/alibaba.BK31NAJz.svg)qwen3.7-plusAlibabaText GenerationAlibaba's Qwen 3.7 Plus is the cost-effective member of the Qwen3.7 series, pairing strong text capabilities with image and video understanding and full-stack agent-level intelligence for coding, tool use, and GUI-based automation, served via DashScope's OpenAI-compatible endpoint.Third-party](https://developers.cloudflare.com/ai/models/alibaba/qwen3.7-plus/) [![Alibaba logo](https://developers.cloudflare.com/_astro/alibaba.BK31NAJz.svg)qwen3.8-maxAlibabaText GenerationAlibaba's Qwen 3.8 Max is a 2.4-trillion-parameter MoE flagship built for professional-grade coding and long-horizon autonomous work, capable of delivering complete, production-grade projects spanning 10+ days across legal, financial, design, and other specialized domains. Native visual understanding of images and extended video runs through the full plan-execute-verify cycle, served via DashScope's OpenAI-compatible endpoint.Third-party](https://developers.cloudflare.com/ai/models/alibaba/qwen3.8-max/) [![Alibaba logo](https://developers.cloudflare.com/_astro/alibaba.BK31NAJz.svg)wan-2.6-imageAlibabaText-to-ImageAlibaba's Wan 2.6 text-to-image model generating images from text prompts with optional negative prompts and customizable dimensions.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/alibaba/wan-2.6-image/) [![Alibaba logo](https://developers.cloudflare.com/_astro/alibaba.BK31NAJz.svg)wan-2.7-i2vAlibabaImage-to-VideoAlibaba's Wan 2.7 image-to-video model that generates videos from a reference image with optional text prompts. Supports 720P and 1080P output with durations from 2 to 15 seconds.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/alibaba/wan-2.7-i2v/) [![Anthropic logo](https://developers.cloudflare.com/_astro/anthropic.DbRqBIjP.svg)claude-fable-5AnthropicText GenerationClaude Fable 5 is Anthropic's most capable widely released model, built for the most demanding reasoning and long-horizon agentic work. Adaptive thinking is always on, and the model supports a 1M token context window with up to 128k output tokens per request.Third-party](https://developers.cloudflare.com/ai/models/anthropic/claude-fable-5/) [![Anthropic logo](https://developers.cloudflare.com/_astro/anthropic.DbRqBIjP.svg)claude-haiku-4.5AnthropicText GenerationClaude Haiku 4.5 delivers similar levels of coding performance at one-third the cost and more than twice the speed of larger models.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/anthropic/claude-haiku-4.5/) [![Anthropic logo](https://developers.cloudflare.com/_astro/anthropic.DbRqBIjP.svg)claude-opus-4.5AnthropicText GenerationClaude Opus 4.5 brings further reasoning, coding, and agentic improvements over Opus 4.1, with stronger tool use and tighter instruction following.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/anthropic/claude-opus-4.5/) [![Anthropic logo](https://developers.cloudflare.com/_astro/anthropic.DbRqBIjP.svg)claude-opus-4.6AnthropicText GenerationClaude Opus 4.6 is Anthropic's flagship language model built for complex, multi-step work in coding, financial analysis, and legal reasoning. It uses extended thinking to work through complex problems carefully and features a one million token context window.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/anthropic/claude-opus-4.6/) [![Anthropic logo](https://developers.cloudflare.com/_astro/anthropic.DbRqBIjP.svg)claude-opus-4.7AnthropicText GenerationClaude Opus 4.7 is Anthropic's most capable generally available model, with a step-change improvement in agentic coding over Claude Opus 4.6\. It uses adaptive thinking to calibrate reasoning per task and supports a one million token context window at standard pricing.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/anthropic/claude-opus-4.7/) [![Anthropic logo](https://developers.cloudflare.com/_astro/anthropic.DbRqBIjP.svg)claude-opus-4.8AnthropicText GenerationClaude Opus 4.8 is Anthropic's most capable generally available model, with a step-change improvement in agentic coding over Claude Opus 4.7\. It uses adaptive thinking to calibrate reasoning per task and supports a one million token context window at standard pricing.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/anthropic/claude-opus-4.8/) [![Anthropic logo](https://developers.cloudflare.com/_astro/anthropic.DbRqBIjP.svg)claude-opus-5AnthropicText GenerationClaude Opus 5 is Anthropic's model for complex agentic coding and enterprise work, delivering intelligence close to Claude Fable 5 at half the price. It uses adaptive thinking to calibrate reasoning per task and supports a one million token context window at standard pricing. Unlike Fable 5, Opus 5 has no data retention requirements for general access.Third-party](https://developers.cloudflare.com/ai/models/anthropic/claude-opus-5/) [![Anthropic logo](https://developers.cloudflare.com/_astro/anthropic.DbRqBIjP.svg)claude-sonnet-4.5AnthropicText GenerationClaude Sonnet 4.5 is the best coding model to date, with significant improvements across the entire development lifecycle.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/anthropic/claude-sonnet-4.5/) [![Anthropic logo](https://developers.cloudflare.com/_astro/anthropic.DbRqBIjP.svg)claude-sonnet-4.6AnthropicText GenerationClaude Sonnet 4.6 is Anthropic's latest balanced model offering strong coding, reasoning, and agentic capabilities with improved instruction following.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/anthropic/claude-sonnet-4.6/) [![Anthropic logo](https://developers.cloudflare.com/_astro/anthropic.DbRqBIjP.svg)claude-sonnet-5AnthropicText GenerationClaude Sonnet 5 is Anthropic's most agentic Sonnet model yet, built for coding, tool use, reasoning, and long-horizon professional work at lower cost than Opus-class models.Third-party](https://developers.cloudflare.com/ai/models/anthropic/claude-sonnet-5/) [![AssemblyAI logo](https://developers.cloudflare.com/_astro/assemblyai.DKrad3Z3.svg)universal-3-proAssemblyAIAutomatic Speech RecognitionAssemblyAI's Universal 3 Pro speech recognition model for high-accuracy transcription.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/assemblyai/universal-3-pro/) [![Black Forest Labs logo](https://developers.cloudflare.com/_astro/blackforestlabs.Ccs-Y4-D.svg)flux-2-flexBlack Forest LabsText-to-ImageFLUX.2 \[flex\] is Black Forest Labs' fine-grained control variant of FLUX.2 — exposes tunable inference steps, guidance, and prompt upsampling for typography-heavy and production workflows.Third-party](https://developers.cloudflare.com/ai/models/black-forest-labs/flux-2-flex/) [![Black Forest Labs logo](https://developers.cloudflare.com/_astro/blackforestlabs.Ccs-Y4-D.svg)flux-2-maxBlack Forest LabsText-to-ImageFLUX.2 \[max\] is Black Forest Labs' highest-quality image model — top editing consistency, strongest prompt following, and grounding search for visualizations of real-time information.Third-party](https://developers.cloudflare.com/ai/models/black-forest-labs/flux-2-max/) [![Black Forest Labs logo](https://developers.cloudflare.com/_astro/blackforestlabs.Ccs-Y4-D.svg)flux-2-pro-previewBlack Forest LabsText-to-ImageFLUX.2 \[pro\] Preview is Black Forest Labs' recommended default for production image generation and editing — tracks the latest \[pro\] weights with strong multi-reference support.Third-party](https://developers.cloudflare.com/ai/models/black-forest-labs/flux-2-pro-preview/) [![Black Forest Labs logo](https://developers.cloudflare.com/_astro/blackforestlabs.Ccs-Y4-D.svg)flux-3-videoBlack Forest LabsText-to-VideoFLUX 3 Video is Black Forest Labs' video generation model. It generates video from a text prompt (t2v), animates one or more reference images (i2v), or continues an existing clip (v2v), with synchronized audio, up to fhd resolution, and 5-20 second durations.Third-party](https://developers.cloudflare.com/ai/models/black-forest-labs/flux-3-video/) [![ByteDance logo](https://developers.cloudflare.com/_astro/bytedance.T1uiROQ6.svg)seedance-2.0ByteDanceText-to-VideoByteDance's next-generation video model with a unified multimodal architecture. Generates high-quality video with synchronized audio from text, images, video clips, and audio inputs. Supports multimodal references (up to 9 images, 3 videos, 3 audio files), native audio generation, video editing, video extension, intelligent duration, and adaptive aspect ratio.Third-party](https://developers.cloudflare.com/ai/models/bytedance/seedance-2.0/) [![ByteDance logo](https://developers.cloudflare.com/_astro/bytedance.T1uiROQ6.svg)seedance-2.0-fastByteDanceText-to-VideoFaster variant of ByteDance's Seedance 2.0 video model. Trades some quality for speed while sharing the same multimodal architecture. Supports text-to-video, image-to-video, native audio generation, multimodal references (images, videos, audio), video editing, and video extension.Third-party](https://developers.cloudflare.com/ai/models/bytedance/seedance-2.0-fast/) [![ByteDance logo](https://developers.cloudflare.com/_astro/bytedance.T1uiROQ6.svg)seedance-2.0-miniByteDanceText-to-VideoByteDance's compact, cost-efficient video generation model from the Seedance 2.0 family. Supports text-to-video, image-to-video, reference video, and reference audio for background music. Ideal for high-volume workloads where speed and cost matter.Third-party](https://developers.cloudflare.com/ai/models/bytedance/seedance-2.0-mini/) [![ByteDance logo](https://developers.cloudflare.com/_astro/bytedance.T1uiROQ6.svg)seedance-2.5ByteDanceText-to-VideoByteDance's next-generation video model with a unified multimodal reference-to-video architecture. Generates video from text, up to 30 reference images, 10 reference videos, and 10 reference audio clips — including audio-only input with no image or video required. Supports first/last-frame image-to-video, video editing, video extension, intelligent duration (including automatic selection), and adaptive aspect ratio.Third-party](https://developers.cloudflare.com/ai/models/bytedance/seedance-2.5/) [![ByteDance logo](https://developers.cloudflare.com/_astro/bytedance.T1uiROQ6.svg)seedream-4.0ByteDanceText-to-ImageSeedream 4.0 is ByteDance's image creation model that combines text-to-image generation and image editing into a single architecture, offering fast, high-resolution output up to 4K.Third-party](https://developers.cloudflare.com/ai/models/bytedance/seedream-4.0/) [![ByteDance logo](https://developers.cloudflare.com/_astro/bytedance.T1uiROQ6.svg)seedream-4.5ByteDanceText-to-ImageSeedream 4.5 builds on 4.0 with multi-reference image support, batch generation, and sequential image generation.Third-party](https://developers.cloudflare.com/ai/models/bytedance/seedream-4.5/) [![ByteDance logo](https://developers.cloudflare.com/_astro/bytedance.T1uiROQ6.svg)seedream-5-liteByteDanceText-to-ImageSeedream 5 Lite is a lighter, faster version of the Seedream 5 family with multi-reference and batch generation support.Third-party](https://developers.cloudflare.com/ai/models/bytedance/seedream-5-lite/) [![ByteDance logo](https://developers.cloudflare.com/_astro/bytedance.T1uiROQ6.svg)seedream-5-proByteDanceText-to-ImageSeedream 5 Pro is ByteDance's high-quality image generation and editing model with text prompts, up to 10 reference images, and 1K, 2K, or explicit pixel-size output controls.Third-party](https://developers.cloudflare.com/ai/models/bytedance/seedream-5-pro/) [ddeepseek-v4-prodeepseekText GenerationDeepSeek V4 Pro is a high-capability reasoning model from DeepSeek, served via Fireworks infrastructure for production-grade inference.Third-party](https://developers.cloudflare.com/ai/models/deepseek/deepseek-v4-pro/) [![ElevenLabs logo](https://developers.cloudflare.com/_astro/elevenlabs.0RXw7U95.svg)eleven-flash-v2-5ElevenLabsText-to-SpeechElevenLabs' low-latency Flash v2.5 text-to-speech model for fast multilingual speech generation.Third-party](https://developers.cloudflare.com/ai/models/elevenlabs/eleven-flash-v2-5/) [![ElevenLabs logo](https://developers.cloudflare.com/_astro/elevenlabs.0RXw7U95.svg)eleven-multilingual-v2ElevenLabsText-to-SpeechElevenLabs' multilingual text-to-speech model for generating natural speech across many languages with ElevenLabs voices.Third-party](https://developers.cloudflare.com/ai/models/elevenlabs/eleven-multilingual-v2/) [![ElevenLabs logo](https://developers.cloudflare.com/_astro/elevenlabs.0RXw7U95.svg)eleven-turbo-v2-5ElevenLabsText-to-SpeechElevenLabs' Turbo v2.5 text-to-speech model balancing high-quality voice generation with low latency across 32 languages.Third-party](https://developers.cloudflare.com/ai/models/elevenlabs/eleven-turbo-v2-5/) [![ElevenLabs logo](https://developers.cloudflare.com/_astro/elevenlabs.0RXw7U95.svg)eleven-v3ElevenLabsText-to-SpeechElevenLabs' latest text-to-speech model for highly expressive, natural speech generation with advanced voice control.Third-party](https://developers.cloudflare.com/ai/models/elevenlabs/eleven-v3/) [![ElevenLabs logo](https://developers.cloudflare.com/_astro/elevenlabs.0RXw7U95.svg)music-v2ElevenLabsMusic GenerationElevenLabs Music v2 composes songs and instrumental tracks from a prompt or detailed composition plan.Third-party](https://developers.cloudflare.com/ai/models/elevenlabs/music-v2/) [![Google logo](https://developers.cloudflare.com/_astro/google.DyXKPTPP.svg)gemini-2.5-flashGoogleText GenerationGoogle's fast multimodal Gemini 2.5 model with strong reasoning and a 1M token context window.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/google/gemini-2.5-flash/) [![Google logo](https://developers.cloudflare.com/_astro/google.DyXKPTPP.svg)gemini-2.5-flash-liteGoogleText GenerationGoogle's lightest and most cost-efficient Gemini 2.5 model for high-throughput tasks.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/google/gemini-2.5-flash-lite/) [![Google logo](https://developers.cloudflare.com/_astro/google.DyXKPTPP.svg)gemini-2.5-proGoogleText GenerationGoogle's most capable Gemini 2.5 model with strong reasoning, thinking support, and a 1M token context window.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/google/gemini-2.5-pro/) [![Google logo](https://developers.cloudflare.com/_astro/google.DyXKPTPP.svg)gemini-3-flashGoogleText GenerationGemini 3 Flash is Google's fast multimodal model with frontier intelligence, superior search, and grounding capabilities.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/google/gemini-3-flash/) [![Google logo](https://developers.cloudflare.com/_astro/google.DyXKPTPP.svg)gemini-3.1-flash-liteGoogleText GenerationGoogle's lightest and most cost-efficient Gemini model for high-throughput tasks.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/google/gemini-3.1-flash-lite/) [![Google logo](https://developers.cloudflare.com/_astro/google.DyXKPTPP.svg)gemini-3.1-flash-ttsGoogleText-to-SpeechThird-partyZero data retention](https://developers.cloudflare.com/ai/models/google/gemini-3.1-flash-tts/) [![Google logo](https://developers.cloudflare.com/_astro/google.DyXKPTPP.svg)gemini-3.1-proGoogleText GenerationGoogle's most intelligent Gemini model with improved reasoning, a medium thinking level, and a 1M token context window.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/google/gemini-3.1-pro/) [![Google logo](https://developers.cloudflare.com/_astro/google.DyXKPTPP.svg)gemini-3.5-flashGoogleText GenerationGemini 3.5 Flash is Google's fast multimodal model with frontier intelligence, superior search, and grounding capabilities.Third-party](https://developers.cloudflare.com/ai/models/google/gemini-3.5-flash/) [![Google logo](https://developers.cloudflare.com/_astro/google.DyXKPTPP.svg)gemini-3.5-flash-liteGoogleText GenerationGemini 3.5 Flash-Lite is a low-latency, cost-effective multimodal model optimized for high-throughput, low-cost execution for subagent tasks and document parsing.Third-party](https://developers.cloudflare.com/ai/models/google/gemini-3.5-flash-lite/) [![Google logo](https://developers.cloudflare.com/_astro/google.DyXKPTPP.svg)gemini-3.6-flashGoogleText GenerationGemini 3.6 Flash provides sustained frontier-level intelligence optimized for real-world tasks at a higher speed and lower cost, excelling at code generation, agentic execution, and spatial reasoning.Third-party](https://developers.cloudflare.com/ai/models/google/gemini-3.6-flash/) [![Google logo](https://developers.cloudflare.com/_astro/google.DyXKPTPP.svg)gemini-3.7-flashGoogleText GenerationGemini 3.7 Flash is a highly capable, natively multimodal reasoning model optimized for agentic workflows and real-world tasks.Third-party](https://developers.cloudflare.com/ai/models/google/gemini-3.7-flash/) [![Google logo](https://developers.cloudflare.com/_astro/google.DyXKPTPP.svg)imagen-4GoogleText-to-ImageGoogle's latest image generation model producing high-quality, photorealistic images from text prompts with support for multiple aspect ratios.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/google/imagen-4/) [![Google logo](https://developers.cloudflare.com/_astro/google.DyXKPTPP.svg)nano-bananaGoogleText-to-ImageGoogle's fast image generation model producing high-quality images from text prompts.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/google/nano-banana/) [![Google logo](https://developers.cloudflare.com/_astro/google.DyXKPTPP.svg)nano-banana-2GoogleText-to-ImageGoogle's second-generation image generation model with improved quality and speed.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/google/nano-banana-2/) [![Google logo](https://developers.cloudflare.com/_astro/google.DyXKPTPP.svg)nano-banana-2-liteGoogleText-to-ImageGoogle's fastest Gemini image generation model for rapid image creation and iteration.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/google/nano-banana-2-lite/) [![Google logo](https://developers.cloudflare.com/_astro/google.DyXKPTPP.svg)nano-banana-proGoogleText-to-ImageGoogle's higher-quality image generation model with improved detail and prompt adherence.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/google/nano-banana-pro/) [![Google logo](https://developers.cloudflare.com/_astro/google.DyXKPTPP.svg)veo-3.1GoogleText-to-VideoGoogle's latest video generation model with improved quality, motion, and audio generation.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/google/veo-3.1/) [![Google logo](https://developers.cloudflare.com/_astro/google.DyXKPTPP.svg)veo-3.1-fastGoogleText-to-VideoA faster version of Veo 3.1 optimized for lower latency while maintaining high-quality video and audio output.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/google/veo-3.1-fast/) [![Inworld logo](https://developers.cloudflare.com/_astro/inworld.BDwMAXI2.svg)tts-1.5-maxInworldText-to-SpeechHighest-quality text-to-speech with under 200ms latency, emotion control, and 15-language support.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/inworld/tts-1.5-max/) [![Inworld logo](https://developers.cloudflare.com/_astro/inworld.BDwMAXI2.svg)tts-1.5-miniInworldText-to-SpeechUltra-fast, cost-efficient text-to-speech with approximately 120ms latency and 15-language support.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/inworld/tts-1.5-mini/) [![Inworld logo](https://developers.cloudflare.com/_astro/inworld.BDwMAXI2.svg)tts-2InworldText-to-SpeechInworld's most powerful and expressive text-to-speech model. Builds on TTS 1.5 with rich expressive speech, real-time latency, natural language steering (e.g. \[whisper\], \[say excitedly\]), and stronger multilingual support across 15 production languages plus 90+ experimental languages.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/inworld/tts-2/) [kkrea-2-largekreaText-to-ImageMore than 2x the size of Medium, with softer post-training. Outputs are rawer, more textured, and more flexible — at its best, Large produces results Medium can't match. Strongest on photorealism, raw looks (motion blur, grain, low dynamic range), and expressive and artistic styles.Third-party](https://developers.cloudflare.com/ai/models/krea/krea-2-large/) [kkrea-2-mediumkreaText-to-ImageSmaller, faster, more cost-efficient. Extensive post-training makes outputs especially stable and consistent across generations. Strongest on illustration, anime, painting, and other expressive or artistic styles.Third-party](https://developers.cloudflare.com/ai/models/krea/krea-2-medium/) [kkrea-2-medium-turbokreaText-to-ImageThe fastest Krea 2 model, built for low-cost iteration on expressive illustrations, style-driven concepts, and rapid visual exploration. Keeps the Krea 2 style system and expressive visual range but uses a distilled sampling schedule so you can move through ideas much faster. Especially useful for expressive illustration, graphic styles, typography experiments, and quick campaign or concept directions.Third-party](https://developers.cloudflare.com/ai/models/krea/krea-2-medium-turbo/) [lltx-2-5-fastlightricksText-to-VideoLightricks LTX-2.5 Fast is a fast video generation model for text-to-video and image-to-video workflows, with synchronized audio, configurable duration, resolution, and frame rate.Third-party](https://developers.cloudflare.com/ai/models/lightricks/ltx-2-5-fast/) [![MiniMax logo](https://developers.cloudflare.com/_astro/minimax.B0Y99aoe.svg)hailuo-2.3MiniMaxText-to-VideoA high-fidelity video generation model optimized for realistic human motion, cinematic VFX, expressive characters, and strong prompt and style adherence across text-to-video and image-to-video workflows.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/minimax/hailuo-2.3/) [![MiniMax logo](https://developers.cloudflare.com/_astro/minimax.B0Y99aoe.svg)hailuo-2.3-fastMiniMaxText-to-VideoA lower-latency version of Hailuo 2.3 that preserves core motion quality, visual consistency, and stylization while enabling faster iteration.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/minimax/hailuo-2.3-fast/) [![MiniMax logo](https://developers.cloudflare.com/_astro/minimax.B0Y99aoe.svg)m2.7MiniMaxText GenerationMiniMax's M2.7 language model with multilingual capabilities.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/minimax/m2.7/) [![MiniMax logo](https://developers.cloudflare.com/_astro/minimax.B0Y99aoe.svg)m3MiniMaxText GenerationMiniMax's M3 language model with frontier coding and agentic capabilities, a 1M token context window, and multilingual support.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/minimax/m3/) [![MiniMax logo](https://developers.cloudflare.com/_astro/minimax.B0Y99aoe.svg)music-2.6MiniMaxMusic GenerationMiniMax's music generation model that creates full-length songs with vocals from text prompts and lyrics, or instrumental tracks. Supports BPM/key control and auto-generated lyrics.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/minimax/music-2.6/) [![MiniMax logo](https://developers.cloudflare.com/_astro/minimax.B0Y99aoe.svg)speech-2.8-hdMiniMaxText-to-SpeechMiniMax Speech 2.8 HD focuses on studio-grade audio generation with emotion control, multilingual support (40+ languages), and voice cloning.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/minimax/speech-2.8-hd/) [![MiniMax logo](https://developers.cloudflare.com/_astro/minimax.B0Y99aoe.svg)speech-2.8-turboMiniMaxText-to-SpeechMiniMax Speech 2.8 Turbo turns text into natural, expressive speech with voice cloning, emotion control, and 40+ language support at faster speeds.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/minimax/speech-2.8-turbo/) [![Moonshot AI logo](https://developers.cloudflare.com/_astro/moonshotai.DjWMkXUS.svg)kimi-k3Moonshot AIText GenerationKimi K3 is Moonshot's flagship 2.8 trillion-parameter model, built on Kimi Delta Attention (a hybrid linear attention mechanism) with Attention Residuals. It offers native visual understanding, always-on reasoning, and a 1M-token context window for long-horizon coding, knowledge work, and deep reasoning tasks.Third-party](https://developers.cloudflare.com/ai/models/moonshotai/kimi-k3/) [![OpenAI logo](https://developers.cloudflare.com/_astro/openai.BBwNKzBb.svg)gpt-4.1OpenAIText GenerationOpenAI's flagship GPT model for complex tasks with a million-token context window.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/openai/gpt-4.1/) [![OpenAI logo](https://developers.cloudflare.com/_astro/openai.BBwNKzBb.svg)gpt-4.1-miniOpenAIText GenerationFast, affordable version of GPT-4.1 with a million-token context window.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/openai/gpt-4.1-mini/) [![OpenAI logo](https://developers.cloudflare.com/_astro/openai.BBwNKzBb.svg)gpt-4.1-nanoOpenAIText GenerationGPT-4.1 Nano is OpenAI’s smallest and cheapest GPT-4.1 variant, optimized for high-throughput, low-latency tasks.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/openai/gpt-4.1-nano/) [![OpenAI logo](https://developers.cloudflare.com/_astro/openai.BBwNKzBb.svg)gpt-4oOpenAIText GenerationGPT-4o is OpenAI’s multimodal flagship, accepting text and images and responding quickly across a wide range of tasks.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/openai/gpt-4o/) [![OpenAI logo](https://developers.cloudflare.com/_astro/openai.BBwNKzBb.svg)gpt-4o-miniOpenAIText GenerationGPT-4o Mini is the lightweight, low-cost variant of GPT-4o, well suited to high-volume tasks with multimodal inputs.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/openai/gpt-4o-mini/) [![OpenAI logo](https://developers.cloudflare.com/_astro/openai.BBwNKzBb.svg)gpt-4o-transcribeOpenAIAutomatic Speech RecognitionA speech-to-text model that uses GPT-4o to transcribe audio with improved word error rate and better language recognition compared to original Whisper models.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/openai/gpt-4o-transcribe/) [![OpenAI logo](https://developers.cloudflare.com/_astro/openai.BBwNKzBb.svg)gpt-5OpenAIText GenerationOpenAI's model excelling at coding, writing, and reasoning.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/openai/gpt-5/) [![OpenAI logo](https://developers.cloudflare.com/_astro/openai.BBwNKzBb.svg)gpt-5-miniOpenAIText GenerationGPT-5 Mini is the lightweight, low-cost variant of GPT-5, well suited to high-volume coding and reasoning tasks.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/openai/gpt-5-mini/) [![OpenAI logo](https://developers.cloudflare.com/_astro/openai.BBwNKzBb.svg)gpt-5-nanoOpenAIText GenerationGPT-5 Nano is OpenAI’s smallest GPT-5 variant, optimized for low latency and cheap, high-throughput tasks.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/openai/gpt-5-nano/) [![OpenAI logo](https://developers.cloudflare.com/_astro/openai.BBwNKzBb.svg)gpt-5.1OpenAIText GenerationGPT-5.1 is OpenAI’s incremental improvement over GPT-5, with stronger coding, reasoning, and writing.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/openai/gpt-5.1/) [![OpenAI logo](https://developers.cloudflare.com/_astro/openai.BBwNKzBb.svg)gpt-5.4OpenAIText GenerationGPT-5.4 is OpenAI's flagship model with strong coding, reasoning, and multimodal capabilities.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/openai/gpt-5.4/) [![OpenAI logo](https://developers.cloudflare.com/_astro/openai.BBwNKzBb.svg)gpt-5.4-miniOpenAIText GenerationGPT-5.4 mini is a smaller, faster, and more cost-efficient version of GPT-5.4 for lightweight tasks.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/openai/gpt-5.4-mini/) [![OpenAI logo](https://developers.cloudflare.com/_astro/openai.BBwNKzBb.svg)gpt-5.4-nanoOpenAIText GenerationGPT-5.4 nano is OpenAI's smallest and fastest model, optimized for edge and low-latency use cases.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/openai/gpt-5.4-nano/) [![OpenAI logo](https://developers.cloudflare.com/_astro/openai.BBwNKzBb.svg)gpt-5.4-proOpenAIText GenerationGPT-5.4 pro uses OpenAI's Responses API with built-in tools, improved reasoning, and stateful context management.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/openai/gpt-5.4-pro/) [![OpenAI logo](https://developers.cloudflare.com/_astro/openai.BBwNKzBb.svg)gpt-5.5OpenAIText GenerationGPT-5.5 is OpenAI's flagship model with strong coding, reasoning, and multimodal capabilities.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/openai/gpt-5.5/) [![OpenAI logo](https://developers.cloudflare.com/_astro/openai.BBwNKzBb.svg)gpt-5.5-proOpenAIText GenerationGPT-5.5 pro uses OpenAI's Responses API with built-in tools, improved reasoning, and stateful context management.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/openai/gpt-5.5-pro/) [![OpenAI logo](https://developers.cloudflare.com/_astro/openai.BBwNKzBb.svg)gpt-5.6-lunaOpenAIText GenerationGPT-5.6 Luna is an OpenAI GPT-5.6 model optimized for cost-sensitive workloads, using the Responses API for efficient text generation.Third-party](https://developers.cloudflare.com/ai/models/openai/gpt-5.6-luna/) [![OpenAI logo](https://developers.cloudflare.com/_astro/openai.BBwNKzBb.svg)gpt-5.6-solOpenAIText GenerationGPT-5.6 Sol is OpenAI's frontier GPT-5.6 model for complex professional work, using the Responses API for reasoning and stateful context management.Third-party](https://developers.cloudflare.com/ai/models/openai/gpt-5.6-sol/) [![OpenAI logo](https://developers.cloudflare.com/_astro/openai.BBwNKzBb.svg)gpt-5.6-terraOpenAIText GenerationGPT-5.6 Terra is an OpenAI GPT-5.6 model that balances intelligence and cost, using the Responses API for reasoning and stateful context management.Third-party](https://developers.cloudflare.com/ai/models/openai/gpt-5.6-terra/) [![OpenAI logo](https://developers.cloudflare.com/_astro/openai.BBwNKzBb.svg)gpt-image-1.5OpenAIText-to-ImageOpenAI's image generation model that creates and edits images from text prompts, supporting multiple quality levels and output sizes.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/openai/gpt-image-1.5/) [![OpenAI logo](https://developers.cloudflare.com/_astro/openai.BBwNKzBb.svg)gpt-image-2OpenAIText-to-ImageOpenAI's next-generation image model that creates and edits images from text prompts, with support for multiple quality levels, sizes, and output formats. Note: transparent backgrounds are not supported — use openai/gpt-image-1.5 for transparent PNGs.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/openai/gpt-image-2/) [![OpenAI logo](https://developers.cloudflare.com/_astro/openai.BBwNKzBb.svg)o3OpenAIText Generationo3 is OpenAI’s general-purpose reasoning model, balancing strong analytical performance with reasonable latency and cost.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/openai/o3/) [![OpenAI logo](https://developers.cloudflare.com/_astro/openai.BBwNKzBb.svg)o3-miniOpenAIText Generationo3-mini is the lightweight, low-cost reasoning variant of o3, well suited to quick analytical tasks at scale.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/openai/o3-mini/) [![OpenAI logo](https://developers.cloudflare.com/_astro/openai.BBwNKzBb.svg)o4-miniOpenAIText GenerationOpenAI's fast, lightweight reasoning model optimized for multi-step problem solving at lower cost.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/openai/o4-mini/) [![OpenAI logo](https://developers.cloudflare.com/_astro/openai.BBwNKzBb.svg)tts-1OpenAIText-to-SpeechOpenAI's text-to-speech model optimized for real-time use with low latency.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/openai/tts-1/) [![OpenAI logo](https://developers.cloudflare.com/_astro/openai.BBwNKzBb.svg)tts-1-hdOpenAIText-to-SpeechOpenAI's high-definition text-to-speech model producing higher quality audio output.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/openai/tts-1-hd/) [![PixVerse logo](https://developers.cloudflare.com/_astro/pixverse.DSyGEAYR.svg)v5.6PixVerseText-to-VideoPixverse v5.6 is a video generation model supporting text-to-video and image-to-video with audio generation, customizable aspect ratios, and up to 1080p output.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/pixverse/v5.6/) [![PixVerse logo](https://developers.cloudflare.com/_astro/pixverse.DSyGEAYR.svg)v6PixVerseText-to-VideoPixverse v6 is the latest Pixverse video model with support for up to 15-second videos, customizable duration from 1 to 15 seconds, and audio generation.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/pixverse/v6/) [![Pruna AI logo](https://developers.cloudflare.com/_astro/prunaai.Bv7D31UF.svg)p-imagePruna AIText-to-ImagePruna's P-Image is an ultra-fast text-to-image model with automatic prompt enhancement and 2-stage refinement, combining exceptional speed with high-quality output and flexible aspect ratios.Third-party](https://developers.cloudflare.com/ai/models/pruna/p-image/) [![Pruna AI logo](https://developers.cloudflare.com/_astro/prunaai.Bv7D31UF.svg)p-image-editPruna AIImage-to-ImagePruna's P-Image-Edit edits and composes 1-5 reference images with text instructions. It supports complex compositions, style transfers, and targeted edits with flexible output aspect ratios.Third-party](https://developers.cloudflare.com/ai/models/pruna/p-image-edit/) [![Pruna AI logo](https://developers.cloudflare.com/_astro/prunaai.Bv7D31UF.svg)p-image-try-onPruna AIImage-to-ImagePruna's P-Image Try-On virtually fits one or more garments onto a person's photo. Provide a photo of a person plus garment reference images and the model realistically dresses the person in the provided garments.Third-party](https://developers.cloudflare.com/ai/models/pruna/p-image-try-on/) [![Pruna AI logo](https://developers.cloudflare.com/_astro/prunaai.Bv7D31UF.svg)p-image-upscalePruna AIImage-to-ImagePruna's P-Image-Upscale increases image resolution using AI, targeting 1-128 megapixels with optional detail and realism enhancement for sharper, cleaner results.Third-party](https://developers.cloudflare.com/ai/models/pruna/p-image-upscale/) [![Pruna AI logo](https://developers.cloudflare.com/_astro/prunaai.Bv7D31UF.svg)p-videoPruna AIText-to-VideoPruna's P-Video is a premium video generation model supporting text-to-video, image-to-video, and audio-conditioned generation up to 1080p at 24 or 48 fps, with configurable duration up to 20 seconds.Third-party](https://developers.cloudflare.com/ai/models/pruna/p-video/) [![Pruna AI logo](https://developers.cloudflare.com/_astro/prunaai.Bv7D31UF.svg)p-video-animatePruna AIImage-to-VideoPruna's P-Video-Animate takes a source video and a subject reference image, then animates the referenced subject using the motion and audio from the source video.Third-party](https://developers.cloudflare.com/ai/models/pruna/p-video-animate/) [![Pruna AI logo](https://developers.cloudflare.com/_astro/prunaai.Bv7D31UF.svg)p-video-avatarPruna AIImage-to-VideoPruna's P-Video-Avatar generates talking-head videos from a single portrait image driven by a text script or audio file, with multiple voices, languages, and output resolutions.Third-party](https://developers.cloudflare.com/ai/models/pruna/p-video-avatar/) [![Pruna AI logo](https://developers.cloudflare.com/_astro/prunaai.Bv7D31UF.svg)p-video-replacePruna AIImage-to-VideoPruna's P-Video-Replace takes a source video and one or more identity reference images, then places the referenced person or people into the video while preserving the source motion and audio.Third-party](https://developers.cloudflare.com/ai/models/pruna/p-video-replace/) [![Recraft logo](https://developers.cloudflare.com/_astro/recraft.BhhnJczi.svg)recraftv3RecraftText-to-ImageRecraft V3 is the previous-generation text-to-image model from Recraft, well-suited to design-quality compositions, brand-aware imagery, and accurate text rendering.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/recraft/recraftv3/) [![Recraft logo](https://developers.cloudflare.com/_astro/recraft.BhhnJczi.svg)recraftv4RecraftText-to-ImageRecraft V4 generates art-directed images with strong composition, accurate text rendering, and design taste built in. Fast and cost-efficient at standard resolution.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/recraft/recraftv4/) [![Recraft logo](https://developers.cloudflare.com/_astro/recraft.BhhnJczi.svg)recraftv4-1RecraftText-to-ImageRecraft V4.1 generates art-directed images tuned for high aesthetics, with strong composition, accurate text rendering, and refined design taste. Fast and cost-efficient at standard resolution.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/recraft/recraftv4-1/) [![Recraft logo](https://developers.cloudflare.com/_astro/recraft.BhhnJczi.svg)recraftv4-1-proRecraftText-to-ImageRecraft V4.1 Pro generates high-resolution, art-directed images at 2048px+ tuned for high aesthetics, with strong composition, text rendering, and refined design taste. Built for print and production work.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/recraft/recraftv4-1-pro/) [![Recraft logo](https://developers.cloudflare.com/_astro/recraft.BhhnJczi.svg)recraftv4-1-pro-vectorRecraftText-to-ImageGenerate detailed, high-resolution SVG vector graphics from text prompts with high aesthetic quality, fine geometry, scalable to any size for print and design work.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/recraft/recraftv4-1-pro-vector/) [![Recraft logo](https://developers.cloudflare.com/_astro/recraft.BhhnJczi.svg)recraftv4-1-utilityRecraftText-to-ImageRecraft V4.1 Utility is a general-purpose text-to-image model balancing quality and flexibility for a wide range of everyday use cases at standard resolution.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/recraft/recraftv4-1-utility/) [![Recraft logo](https://developers.cloudflare.com/_astro/recraft.BhhnJczi.svg)recraftv4-1-utility-proRecraftText-to-ImageRecraft V4.1 Utility Pro is a general-purpose text-to-image model producing high-resolution 2048px+ output for a wide range of production and print use cases.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/recraft/recraftv4-1-utility-pro/) [![Recraft logo](https://developers.cloudflare.com/_astro/recraft.BhhnJczi.svg)recraftv4-1-utility-pro-vectorRecraftText-to-ImageGenerate detailed, high-resolution SVG vector graphics from text prompts with a general-purpose model, scalable to any size for print and large-scale design work.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/recraft/recraftv4-1-utility-pro-vector/) [![Recraft logo](https://developers.cloudflare.com/_astro/recraft.BhhnJczi.svg)recraftv4-1-utility-vectorRecraftText-to-ImageGenerate production-ready SVG vector graphics from text prompts with a general-purpose model suited for a wide range of design and illustration tasks.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/recraft/recraftv4-1-utility-vector/) [![Recraft logo](https://developers.cloudflare.com/_astro/recraft.BhhnJczi.svg)recraftv4-1-vectorRecraftText-to-ImageGenerate production-ready SVG vector graphics from text prompts with high aesthetic quality, clean geometry, structured layers, and editable paths.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/recraft/recraftv4-1-vector/) [![Recraft logo](https://developers.cloudflare.com/_astro/recraft.BhhnJczi.svg)recraftv4-proRecraftText-to-ImageRecraft V4 Pro generates high-resolution, art-directed images at 2048px+ with strong composition, text rendering, and design taste. Built for print and production work.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/recraft/recraftv4-pro/) [![Recraft logo](https://developers.cloudflare.com/_astro/recraft.BhhnJczi.svg)recraftv4-pro-vectorRecraftText-to-ImageGenerate detailed, production-ready SVG vector graphics from text prompts with fine geometry, scalable to any size for print and design work.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/recraft/recraftv4-pro-vector/) [![Recraft logo](https://developers.cloudflare.com/_astro/recraft.BhhnJczi.svg)recraftv4-vectorRecraftText-to-ImageGenerate production-ready SVG vector graphics from text prompts with clean geometry, structured layers, and editable paths.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/recraft/recraftv4-vector/) [![RunwayML logo](https://developers.cloudflare.com/_astro/runway.Cq8Cjov4.svg)aleph-2RunwayMLText-to-VideoRunwayML's video editing model. Edit one frame to update your whole video, make changes across multiple shots, and work with up to 30 seconds of video. Supports keyframe-guided editing for precise control over specific moments in the clip.Third-party](https://developers.cloudflare.com/ai/models/runwayml/aleph-2/) [![RunwayML logo](https://developers.cloudflare.com/_astro/runway.Cq8Cjov4.svg)gen-4.5RunwayMLText-to-VideoRunwayML's video generation model supporting both text-to-video and image-to-video with customizable duration, aspect ratio, and content moderation controls.Third-party](https://developers.cloudflare.com/ai/models/runwayml/gen-4.5/) [tinklingthinkingmachinesText GenerationInkling is Thinking Machines' open-weights hybrid reasoning model, built on a mixture-of-experts architecture. It reasons by default, exposing its chain-of-thought as leading thinking content blocks, with reasoning effort tunable via a Tinker-specific output\_config.effort parameter. Available through Tinker's beta Anthropic Messages-compatible endpoint alongside tool use, streaming, and multi-turn conversations. Currently intended for low-traffic testing and internal use rather than high-throughput production deployments; prompt caching, citations, and audio input are not supported through this endpoint.Third-party](https://developers.cloudflare.com/ai/models/thinkingmachines/inkling/) [tinkling-256kthinkingmachinesText GenerationThe 256K-context variant of Inkling, Thinking Machines' open-weights hybrid reasoning MoE model. Same hybrid reasoning, tool-use, and streaming support as the base model, with an extended context window for longer conversations and documents. Currently intended for low-traffic testing and internal use rather than high-throughput production deployments.Third-party](https://developers.cloudflare.com/ai/models/thinkingmachines/inkling-256k/) [![Vidu logo](https://developers.cloudflare.com/_astro/vidu.CcN5bM2x.svg)q3-proViduText-to-VideoVidu Q3 Pro is a high-quality video generation model supporting text-to-video, image-to-video, and start/end-frame-to-video workflows with audio and up to 16-second clips.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/vidu/q3-pro/) [![Vidu logo](https://developers.cloudflare.com/_astro/vidu.CcN5bM2x.svg)q3-turboViduText-to-VideoVidu Q3 Turbo is a faster version of Vidu Q3 optimized for lower latency video generation while maintaining audio support and up to 16-second clips.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/vidu/q3-turbo/) [![xAI logo](https://developers.cloudflare.com/_astro/xai.2Y8IhZGx.svg)grok-4.20-0309-non-reasoningxAIText GenerationxAI's Grok 4.20 non-reasoning model. Skips the thinking trace for fast, single-pass responses while keeping the same training as the reasoning variant.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/xai/grok-4.20-0309-non-reasoning/) [![xAI logo](https://developers.cloudflare.com/_astro/xai.2Y8IhZGx.svg)grok-4.20-0309-reasoningxAIText GenerationxAI's Grok 4.20 reasoning model. Uses extended thinking to work through complex problems, returning a reasoning trace alongside the final answer.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/xai/grok-4.20-0309-reasoning/) [![xAI logo](https://developers.cloudflare.com/_astro/xai.2Y8IhZGx.svg)grok-4.20-multi-agent-0309xAIText GenerationxAI's Grok 4.20 multi-agent model with a 2M-token context window. Multiple agents collaborate in parallel to perform deep research tasks, with function calling, structured outputs, and reasoning capabilities.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/xai/grok-4.20-multi-agent-0309/) [![xAI logo](https://developers.cloudflare.com/_astro/xai.2Y8IhZGx.svg)grok-4.3xAIText GenerationxAI's Grok 4.3 model with a 1M-token context window and strong agentic tool calling with minimal hallucinations. Accepts text and image inputs, and supports function calling, structured outputs, and configurable reasoning effort (none, low, medium, high).Third-partyZero data retention](https://developers.cloudflare.com/ai/models/xai/grok-4.3/) [![xAI logo](https://developers.cloudflare.com/_astro/xai.2Y8IhZGx.svg)grok-4.5xAIText GenerationxAI's Grok 4.5, a frontier model built for coding, agentic tasks, and knowledge work. Accepts text and image inputs, and supports function calling, structured outputs, and configurable reasoning effort (low, medium, high).Third-partyZero data retention](https://developers.cloudflare.com/ai/models/xai/grok-4.5/) [![xAI logo](https://developers.cloudflare.com/_astro/xai.2Y8IhZGx.svg)grok-4.6xAIText GenerationxAI's Grok 4.6, a flagship reasoning model for coding, agentic tasks, and visual work. Accepts text and image inputs, and supports function calling and structured outputs.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/xai/grok-4.6/) [![xAI logo](https://developers.cloudflare.com/_astro/xai.2Y8IhZGx.svg)grok-imagine-imagexAIText-to-ImagexAI's Grok Imagine image model. Generates and edits images from text and reference-image inputs with configurable aspect ratio and resolution.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/xai/grok-imagine-image/) [![xAI logo](https://developers.cloudflare.com/_astro/xai.2Y8IhZGx.svg)grok-imagine-image-2.0xAIText-to-ImagexAI's Grok Imagine Image 2.0 is a precise image generation and editing model for creative work, with strong instruction following, typography, layout, and reference-image preservation.Third-party](https://developers.cloudflare.com/ai/models/xai/grok-imagine-image-2.0/) [![xAI logo](https://developers.cloudflare.com/_astro/xai.2Y8IhZGx.svg)grok-imagine-image-qualityxAIText-to-ImagexAI's higher-fidelity text-to-image model optimized for sharper details, more accurate compositions, and stronger text rendering. Supports image editing via reference images and masks. Trades speed for quality compared to grok-imagine-image. Default output at 2k resolution.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/xai/grok-imagine-image-quality/) [![xAI logo](https://developers.cloudflare.com/_astro/xai.2Y8IhZGx.svg)grok-imagine-videoxAIText-to-VideoxAI's video generation model. Generates, edits, and extends videos from text and image inputs with native synchronized audio including dialogue, sound effects, and music. Supports multiple creative modes (normal, fun, custom).Third-partyZero data retention](https://developers.cloudflare.com/ai/models/xai/grok-imagine-video/) [![xAI logo](https://developers.cloudflare.com/_astro/xai.2Y8IhZGx.svg)grok-imagine-video-1.5-previewxAIImage-to-VideoxAI's next-generation video generation model. Generates, edits, and extends videos from text and image inputs. Supports multiple aspect ratios and resolutions with improved quality over the previous generation.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/xai/grok-imagine-video-1.5-preview/) [![xAI logo](https://developers.cloudflare.com/_astro/xai.2Y8IhZGx.svg)grok-sttxAIAutomatic Speech RecognitionxAI's Grok speech-to-text model. Transcribes audio files into text across 25 languages with word-level timestamps, multichannel transcription, speaker diarization, and key-term biasing.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/xai/grok-stt/) [![xAI logo](https://developers.cloudflare.com/_astro/xai.2Y8IhZGx.svg)grok-ttsxAIText-to-SpeechxAI's Grok text-to-speech model. Generates high-fidelity spoken audio in 5 expressive voices (eve, ara, rex, sal, leo) with 20+ supported languages. Supports inline speech tags for laughter, whispers, and pauses.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/xai/grok-tts/) [![xAI logo](https://developers.cloudflare.com/_astro/xai.2Y8IhZGx.svg)grok-voicexAIwebsocketxAI's real-time voice conversation model with low-latency audio input and output streaming.Third-partyZero data retention](https://developers.cloudflare.com/ai/models/xai/grok-voice/) [![Deepgram logo](https://developers.cloudflare.com/_astro/deepgram.BYzW8KfF.svg)aura-1DeepgramText-to-SpeechAura is a context-aware text-to-speech (TTS) model that applies natural pacing, expressiveness, and fillers based on the context of the provided text. The quality of your text input directly impacts the naturalness of the audio output.Cloudflare-hostedBatchPartnerReal-time](https://developers.cloudflare.com/ai/models/@cf/deepgram/aura-1/) [![Deepgram logo](https://developers.cloudflare.com/_astro/deepgram.BYzW8KfF.svg)aura-2-enDeepgramText-to-SpeechAura-2 is a context-aware text-to-speech (TTS) model that applies natural pacing, expressiveness, and fillers based on the context of the provided text. The quality of your text input directly impacts the naturalness of the audio output.Cloudflare-hostedBatchPartnerReal-time](https://developers.cloudflare.com/ai/models/@cf/deepgram/aura-2-en/) [![Deepgram logo](https://developers.cloudflare.com/_astro/deepgram.BYzW8KfF.svg)aura-2-esDeepgramText-to-SpeechAura-2 is a context-aware text-to-speech (TTS) model that applies natural pacing, expressiveness, and fillers based on the context of the provided text. The quality of your text input directly impacts the naturalness of the audio output.Cloudflare-hostedBatchPartnerReal-time](https://developers.cloudflare.com/ai/models/@cf/deepgram/aura-2-es/) [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)bart-large-cnnBetaMetaSummarizationBART is a transformer encoder-encoder (seq2seq) model with a bidirectional (BERT-like) encoder and an autoregressive (GPT-like) decoder. You can use this model for text summarization.Cloudflare-hostedDeprecated](https://developers.cloudflare.com/ai/models/@cf/facebook/bart-large-cnn/) [![BAAI logo](https://developers.cloudflare.com/_astro/baai.BooZR_xF.svg)bge-base-en-v1.5BAAIText EmbeddingsBAAI general embedding (Base) model that transforms any given text into a 768-dimensional vectorCloudflare-hostedBatch](https://developers.cloudflare.com/ai/models/@cf/baai/bge-base-en-v1.5/) [![BAAI logo](https://developers.cloudflare.com/_astro/baai.BooZR_xF.svg)bge-large-en-v1.5BAAIText EmbeddingsBAAI general embedding (Large) model that transforms any given text into a 1024-dimensional vectorCloudflare-hostedBatch](https://developers.cloudflare.com/ai/models/@cf/baai/bge-large-en-v1.5/) [![BAAI logo](https://developers.cloudflare.com/_astro/baai.BooZR_xF.svg)bge-m3BAAIText EmbeddingsMulti-Functionality, Multi-Linguality, and Multi-Granularity embeddings model.Cloudflare-hosted](https://developers.cloudflare.com/ai/models/@cf/baai/bge-m3/) [![BAAI logo](https://developers.cloudflare.com/_astro/baai.BooZR_xF.svg)bge-reranker-baseBAAIText ClassificationDifferent from embedding model, reranker uses question and document as input and directly output similarity instead of embedding. You can get a relevance score by inputting query and passage to the reranker. And the score can be mapped to a float value in \[0,1\] by sigmoid function. Cloudflare-hosted](https://developers.cloudflare.com/ai/models/@cf/baai/bge-reranker-base/) [![BAAI logo](https://developers.cloudflare.com/_astro/baai.BooZR_xF.svg)bge-small-en-v1.5BAAIText EmbeddingsBAAI general embedding (Small) model that transforms any given text into a 384-dimensional vectorCloudflare-hostedBatch](https://developers.cloudflare.com/ai/models/@cf/baai/bge-small-en-v1.5/) [![DeepSeek logo](https://developers.cloudflare.com/_astro/deepseek.CkzAgvN6.svg)deepseek-r1-distill-qwen-32bDeepSeekText GenerationDeepSeek-R1-Distill-Qwen-32B is a model distilled from DeepSeek-R1 based on Qwen2.5\. It outperforms OpenAI-o1-mini across various benchmarks, achieving new state-of-the-art results for dense models.Cloudflare-hostedReasoning](https://developers.cloudflare.com/ai/models/@cf/deepseek-ai/deepseek-r1-distill-qwen-32b/) [![DeepSeek logo](https://developers.cloudflare.com/_astro/deepseek.CkzAgvN6.svg)deepseek-v4-flash-0731DeepSeekText GenerationDeepSeek-V4-Flash-0731 is the official release of DeepSeek-V4-Flash, superseding the preview version, with substantially enhanced agentic capabilities. Cloudflare-hostedFunction callingReasoning](https://developers.cloudflare.com/ai/models/@cf/deepseek-ai/deepseek-v4-flash-0731/) [![DeepSeek logo](https://developers.cloudflare.com/_astro/deepseek.CkzAgvN6.svg)deepseek-v4-pro-0813DeepSeekText GenerationDeepSeek V4 Pro is a high-capability reasoning model from DeepSeek with a one million token context window, built for long-horizon agentic workflows and complex, multi-step problem-solvingCloudflare-hostedFunction callingReasoning](https://developers.cloudflare.com/ai/models/@cf/deepseek-ai/deepseek-v4-pro-0813/) [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)detr-resnet-50BetaMetaObject DetectionDEtection TRansformer (DETR) model trained end-to-end on COCO 2017 object detection (118k annotated images).Cloudflare-hosted](https://developers.cloudflare.com/ai/models/@cf/facebook/detr-resnet-50/) [![HuggingFace logo](https://developers.cloudflare.com/_astro/huggingface.DMS-v5TA.svg)distilbert-sst-2-int8HuggingFaceText ClassificationDistilled BERT model that was finetuned on SST-2 for sentiment classificationCloudflare-hosted](https://developers.cloudflare.com/ai/models/@cf/huggingface/distilbert-sst-2-int8/) [ldreamshaper-8-lcmlykonText-to-ImageStable Diffusion model that has been fine-tuned to be better at photorealism without sacrificing range.Cloudflare-hosted](https://developers.cloudflare.com/ai/models/@cf/lykon/dreamshaper-8-lcm/) [![Google logo](https://developers.cloudflare.com/_astro/google.DyXKPTPP.svg)embeddinggemma-300mGoogleText EmbeddingsEmbeddingGemma is a 300M parameter, state-of-the-art for its size, open embedding model from Google, built from Gemma 3 (with T5Gemma initialization) and the same research and technology used to create Gemini models. EmbeddingGemma produces vector representations of text, making it well-suited for search and retrieval tasks, including classification, clustering, and semantic similarity search. This model was trained with data in 100+ spoken languages.Cloudflare-hosted](https://developers.cloudflare.com/ai/models/@cf/google/embeddinggemma-300m/) [![Deepgram logo](https://developers.cloudflare.com/_astro/deepgram.BYzW8KfF.svg)fluxDeepgramAutomatic Speech RecognitionFlux is the first conversational speech recognition model built specifically for voice agents.Cloudflare-hostedPartnerReal-time](https://developers.cloudflare.com/ai/models/@cf/deepgram/flux/) [![Black Forest Labs logo](https://developers.cloudflare.com/_astro/blackforestlabs.Ccs-Y4-D.svg)flux-1-schnellBlack Forest LabsText-to-ImageFLUX.1 \[schnell\] is a 12 billion parameter rectified flow transformer capable of generating images from text descriptions. Cloudflare-hosted](https://developers.cloudflare.com/ai/models/@cf/black-forest-labs/flux-1-schnell/) [![Black Forest Labs logo](https://developers.cloudflare.com/_astro/blackforestlabs.Ccs-Y4-D.svg)flux-2-devBlack Forest LabsText-to-ImageFLUX.2 \[dev\] is an image model from Black Forest Labs where you can generate highly realistic and detailed images, with multi-reference support.Cloudflare-hostedPartner](https://developers.cloudflare.com/ai/models/@cf/black-forest-labs/flux-2-dev/) [![Black Forest Labs logo](https://developers.cloudflare.com/_astro/blackforestlabs.Ccs-Y4-D.svg)flux-2-klein-4bBlack Forest LabsText-to-ImageFLUX.2 \[klein\] is an ultra-fast, distilled image model. It unifies image generation and editing in a single model, delivering state-of-the-art quality enabling interactive workflows, real-time previews, and latency-critical applications.Cloudflare-hostedPartner](https://developers.cloudflare.com/ai/models/@cf/black-forest-labs/flux-2-klein-4b/) [![Black Forest Labs logo](https://developers.cloudflare.com/_astro/blackforestlabs.Ccs-Y4-D.svg)flux-2-klein-9bBlack Forest LabsText-to-ImageFLUX.2 \[klein\] 9B is an ultra-fast, distilled image model with enhanced quality. It unifies image generation and editing in a single model, delivering state-of-the-art quality enabling interactive workflows, real-time previews, and latency-critical applications.Cloudflare-hostedPartner](https://developers.cloudflare.com/ai/models/@cf/black-forest-labs/flux-2-klein-9b/) [![Google logo](https://developers.cloudflare.com/_astro/google.DyXKPTPP.svg)gemma-2b-it-loraBetaGoogleText GenerationThis is a Gemma-2B base model that Cloudflare dedicates for inference with LoRA adapters. Gemma is a family of lightweight, state-of-the-art open models from Google, built from the same research and technology used to create the Gemini models.Cloudflare-hostedLoRA](https://developers.cloudflare.com/ai/models/@cf/google/gemma-2b-it-lora/) [![Google logo](https://developers.cloudflare.com/_astro/google.DyXKPTPP.svg)gemma-3-12b-itGoogleText GenerationGemma 3 models are well-suited for a variety of text generation and image understanding tasks, including question answering, summarization, and reasoning. Gemma 3 models are multimodal, handling text and image input and generating text output, with a large, 128K context window, multilingual support in over 140 languages, and is available in more sizes than previous versions.Cloudflare-hostedLoRADeprecated](https://developers.cloudflare.com/ai/models/@cf/google/gemma-3-12b-it/) [![Google logo](https://developers.cloudflare.com/_astro/google.DyXKPTPP.svg)gemma-4-26b-a4b-itGoogleText GenerationGemma 4 is Google's most intelligent family of open models, built from Gemini 3 research to maximize intelligence-per-parameter.Cloudflare-hostedFunction callingReasoningVision](https://developers.cloudflare.com/ai/models/@cf/google/gemma-4-26b-a4b-it/) [![Google logo](https://developers.cloudflare.com/_astro/google.DyXKPTPP.svg)gemma-7b-itBetaGoogleText GenerationGemma is a family of lightweight, state-of-the-art open models from Google, built from the same research and technology used to create the Gemini models. They are text-to-text, decoder-only large language models, available in English, with open weights, pre-trained variants, and instruction-tuned variants.Cloudflare-hostedLoRADeprecated](https://developers.cloudflare.com/ai/models/@hf/google/gemma-7b-it/) [![Google logo](https://developers.cloudflare.com/_astro/google.DyXKPTPP.svg)gemma-7b-it-loraBetaGoogleText Generation This is a Gemma-7B base model that Cloudflare dedicates for inference with LoRA adapters. Gemma is a family of lightweight, state-of-the-art open models from Google, built from the same research and technology used to create the Gemini models.Cloudflare-hostedLoRA](https://developers.cloudflare.com/ai/models/@cf/google/gemma-7b-it-lora/) [agemma-sea-lion-v4-27b-itaisingaporeText GenerationSEA-LION stands for Southeast Asian Languages In One Network, which is a collection of Large Language Models (LLMs) which have been pretrained and instruct-tuned for the Southeast Asia (SEA) region.Cloudflare-hosted](https://developers.cloudflare.com/ai/models/@cf/aisingapore/gemma-sea-lion-v4-27b-it/) [Pinned![Zhipu AI logo](https://developers.cloudflare.com/_astro/zai-org.Dj2vcayE.svg)glm-4.7-flashZhipu AIText GenerationGLM-4.7-Flash is a fast and efficient multilingual text generation model with a 131,072 token context window. Optimized for dialogue, instruction-following, and multi-turn tool calling across 100+ languages.Cloudflare-hostedFunction callingReasoning](https://developers.cloudflare.com/ai/models/@cf/zai-org/glm-4.7-flash/) [![Zhipu AI logo](https://developers.cloudflare.com/_astro/zai-org.Dj2vcayE.svg)glm-5.2Zhipu AIText GenerationZ.ai's flagship agentic coding modelCloudflare-hostedFunction callingReasoning](https://developers.cloudflare.com/ai/models/@cf/zai-org/glm-5.2/) [Pinned![OpenAI logo](https://developers.cloudflare.com/_astro/openai.BBwNKzBb.svg)gpt-oss-120bOpenAIText GenerationOpenAI's open-weight models designed for powerful reasoning, agentic tasks, and versatile developer use cases – gpt-oss-120b is for production, general purpose, high reasoning use-cases.Cloudflare-hostedFunction callingReasoning](https://developers.cloudflare.com/ai/models/@cf/openai/gpt-oss-120b/) [![OpenAI logo](https://developers.cloudflare.com/_astro/openai.BBwNKzBb.svg)gpt-oss-20bOpenAIText GenerationOpenAI's open-weight models designed for powerful reasoning, agentic tasks, and versatile developer use cases – gpt-oss-20b is for lower latency, and local or specialized use-cases.Cloudflare-hostedFunction callingReasoning](https://developers.cloudflare.com/ai/models/@cf/openai/gpt-oss-20b/) [![IBM logo](https://developers.cloudflare.com/_astro/ibm.CYdLt4EI.svg)granite-4.0-h-microIBMText GenerationGranite 4.0 instruct models deliver strong performance across benchmarks, achieving industry-leading results in key agentic tasks like instruction following and function calling. These efficiencies make the models well-suited for a wide range of use cases like retrieval-augmented generation (RAG), multi-agent workflows, and edge deployments.Cloudflare-hostedFunction calling](https://developers.cloudflare.com/ai/models/@cf/ibm-granite/granite-4.0-h-micro/) [nhermes-2-pro-mistral-7bBetanousresearchText GenerationHermes 2 Pro on Mistral 7B is the new flagship 7B Hermes! Hermes 2 Pro is an upgraded, retrained version of Nous Hermes 2, consisting of an updated and cleaned version of the OpenHermes 2.5 Dataset, as well as a newly introduced Function Calling and JSON Mode dataset developed in-house.Cloudflare-hostedFunction callingDeprecated](https://developers.cloudflare.com/ai/models/@hf/nousresearch/hermes-2-pro-mistral-7b/) [aindictrans2-en-indic-1Bai4bharatTranslationIndicTrans2 is the first open-source transformer-based multilingual NMT model that supports high-quality translations across all the 22 scheduled Indic languagesCloudflare-hosted](https://developers.cloudflare.com/ai/models/@cf/ai4bharat/indictrans2-en-indic-1B/) [![Moonshot AI logo](https://developers.cloudflare.com/_astro/moonshotai.DjWMkXUS.svg)kimi-k2.5Moonshot AIText GenerationKimi K2.5 is a frontier-scale open-source model with a 256k context window, multi-turn tool calling, vision inputs, and structured outputs for agentic workloads.Cloudflare-hostedFunction callingDeprecatedReasoningVision](https://developers.cloudflare.com/ai/models/@cf/moonshotai/kimi-k2.5/) [![Moonshot AI logo](https://developers.cloudflare.com/_astro/moonshotai.DjWMkXUS.svg)kimi-k2.6Moonshot AIText GenerationKimi K2.6 is a frontier-scale open-source 1T parameter model with a 262.1k context window, multi-turn tool calling, vision inputs, and structured outputs for agentic workloads.Cloudflare-hostedFunction callingReasoningVision](https://developers.cloudflare.com/ai/models/@cf/moonshotai/kimi-k2.6/) [Pinned![Moonshot AI logo](https://developers.cloudflare.com/_astro/moonshotai.DjWMkXUS.svg)kimi-k2.7-codeMoonshot AIText GenerationKimi K2.7 is a frontier-scale open-source 1T parameter model with a 262.1k context window, multi-turn tool calling, vision inputs, and structured outputs for agentic workloads.Cloudflare-hostedFunction callingReasoningVision](https://developers.cloudflare.com/ai/models/@cf/moonshotai/kimi-k2.7-code/) [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)llama-2-7b-chat-fp16MetaText GenerationFull precision (fp16) generative text model with 7 billion parameters from MetaCloudflare-hostedDeprecated](https://developers.cloudflare.com/ai/models/@cf/meta/llama-2-7b-chat-fp16/) [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)llama-2-7b-chat-hf-loraBetaMetaText GenerationThis is a Llama2 base model that Cloudflare dedicated for inference with LoRA adapters. Llama 2 is a collection of pretrained and fine-tuned generative text models ranging in scale from 7 billion to 70 billion parameters. This is the repository for the 7B fine-tuned model, optimized for dialogue use cases and converted for the Hugging Face Transformers format. Cloudflare-hostedLoRA](https://developers.cloudflare.com/ai/models/@cf/meta-llama/llama-2-7b-chat-hf-lora/) [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)llama-2-7b-chat-int8MetaText GenerationQuantized (int8) generative text model with 7 billion parameters from MetaCloudflare-hostedDeprecated](https://developers.cloudflare.com/ai/models/@cf/meta/llama-2-7b-chat-int8/) [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)llama-3-8b-instructMetaText GenerationGeneration over generation, Meta Llama 3 demonstrates state-of-the-art performance on a wide range of industry benchmarks and offers new capabilities, including improved reasoning.Cloudflare-hostedDeprecated](https://developers.cloudflare.com/ai/models/@cf/meta/llama-3-8b-instruct/) [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)llama-3-8b-instruct-awqMetaText GenerationQuantized (int4) generative text model with 8 billion parameters from Meta.Cloudflare-hostedDeprecated](https://developers.cloudflare.com/ai/models/@cf/meta/llama-3-8b-instruct-awq/) [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)llama-3.1-70b-instructMetaText GenerationThe Meta Llama 3.1 collection of multilingual large language models (LLMs) is a collection of pretrained and instruction tuned generative models. The Llama 3.1 instruction tuned text only models are optimized for multilingual dialogue use cases and outperform many of the available open source and closed chat models on common industry benchmarks.Cloudflare-hostedDeprecated](https://developers.cloudflare.com/ai/models/@cf/meta/llama-3.1-70b-instruct/) [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)llama-3.1-8b-instructMetaText GenerationThe Meta Llama 3.1 collection of multilingual large language models (LLMs) is a collection of pretrained and instruction tuned generative models. The Llama 3.1 instruction tuned text only models are optimized for multilingual dialogue use cases and outperform many of the available open source and closed chat models on common industry benchmarks.Cloudflare-hostedDeprecated](https://developers.cloudflare.com/ai/models/@cf/meta/llama-3.1-8b-instruct/) [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)llama-3.1-8b-instruct-awqMetaText GenerationQuantized (int4) generative text model with 8 billion parameters from Meta. Cloudflare-hostedDeprecated](https://developers.cloudflare.com/ai/models/@cf/meta/llama-3.1-8b-instruct-awq/) [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)llama-3.1-8b-instruct-fastMetaText Generation\[Fast version\] The Meta Llama 3.1 collection of multilingual large language models (LLMs) is a collection of pretrained and instruction tuned generative models. The Llama 3.1 instruction tuned text only models are optimized for multilingual dialogue use cases and outperform many of the available open source and closed chat models on common industry benchmarks.Cloudflare-hosted](https://developers.cloudflare.com/ai/models/@cf/meta/llama-3.1-8b-instruct-fast/) [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)llama-3.1-8b-instruct-fp8MetaText GenerationLlama 3.1 8B quantized to FP8 precisionCloudflare-hosted](https://developers.cloudflare.com/ai/models/@cf/meta/llama-3.1-8b-instruct-fp8/) [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)llama-3.2-11b-vision-instructMetaText Generation The Llama 3.2-Vision instruction-tuned models are optimized for visual recognition, image reasoning, captioning, and answering general questions about an image.Cloudflare-hostedLoRAVision](https://developers.cloudflare.com/ai/models/@cf/meta/llama-3.2-11b-vision-instruct/) [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)llama-3.2-1b-instructMetaText GenerationThe Llama 3.2 instruction-tuned text only models are optimized for multilingual dialogue use cases, including agentic retrieval and summarization tasks.Cloudflare-hosted](https://developers.cloudflare.com/ai/models/@cf/meta/llama-3.2-1b-instruct/) [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)llama-3.2-3b-instructMetaText GenerationThe Llama 3.2 instruction-tuned text only models are optimized for multilingual dialogue use cases, including agentic retrieval and summarization tasks.Cloudflare-hosted](https://developers.cloudflare.com/ai/models/@cf/meta/llama-3.2-3b-instruct/) [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)llama-3.3-70b-instruct-fp8-fastMetaText GenerationLlama 3.3 70B quantized to fp8 precision, optimized to be faster.Cloudflare-hostedBatchFunction calling](https://developers.cloudflare.com/ai/models/@cf/meta/llama-3.3-70b-instruct-fp8-fast/) [Pinned![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)llama-4-scout-17b-16e-instructMetaText GenerationMeta's Llama 4 Scout is a 17 billion parameter model with 16 experts that is natively multimodal. These models leverage a mixture-of-experts architecture to offer industry-leading performance in text and image understanding.Cloudflare-hostedBatchFunction callingVision](https://developers.cloudflare.com/ai/models/@cf/meta/llama-4-scout-17b-16e-instruct/) [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)llama-guard-3-8bMetaText GenerationLlama Guard 3 is a Llama-3.1-8B pretrained model, fine-tuned for content safety classification. Similar to previous versions, it can be used to classify content in both LLM inputs (prompt classification) and in LLM responses (response classification). It acts as an LLM – it generates text in its output that indicates whether a given prompt or response is safe or unsafe, and if unsafe, it also lists the content categories violated.Cloudflare-hostedLoRA](https://developers.cloudflare.com/ai/models/@cf/meta/llama-guard-3-8b/) [lllava-1.5-7b-hfBetallava-hfImage-to-TextLLaVA is an open-source chatbot trained by fine-tuning LLaMA/Vicuna on GPT-generated multimodal instruction-following data. It is an auto-regressive language model, based on the transformer architecture.Cloudflare-hosted](https://developers.cloudflare.com/ai/models/@cf/llava-hf/llava-1.5-7b-hf/) [![Leonardo logo](https://developers.cloudflare.com/_astro/leonardo.JZysY-g3.svg)lucid-originLeonardoText-to-ImageLucid Origin from Leonardo.AI is their most adaptable and prompt-responsive model to date. Whether you're generating images with sharp graphic design, stunning full-HD renders, or highly specific creative direction, it adheres closely to your prompts, renders text with accuracy, and supports a wide array of visual styles and aesthetics – from stylized concept art to crisp product mockups. Cloudflare-hostedPartner](https://developers.cloudflare.com/ai/models/@cf/leonardo/lucid-origin/) [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)m2m100-1.2bMetaTranslationMultilingual encoder-decoder (seq-to-seq) model trained for Many-to-Many multilingual translationCloudflare-hostedBatch](https://developers.cloudflare.com/ai/models/@cf/meta/m2m100-1.2b/) [![MyShell logo](https://developers.cloudflare.com/_astro/myshell.6ROagMV2.svg)melottsMyShellText-to-SpeechMeloTTS is a high-quality multi-lingual text-to-speech library by MyShell.ai.Cloudflare-hosted](https://developers.cloudflare.com/ai/models/@cf/myshell-ai/melotts/) [![Meta logo](https://developers.cloudflare.com/_astro/meta.CTzB_ysm.svg)meta-llama-3-8b-instructMetaText GenerationGeneration over generation, Meta Llama 3 demonstrates state-of-the-art performance on a wide range of industry benchmarks and offers new capabilities, including improved reasoning. Cloudflare-hostedDeprecated](https://developers.cloudflare.com/ai/models/@hf/meta-llama/meta-llama-3-8b-instruct/) [![MistralAI logo](https://developers.cloudflare.com/_astro/mistralai.Bn9UMUMu.svg)mistral-7b-instruct-v0.1MistralAIText GenerationInstruct fine-tuned version of the Mistral-7b generative text model with 7 billion parametersCloudflare-hostedLoRADeprecated](https://developers.cloudflare.com/ai/models/@cf/mistral/mistral-7b-instruct-v0.1/) [![MistralAI logo](https://developers.cloudflare.com/_astro/mistralai.Bn9UMUMu.svg)mistral-7b-instruct-v0.2BetaMistralAIText GenerationThe Mistral-7B-Instruct-v0.2 Large Language Model (LLM) is an instruct fine-tuned version of the Mistral-7B-v0.2\. Mistral-7B-v0.2 has the following changes compared to Mistral-7B-v0.1: 32k context window (vs 8k context in v0.1), rope-theta = 1e6, and no Sliding-Window Attention.Cloudflare-hostedLoRADeprecated](https://developers.cloudflare.com/ai/models/@hf/mistral/mistral-7b-instruct-v0.2/) [![MistralAI logo](https://developers.cloudflare.com/_astro/mistralai.Bn9UMUMu.svg)mistral-7b-instruct-v0.2-loraBetaMistralAIText GenerationThe Mistral-7B-Instruct-v0.2 Large Language Model (LLM) is an instruct fine-tuned version of the Mistral-7B-v0.2.Cloudflare-hostedLoRA](https://developers.cloudflare.com/ai/models/@cf/mistral/mistral-7b-instruct-v0.2-lora/) [![MistralAI logo](https://developers.cloudflare.com/_astro/mistralai.Bn9UMUMu.svg)mistral-small-3.1-24b-instructMistralAIText GenerationBuilding upon Mistral Small 3 (2501), Mistral Small 3.1 (2503) adds state-of-the-art vision understanding and enhances long context capabilities up to 128k tokens without compromising text performance. With 24 billion parameters, this model achieves top-tier capabilities in both text and vision tasks.Cloudflare-hostedFunction calling](https://developers.cloudflare.com/ai/models/@cf/mistralai/mistral-small-3.1-24b-instruct/) [mmoondream3.1-9B-A2BmoondreamImage-to-TextMoondream 3 is a fast, efficient 9B mixture-of-experts vision language model (2B active parameters) that delivers frontier-level visual reasoning for tasks like object detection, pointing, OCR, and structured output.Cloudflare-hostedVision](https://developers.cloudflare.com/ai/models/@cf/moondream/moondream3.1-9B-A2B/) [![NVIDIA logo](https://developers.cloudflare.com/_astro/nvidia.DI1bb8hH.svg)nemotron-3-120b-a12bNVIDIAText GenerationNVIDIA Nemotron 3 Super is a hybrid MoE model with leading accuracy for multi-agent applications and specialized agentic AI systems.Cloudflare-hostedFunction callingReasoning](https://developers.cloudflare.com/ai/models/@cf/nvidia/nemotron-3-120b-a12b/) [![Deepgram logo](https://developers.cloudflare.com/_astro/deepgram.BYzW8KfF.svg)nova-3DeepgramAutomatic Speech RecognitionTranscribe audio using Deepgram’s speech-to-text modelCloudflare-hostedBatchPartnerReal-time](https://developers.cloudflare.com/ai/models/@cf/deepgram/nova-3/) [![Microsoft logo](https://developers.cloudflare.com/_astro/microsoft.LujcDJ--.svg)phi-2BetaMicrosoftText GenerationPhi-2 is a Transformer-based model with a next-word prediction objective, trained on 1.4T tokens from multiple passes on a mixture of Synthetic and Web datasets for NLP and coding.Cloudflare-hostedDeprecated](https://developers.cloudflare.com/ai/models/@cf/microsoft/phi-2/) [![Leonardo logo](https://developers.cloudflare.com/_astro/leonardo.JZysY-g3.svg)phoenix-1.0LeonardoText-to-ImagePhoenix 1.0 is a model by Leonardo.Ai that generates images with exceptional prompt adherence and coherent text.Cloudflare-hostedPartner](https://developers.cloudflare.com/ai/models/@cf/leonardo/phoenix-1.0/) [pplamo-embedding-1bpfnetText EmbeddingsPLaMo-Embedding-1B is a Japanese text embedding model developed by Preferred Networks, Inc. It can convert Japanese text input into numerical vectors and can be used for a wide range of applications, including information retrieval, text classification, and clustering.Cloudflare-hosted](https://developers.cloudflare.com/ai/models/@cf/pfnet/plamo-embedding-1b/) [![Qwen logo](https://developers.cloudflare.com/_astro/qwen.ByCZjtXU.svg)qwen2.5-coder-32b-instructQwenText GenerationQwen2.5-Coder is the latest series of Code-Specific Qwen large language models (formerly known as CodeQwen). As of now, Qwen2.5-Coder has covered six mainstream model sizes, 0.5, 1.5, 3, 7, 14, 32 billion parameters, to meet the needs of different developers. Qwen2.5-Coder brings the following improvements upon CodeQwen1.5:Cloudflare-hostedLoRA](https://developers.cloudflare.com/ai/models/@cf/qwen/qwen2.5-coder-32b-instruct/) [![Qwen logo](https://developers.cloudflare.com/_astro/qwen.ByCZjtXU.svg)qwen3-30b-a3b-fp8QwenText GenerationQwen3 is the latest generation of large language models in Qwen series, offering a comprehensive suite of dense and mixture-of-experts (MoE) models. Built upon extensive training, Qwen3 delivers groundbreaking advancements in reasoning, instruction-following, agent capabilities, and multilingual support.Cloudflare-hostedBatchFunction callingReasoning](https://developers.cloudflare.com/ai/models/@cf/qwen/qwen3-30b-a3b-fp8/) [![Qwen logo](https://developers.cloudflare.com/_astro/qwen.ByCZjtXU.svg)qwen3-embedding-0.6bQwenText EmbeddingsThe Qwen3 Embedding model series is the latest proprietary model of the Qwen family, specifically designed for text embedding and ranking tasks. Cloudflare-hosted](https://developers.cloudflare.com/ai/models/@cf/qwen/qwen3-embedding-0.6b/) [![Qwen logo](https://developers.cloudflare.com/_astro/qwen.ByCZjtXU.svg)qwen3.8-27bQwenImage-Text-to-TextQwen 3.8 27B is a 27-billion-parameter instruction-tuned language model from Alibaba's Qwen family, designed for vision, efficient general-purpose text generation and agentic workloads.Cloudflare-hostedFunction callingReasoningVision](https://developers.cloudflare.com/ai/models/@cf/qwen/qwen3.8-27b/) [![Qwen logo](https://developers.cloudflare.com/_astro/qwen.ByCZjtXU.svg)qwq-32bQwenText GenerationQwQ is the reasoning model of the Qwen series. Compared with conventional instruction-tuned models, QwQ, which is capable of thinking and reasoning, can achieve significantly enhanced performance in downstream tasks, especially hard problems. QwQ-32B is the medium-sized reasoning model, which is capable of achieving competitive performance against state-of-the-art reasoning models, e.g., DeepSeek-R1, o1-mini.Cloudflare-hostedLoRAReasoning](https://developers.cloudflare.com/ai/models/@cf/qwen/qwq-32b/) [![Microsoft logo](https://developers.cloudflare.com/_astro/microsoft.LujcDJ--.svg)resnet-50MicrosoftImage Classification50 layers deep image classification CNN trained on more than 1M images from ImageNetCloudflare-hosted](https://developers.cloudflare.com/ai/models/@cf/microsoft/resnet-50/) [![Pipecat logo](https://developers.cloudflare.com/_astro/pipecat.B-PNBdef.svg)smart-turn-v2PipecatVoice Activity DetectionAn open source, community-driven, native audio turn detection model in 2nd versionCloudflare-hostedBatchReal-time](https://developers.cloudflare.com/ai/models/@cf/pipecat-ai/smart-turn-v2/) [![Defog logo](https://developers.cloudflare.com/_astro/defog.C0vfV4et.svg)sqlcoder-7b-2BetaDefogText GenerationThis model is intended to be used by non-technical users to understand data inside their SQL databases. Cloudflare-hostedDeprecated](https://developers.cloudflare.com/ai/models/@cf/defog/sqlcoder-7b-2/) [![RunwayML logo](https://developers.cloudflare.com/_astro/runway.Cq8Cjov4.svg)stable-diffusion-v1-5-img2imgBetaRunwayMLText-to-ImageStable Diffusion is a latent text-to-image diffusion model capable of generating photo-realistic images. Img2img generate a new image from an input image with Stable Diffusion. Cloudflare-hosted](https://developers.cloudflare.com/ai/models/@cf/runwayml/stable-diffusion-v1-5-img2img/) [![RunwayML logo](https://developers.cloudflare.com/_astro/runway.Cq8Cjov4.svg)stable-diffusion-v1-5-inpaintingBetaRunwayMLText-to-ImageStable Diffusion Inpainting is a latent text-to-image diffusion model capable of generating photo-realistic images given any text input, with the extra capability of inpainting the pictures by using a mask.Cloudflare-hosted](https://developers.cloudflare.com/ai/models/@cf/runwayml/stable-diffusion-v1-5-inpainting/) [![Stability.ai logo](https://developers.cloudflare.com/_astro/stabilityai.VsBx3CKv.svg)stable-diffusion-xl-base-1.0BetaStability.aiText-to-ImageDiffusion-based text-to-image generative model by Stability AI. Generates and modify images based on text prompts.Cloudflare-hosted](https://developers.cloudflare.com/ai/models/@cf/stabilityai/stable-diffusion-xl-base-1.0/) [![ByteDance logo](https://developers.cloudflare.com/_astro/bytedance.T1uiROQ6.svg)stable-diffusion-xl-lightningBetaByteDanceText-to-ImageSDXL-Lightning is a lightning-fast text-to-image generation model. It can generate high-quality 1024px images in a few steps.Cloudflare-hosted](https://developers.cloudflare.com/ai/models/@cf/bytedance/stable-diffusion-xl-lightning/) [![Unum logo](https://developers.cloudflare.com/_astro/unum.CWYcwnUh.svg)uform-gen2-qwen-500mBetaUnumImage-to-TextUForm-Gen is a small generative vision-language model primarily designed for Image Captioning and Visual Question Answering. The model was pre-trained on the internal image captioning dataset and fine-tuned on public instructions datasets: SVIT, LVIS, VQAs datasets.Cloudflare-hostedDeprecated](https://developers.cloudflare.com/ai/models/@cf/unum/uform-gen2-qwen-500m/) [![OpenAI logo](https://developers.cloudflare.com/_astro/openai.BBwNKzBb.svg)whisperOpenAIAutomatic Speech RecognitionWhisper is a general-purpose speech recognition model. It is trained on a large dataset of diverse audio and is also a multitasking model that can perform multilingual speech recognition, speech translation, and language identification.Cloudflare-hosted](https://developers.cloudflare.com/ai/models/@cf/openai/whisper/) [![OpenAI logo](https://developers.cloudflare.com/_astro/openai.BBwNKzBb.svg)whisper-large-v3-turboOpenAIAutomatic Speech RecognitionWhisper is a pre-trained model for automatic speech recognition (ASR) and speech translation. Cloudflare-hostedBatch](https://developers.cloudflare.com/ai/models/@cf/openai/whisper-large-v3-turbo/) [![OpenAI logo](https://developers.cloudflare.com/_astro/openai.BBwNKzBb.svg)whisper-tiny-enBetaOpenAIAutomatic Speech RecognitionWhisper is a pre-trained model for automatic speech recognition (ASR) and speech translation. Trained on 680k hours of labelled data, Whisper models demonstrate a strong ability to generalize to many datasets and domains without the need for fine-tuning. This is the English-only version of the Whisper Tiny model which was trained on the task of speech recognition.Cloudflare-hosted](https://developers.cloudflare.com/ai/models/@cf/openai/whisper-tiny-en/) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"WebPage","@id":"https://developers.cloudflare.com/ai/models/#page","headline":"Models · Cloudflare AI docs","description":"Browse AI models available through Cloudflare, including hosted models on Workers AI and external providers via AI Gateway.","url":"https://developers.cloudflare.com/ai/models/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-08-12","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Connect your AI applications to AI Gateway using the unified API, provider-native endpoints, or WebSockets. title: Using AI Gateway image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Using AI Gateway Last updated Apr 20, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/usage/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) ## Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/usage/#page","headline":"Using AI Gateway · Cloudflare AI Gateway docs","description":"Connect your AI applications to AI Gateway using the unified API, provider-native endpoints, or WebSockets.","url":"https://developers.cloudflare.com/ai-gateway/usage/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-20","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Send requests to multiple AI providers through a single OpenAI-compatible endpoint on AI Gateway. title: Unified API (OpenAI compat) image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Unified API (OpenAI compat) Last updated Aug 7, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/usage/chat-completion/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Deprecated for single-model calls For standard single-model chat completions, this endpoint is deprecated. Use the [REST API](https://developers.cloudflare.com/ai-gateway/usage/rest-api/) instead, which provides OpenAI-compatible endpoints at `api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/v1/chat/completions`. The `/compat/chat/completions` endpoint will continue to work for existing integrations. Required for dynamic routing [Dynamic routes](https://developers.cloudflare.com/ai-gateway/features/dynamic-routing/) (`dynamic/{route}`) are invoked through this `/compat/chat/completions` endpoint. The REST API does not currently cover dynamic routing, so continue to use this endpoint when calling a dynamic route. See [Using a dynamic route](https://developers.cloudflare.com/ai-gateway/features/dynamic-routing/usage/) for examples. Cloudflare's AI Gateway offers an OpenAI-compatible `/chat/completions` endpoint, enabling integration with multiple AI providers using a single URL. This feature simplifies the integration process, allowing for seamless switching between different models without significant code modifications. ## Endpoint URL ```txt https://gateway.ai.cloudflare.com/v1/{account_id}/default/compat/chat/completions ``` Replace `{account_id}` with your Cloudflare account ID. The `default` gateway is created automatically on your first request — no setup needed. You can also replace `default` with a specific gateway ID if you have already created one. ## Parameters Switch providers by changing the `model` and `apiKey` parameters. Specify the model using `{provider}/{model}` format. For example: * `openai/gpt-5-mini` * `google-ai-studio/gemini-2.5-flash` * `anthropic/claude-sonnet-4-5` ## Examples Make a request to ![]() OpenAI using OpenAI JS SDK with Stored Key (BYOK) ```javascript import OpenAI from "openai"; const client = new OpenAI({ apiKey: "{cf_api_token}", baseURL: "https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat", }); const response = await client.chat.completions.create({ model: "openai/gpt-5.2", messages: [{ role: "user", content: "Hello, world!" }], }); ``` ```javascript import OpenAI from "openai"; const client = new OpenAI({ apiKey: "{cf_api_token}", baseURL: "https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat", }); const response = await client.chat.completions.create({ model: "anthropic/claude-4-5-sonnet", messages: [{ role: "user", content: "Hello, world!" }], }); ``` ```javascript import OpenAI from "openai"; const client = new OpenAI({ apiKey: "{cf_api_token}", baseURL: "https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat", }); const response = await client.chat.completions.create({ model: "google/gemini-2.5-pro", messages: [{ role: "user", content: "Hello, world!" }], }); ``` ```javascript import OpenAI from "openai"; const client = new OpenAI({ apiKey: "{cf_api_token}", baseURL: "https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat", }); const response = await client.chat.completions.create({ model: "grok/grok-4", messages: [{ role: "user", content: "Hello, world!" }], }); ``` ```javascript import OpenAI from "openai"; const client = new OpenAI({ apiKey: "{cf_api_token}", baseURL: "https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat", }); const response = await client.chat.completions.create({ model: "dynamic/customer-support", messages: [{ role: "user", content: "Hello, world!" }], }); ``` ```javascript import OpenAI from "openai"; const client = new OpenAI({ apiKey: "{cf_api_token}", baseURL: "https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat", }); const response = await client.chat.completions.create({ model: "workers-ai/@cf/meta/llama-3.3-70b-instruct-fp8-fast", messages: [{ role: "user", content: "Hello, world!" }], }); ``` ```javascript import OpenAI from "openai"; const client = new OpenAI({ apiKey: "{openai_api_token}", defaultHeaders: { // if gateway is authenticated "cf-aig-authorization": `Bearer {cf_api_token}`, }, baseURL: "https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat", }); const response = await client.chat.completions.create({ model: "openai/gpt-5.2", messages: [{ role: "user", content: "Hello, world!" }], }); ``` ```javascript import OpenAI from "openai"; const client = new OpenAI({ apiKey: "{anthropic_api_token}", defaultHeaders: { // if gateway is authenticated "cf-aig-authorization": `Bearer {cf_api_token}`, }, baseURL: "https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat", }); const response = await client.chat.completions.create({ model: "anthropic/claude-4-5-sonnet", messages: [{ role: "user", content: "Hello, world!" }], }); ``` ```javascript import OpenAI from "openai"; const client = new OpenAI({ apiKey: "{google_api_token}", defaultHeaders: { // if gateway is authenticated "cf-aig-authorization": `Bearer {cf_api_token}`, }, baseURL: "https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat", }); const response = await client.chat.completions.create({ model: "google/gemini-2.5-pro", messages: [{ role: "user", content: "Hello, world!" }], }); ``` ```javascript import OpenAI from "openai"; const client = new OpenAI({ apiKey: "{grok_api_token}", defaultHeaders: { // if gateway is authenticated "cf-aig-authorization": `Bearer {cf_api_token}`, }, baseURL: "https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat", }); const response = await client.chat.completions.create({ model: "grok/grok-4", messages: [{ role: "user", content: "Hello, world!" }], }); ``` ```javascript import OpenAI from "openai"; const client = new OpenAI({ apiKey: "{dynamic_api_token}", defaultHeaders: { // if gateway is authenticated "cf-aig-authorization": `Bearer {cf_api_token}`, }, baseURL: "https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat", }); const response = await client.chat.completions.create({ model: "dynamic/customer-support", messages: [{ role: "user", content: "Hello, world!" }], }); ``` ```javascript import OpenAI from "openai"; const client = new OpenAI({ apiKey: "{workers-ai_api_token}", defaultHeaders: { // if gateway is authenticated "cf-aig-authorization": `Bearer {cf_api_token}`, }, baseURL: "https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat", }); const response = await client.chat.completions.create({ model: "workers-ai/@cf/meta/llama-3.3-70b-instruct-fp8-fast", messages: [{ role: "user", content: "Hello, world!" }], }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createUnified } from 'ai-gateway-provider/providers/unified'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const unified = createUnified(); const { text } = await generateText({ model: aigateway(unified('openai/gpt-5.2')), prompt: 'What is Cloudflare?', }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createUnified } from 'ai-gateway-provider/providers/unified'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const unified = createUnified(); const { text } = await generateText({ model: aigateway(unified('anthropic/claude-4-5-sonnet')), prompt: 'What is Cloudflare?', }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createUnified } from 'ai-gateway-provider/providers/unified'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const unified = createUnified(); const { text } = await generateText({ model: aigateway(unified('google/gemini-2.5-pro')), prompt: 'What is Cloudflare?', }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createUnified } from 'ai-gateway-provider/providers/unified'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const unified = createUnified(); const { text } = await generateText({ model: aigateway(unified('grok/grok-4')), prompt: 'What is Cloudflare?', }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createUnified } from 'ai-gateway-provider/providers/unified'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const unified = createUnified(); const { text } = await generateText({ model: aigateway(unified('dynamic/customer-support')), prompt: 'What is Cloudflare?', }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createUnified } from 'ai-gateway-provider/providers/unified'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const unified = createUnified(); const { text } = await generateText({ model: aigateway(unified('workers-ai/@cf/meta/llama-3.3-70b-instruct-fp8-fast')), prompt: 'What is Cloudflare?', }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createUnified } from 'ai-gateway-provider/providers/unified'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const unified = createUnified({ apiKey: '{API_KEY}' }); const { text } = await generateText({ model: aigateway(unified('openai/gpt-5.2')), prompt: 'What is Cloudflare?', }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createUnified } from 'ai-gateway-provider/providers/unified'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const unified = createUnified({ apiKey: '{API_KEY}' }); const { text } = await generateText({ model: aigateway(unified('anthropic/claude-4-5-sonnet')), prompt: 'What is Cloudflare?', }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createUnified } from 'ai-gateway-provider/providers/unified'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const unified = createUnified({ apiKey: '{API_KEY}' }); const { text } = await generateText({ model: aigateway(unified('google/gemini-2.5-pro')), prompt: 'What is Cloudflare?', }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createUnified } from 'ai-gateway-provider/providers/unified'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const unified = createUnified({ apiKey: '{API_KEY}' }); const { text } = await generateText({ model: aigateway(unified('grok/grok-4')), prompt: 'What is Cloudflare?', }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createUnified } from 'ai-gateway-provider/providers/unified'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const unified = createUnified({ apiKey: '{API_KEY}' }); const { text } = await generateText({ model: aigateway(unified('dynamic/customer-support')), prompt: 'What is Cloudflare?', }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createUnified } from 'ai-gateway-provider/providers/unified'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const unified = createUnified({ apiKey: '{API_KEY}' }); const { text } = await generateText({ model: aigateway(unified('workers-ai/@cf/meta/llama-3.3-70b-instruct-fp8-fast')), prompt: 'What is Cloudflare?', }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createOpenAI } from 'ai-gateway-provider/providers/openai'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const openai = createOpenAI(); const { text } = await generateText({ model: aigateway(openai.chat('gpt-5.2')), prompt: 'What is Cloudflare?', }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createAnthropic } from 'ai-gateway-provider/providers/anthropic'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const anthropic = createAnthropic(); const { text } = await generateText({ model: aigateway(anthropic('claude-4-5-sonnet')), prompt: 'What is Cloudflare?', }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createGoogle } from 'ai-gateway-provider/providers/google'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const google = createGoogle(); const { text } = await generateText({ model: aigateway(google('gemini-2.5-pro')), prompt: 'What is Cloudflare?', }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createXai } from 'ai-gateway-provider/providers/xai'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const xai = createXai(); const { text } = await generateText({ model: aigateway(xai('grok-4')), prompt: 'What is Cloudflare?', }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createUnified } from 'ai-gateway-provider/providers/unified'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const unified = createUnified(); const { text } = await generateText({ model: aigateway(unified('customer-support')), prompt: 'What is Cloudflare?', }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createUnified } from 'ai-gateway-provider/providers/unified'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const unified = createUnified(); const { text } = await generateText({ model: aigateway(unified('@cf/meta/llama-3.3-70b-instruct-fp8-fast')), prompt: 'What is Cloudflare?', }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createOpenAI } from 'ai-gateway-provider/providers/openai'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const openai = createOpenAI({ apiKey: '{API_KEY}' }); const { text } = await generateText({ model: aigateway(openai.chat('gpt-5.2')), prompt: 'What is Cloudflare?', }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createAnthropic } from 'ai-gateway-provider/providers/anthropic'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const anthropic = createAnthropic({ apiKey: '{API_KEY}' }); const { text } = await generateText({ model: aigateway(anthropic('claude-4-5-sonnet')), prompt: 'What is Cloudflare?', }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createGoogle } from 'ai-gateway-provider/providers/google'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const google = createGoogle({ apiKey: '{API_KEY}' }); const { text } = await generateText({ model: aigateway(google('gemini-2.5-pro')), prompt: 'What is Cloudflare?', }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createXai } from 'ai-gateway-provider/providers/xai'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const xai = createXai({ apiKey: '{API_KEY}' }); const { text } = await generateText({ model: aigateway(xai('grok-4')), prompt: 'What is Cloudflare?', }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createUnified } from 'ai-gateway-provider/providers/unified'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const unified = createUnified({ apiKey: '{API_KEY}' }); const { text } = await generateText({ model: aigateway(unified('customer-support')), prompt: 'What is Cloudflare?', }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createUnified } from 'ai-gateway-provider/providers/unified'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const unified = createUnified({ apiKey: '{API_KEY}' }); const { text } = await generateText({ model: aigateway(unified('@cf/meta/llama-3.3-70b-instruct-fp8-fast')), prompt: 'What is Cloudflare?', }); ``` ```bash curl -X POST https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat/chat/completions \ --header 'cf-aig-authorization: Bearer {CF_AIG_TOKEN}' \ --header 'Content-Type: application/json' \ --data '{ "model": "openai/gpt-5.2", "messages": [ { "role": "user", "content": "What is Cloudflare?" } ] }' ``` ```bash curl -X POST https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat/chat/completions \ --header 'cf-aig-authorization: Bearer {CF_AIG_TOKEN}' \ --header 'Content-Type: application/json' \ --data '{ "model": "anthropic/claude-4-5-sonnet", "messages": [ { "role": "user", "content": "What is Cloudflare?" } ] }' ``` ```bash curl -X POST https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat/chat/completions \ --header 'cf-aig-authorization: Bearer {CF_AIG_TOKEN}' \ --header 'Content-Type: application/json' \ --data '{ "model": "google/gemini-2.5-pro", "messages": [ { "role": "user", "content": "What is Cloudflare?" } ] }' ``` ```bash curl -X POST https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat/chat/completions \ --header 'cf-aig-authorization: Bearer {CF_AIG_TOKEN}' \ --header 'Content-Type: application/json' \ --data '{ "model": "grok/grok-4", "messages": [ { "role": "user", "content": "What is Cloudflare?" } ] }' ``` ```bash curl -X POST https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat/chat/completions \ --header 'cf-aig-authorization: Bearer {CF_AIG_TOKEN}' \ --header 'Content-Type: application/json' \ --data '{ "model": "dynamic/customer-support", "messages": [ { "role": "user", "content": "What is Cloudflare?" } ] }' ``` ```bash curl -X POST https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat/chat/completions \ --header 'cf-aig-authorization: Bearer {CF_AIG_TOKEN}' \ --header 'Content-Type: application/json' \ --data '{ "model": "workers-ai/@cf/meta/llama-3.3-70b-instruct-fp8-fast", "messages": [ { "role": "user", "content": "What is Cloudflare?" } ] }' ``` ```bash curl -X POST https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat/chat/completions \ --header 'cf-aig-authorization: Bearer {CF_AIG_TOKEN}' \ --header 'Authorization: Bearer {openai_api_token}' \ --header 'Content-Type: application/json' \ --data '{ "model": "openai/gpt-5.2", "messages": [ { "role": "user", "content": "What is Cloudflare?" } ] }' ``` ```bash curl -X POST https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat/chat/completions \ --header 'cf-aig-authorization: Bearer {CF_AIG_TOKEN}' \ --header 'Authorization: Bearer {anthropic_api_token}' \ --header 'Content-Type: application/json' \ --data '{ "model": "anthropic/claude-4-5-sonnet", "messages": [ { "role": "user", "content": "What is Cloudflare?" } ] }' ``` ```bash curl -X POST https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat/chat/completions \ --header 'cf-aig-authorization: Bearer {CF_AIG_TOKEN}' \ --header 'Authorization: Bearer {google_api_token}' \ --header 'Content-Type: application/json' \ --data '{ "model": "google/gemini-2.5-pro", "messages": [ { "role": "user", "content": "What is Cloudflare?" } ] }' ``` ```bash curl -X POST https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat/chat/completions \ --header 'cf-aig-authorization: Bearer {CF_AIG_TOKEN}' \ --header 'Authorization: Bearer {grok_api_token}' \ --header 'Content-Type: application/json' \ --data '{ "model": "grok/grok-4", "messages": [ { "role": "user", "content": "What is Cloudflare?" } ] }' ``` ```bash curl -X POST https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat/chat/completions \ --header 'cf-aig-authorization: Bearer {CF_AIG_TOKEN}' \ --header 'Authorization: Bearer {dynamic_api_token}' \ --header 'Content-Type: application/json' \ --data '{ "model": "dynamic/customer-support", "messages": [ { "role": "user", "content": "What is Cloudflare?" } ] }' ``` ```bash curl -X POST https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat/chat/completions \ --header 'cf-aig-authorization: Bearer {CF_AIG_TOKEN}' \ --header 'Authorization: Bearer {workers-ai_api_token}' \ --header 'Content-Type: application/json' \ --data '{ "model": "workers-ai/@cf/meta/llama-3.3-70b-instruct-fp8-fast", "messages": [ { "role": "user", "content": "What is Cloudflare?" } ] }' ``` ## Supported Providers The OpenAI-compatible endpoint supports models from the following providers: * [Anthropic](https://developers.cloudflare.com/ai-gateway/usage/providers/anthropic/) * [OpenAI](https://developers.cloudflare.com/ai-gateway/usage/providers/openai/) * [Groq](https://developers.cloudflare.com/ai-gateway/usage/providers/groq/) * [Mistral](https://developers.cloudflare.com/ai-gateway/usage/providers/mistral/) * [Cohere](https://developers.cloudflare.com/ai-gateway/usage/providers/cohere/) * [Perplexity](https://developers.cloudflare.com/ai-gateway/usage/providers/perplexity/) * [Workers AI](https://developers.cloudflare.com/ai-gateway/usage/providers/workersai/) * [Google-AI-Studio](https://developers.cloudflare.com/ai-gateway/usage/providers/google-ai-studio/) * [Google Vertex AI](https://developers.cloudflare.com/ai-gateway/usage/providers/vertex/) * [xAI](https://developers.cloudflare.com/ai-gateway/usage/providers/grok/) * [DeepSeek](https://developers.cloudflare.com/ai-gateway/usage/providers/deepseek/) * [Cerebras](https://developers.cloudflare.com/ai-gateway/usage/providers/cerebras/) * [Baseten](https://developers.cloudflare.com/ai-gateway/usage/providers/baseten/) * [Parallel](https://developers.cloudflare.com/ai-gateway/usage/providers/parallel/) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/usage/chat-completion/#page","headline":"Unified API (OpenAI compat) · Cloudflare AI Gateway docs","description":"Send requests to multiple AI providers through a single OpenAI-compatible endpoint on AI Gateway.","url":"https://developers.cloudflare.com/ai-gateway/usage/chat-completion/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-08-07","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"},"keywords":["AI"]} ``` --- --- description: Route Anthropic API requests through AI Gateway for observability and control. title: Anthropic image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Anthropic Last updated Jul 28, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/usage/providers/anthropic/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) [Anthropic ↗](https://www.anthropic.com/) helps build reliable, interpretable, and steerable AI systems. ## Endpoint **Base URL** ```txt https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/anthropic ``` ## Examples ### cURL With API Key in Request ```bash curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/anthropic/v1/messages \ --header 'x-api-key: {anthropic_api_key}' \ --header 'cf-aig-authorization: Bearer {CF_AIG_TOKEN}' \ --header 'anthropic-version: 2023-06-01' \ --header 'Content-Type: application/json' \ --data '{ "model": "claude-sonnet-4-5", "max_tokens": 1024, "messages": [ {"role": "user", "content": "What is Cloudflare?"} ] }' ``` ```bash curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/anthropic/v1/messages \ --header 'x-api-key: {anthropic_api_key}' \ --header 'anthropic-version: 2023-06-01' \ --header 'Content-Type: application/json' \ --data '{ "model": "claude-sonnet-4-5", "max_tokens": 1024, "messages": [ {"role": "user", "content": "What is Cloudflare?"} ] }' ``` With Stored Keys (BYOK) / Unified Billing ```bash curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/anthropic/v1/messages \ --header 'cf-aig-authorization: Bearer {CF_AIG_TOKEN}' \ --header 'anthropic-version: 2023-06-01' \ --header 'Content-Type: application/json' \ --data '{ "model": "claude-sonnet-4-5", "max_tokens": 1024, "messages": [ {"role": "user", "content": "What is Cloudflare?"} ] }' ``` ### Anthropic SDK With Key in Request ```js import Anthropic from "@anthropic-ai/sdk"; const baseURL = `https://gateway.ai.cloudflare.com/v1/{accountId}/{gatewayId}/anthropic`; const anthropic = new Anthropic({ apiKey: "{ANTHROPIC_API_KEY}", baseURL, defaultHeaders: { Authorization: `Bearer {cf_api_token}`, }, }); const message = await anthropic.messages.create({ model: "claude-sonnet-4-5", messages: [{ role: "user", content: "What is Cloudflare?" }], max_tokens: 1024, }); ``` ```js import Anthropic from "@anthropic-ai/sdk"; const baseURL = `https://gateway.ai.cloudflare.com/v1/{accountId}/{gatewayId}/anthropic`; const anthropic = new Anthropic({ apiKey: "{ANTHROPIC_API_KEY}", baseURL, }); const message = await anthropic.messages.create({ model: "claude-sonnet-4-5", messages: [{ role: "user", content: "What is Cloudflare?" }], max_tokens: 1024, }); ``` With Stored Keys (BYOK) / Unified Billing ```js import Anthropic from "@anthropic-ai/sdk"; const baseURL = `https://gateway.ai.cloudflare.com/v1/{accountId}/{gatewayId}/anthropic`; const anthropic = new Anthropic({ apiKey: "placeholder", // Ignored by AI Gateway when using BYOK or Unified Billing, but the SDK requires a value. baseURL, defaultHeaders: { Authorization: `Bearer {cf_api_token}`, }, }); const message = await anthropic.messages.create({ model: "claude-sonnet-4-5", messages: [{ role: "user", content: "What is Cloudflare?" }], max_tokens: 1024, }); ``` Note When using BYOK or Unified Billing, do not set `x-api-key` in `defaultHeaders`. AI Gateway supplies the Anthropic key for you, and adding your own `x-api-key` header will cause the request to fail. The `apiKey` value in the example is a placeholder to satisfy the Anthropic SDK, which requires either the `apiKey` option or the `ANTHROPIC_API_KEY` environment variable to be set. ## OpenAI-Compatible Endpoint You can also access Anthropic models using the OpenAI API schema through the [REST API](https://developers.cloudflare.com/ai-gateway/usage/rest-api/). Send your requests to: ```txt https://api.cloudflare.com/client/v4/accounts/{account_id}/ai/v1/chat/completions ``` Specify: ```json { "model": "anthropic/{model}" } ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/usage/providers/anthropic/#page","headline":"Anthropic · Cloudflare AI Gateway docs","description":"Route Anthropic API requests through AI Gateway for observability and control.","url":"https://developers.cloudflare.com/ai-gateway/usage/providers/anthropic/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-07-28","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Route Azure OpenAI requests through AI Gateway for observability and control. title: Azure OpenAI image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Azure OpenAI Last updated Apr 20, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/usage/providers/azureopenai/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) [Azure OpenAI ↗](https://azure.microsoft.com/en-gb/products/ai-services/openai-service/) allows you apply natural language algorithms on your data. ## Endpoint ```txt https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/azure-openai/{resource_name}/{deployment_name} ``` ## Prerequisites When making requests to Azure OpenAI, you will need: * AI Gateway account ID * AI Gateway gateway name * Azure OpenAI API key * Azure OpenAI resource name * Azure OpenAI deployment name (aka model name) ## URL structure Your new base URL will use the data above in this structure: `https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/azure-openai/{resource_name}/{deployment_name}`. Then, you can append your endpoint and api-version at the end of the base URL, like `.../chat/completions?api-version=2023-05-15`. ## Examples ### cURL ```bash curl 'https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway}/azure-openai/{resource_name}/{deployment_name}/chat/completions?api-version=2023-05-15' \ --header 'Content-Type: application/json' \ --header 'api-key: {azure_api_key}' \ --data '{ "messages": [ { "role": "user", "content": "What is Cloudflare?" } ] }' ``` ### Use `openai` JavaScript SDK ```js import { AzureOpenAI } from "openai"; const azure_openai = new AzureOpenAI({ apiKey: "{azure_api_key}", baseURL: `https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway}/azure-openai/{resource_name}/`, apiVersion: "2023-05-15", defaultHeaders: { "cf-aig-authorization": "{cf-api-token}" }, // if authenticated }); const result = await azure_openai.chat.completions.create({ model: '{deployment_name}', messages: [{ role: "user", content: "Hello" }], }); ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/usage/providers/azureopenai/#page","headline":"Azure OpenAI · Cloudflare AI Gateway docs","description":"Route Azure OpenAI requests through AI Gateway for observability and control.","url":"https://developers.cloudflare.com/ai-gateway/usage/providers/azureopenai/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-20","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Route Baseten model inference requests through AI Gateway for observability and control. title: Baseten image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Baseten Last updated Apr 20, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/usage/providers/baseten/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) [Baseten ↗](https://www.baseten.co/) provides infrastructure for building and deploying machine learning models at scale. Baseten offers access to various language models through a unified chat completions API. ## Endpoint ```txt https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/baseten ``` ## Prerequisites When making requests to Baseten, ensure you have the following: * Your AI Gateway Account ID. * Your AI Gateway gateway name. * An active Baseten API token. * The name of the Baseten model you want to use. ## OpenAI-compatible chat completions API Baseten provides an OpenAI-compatible chat completions API for supported models. ### cURL ```bash curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/baseten/v1/chat/completions \ --header 'Authorization: Bearer {baseten_api_token}' \ --header 'Content-Type: application/json' \ --data '{ "model": "openai/gpt-oss-120b", "messages": [ { "role": "user", "content": "What is Cloudflare?" } ] }' ``` ### Use OpenAI SDK with JavaScript ```js import OpenAI from "openai"; const apiKey = "{baseten_api_token}"; const accountId = "{account_id}"; const gatewayId = "{gateway_id}"; const baseURL = `https://gateway.ai.cloudflare.com/v1/${accountId}/${gatewayId}/baseten`; const openai = new OpenAI({ apiKey, baseURL, }); const model = "openai/gpt-oss-120b"; const messages = [{ role: "user", content: "What is Cloudflare?" }]; const chatCompletion = await openai.chat.completions.create({ model, messages, }); console.log(chatCompletion); ``` ## OpenAI-Compatible Endpoint You can also access Baseten models using the OpenAI API schema through the [REST API](https://developers.cloudflare.com/ai-gateway/usage/rest-api/). Send your requests to: ```txt https://api.cloudflare.com/client/v4/accounts/{account_id}/ai/v1/chat/completions ``` Specify: ```json { "model": "baseten/{model}" } ``` ## Model-specific endpoints For models that don't use the OpenAI-compatible API, you can access them through their specific model endpoints. ### cURL ```bash curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/baseten/model/{model_id} \ --header 'Authorization: Bearer {baseten_api_token}' \ --header 'Content-Type: application/json' \ --data '{ "prompt": "What is Cloudflare?", "max_tokens": 100 }' ``` ### Use with JavaScript ```js const accountId = "{account_id}"; const gatewayId = "{gateway_id}"; const basetenApiToken = "{baseten_api_token}"; const modelId = "{model_id}"; const baseURL = `https://gateway.ai.cloudflare.com/v1/${accountId}/${gatewayId}/baseten`; const response = await fetch(`${baseURL}/model/${modelId}`, { method: "POST", headers: { "Authorization": `Bearer ${basetenApiToken}`, "Content-Type": "application/json", }, body: JSON.stringify({ prompt: "What is Cloudflare?", max_tokens: 100, }), }); const result = await response.json(); console.log(result); ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/usage/providers/baseten/#page","headline":"Baseten · Cloudflare AI Gateway docs","description":"Route Baseten model inference requests through AI Gateway for observability and control.","url":"https://developers.cloudflare.com/ai-gateway/usage/providers/baseten/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-20","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Route Amazon Bedrock requests through AI Gateway for observability and control. title: Amazon Bedrock image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Amazon Bedrock Last updated May 7, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/usage/providers/bedrock/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) [Amazon Bedrock ↗](https://aws.amazon.com/bedrock/) allows you to build and scale generative AI applications with foundation models. ## Endpoint ```txt https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/aws-bedrock ``` ## Prerequisites When making requests to Amazon Bedrock, you will need: * AI Gateway account ID * AI Gateway gateway name * AWS credentials (`accessKeyId`, `secretAccessKey`, and `region`) with permissions for Amazon Bedrock * The name of the Amazon Bedrock model you want to use ## URL structure When making requests to Amazon Bedrock, replace `https://bedrock-runtime.us-east-1.amazonaws.com/` in the URL you are currently using with `https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/aws-bedrock/bedrock-runtime/us-east-1/`, then append the model you want to use. For example, to invoke the Anthropic Claude model in `us-east-1`: ```txt https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/aws-bedrock/bedrock-runtime/us-east-1/model/us.anthropic.claude-haiku-4-5-20251001-v1:0/invoke ``` ## Authenticating with Amazon Bedrock Amazon Bedrock uses [AWS Signature Version 4 (SigV4) ↗](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference%5Faws-signing.html) to authenticate API requests. Unlike providers such as OpenAI or Anthropic that use a simple API key, AWS requires each request to be cryptographically signed with your credentials. AI Gateway handles this complexity for you. When you store your AWS credentials using BYOK, the gateway automatically signs each request before forwarding it to AWS. ### Authentication methods comparison | Method | cf-aig-authorization header | Authorization header | Signing | | ----------------------- | --------------------------- | ---------------------- | ---------------------------------- | | **BYOK (Recommended)** | Bearer {CF\_AIG\_TOKEN} | Not needed | Gateway signs automatically | | **Client-side signing** | Bearer {CF\_AIG\_TOKEN} | Pre-signed AWS headers | You sign with aws4fetch or AWS SDK | Do not confuse the headers `cf-aig-authorization` authenticates your request to AI Gateway. When using BYOK, you do not need to include any AWS authorization headers because AI Gateway signs the request for you. ### Option 1: BYOK (Recommended) The recommended approach is to store your AWS credentials using AI Gateway's [Bring Your Own Keys (BYOK)](https://developers.cloudflare.com/ai-gateway/configuration/bring-your-own-keys/) feature. This keeps your credentials secure and eliminates the need for client-side request signing. 1. In the Cloudflare dashboard, go to **AI** \> **AI Gateway** \> your gateway > **Provider Keys**. 2. Select **Add API Key** and choose **Amazon Bedrock** as the provider. 3. Enter your AWS credentials as a JSON object with the following structure: ```json { "accessKeyId": "AKIAIOSFODNN7EXAMPLE", "secretAccessKey": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY", "region": "us-east-1" } ``` 4. Select **Save**. If you are using temporary credentials from AWS STS (for example, from assuming an IAM role), include the `sessionToken` field: ```json { "accessKeyId": "ASIAIOSFODNN7EXAMPLE", "secretAccessKey": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY", "region": "us-east-1", "sessionToken": "FwoGZXIvYXdzEBY..." } ``` With BYOK configured, you only need to include the `cf-aig-authorization` header in your requests. AI Gateway handles the AWS SigV4 signing automatically. ### Option 2: Client-side signing If you prefer to sign requests yourself, you can use the [aws4fetch ↗](https://github.com/mhart/aws4fetch) library or any AWS SDK to sign the request before sending it through AI Gateway. Refer to the [client-side signing example](#client-side-signing-with-aws4fetch) below. ## Examples ### cURL with BYOK With your AWS credentials [stored as a provider key](https://developers.cloudflare.com/ai-gateway/configuration/bring-your-own-keys/), requests are simple — no AWS signing required: ```bash curl "https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/aws-bedrock/bedrock-runtime/us-east-1/model/us.anthropic.claude-haiku-4-5-20251001-v1:0/invoke" \ -H "cf-aig-authorization: Bearer {CF_AIG_TOKEN}" \ -H "Content-Type: application/json" \ -d '{ "messages": [ { "role": "user", "content": "What is Cloudflare?" } ], "max_tokens": 256, "anthropic_version": "bedrock-2023-05-31" }' ``` ### Client-side signing with aws4fetch If you are not using BYOK, you must sign the request before sending it through AI Gateway. The following example uses the `aws4fetch` library in a Cloudflare Worker: ```typescript import { AwsClient } from "aws4fetch"; interface Env { accessKey: string; secretAccessKey: string; } export default { async fetch( request: Request, env: Env, ctx: ExecutionContext, ): Promise { const cfAccountId = "{account_id}"; const gatewayName = "{gateway_id}"; const region = "us-east-1"; const awsClient = new AwsClient({ accessKeyId: env.accessKey, secretAccessKey: env.secretAccessKey, region: region, service: "bedrock", }); const body = JSON.stringify({ messages: [{ role: "user", content: "What does ethereal mean?" }], max_tokens: 256, anthropic_version: "bedrock-2023-05-31", }); // Sign against the original AWS URL const awsUrl = `https://bedrock-runtime.${region}.amazonaws.com/model/us.anthropic.claude-haiku-4-5-20251001-v1:0/invoke`; const presignedRequest = await awsClient.sign(awsUrl, { method: "POST", headers: { "Content-Type": "application/json" }, body: body, }); // Send through AI Gateway const gatewayUrl = `https://gateway.ai.cloudflare.com/v1/${cfAccountId}/${gatewayName}/aws-bedrock/bedrock-runtime/${region}/model/us.anthropic.claude-haiku-4-5-20251001-v1:0/invoke`; const response = await fetch(gatewayUrl, { method: "POST", headers: presignedRequest.headers, body: body, }); if ( response.ok && response.headers.get("content-type")?.includes("application/json") ) { const data = await response.json(); return new Response(JSON.stringify(data)); } return new Response("Invalid response", { status: 500 }); }, }; ``` ## Using the Unified API (OpenAI compatible) AI Gateway provides a [Unified API](https://developers.cloudflare.com/ai-gateway/usage/chat-completion/) that lets you use the OpenAI chat completions format with Bedrock models. This is currently supported for **Anthropic Claude** and **Amazon Nova** model families. You can use the OpenAI SDK to access these models running on Bedrock without changing your request format. ### Endpoint ```txt https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat/chat/completions ``` ### cURL With your AWS credentials [stored as a provider key](https://developers.cloudflare.com/ai-gateway/configuration/bring-your-own-keys/), specify the model using the `aws-bedrock/{model}` format: ```bash curl "https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat/chat/completions" \ -H "cf-aig-authorization: Bearer {CF_AIG_TOKEN}" \ -H "Content-Type: application/json" \ -d '{ "model": "aws-bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0", "messages": [ { "role": "user", "content": "What is Cloudflare?" } ] }' ``` ### OpenAI SDK ```javascript import OpenAI from "openai"; const client = new OpenAI({ apiKey: "{CF_AIG_TOKEN}", baseURL: "https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat", }); const response = await client.chat.completions.create({ model: "aws-bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0", messages: [ { role: "user", content: "What is Cloudflare?", }, ], }); console.log(response.choices[0].message.content); ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/usage/providers/bedrock/#page","headline":"Amazon Bedrock · Cloudflare AI Gateway docs","description":"Route Amazon Bedrock requests through AI Gateway for observability and control.","url":"https://developers.cloudflare.com/ai-gateway/usage/providers/bedrock/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-05-07","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Route Cartesia text-to-speech requests through AI Gateway for observability and control. title: Cartesia image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Cartesia Last updated Apr 20, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/usage/providers/cartesia/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) [Cartesia ↗](https://docs.cartesia.ai/) provides advanced text-to-speech services with customizable voice models. ## Endpoint ```txt https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/cartesia ``` ## URL Structure When making requests to Cartesia, replace `https://api.cartesia.ai/v1` in the URL you are currently using with `https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/cartesia`. ## Prerequisites When making requests to Cartesia, ensure you have the following: * Your AI Gateway Account ID. * Your AI Gateway gateway name. * An active Cartesia API token. * The model ID and voice ID for the Cartesia voice model you want to use. ## Example ### cURL ```bash curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/cartesia/tts/bytes \ --header 'Content-Type: application/json' \ --header 'Cartesia-Version: 2024-06-10' \ --header 'X-API-Key: {cartesia_api_token}' \ --data '{ "transcript": "Welcome to Cloudflare - AI Gateway!", "model_id": "sonic-english", "voice": { "mode": "id", "id": "694f9389-aac1-45b6-b726-9d9369183238" }, "output_format": { "container": "wav", "encoding": "pcm_f32le", "sample_rate": 44100 } } ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/usage/providers/cartesia/#page","headline":"Cartesia · Cloudflare AI Gateway docs","description":"Route Cartesia text-to-speech requests through AI Gateway for observability and control.","url":"https://developers.cloudflare.com/ai-gateway/usage/providers/cartesia/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-20","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Route Cerebras inference requests through AI Gateway for observability and control. title: Cerebras image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Cerebras Last updated Aug 18, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/usage/providers/cerebras/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) [Cerebras ↗](https://inference-docs.cerebras.ai/) offers developers a low-latency solution for AI model inference. ## Endpoint ```txt https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/cerebras ``` ## Prerequisites When making requests to Cerebras, ensure you have the following: * Your AI Gateway Account ID. * Your AI Gateway gateway name. * An active Cerebras API token. * The name of the Cerebras model you want to use. ## Examples ### cURL ```bash curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/cerebras/chat/completions \ --header 'content-type: application/json' \ --header 'Authorization: Bearer CEREBRAS_TOKEN' \ --data '{ "model": "gpt-oss-120b", "messages": [ { "role": "user", "content": "What is Cloudflare?" } ] }' ``` ## OpenAI-Compatible Endpoint You can also access Cerebras models using the OpenAI API schema through the [REST API](https://developers.cloudflare.com/ai-gateway/usage/rest-api/). Send your requests to: ```txt https://api.cloudflare.com/client/v4/accounts/{account_id}/ai/v1/chat/completions ``` Specify: ```json { "model": "cerebras/{model}" } ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/usage/providers/cerebras/#page","headline":"Cerebras · Cloudflare AI Gateway docs","description":"Route Cerebras inference requests through AI Gateway for observability and control.","url":"https://developers.cloudflare.com/ai-gateway/usage/providers/cerebras/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-08-18","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Route Cohere API requests through AI Gateway for observability and control. title: Cohere image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Cohere Last updated Apr 20, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/usage/providers/cohere/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) [Cohere ↗](https://cohere.com/) build AI models designed to solve real-world business challenges. ## Endpoint ```txt https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/cohere ``` ## URL structure When making requests to [Cohere ↗](https://cohere.com/), replace `https://api.cohere.ai/v1` in the URL you're currently using with `https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/cohere`. ## Prerequisites When making requests to Cohere, ensure you have the following: * Your AI Gateway Account ID. * Your AI Gateway gateway name. * An active Cohere API token. * The name of the Cohere model you want to use. ## Examples ### cURL ```bash curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/cohere/v1/chat \ --header 'Authorization: Token {cohere_api_token}' \ --header 'Content-Type: application/json' \ --data '{ "chat_history": [ {"role": "USER", "message": "Who discovered gravity?"}, {"role": "CHATBOT", "message": "The man who is widely credited with discovering gravity is Sir Isaac Newton"} ], "message": "What year was he born?", "connectors": [{"id": "web-search"}] }' ``` ### Use Cohere SDK with Python If using the [cohere-python-sdk ↗](https://github.com/cohere-ai/cohere-python), set your endpoint like this: ```js import cohere import os api_key = os.getenv('API_KEY') account_id = '{account_id}' gateway_id = '{gateway_id}' base_url = f"https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/cohere/v1" co = cohere.Client( api_key=api_key, base_url=base_url, ) message = "hello world!" model = "command-r-plus" chat = co.chat( message=message, model=model ) print(chat) ``` ## OpenAI-Compatible Endpoint You can also access Cohere models using the OpenAI API schema through the [REST API](https://developers.cloudflare.com/ai-gateway/usage/rest-api/). Send your requests to: ```txt https://api.cloudflare.com/client/v4/accounts/{account_id}/ai/v1/chat/completions ``` Specify: ```json { "model": "cohere/{model}" } ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/usage/providers/cohere/#page","headline":"Cohere · Cloudflare AI Gateway docs","description":"Route Cohere API requests through AI Gateway for observability and control.","url":"https://developers.cloudflare.com/ai-gateway/usage/providers/cohere/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-20","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Route Deepgram speech-to-text and text-to-speech requests through AI Gateway for observability and control. title: Deepgram image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Deepgram Last updated Apr 20, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/usage/providers/deepgram/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) [Deepgram ↗](https://developers.deepgram.com/home) provides Voice AI APIs for speech-to-text, text-to-speech, and voice agents. Note Deepgram is also available through Workers AI, see [Deepgram Workers AI](https://developers.cloudflare.com/ai-gateway/usage/websockets-api/realtime-api/#deepgram-workers-ai). ## Endpoint ```txt https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/deepgram ``` ## URL Structure When making requests to Deepgram, replace `https://api.deepgram.com/` in the URL you are currently using with `https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/deepgram/`. ## Prerequisites When making requests to Deepgram, ensure you have the following: * Your AI Gateway Account ID. * Your AI Gateway gateway name. * An active Deepgram API token. ## Example ### SDK ```ts import { createClient, LiveTranscriptionEvents } from "@deepgram/sdk"; const deepgram = createClient("{deepgram_api_key}", { global: { websocket: { options: { url: "wss://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/deepgram/", _nodeOnlyHeaders: { "cf-aig-authorization": "Bearer {CF_AIG_TOKEN}" } } } } }); const connection = deepgram.listen.live({ model: "nova-3", language: "en-US", smart_format: true, }); connection.send(...); ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/usage/providers/deepgram/#page","headline":"Deepgram · Cloudflare AI Gateway docs","description":"Route Deepgram speech-to-text and text-to-speech requests through AI Gateway for observability and control.","url":"https://developers.cloudflare.com/ai-gateway/usage/providers/deepgram/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-20","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Route DeepSeek API requests through AI Gateway for observability and control. title: DeepSeek image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # DeepSeek Last updated Apr 20, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/usage/providers/deepseek/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) [DeepSeek ↗](https://www.deepseek.com/) helps you build quickly with DeepSeek's advanced AI models. ## Endpoint ```txt https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/deepseek ``` ## Prerequisites When making requests to DeepSeek, ensure you have the following: * Your AI Gateway Account ID. * Your AI Gateway gateway name. * An active DeepSeek AI API token. * The name of the DeepSeek AI model you want to use. ## URL structure Your new base URL will use the data above in this structure: `https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/deepseek/`. You can then append the endpoint you want to hit, for example: `chat/completions`. So your final URL will come together as: `https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/deepseek/chat/completions`. ## Examples ### cURL ```bash curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/deepseek/chat/completions \ --header 'content-type: application/json' \ --header 'Authorization: Bearer DEEPSEEK_TOKEN' \ --data '{ "model": "deepseek-chat", "messages": [ { "role": "user", "content": "What is Cloudflare?" } ] }' ``` ### Use DeepSeek with JavaScript If you are using the OpenAI SDK, you can set your endpoint like this: ```js import OpenAI from "openai"; const openai = new OpenAI({ apiKey: env.DEEPSEEK_TOKEN, baseURL: "https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/deepseek", }); try { const chatCompletion = await openai.chat.completions.create({ model: "deepseek-chat", messages: [{ role: "user", content: "What is Cloudflare?" }], }); const response = chatCompletion.choices[0].message; return new Response(JSON.stringify(response)); } catch (e) { return new Response(e); } ``` ## OpenAI-Compatible Endpoint You can also access DeepSeek models using the OpenAI API schema through the [REST API](https://developers.cloudflare.com/ai-gateway/usage/rest-api/). Send your requests to: ```txt https://api.cloudflare.com/client/v4/accounts/{account_id}/ai/v1/chat/completions ``` Specify: ```json { "model": "deepseek/{model}" } ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/usage/providers/deepseek/#page","headline":"DeepSeek · Cloudflare AI Gateway docs","description":"Route DeepSeek API requests through AI Gateway for observability and control.","url":"https://developers.cloudflare.com/ai-gateway/usage/providers/deepseek/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-20","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Route ElevenLabs text-to-speech requests through AI Gateway for observability and control. title: ElevenLabs image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # ElevenLabs Last updated Apr 20, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/usage/providers/elevenlabs/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) [ElevenLabs ↗](https://elevenlabs.io/) offers advanced text-to-speech services, enabling high-quality voice synthesis in multiple languages. ## Endpoint ```txt https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/elevenlabs ``` ## Prerequisites When making requests to ElevenLabs, ensure you have the following: * Your AI Gateway Account ID. * Your AI Gateway gateway name. * An active ElevenLabs API token. * The model ID of the ElevenLabs voice model you want to use. ## Example ### cURL ```bash curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/elevenlabs/v1/text-to-speech/JBFqnCBsd6RMkjVDRZzb?output_format=mp3_44100_128 \ --header 'Content-Type: application/json' \ --header 'xi-api-key: {elevenlabs_api_token}' \ --data '{ "text": "Welcome to Cloudflare - AI Gateway!", "model_id": "eleven_multilingual_v2" }' ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/usage/providers/elevenlabs/#page","headline":"ElevenLabs · Cloudflare AI Gateway docs","description":"Route ElevenLabs text-to-speech requests through AI Gateway for observability and control.","url":"https://developers.cloudflare.com/ai-gateway/usage/providers/elevenlabs/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-20","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Route Fal AI generative media requests through AI Gateway for observability and control. title: Fal AI image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Fal AI Last updated Apr 20, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/usage/providers/fal/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) [Fal AI ↗](https://fal.ai/) provides access to 600+ production-ready generative media models through a single, unified API. The service offers the world's largest collection of open image, video, voice, and audio generation models, all accessible with one line of code. ## Endpoint ```txt https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/fal ``` ## URL structure When making requests to Fal AI, replace `https://fal.run` in the URL you're currently using with `https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/fal`. ## Prerequisites When making requests to Fal AI, ensure you have the following: * Your AI Gateway Account ID. * Your AI Gateway gateway name. * An active Fal AI API token. * The name of the Fal AI model you want to use. ## Default synchronous API By default, requests to the Fal AI endpoint will hit the synchronous API at `https://fal.run/`. ### cURL example ```bash curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/fal/fal-ai/fast-sdxl \ --header 'Authorization: Key {fal_ai_token}' \ --header 'Content-Type: application/json' \ --data '{ "prompt": "Make an image of a cat flying an aeroplane" }' ``` ## Custom target URLs If you need to hit a different target URL, you can supply the entire Fal target URL in the `x-fal-target-url` header. ### cURL example with custom target URL ```bash curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/fal \ --header 'Authorization: Bearer {fal_ai_token}' \ --header 'x-fal-target-url: https://queue.fal.run/fal-ai/bytedance/seedream/v4/edit' \ --header 'Content-Type: application/json' \ --data '{ "prompt": "Dress the model in the clothes and hat. Add a cat to the scene and change the background to a Victorian era building.", "image_urls": [ "https://storage.googleapis.com/falserverless/example_inputs/seedream4_edit_input_1.png", "https://storage.googleapis.com/falserverless/example_inputs/seedream4_edit_input_2.png", "https://storage.googleapis.com/falserverless/example_inputs/seedream4_edit_input_3.png", "https://storage.googleapis.com/falserverless/example_inputs/seedream4_edit_input_4.png" ] }' ``` ## WebSocket API Fal AI also supports real-time interactions through WebSockets. For WebSocket connections and examples, see the [Realtime WebSockets API documentation](https://developers.cloudflare.com/ai-gateway/usage/websockets-api/realtime-api/#fal-ai). ## JavaScript SDK integration The `x-fal-target-url` format is compliant with the Fal SDKs, so AI Gateway can be easily passed as a `proxyUrl` in the SDKs. ### JavaScript SDK example ```js import { fal } from "@fal-ai/client"; fal.config({ credentials: "{fal_ai_token}", // OR pass a cloudflare api token if using BYOK on AI Gateway proxyUrl: "https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/fal" }); const result = await fal.subscribe("fal-ai/bytedance/seedream/v4/edit", { "input": { "prompt": "Dress the model in the clothes and hat. Add a cat to the scene and change the background to a Victorian era building.", "image_urls": [ "https://storage.googleapis.com/falserverless/example_inputs/seedream4_edit_input_1.png", "https://storage.googleapis.com/falserverless/example_inputs/seedream4_edit_input_2.png", "https://storage.googleapis.com/falserverless/example_inputs/seedream4_edit_input_3.png", "https://storage.googleapis.com/falserverless/example_inputs/seedream4_edit_input_4.png" ] } }); console.log(result.data.images[0]); ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/usage/providers/fal/#page","headline":"Fal AI · Cloudflare AI Gateway docs","description":"Route Fal AI generative media requests through AI Gateway for observability and control.","url":"https://developers.cloudflare.com/ai-gateway/usage/providers/fal/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-20","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Route Google AI Studio and Gemini requests through AI Gateway for observability and control. title: Google AI Studio image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Google AI Studio Last updated Apr 20, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/usage/providers/google-ai-studio/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) [Google AI Studio ↗](https://ai.google.dev/aistudio) helps you build quickly with Google Gemini models. ## Endpoint **Base URL:** ```txt https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/google-ai-studio ``` Then you can append the endpoint you want to hit, for example: `v1/models/{model}:{generative_ai_rest_resource}` So your final URL will come together as: `https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/google-ai-studio/v1/models/{model}:{generative_ai_rest_resource}`. ## Examples ### cURL With API Key in Request ```bash curl "https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_name}/google-ai-studio/v1/models/gemini-2.5-flash:generateContent" \ --header 'content-type: application/json' \ --header 'cf-aig-authorization: Bearer {CF_AIG_TOKEN}' \ --header 'x-goog-api-key: {google_studio_api_key}' \ --data '{ "contents": [ { "role":"user", "parts": [ {"text":"What is Cloudflare?"} ] } ] }' ``` ```bash curl "https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_name}/google-ai-studio/v1/models/gemini-2.5-flash:generateContent" \ --header 'content-type: application/json' \ --header 'x-goog-api-key: {google_studio_api_key}' \ --data '{ "contents": [ { "role":"user", "parts": [ {"text":"What is Cloudflare?"} ] } ] }' ``` With Stored Keys (BYOK) / Unified Billing ```bash curl "https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_name}/google-ai-studio/v1/models/gemini-2.5-flash:generateContent" \ --header 'content-type: application/json' \ --header 'cf-aig-authorization: Bearer {CF_AIG_TOKEN}' \ --data '{ "contents": [ { "role":"user", "parts": [ {"text":"What is Cloudflare?"} ] } ] }' ``` ### `@google/genai` If you are using the `@google/genai` package, you can set your endpoint like this: With Key in Request ```js import { GoogleGenAI } from "@google/genai"; const ai = new GoogleGenAI({ apiKey: "{google_studio_api_key}", httpOptions: { baseUrl: `https://gateway.ai.cloudflare.com/v1/${account_id}/${gateway_name}/google-ai-studio`, headers: { 'cf-aig-authorization': 'Bearer {cf_aig_token}', } } }); const response = await ai.models.generateContent({ model: "gemini-2.5-flash", contents: "What is Cloudflare?", }); console.log(response.text); ``` ```js import { GoogleGenAI } from "@google/genai"; const ai = new GoogleGenAI({ apiKey: "{google_studio_api_key}", httpOptions: { baseUrl: `https://gateway.ai.cloudflare.com/v1/${account_id}/${gateway_name}/google-ai-studio`, } }); const response = await ai.models.generateContent({ model: "gemini-2.5-flash", contents: "What is Cloudflare?", }); console.log(response.text); ``` With Stored Keys (BYOK) / Unified Billing ```js import { GoogleGenAI } from "@google/genai"; const ai = new GoogleGenAI({ apiKey: "{cf_aig_token}", httpOptions: { baseUrl: `https://gateway.ai.cloudflare.com/v1/${account_id}/${gateway_name}/google-ai-studio`, } }); const response = await ai.models.generateContent({ model: "gemini-2.5-flash", contents: "What is Cloudflare?", }); console.log(response.text); ``` ## OpenAI-Compatible Endpoint You can also access Google AI Studio models using the OpenAI API schema through the [REST API](https://developers.cloudflare.com/ai-gateway/usage/rest-api/). Send your requests to: ```txt https://api.cloudflare.com/client/v4/accounts/{account_id}/ai/v1/chat/completions ``` Specify: ```json { "model": "google-ai-studio/{model}" } ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/usage/providers/google-ai-studio/#page","headline":"Google AI Studio · Cloudflare AI Gateway docs","description":"Route Google AI Studio and Gemini requests through AI Gateway for observability and control.","url":"https://developers.cloudflare.com/ai-gateway/usage/providers/google-ai-studio/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-20","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Route xAI (Grok) API requests through AI Gateway for observability and control. title: xAI image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # xAI Last updated Apr 20, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/usage/providers/grok/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) ## Endpoint ```txt https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/grok ``` ## URL structure When making requests to [Grok ↗](https://docs.x.ai/docs#getting-started), replace `https://api.x.ai/v1` in the URL you are currently using with `https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/grok`. ## Prerequisites When making requests to Grok, ensure you have the following: * Your AI Gateway Account ID. * Your AI Gateway gateway name. * An active xAI API token. * The name of the xAI model you want to use. ## Examples ### cURL ```bash curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/grok/v1/chat/completions \ --header 'content-type: application/json' \ --header 'Authorization: Bearer {xai_api_token}' \ --data '{ "model": "grok-4", "messages": [ { "role": "user", "content": "What is Cloudflare?" } ] }' ``` ### Use OpenAI SDK with JavaScript If you are using the OpenAI SDK with JavaScript, you can set your endpoint like this: ```js import OpenAI from "openai"; const openai = new OpenAI({ apiKey: "", baseURL: "https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/grok", }); const completion = await openai.chat.completions.create({ model: "grok-4", messages: [ { role: "system", content: "You are Grok, a chatbot inspired by the Hitchhiker's Guide to the Galaxy.", }, { role: "user", content: "What is the meaning of life, the universe, and everything?", }, ], }); console.log(completion.choices[0].message); ``` ### Use OpenAI SDK with Python If you are using the OpenAI SDK with Python, you can set your endpoint like this: ```python import os from openai import OpenAI XAI_API_KEY = os.getenv("XAI_API_KEY") client = OpenAI( api_key=XAI_API_KEY, base_url="https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/grok", ) completion = client.chat.completions.create( model="grok-4", messages=[ {"role": "system", "content": "You are Grok, a chatbot inspired by the Hitchhiker's Guide to the Galaxy."}, {"role": "user", "content": "What is the meaning of life, the universe, and everything?"}, ], ) print(completion.choices[0].message) ``` ### Use Anthropic SDK with JavaScript If you are using the Anthropic SDK with JavaScript, you can set your endpoint like this: ```js import Anthropic from "@anthropic-ai/sdk"; const anthropic = new Anthropic({ apiKey: "", baseURL: "https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/grok", }); const msg = await anthropic.messages.create({ model: "grok-beta", max_tokens: 128, system: "You are Grok, a chatbot inspired by the Hitchhiker's Guide to the Galaxy.", messages: [ { role: "user", content: "What is the meaning of life, the universe, and everything?", }, ], }); console.log(msg); ``` ### Use Anthropic SDK with Python If you are using the Anthropic SDK with Python, you can set your endpoint like this: ```python import os from anthropic import Anthropic XAI_API_KEY = os.getenv("XAI_API_KEY") client = Anthropic( api_key=XAI_API_KEY, base_url="https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/grok", ) message = client.messages.create( model="grok-beta", max_tokens=128, system="You are Grok, a chatbot inspired by the Hitchhiker's Guide to the Galaxy.", messages=[ { "role": "user", "content": "What is the meaning of life, the universe, and everything?", }, ], ) print(message.content) ``` ## OpenAI-Compatible Endpoint You can also access Grok models using the OpenAI API schema through the [REST API](https://developers.cloudflare.com/ai-gateway/usage/rest-api/). Send your requests to: ```txt https://api.cloudflare.com/client/v4/accounts/{account_id}/ai/v1/chat/completions ``` Specify: ```json { "model": "grok/{model}" } ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/usage/providers/grok/#page","headline":"xAI · Cloudflare AI Gateway docs","description":"Route xAI (Grok) API requests through AI Gateway for observability and control.","url":"https://developers.cloudflare.com/ai-gateway/usage/providers/grok/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-20","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Route Groq API requests through AI Gateway for observability and control. title: Groq image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Groq Last updated Apr 20, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/usage/providers/groq/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) [Groq ↗](https://groq.com/) delivers high-speed processing and low-latency performance. ## Endpoint ```txt https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/groq ``` ## URL structure When making requests to [Groq ↗](https://groq.com/), replace `https://api.groq.com/openai/v1` in the URL you're currently using with `https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/groq`. ## Prerequisites When making requests to Groq, ensure you have the following: * Your AI Gateway Account ID. * Your AI Gateway gateway name. * An active Groq API token. * The name of the Groq model you want to use. ## Examples ### cURL ```bash curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/groq/chat/completions \ --header 'Authorization: Bearer {groq_api_key}' \ --header 'Content-Type: application/json' \ --data '{ "messages": [ { "role": "user", "content": "What is Cloudflare?" } ], "model": "llama3-8b-8192" }' ``` ### Use Groq SDK with JavaScript If using the [groq-sdk ↗](https://www.npmjs.com/package/groq-sdk), set your endpoint like this: ```js import Groq from "groq-sdk"; const apiKey = env.GROQ_API_KEY; const accountId = "{account_id}"; const gatewayId = "{gateway_id}"; const baseURL = `https://gateway.ai.cloudflare.com/v1/${accountId}/${gatewayId}/groq`; const groq = new Groq({ apiKey, baseURL, }); const messages = [{ role: "user", content: "What is Cloudflare?" }]; const model = "llama3-8b-8192"; const chatCompletion = await groq.chat.completions.create({ messages, model, }); ``` ## OpenAI-Compatible Endpoint You can also access Groq models using the OpenAI API schema through the [REST API](https://developers.cloudflare.com/ai-gateway/usage/rest-api/). Send your requests to: ```txt https://api.cloudflare.com/client/v4/accounts/{account_id}/ai/v1/chat/completions ``` Specify: ```json { "model": "groq/{model}" } ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/usage/providers/groq/#page","headline":"Groq · Cloudflare AI Gateway docs","description":"Route Groq API requests through AI Gateway for observability and control.","url":"https://developers.cloudflare.com/ai-gateway/usage/providers/groq/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-20","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Route HuggingFace Inference API requests through AI Gateway for observability and control. title: HuggingFace image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # HuggingFace Last updated Apr 20, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/usage/providers/huggingface/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) [HuggingFace ↗](https://huggingface.co/) helps users build, deploy and train machine learning models. ## Endpoint ```txt https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/huggingface ``` ## URL structure When making requests to HuggingFace Inference API, replace `https://api-inference.huggingface.co/models/` in the URL you're currently using with `https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/huggingface`. Note that the model you're trying to access should come right after, for example `https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/huggingface/bigcode/starcoder`. ## Prerequisites When making requests to HuggingFace, ensure you have the following: * Your AI Gateway Account ID. * Your AI Gateway gateway name. * An active HuggingFace API token. * The name of the HuggingFace model you want to use. ## Examples ### cURL ```bash curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/huggingface/bigcode/starcoder \ --header 'Authorization: Bearer {hf_api_token}' \ --header 'Content-Type: application/json' \ --data '{ "inputs": "console.log" }' ``` ### Use HuggingFace.js library with JavaScript If you are using the HuggingFace.js library, you can set your inference endpoint like this: ```js import { HfInferenceEndpoint } from "@huggingface/inference"; const accountId = "{account_id}"; const gatewayId = "{gateway_id}"; const model = "gpt2"; const baseURL = `https://gateway.ai.cloudflare.com/v1/${accountId}/${gatewayId}/huggingface/${model}`; const apiToken = env.HF_API_TOKEN; const hf = new HfInferenceEndpoint(baseURL, apiToken); ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/usage/providers/huggingface/#page","headline":"HuggingFace · Cloudflare AI Gateway docs","description":"Route HuggingFace Inference API requests through AI Gateway for observability and control.","url":"https://developers.cloudflare.com/ai-gateway/usage/providers/huggingface/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-20","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Route Ideogram image generation requests through AI Gateway for observability and control. title: Ideogram image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Ideogram Last updated Apr 20, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/usage/providers/ideogram/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) [Ideogram ↗](https://ideogram.ai/) provides advanced text-to-image generation models with exceptional text rendering capabilities and visual quality. ## Endpoint ```txt https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/ideogram ``` ## Prerequisites When making requests to Ideogram, ensure you have the following: * Your AI Gateway Account ID. * Your AI Gateway gateway name. * An active Ideogram API key. * The name of the Ideogram model you want to use (e.g., `V_3`). ## Examples ### cURL ```bash curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/ideogram/v1/ideogram-v3/generate \ --header 'Api-Key: {ideogram_api_key}' \ --header 'Content-Type: application/json' \ --data '{ "prompt": "A serene landscape with mountains and a lake at sunset", "model": "V_3" }' ``` ### Use with JavaScript ```js const accountId = "{account_id}"; const gatewayId = "{gateway_id}"; const ideogramApiKey = "{ideogram_api_key}"; const baseURL = `https://gateway.ai.cloudflare.com/v1/${accountId}/${gatewayId}/ideogram`; const response = await fetch(`${baseURL}/v1/ideogram-v3/generate`, { method: "POST", headers: { "Api-Key": ideogramApiKey, "Content-Type": "application/json", }, body: JSON.stringify({ prompt: "A serene landscape with mountains and a lake at sunset", model: "V_3", }), }); const result = await response.json(); console.log(result); ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/usage/providers/ideogram/#page","headline":"Ideogram · Cloudflare AI Gateway docs","description":"Route Ideogram image generation requests through AI Gateway for observability and control.","url":"https://developers.cloudflare.com/ai-gateway/usage/providers/ideogram/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-20","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Route Mistral AI requests through AI Gateway for observability and control. title: Mistral AI image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Mistral AI Last updated Apr 20, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/usage/providers/mistral/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) [Mistral AI ↗](https://mistral.ai) helps you build quickly with Mistral's advanced AI models. ## Endpoint ```txt https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/mistral ``` ## Prerequisites When making requests to the Mistral AI, you will need: * AI Gateway Account ID * AI Gateway gateway name * Mistral AI API token * Mistral AI model name ## URL structure Your new base URL will use the data above in this structure: `https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/mistral/`. Then you can append the endpoint you want to hit, for example: `v1/chat/completions` So your final URL will come together as: `https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/mistral/v1/chat/completions`. ## Examples ### cURL ```bash curl -X POST https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/mistral/v1/chat/completions \ --header 'content-type: application/json' \ --header 'Authorization: Bearer MISTRAL_TOKEN' \ --data '{ "model": "mistral-large-latest", "messages": [ { "role": "user", "content": "What is Cloudflare?" } ] }' ``` ### Use `@mistralai/mistralai` package with JavaScript If you are using the `@mistralai/mistralai` package, you can set your endpoint like this: ```js import { Mistral } from "@mistralai/mistralai"; const client = new Mistral({ apiKey: MISTRAL_TOKEN, serverURL: `https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/mistral`, }); await client.chat.create({ model: "mistral-large-latest", messages: [ { role: "user", content: "What is Cloudflare?", }, ], }); ``` ## OpenAI-Compatible Endpoint You can also access Mistral models using the OpenAI API schema through the [REST API](https://developers.cloudflare.com/ai-gateway/usage/rest-api/). Send your requests to: ```txt https://api.cloudflare.com/client/v4/accounts/{account_id}/ai/v1/chat/completions ``` Specify: ```json { "model": "mistral/{model}" } ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/usage/providers/mistral/#page","headline":"Mistral AI · Cloudflare AI Gateway docs","description":"Route Mistral AI requests through AI Gateway for observability and control.","url":"https://developers.cloudflare.com/ai-gateway/usage/providers/mistral/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-20","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Route OpenAI API requests through AI Gateway for observability and control. title: OpenAI image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # OpenAI Last updated Apr 20, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/usage/providers/openai/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) [OpenAI ↗](https://openai.com/about/) helps you build with GPT models. ## Endpoint **Base URL** ```plaintext https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/openai ``` When making requests to OpenAI, replace `https://api.openai.com/v1` in the URL you are currently using with `https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/openai`. **Chat completions endpoint** `https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/openai/chat/completions` **Responses endpoint** `https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/openai/responses` ## Examples ### OpenAI SDK With Key in Request ```js import OpenAI from "openai"; const client = new OpenAI({ apiKey: "YOUR_OPENAI_API_KEY", defaultHeaders: { "cf-aig-authorization": `Bearer {cf_api_token}`, }, baseURL: "https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/openai", }); const response = await client.chat.completions.create({ model: "gpt-4o-mini", messages: [{ role: "user", content: "Hello, world!" }], }); ``` ```js import OpenAI from "openai"; const client = new OpenAI({ apiKey: "YOUR_OPENAI_API_KEY", baseURL: "https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/openai", }); const response = await client.chat.completions.create({ model: "gpt-4o-mini", messages: [{ role: "user", content: "Hello, world!" }], }); ``` With Stored Keys (BYOK) / Unified Billing ```js import OpenAI from "openai"; const client = new OpenAI({ apiKey: "{cf_api_token}", baseURL: "https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/openai", }); // Ensure your OpenAI API key is stored with BYOK // or Unified Billing has credits const response = await client.chat.completions.create({ model: "gpt-4o-mini", messages: [{ role: "user", content: "Hello, world!" }], }); ``` ### cURL Responses API with API Key in Request ```bash curl -X POST https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/openai/responses \ --header 'Authorization: Bearer {OPENAI_API_KEY}' \ --header 'cf-aig-authorization: Bearer {CF_AIG_TOKEN}' \ --header 'Content-Type: application/json' \ --data '{ "model": "gpt-5.1", "input": [ { "role": "user", "content": "Write a one-sentence bedtime story about a unicorn." } ] }' ``` ```bash curl -X POST https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/openai/responses \ --header 'Authorization: Bearer {OPENAI_API_KEY}' \ --header 'Content-Type: application/json' \ --data '{ "model": "gpt-5.1", "input": [ { "role": "user", "content": "Write a one-sentence bedtime story about a unicorn." } ] }' ``` Chat Completions with API Key in Request ```bash curl -X POST https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/openai/chat/completions \ --header 'Authorization: Bearer {OPENAI_API_KEY}' \ --header 'cf-aig-authorization: Bearer {CF_AIG_TOKEN}' \ --header 'Content-Type: application/json' \ --data '{ "model": "gpt-4o-mini", "messages": [ { "role": "user", "content": "What is Cloudflare?" } ] }' ``` ```bash curl -X POST https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/openai/chat/completions \ --header 'Authorization: Bearer {OPENAI_API_KEY}' \ --header 'Content-Type: application/json' \ --data '{ "model": "gpt-4o-mini", "messages": [ { "role": "user", "content": "What is Cloudflare?" } ] }' ``` Responses API with Stored Keys (BYOK) / Unified Billing ```bash curl -X POST https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/openai/responses \ --header 'cf-aig-authorization: Bearer {CF_AIG_TOKEN}' \ --header 'Content-Type: application/json' \ --data '{ "model": "gpt-5.1", "input": [ { "role": "user", "content": "Write a one-sentence bedtime story about a unicorn." } ] }' ``` Chat Completions with Stored Keys (BYOK) / Unified Billing ```bash curl -X POST https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/openai/chat/completions \ --header 'cf-aig-authorization: Bearer {CF_AIG_TOKEN}' \ --header 'Content-Type: application/json' \ --data '{ "model": "gpt-4o-mini", "messages": [ { "role": "user", "content": "What is Cloudflare?" } ] }' ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/usage/providers/openai/#page","headline":"OpenAI · Cloudflare AI Gateway docs","description":"Route OpenAI API requests through AI Gateway for observability and control.","url":"https://developers.cloudflare.com/ai-gateway/usage/providers/openai/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-20","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Route OpenRouter API requests through AI Gateway for observability and control. title: OpenRouter image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # OpenRouter Last updated Apr 20, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/usage/providers/openrouter/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) [OpenRouter ↗](https://openrouter.ai/) is a platform that provides a unified interface for accessing and using large language models (LLMs). ## Endpoint ```txt https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/openrouter ``` ## URL structure When making requests to [OpenRouter ↗](https://openrouter.ai/), replace `https://openrouter.ai/api/v1/chat/completions` in the URL you are currently using with `https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/openrouter/chat/completions`. ## Prerequisites When making requests to OpenRouter, ensure you have the following: * Your AI Gateway Account ID. * Your AI Gateway gateway name. * An active OpenRouter API token or a token from the original model provider. * The name of the OpenRouter model you want to use. ## Examples ### cURL ```bash curl -X POST https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/openrouter/v1/chat/completions \ --header 'content-type: application/json' \ --header 'Authorization: Bearer OPENROUTER_TOKEN' \ --data '{ "model": "openai/gpt-5-mini", "messages": [ { "role": "user", "content": "What is Cloudflare?" } ] }' ``` ### Use OpenAI SDK with JavaScript If you are using the OpenAI SDK with JavaScript, you can set your endpoint like this: ```js import OpenAI from "openai"; const openai = new OpenAI({ apiKey: env.OPENROUTER_TOKEN, baseURL: "https://gateway.ai.cloudflare.com/v1/ACCOUNT_TAG/GATEWAY/openrouter", }); try { const chatCompletion = await openai.chat.completions.create({ model: "openai/gpt-5-mini", messages: [{ role: "user", content: "What is Cloudflare?" }], }); const response = chatCompletion.choices[0].message; return new Response(JSON.stringify(response)); } catch (e) { return new Response(e); } ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/usage/providers/openrouter/#page","headline":"OpenRouter · Cloudflare AI Gateway docs","description":"Route OpenRouter API requests through AI Gateway for observability and control.","url":"https://developers.cloudflare.com/ai-gateway/usage/providers/openrouter/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-20","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Route Parallel API requests through AI Gateway for observability and control. title: Parallel image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Parallel Last updated Apr 20, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/usage/providers/parallel/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) [Parallel ↗](https://parallel.ai/) is a web API purpose-built for AIs, providing production-ready outputs with minimal hallucination and evidence-based results. ## Endpoint ```txt https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/parallel ``` ## URL structure When making requests to Parallel, you can route to any Parallel endpoint through AI Gateway by appending the path after `parallel`. For example, to access the Tasks API at `/v1/tasks/runs`, use: ```txt https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/parallel/v1/tasks/runs ``` ## Prerequisites When making requests to Parallel, ensure you have the following: * Your AI Gateway Account ID. * Your AI Gateway gateway name. * An active Parallel API key. ## Examples ### Tasks API The [Tasks API ↗](https://docs.parallel.ai/task-api/task-quickstart) allows you to create comprehensive research and analysis tasks. #### cURL example ```bash curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/parallel/v1/tasks/runs \ --header 'x-api-key: {parallel_api_key}' \ --header 'Content-Type: application/json' \ --data '{ "input": "Create a comprehensive market research report on the HVAC industry in the USA including an analysis of recent M&A activity and other relevant details.", "processor": "ultra" }' ``` ### Search API The [Search API ↗](https://docs.parallel.ai/search-api/search-quickstart) enables advanced search with configurable parameters. #### cURL example ```bash curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/parallel/v1beta/search \ --header 'x-api-key: {parallel_api_key}' \ --header 'Content-Type: application/json' \ --data '{ "objective": "When was the United Nations established? Prefer UN'\''s websites.", "search_queries": [ "Founding year UN", "Year of founding United Nations" ], "processor": "base", "max_results": 10, "max_chars_per_result": 6000 }' ``` ## Chat API The [Chat API ↗](https://docs.parallel.ai/chat-api/chat-quickstart) is supported through AI Gateway's Unified Chat Completions API. See below for more details: ## OpenAI-Compatible Endpoint You can also access Parallel models using the OpenAI API schema through the [REST API](https://developers.cloudflare.com/ai-gateway/usage/rest-api/). Send your requests to: ```txt https://api.cloudflare.com/client/v4/accounts/{account_id}/ai/v1/chat/completions ``` Specify: ```json { "model": "parallel/{model}" } ``` #### JavaScript SDK example ```js import OpenAI from "openai"; const apiKey = "{parallel_api_key}"; const accountId = "{account_id}"; const gatewayId = "{gateway_id}"; const baseURL = `https://gateway.ai.cloudflare.com/v1/${accountId}/${gatewayId}/compat`; const client = new OpenAI({ apiKey, baseURL, }); try { const model = "parallel/speed"; const messages = [{ role: "user", content: "Hello!" }]; const chatCompletion = await client.chat.completions.create({ model, messages, }); const response = chatCompletion.choices[0].message; console.log(response); } catch (e) { console.error(e); } ``` ### FindAll API The [FindAll API ↗](https://docs.parallel.ai/findall-api/findall-quickstart) enables structured data extraction from complex queries. #### cURL example ```bash curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/parallel/v1beta/findall/ingest \ --header 'x-api-key: {parallel_api_key}' \ --header 'Content-Type: application/json' \ --data '{ "query": "Find all AI companies that recently raised money and get their website, CEO name, and CTO name." }' ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/usage/providers/parallel/#page","headline":"Parallel · Cloudflare AI Gateway docs","description":"Route Parallel API requests through AI Gateway for observability and control.","url":"https://developers.cloudflare.com/ai-gateway/usage/providers/parallel/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-20","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Route Perplexity API requests through AI Gateway for observability and control. title: Perplexity image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Perplexity Last updated Apr 20, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/usage/providers/perplexity/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) [Perplexity ↗](https://www.perplexity.ai/) is an AI powered answer engine. ## Endpoint ```txt https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/perplexity-ai ``` ## Prerequisites When making requests to Perplexity, ensure you have the following: * Your AI Gateway Account ID. * Your AI Gateway gateway name. * An active Perplexity API token. * The name of the Perplexity model you want to use. ## Examples ### cURL ```bash curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/perplexity-ai/chat/completions \ --header 'accept: application/json' \ --header 'content-type: application/json' \ --header 'Authorization: Bearer {perplexity_token}' \ --data '{ "model": "mistral-7b-instruct", "messages": [ { "role": "user", "content": "What is Cloudflare?" } ] }' ``` ### Use Perplexity through OpenAI SDK with JavaScript Perplexity does not have their own SDK, but they have compatibility with the OpenAI SDK. You can use the OpenAI SDK to make a Perplexity call through AI Gateway as follows: ```js import OpenAI from "openai"; const apiKey = env.PERPLEXITY_API_KEY; const accountId = "{account_id}"; const gatewayId = "{gateway_id}"; const baseURL = `https://gateway.ai.cloudflare.com/v1/${accountId}/${gatewayId}/perplexity-ai`; const perplexity = new OpenAI({ apiKey, baseURL, }); const model = "mistral-7b-instruct"; const messages = [{ role: "user", content: "What is Cloudflare?" }]; const maxTokens = 20; const chatCompletion = await perplexity.chat.completions.create({ model, messages, max_tokens: maxTokens, }); ``` ## OpenAI-Compatible Endpoint You can also access Perplexity models using the OpenAI API schema through the [REST API](https://developers.cloudflare.com/ai-gateway/usage/rest-api/). Send your requests to: ```txt https://api.cloudflare.com/client/v4/accounts/{account_id}/ai/v1/chat/completions ``` Specify: ```json { "model": "perplexity/{model}" } ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/usage/providers/perplexity/#page","headline":"Perplexity · Cloudflare AI Gateway docs","description":"Route Perplexity API requests through AI Gateway for observability and control.","url":"https://developers.cloudflare.com/ai-gateway/usage/providers/perplexity/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-20","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Route Replicate API requests through AI Gateway for observability and control. title: Replicate image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Replicate Last updated Apr 20, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/usage/providers/replicate/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) [Replicate ↗](https://replicate.com/) runs and fine tunes open-source models. ## Endpoint ```txt https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/replicate ``` ## URL structure When making requests to Replicate, replace `https://api.replicate.com/v1` in the URL you're currently using with `https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/replicate`. ## Prerequisites When making requests to Replicate, ensure you have the following: * Your AI Gateway Account ID. * Your AI Gateway gateway name. * An active Replicate API token. You can create one at [replicate.com/account/api-tokens ↗](https://replicate.com/account/api-tokens) * The name of the Replicate model you want to use, like `anthropic/claude-4.5-haiku` or `google/nano-banana`. ## Example ### cURL ```bash curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/replicate/predictions \ --header 'Authorization: Bearer {replicate_api_token}' \ --header 'Content-Type: application/json' \ --data '{ "version": "anthropic/claude-4.5-haiku", "input": { "prompt": "Write a haiku about Cloudflare" } }' ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/usage/providers/replicate/#page","headline":"Replicate · Cloudflare AI Gateway docs","description":"Route Replicate API requests through AI Gateway for observability and control.","url":"https://developers.cloudflare.com/ai-gateway/usage/providers/replicate/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-20","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Route Google Vertex AI requests through AI Gateway for observability and control. title: Google Vertex AI image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Google Vertex AI Last updated Apr 20, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/usage/providers/vertex/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) [Google Vertex AI ↗](https://cloud.google.com/vertex-ai) enables developers to easily build and deploy enterprise ready generative AI experiences. Below is a quick guide on how to set your Google Cloud Account: 1. Google Cloud Platform (GCP) Account * Sign up for a [GCP account ↗](https://cloud.google.com/vertex-ai). New users may be eligible for credits (valid for 90 days). 2. Enable the Vertex AI API * Go to [Enable Vertex AI API ↗](https://console.cloud.google.com/marketplace/product/google/aiplatform.googleapis.com) and activate the API for your project. 3. Apply for access to desired models. ## Endpoint ```txt https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/google-vertex-ai ``` ## Prerequisites When making requests to Google Vertex AI, you will need: * AI Gateway account tag * AI Gateway gateway name * Google Vertex AI credentials (service account JSON or access token) * Google Vertex AI Project Name * Google Vertex AI Region (for example, `us-central1`) * Google Vertex AI model ## URL structure Your new base URL will use the data above in this structure: `https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/google-vertex-ai/v1/projects/{project_name}/locations/{region}`. Then you can append the endpoint you want to hit, for example: `/publishers/google/models/{model}:{generative_ai_rest_resource}` So your final URL will come together as: `https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/google-vertex-ai/v1/projects/{project_name}/locations/{region}/publishers/google/models/gemini-2.5-flash:generateContent` Use a specific region Use a specific regional endpoint like `us-central1` or `us-east4` rather than `global`. The `global` endpoint has limited model support and may not work with all Vertex AI operations. ## Authenticating with Vertex AI Authenticating with Vertex AI normally requires generating short-term credentials using the [Google Cloud SDKs ↗](https://cloud.google.com/vertex-ai/docs/authentication) with a complicated setup, but AI Gateway simplifies this for you with multiple options. ### Authentication methods comparison | Method | cf-aig-authorization header | Authorization header | Region handling | | ---------------------------------- | --------------------------- | ----------------------------------- | ---------------------------- | | **BYOK (Recommended)** | Bearer {CF\_AIG\_TOKEN} | Not needed | Select in dashboard dropdown | | **Service account JSON in header** | Bearer {CF\_AIG\_TOKEN} | Base64-encoded JSON with region key | Include region key in JSON | | **Direct access token** | Bearer {CF\_AIG\_TOKEN} | Bearer {gcloud\_access\_token} | Included in URL path | Do not confuse the headers `cf-aig-authorization` authenticates your request to AI Gateway. `Authorization` passes credentials to the upstream provider (Google). When using BYOK, you only need `cf-aig-authorization` because AI Gateway injects the stored Google credentials for you. ### Option 1: BYOK (Recommended) The recommended approach is to store your Google service account credentials using AI Gateway's [Bring Your Own Keys (BYOK)](https://developers.cloudflare.com/ai-gateway/configuration/bring-your-own-keys/) feature. This keeps your credentials secure and out of your application code. 1. [Create a service account key ↗](https://cloud.google.com/iam/docs/keys-create-delete) in the Google Cloud Console. Ensure that the service account has the required permissions for the Vertex AI endpoints and models you plan to use. 2. In the Cloudflare dashboard, go to **AI** \> **AI Gateway** \> your gateway > **Provider Keys**. 3. Select **Add API Key** and choose **Google Vertex AI** as the provider. 4. Paste your service account JSON and select your region from the dropdown. AI Gateway automatically applies this selected region to your stored credentials, so you do not need to manually add a `region` field to the JSON. 5. Select **Save**. With BYOK configured, you only need to include the `cf-aig-authorization` header in your requests. AI Gateway handles the Vertex AI authentication automatically. ```bash curl "https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/google-vertex-ai/v1/projects/{project_name}/locations/{region}/publishers/google/models/gemini-2.5-flash:generateContent" \ -H 'cf-aig-authorization: Bearer {CF_AIG_TOKEN}' \ -H 'Content-Type: application/json' \ -d '{ "contents": [ { "role": "user", "parts": [ { "text": "Tell me more about Cloudflare" } ] } ] }' ``` ### Option 2: Service Account JSON in Header You can pass a Google service account JSON directly in the `Authorization` header on each request with a base64-encoded version of the JSON. This option is useful for testing or when you cannot use BYOK. [Create a service account key ↗](https://cloud.google.com/iam/docs/keys-create-delete) in the Google Cloud Console. Ensure that the service account has the required permissions for the Vertex AI endpoints and models you plan to use. AI Gateway uses your service account JSON to generate short-term access tokens which are cached and used for consecutive requests, and are automatically refreshed when they expire. Note When passing the service account JSON directly in the header (not using BYOK), you must include an additional key called `region` in the JSON with the GCP region code (for example, `us-central1`) you intend to use for your [Vertex AI endpoint ↗](https://cloud.google.com/vertex-ai/docs/reference/rest#service-endpoint). #### Example service account JSON structure ```json { "type": "service_account", "project_id": "your-project-id", "private_key_id": "your-private-key-id", "private_key": "-----BEGIN PRIVATE KEY-----\nYOUR_PRIVATE_KEY\n-----END PRIVATE KEY-----\n", "client_email": "your-service-account@your-project.iam.gserviceaccount.com", "client_id": "your-client-id", "auth_uri": "https://accounts.google.com/o/oauth2/auth", "token_uri": "https://oauth2.googleapis.com/token", "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs", "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/your-service-account%40your-project.iam.gserviceaccount.com", "region": "us-central1" } ``` ### Option 3: Direct Access Token If you are already using the Google Cloud SDKs and generating a short-term access token (for example, with `gcloud auth print-access-token`), you can directly pass this as a Bearer token in the `Authorization` header of the request. Note This option is only supported for the provider-specific endpoint, not for the unified chat completions endpoint. ```bash curl "https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/google-vertex-ai/v1/projects/{project_name}/locations/{region}/publishers/google/models/gemini-2.5-flash:generateContent" \ -H 'cf-aig-authorization: Bearer {CF_AIG_TOKEN}' \ -H "Authorization: Bearer ya29.c.b0Aaekm1K..." \ -H 'Content-Type: application/json' \ -d '{ "contents": [ { "role": "user", "parts": [ { "text": "Tell me more about Cloudflare" } ] } ] }' ``` ## Using Unified Chat Completions API AI Gateway provides a [Unified API](https://developers.cloudflare.com/ai-gateway/usage/chat-completion/) that works across providers. For Google Vertex AI, you can use the standard chat completions format. Note that the model field includes the provider prefix, so your model string will look like `google-vertex-ai/google/gemini-2.5-pro`. ### Endpoint ```txt https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat/chat/completions ``` ### Example with BYOK With BYOK configured, you only need to include the `cf-aig-authorization` header: ```bash curl "https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat/chat/completions" \ -H 'cf-aig-authorization: Bearer {CF_AIG_TOKEN}' \ -H 'Content-Type: application/json' \ -d '{ "model": "google-vertex-ai/google/gemini-2.5-pro", "messages": [ { "role": "user", "content": "What is Cloudflare?" } ] }' ``` ### Example with OpenAI SDK If not using BYOK, pass the base64-encoded service account JSON (with `region` key included) as the API key: ```javascript import OpenAI from "openai"; // Service account JSON must include "region" key when not using BYOK const serviceAccountJson = JSON.stringify({ type: "service_account", project_id: "your-project-id", // ... other fields from your downloaded JSON region: "us-central1", // Required: add this to your service account JSON }); const client = new OpenAI({ apiKey: Buffer.from(serviceAccountJson).toString("base64"), baseURL: "https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat", defaultHeaders: { "cf-aig-authorization": `Bearer {cf_aig_token}`, }, }); const response = await client.chat.completions.create({ model: "google-vertex-ai/google/gemini-2.5-pro", messages: [ { role: "user", content: "What is Cloudflare?", }, ], }); console.log(response.choices[0].message.content); ``` ### Example with cURL ```bash # First, base64-encode your service account JSON (must include "region" key) SERVICE_ACCOUNT_BASE64=$(base64 < service-account.json | tr -d '\n') curl "https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat/chat/completions" \ -H 'cf-aig-authorization: Bearer {CF_AIG_TOKEN}' \ -H "Authorization: Bearer $SERVICE_ACCOUNT_BASE64" \ -H 'Content-Type: application/json' \ -d '{ "model": "google-vertex-ai/google/gemini-2.5-pro", "messages": [ { "role": "user", "content": "What is Cloudflare?" } ] }' ``` Note When not using BYOK, the service account JSON must include the `region` key and be base64-encoded. Refer to [Option 2: Service Account JSON in Header](#option-2-service-account-json-in-header) for the required JSON structure. ## Using Provider-Specific Endpoint You can also use the provider-specific endpoint to access the full Vertex AI API. ### cURL with BYOK With BYOK configured, you only need the `cf-aig-authorization` header: ```bash curl "https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/google-vertex-ai/v1/projects/{project_name}/locations/{region}/publishers/google/models/gemini-2.5-flash:generateContent" \ -H 'cf-aig-authorization: Bearer {CF_AIG_TOKEN}' \ -H 'Content-Type: application/json' \ -d '{ "contents": [ { "role": "user", "parts": [ { "text": "Tell me more about Cloudflare" } ] } ] }' ``` ### cURL with Service Account JSON If not using BYOK, pass the base64-encoded service account JSON (with `region` key included) in the Authorization header: ```bash # First, base64-encode your service account JSON (must include "region" key) as a single line SERVICE_ACCOUNT_BASE64=$(base64 < service-account.json | tr -d '\n') curl "https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/google-vertex-ai/v1/projects/{project_name}/locations/{region}/publishers/google/models/gemini-2.5-flash:generateContent" \ -H 'cf-aig-authorization: Bearer {CF_AIG_TOKEN}' \ -H "Authorization: Bearer $SERVICE_ACCOUNT_BASE64" \ -H 'Content-Type: application/json' \ -d '{ "contents": [ { "role": "user", "parts": [ { "text": "Tell me more about Cloudflare" } ] } ] }' ``` ## Troubleshooting For general AI Gateway troubleshooting, refer to [Troubleshooting](https://developers.cloudflare.com/ai-gateway/reference/troubleshooting/). ### 401 Unauthenticated errors If you receive a `CREDENTIALS_MISSING` or `UNAUTHENTICATED` error from Google, check the following Vertex AI-specific issues: 1. **Check your region**: Use a specific regional endpoint (like `us-central1`) in your URL, not `global`. The `global` endpoint has limited model support. 2. **Verify BYOK configuration**: If using BYOK, confirm in the dashboard that: * Your service account JSON was saved correctly * A region was selected from the dropdown 3. **Check service account permissions**: Ensure your service account has the `Vertex AI User` role or equivalent permissions in Google Cloud. 4. **Verify the region key** (non-BYOK only): If passing service account JSON directly in the `Authorization` header, make sure the JSON includes the `region` key. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/usage/providers/vertex/#page","headline":"Google Vertex AI · Cloudflare AI Gateway docs","description":"Route Google Vertex AI requests through AI Gateway for observability and control.","url":"https://developers.cloudflare.com/ai-gateway/usage/providers/vertex/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-20","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Route Workers AI requests through AI Gateway for analytics, caching, and rate limiting. title: Workers AI image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Workers AI Last updated Aug 7, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/usage/providers/workersai/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Use AI Gateway as a unified control layer for [Workers AI](https://developers.cloudflare.com/workers-ai/) requests, with analytics, logging, caching, security, and prepaid billing. To use prepaid [AI Gateway credits](https://developers.cloudflare.com/ai-gateway/features/unified-billing/), set the gateway's [Workers AI billing setting](https://developers.cloudflare.com/ai-gateway/configuration/manage-gateway/#configure-workers-ai-billing) to **Unified billing**. Requests to frontier models billed with prepaid credits receive [higher rate limits](https://developers.cloudflare.com/workers-ai/platform/limits/#frontier-models). ## REST API Use the [REST API](https://developers.cloudflare.com/ai-gateway/usage/rest-api/) to call Workers AI models. Workers AI models use the `@cf/` prefix in the model name and require the `cf-aig-gateway-id` header to specify which gateway to route through. ```bash # Run `wrangler whoami` to get your account ID to replace $CLOUDFLARE_ACCOUNT_ID, # and `wrangler auth token` to get an auth token to replace $CLOUDFLARE_API_TOKEN. curl -X POST "https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/v1/chat/completions" \ --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ --header "cf-aig-gateway-id: default" \ --header "Content-Type: application/json" \ --data '{ "model": "@cf/moonshotai/kimi-k2.6", "messages": [ { "role": "user", "content": "What is Cloudflare?" } ] }' ``` ## Workers binding You can integrate Workers AI with AI Gateway using an environment binding. To include an AI Gateway within your Worker, add the gateway as an object in your Workers AI request. ```js export default { async fetch(request, env) { const response = await env.AI.run( "@cf/meta/llama-3.1-8b-instruct", { prompt: "Why should you use Cloudflare for your AI inference?", }, { gateway: { id: "{gateway_id}", skipCache: false, cacheTtl: 3360, }, }, ); return new Response(JSON.stringify(response)); }, }; ``` ```ts export interface Env { AI: Ai; } export default { async fetch(request: Request, env: Env): Promise { const response = await env.AI.run( "@cf/meta/llama-3.1-8b-instruct", { prompt: "Why should you use Cloudflare for your AI inference?", }, { gateway: { id: "{gateway_id}", skipCache: false, cacheTtl: 3360, }, }, ); return new Response(JSON.stringify(response)); }, } satisfies ExportedHandler; ``` For a detailed step-by-step guide on integrating Workers AI with AI Gateway using a binding, refer to [Integrations in AI Gateway](https://developers.cloudflare.com/ai-gateway/integrations/aig-workers-ai-binding/). Workers AI supports the following parameters for AI gateways: * `id` string * Name of your existing [AI Gateway](https://developers.cloudflare.com/ai-gateway/get-started/). Must be in the same account as your Worker. * `skipCache` boolean(default: false) * Controls whether the request should [skip the cache](https://developers.cloudflare.com/ai-gateway/features/caching/#skip-cache-cf-aig-skip-cache). * `cacheTtl` number * Controls the [Cache TTL](https://developers.cloudflare.com/ai-gateway/features/caching/#cache-ttl-cf-aig-cache-ttl). Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/usage/providers/workersai/#page","headline":"Workers AI · Cloudflare AI Gateway docs","description":"Route Workers AI requests through AI Gateway for analytics, caching, and rate limiting.","url":"https://developers.cloudflare.com/ai-gateway/usage/providers/workersai/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-08-07","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"},"keywords":["AI"]} ``` --- --- description: Call third-party and Workers AI models through the Cloudflare API with AI Gateway features like logging, caching, and rate limiting. title: REST API image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # REST API Last updated Aug 12, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/usage/rest-api/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) The REST API lets you call any model — whether hosted on Cloudflare or by a third-party provider like OpenAI, Anthropic, or Google — through the same Cloudflare API, with all AI Gateway features — logging, caching, rate limiting, and more — applied automatically. No provider SDKs or API keys are needed. Authentication and billing are handled through your Cloudflare account. Third-party models are billed via [Unified Billing](https://developers.cloudflare.com/ai-gateway/features/unified-billing/). Workers AI models can use prepaid AI Gateway credits or [Workers AI billing](https://developers.cloudflare.com/workers-ai/platform/pricing/). ## Endpoints Four endpoints are available, each suited to different use cases: | Endpoint | Format | Use case | Third-Party Models | Workers AI Models (@cf/) | | ---------------------------- | -------------------------- | ------------------------------------------------ | ------------------ | ------------------------ | | POST /ai/run | Envelope with model, input | All models and modalities (LLM, image, TTS, ASR) | ✅ Yes | ✅ Yes | | POST /ai/v1/chat/completions | OpenAI chat completions | LLMs — OpenAI SDK compatible | ✅ Yes | ✅ Yes | | POST /ai/v1/responses | OpenAI Responses API | Agentic workflows — OpenAI SDK compatible | ✅ Yes | ✅ Model dependent | | POST /ai/v1/messages | Anthropic Messages API | LLMs — Anthropic SDK compatible | ✅ Yes | ❌ No | Note The `/ai/v1/messages` endpoint strictly uses Anthropic's API schema and supports routing to Anthropic and other third-party models. Workers AI models (`@cf/`) do not support this schema. Use `/ai/run` or `/ai/v1/chat/completions` for Workers AI models, or `/ai/v1/responses` only for Workers AI models that support the Responses API, such as GPT-OSS. ## Authentication Authenticate with a [Cloudflare API token](https://developers.cloudflare.com/fundamentals/api/get-started/create-token/) that has the **Account** \> **Workers AI** \> **Read** permission. Pass it in the `Authorization` header. All `/accounts/{account_id}/ai/*` endpoints require the Workers AI permission. This applies to third-party models and to Workers AI (`@cf/`) models. A token that holds only an `AI Gateway` permission returns `401` with error code `10000`. The `AI Gateway` permissions apply to the `/accounts/{account_id}/ai-gateway/*` endpoints, which manage gateway configuration, logs, and routes. Note Ensure your Cloudflare account has [sufficient credits loaded](https://developers.cloudflare.com/ai-gateway/features/unified-billing/#load-credits) before calling third-party models or using prepaid credits for Workers AI. ## Model naming Third-party models use the `author/model` format: * `openai/gpt-4.1` — OpenAI * `anthropic/claude-sonnet-4` — Anthropic * `google/gemini-3-flash` — Google * `xai/grok-3` — xAI Workers AI models use the `@cf/author/model` format (for example, `@cf/moonshotai/kimi-k2.6`). Workers AI requests also require the `cf-aig-gateway-id` header — refer to [Call a Workers AI model](#call-a-workers-ai-model) for details. Browse available models in the [model catalog](https://developers.cloudflare.com/ai/models/). ## `/ai/run` — universal endpoint Accepts any model with its per-model schema. Model-specific parameters go inside `input`. ```bash # Run `wrangler whoami` to get your account ID to replace $CLOUDFLARE_ACCOUNT_ID, # and `wrangler auth token` to get an auth token to replace $CLOUDFLARE_API_TOKEN. curl -X POST "https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run" \ --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ --header "Content-Type: application/json" \ --data '{ "model": "openai/gpt-4.1", "input": { "messages": [ { "role": "user", "content": "What is Cloudflare?" } ], "max_tokens": 512 } }' ``` ### Call a Workers AI model To call a Workers AI model, use the `@cf/` prefix in the model name and include the `cf-aig-gateway-id` header to specify which gateway to route through. ```bash # Run `wrangler whoami` to get your account ID to replace $CLOUDFLARE_ACCOUNT_ID, # and `wrangler auth token` to get an auth token to replace $CLOUDFLARE_API_TOKEN. curl -X POST "https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run" \ --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ --header "cf-aig-gateway-id: default" \ --header "Content-Type: application/json" \ --data '{ "model": "@cf/moonshotai/kimi-k2.6", "input": { "messages": [ { "role": "user", "content": "What is Cloudflare?" } ] } }' ``` The existing Workers AI endpoint with the model ID in the URL path also continues to work: ```bash # Run `wrangler whoami` to get your account ID to replace $CLOUDFLARE_ACCOUNT_ID, # and `wrangler auth token` to get an auth token to replace $CLOUDFLARE_API_TOKEN. curl -X POST "https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/moonshotai/kimi-k2.6" \ --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ --header "cf-aig-gateway-id: default" \ --header "Content-Type: application/json" \ --data '{ "messages": [ { "role": "user", "content": "What is Cloudflare?" } ] }' ``` To use prepaid AI Gateway credits for Workers AI, use the model-in-path endpoint shown above, set the gateway's [Workers AI billing setting](https://developers.cloudflare.com/ai-gateway/configuration/manage-gateway/#configure-workers-ai-billing) to **Unified billing**, and include its ID in the `cf-aig-gateway-id` header. Requests to frontier models billed with prepaid credits receive [higher rate limits](https://developers.cloudflare.com/workers-ai/platform/limits/#frontier-models). ### Background requests and webhooks By default, `/ai/run` requests are synchronous — the connection stays open until the model finishes and the result comes back in the response. For long-running models — such as image, video, or audio generation — or when you do not want to hold a connection open, run the request in the background and have AI Gateway notify a webhook when it completes. Set `background` to `true` and provide a `webhookUrl`. Both are fields of the `options` object in the `/ai/run` body, alongside `model` and `input`. `webhookUrl` can only be provided when `background` is `true`. Providing a `webhookUrl` without `background: true` returns a `400` error. ```bash # Run `wrangler whoami` to get your account ID to replace $CLOUDFLARE_ACCOUNT_ID, # and `wrangler auth token` to get an auth token to replace $CLOUDFLARE_API_TOKEN. curl -X POST "https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run" \ --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ --header "Content-Type: application/json" \ --data '{ "model": "google/nano-banana", "input": { "prompt": "A cozy coffee shop interior with warm lighting, plants hanging from the ceiling, and a cat sleeping on a velvet armchair by the window", "aspect_ratio": "16:9" }, "options": { "background": true, "webhookUrl": "https://example.com/my-webhook" } }' ``` A background request returns immediately while the model runs. The result is delivered to your webhook when the run completes. #### Webhook payload When the run completes, AI Gateway sends a single `POST` request to your `webhookUrl` with the run outcome: ```json { "id": "", "state": "", "result": {}, "error": null, "provider": "google", "model": "google/nano-banana", "usage": {} } ``` Webhook delivery is best-effort and is not retried. The destination must be an HTTPS URL that does not resolve to a private network address. #### Webhook format Use the optional `webhookFormat` field in the `options` object to control the shape of the webhook body. The default is `raw`. `webhookFormat` can only be provided when `webhookUrl` is present. Otherwise, the request returns a `400` error. | Format | Description | | ------ | --------------------------------------------------------------------------------------------------------------------------- | | raw | Sends the payload as-is (default). | | chat | Wraps the payload in { "text": "" }, matching the incoming-webhook body accepted by Google Chat and Slack. | ## `/ai/v1/chat/completions` — OpenAI compatible Uses the standard OpenAI chat completions format. The `model` field uses the same `author/model` naming. This endpoint is compatible with the OpenAI SDK and other OpenAI-compatible clients. ```bash # Run `wrangler whoami` to get your account ID to replace $CLOUDFLARE_ACCOUNT_ID, # and `wrangler auth token` to get an auth token to replace $CLOUDFLARE_API_TOKEN. curl -X POST "https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/v1/chat/completions" \ --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ --header "Content-Type: application/json" \ --data '{ "model": "openai/gpt-4.1", "messages": [ { "role": "system", "content": "You are a helpful assistant." }, { "role": "user", "content": "What is Cloudflare?" } ], "max_tokens": 512, "temperature": 0.7, "stream": true }' ``` ### OpenAI SDK Point the OpenAI SDK `baseURL` at the Cloudflare API: ```javascript import OpenAI from "openai"; const openai = new OpenAI({ apiKey: CLOUDFLARE_API_TOKEN, baseURL: `https://api.cloudflare.com/client/v4/accounts/${ACCOUNT_ID}/ai/v1`, }); const response = await openai.chat.completions.create({ model: "openai/gpt-4.1", messages: [{ role: "user", content: "What is Cloudflare?" }], }); ``` ## `/ai/v1/responses` — OpenAI Responses API Uses the OpenAI Responses API format for agentic workflows. Compatible with the OpenAI SDK. ```javascript import OpenAI from "openai"; const openai = new OpenAI({ apiKey: CLOUDFLARE_API_TOKEN, baseURL: `https://api.cloudflare.com/client/v4/accounts/${ACCOUNT_ID}/ai/v1`, }); const response = await openai.responses.create({ model: "openai/gpt-4.1", input: "What is Cloudflare?", }); ``` ## `/ai/v1/messages` — Anthropic compatible Uses the Anthropic Messages API format. Compatible with the Anthropic SDK. ```bash # Run `wrangler whoami` to get your account ID to replace $CLOUDFLARE_ACCOUNT_ID, # and `wrangler auth token` to get an auth token to replace $CLOUDFLARE_API_TOKEN. curl -X POST "https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/v1/messages" \ --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ --header "Content-Type: application/json" \ --data '{ "model": "anthropic/claude-sonnet-4-5", "max_tokens": 512, "messages": [ { "role": "user", "content": "What is Cloudflare?" } ] }' ``` Point the Anthropic SDK `baseURL` at the Cloudflare API: ```javascript import Anthropic from "@anthropic-ai/sdk"; const anthropic = new Anthropic({ apiKey: CLOUDFLARE_API_TOKEN, baseURL: `https://api.cloudflare.com/client/v4/accounts/${ACCOUNT_ID}/ai/v1`, }); const message = await anthropic.messages.create({ model: "anthropic/claude-sonnet-4-5", max_tokens: 512, messages: [{ role: "user", content: "What is Cloudflare?" }], }); ``` ## Provider tools and web search Some providers expose native tools — including server-side web search — through these endpoints. Refer to [Web Search](https://developers.cloudflare.com/ai-gateway/usage/web-search/) for the supported models per provider and the request shape each one uses. Browse the [model catalog](https://developers.cloudflare.com/ai/models/) for canonical model IDs. ## Specify a gateway By default, third-party model requests route through your account's default AI Gateway. To use a specific gateway, include the `cf-aig-gateway-id` header. Workers AI requests always require this header. ```bash # Run `wrangler whoami` to get your account ID to replace $CLOUDFLARE_ACCOUNT_ID, # and `wrangler auth token` to get an auth token to replace $CLOUDFLARE_API_TOKEN. curl -X POST "https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/v1/chat/completions" \ --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ --header "cf-aig-gateway-id: default" \ --header "Content-Type: application/json" \ --data '{ "model": "anthropic/claude-sonnet-4", "messages": [ { "role": "user", "content": "Hello" } ] }' ``` With the OpenAI SDK, set the header via `defaultHeaders`: ```javascript const openai = new OpenAI({ apiKey: CLOUDFLARE_API_TOKEN, baseURL: `https://api.cloudflare.com/client/v4/accounts/${ACCOUNT_ID}/ai/v1`, defaultHeaders: { "cf-aig-gateway-id": "default", }, }); ``` All AI Gateway features configured on that gateway — caching, rate limiting, guardrails, and logging — apply to the request. ## Per-request configuration Use `cf-aig-*` headers to control AI Gateway behavior on a per-request basis: | Header | Type | Description | | ---------------------- | ----------- | ------------------------------------------------- | | cf-aig-skip-cache | boolean | Skip the cache for this request. | | cf-aig-cache-ttl | number | Cache TTL in seconds. | | cf-aig-cache-key | string | Custom cache key. | | cf-aig-collect-log | boolean | Turn logging on or off for this request. | | cf-aig-request-timeout | number | Request timeout in milliseconds. | | cf-aig-max-attempts | number | Retry attempts (max 5). | | cf-aig-retry-delay | number | Retry delay in milliseconds (max 5000). | | cf-aig-backoff | string | Backoff method: constant, linear, or exponential. | | cf-aig-metadata | JSON string | Custom metadata to attach to the log entry. | For more details on these options, refer to [Request handling](https://developers.cloudflare.com/ai-gateway/configuration/request-handling/) and [Caching](https://developers.cloudflare.com/ai-gateway/features/caching/). ## Related resources * [Unified Billing](https://developers.cloudflare.com/ai-gateway/features/unified-billing/) — load credits and pay for inference requests with a single Cloudflare bill. * [Workers AI binding](https://developers.cloudflare.com/ai-gateway/usage/worker-binding-methods/) — call models from within a Cloudflare Worker using `env.AI.run()`. * [Model catalog](https://developers.cloudflare.com/ai/models/) — browse models supported by the REST API. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/usage/rest-api/#page","headline":"REST API · Cloudflare AI Gateway docs","description":"Call third-party and Workers AI models through the Cloudflare API with AI Gateway features like logging, caching, and rate limiting.","url":"https://developers.cloudflare.com/ai-gateway/usage/rest-api/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-08-12","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"},"keywords":["AI"]} ``` --- --- description: Route requests to any AI provider through a single AI Gateway endpoint with support for fallbacks and retries. title: Universal Endpoint (Deprecated) image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Universal Endpoint (Deprecated) Last updated May 8, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/usage/universal/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Deprecated The Universal Endpoint is deprecated. Use the [OpenAI-compatible endpoint](https://developers.cloudflare.com/ai-gateway/usage/chat-completion/) for new integrations, and [Dynamic Routing](https://developers.cloudflare.com/ai-gateway/features/dynamic-routing/) for fallbacks, retries, and conditional routing. The Universal Endpoint will continue to work for existing integrations. The Universal Endpoint allows you to contact every provider through a single endpoint. ```txt https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id} ``` The payload expects an array of messages. Each message is an object with the following parameters: * `provider`: the name of the provider you would like to direct this message to. Can be OpenAI, workers-ai, or any of our supported providers. * `endpoint`: the pathname of the provider API you are trying to reach. For example, on OpenAI it can be `chat/completions`, and for Workers AI this might be [@cf/meta/llama-3.1-8b-instruct](https://developers.cloudflare.com/workers-ai/models/llama-3.1-8b-instruct/). Refer to the sections that are specific to [each provider](https://developers.cloudflare.com/ai-gateway/usage/providers/). * `authorization`: the content of the Authorization HTTP Header that should be used when contacting this provider. This usually starts with `Token` or `Bearer`. * `query`: the payload as the provider expects it in their official API. ## cURL example ```bash curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id} \ --header 'Content-Type: application/json' \ --data '[ { "provider": "workers-ai", "endpoint": "@cf/meta/llama-3.1-8b-instruct", "headers": { "Authorization": "Bearer {cloudflare_token}", "Content-Type": "application/json" }, "query": { "messages": [ { "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "What is Cloudflare?" } ] } }, { "provider": "openai", "endpoint": "chat/completions", "headers": { "Authorization": "Bearer {open_ai_token}", "Content-Type": "application/json" }, "query": { "model": "gpt-4o-mini", "stream": true, "messages": [ { "role": "user", "content": "What is Cloudflare?" } ] } } ]' ``` The above will send a request to Workers AI Inference API. If it fails, it will proceed to OpenAI. You can add as many fallbacks as you need by adding another object in the array. ## Fallbacks You can specify model or provider fallbacks to handle request failures and ensure reliability. The payload array defines the fallback sequence — if the first provider fails, the request falls to the next entry in the array. For more details, refer to [Fallbacks](https://developers.cloudflare.com/ai-gateway/configuration/fallbacks/). By default, Cloudflare triggers your fallback if a model request returns an error. You can also configure [request timeouts](#request-timeouts) to trigger fallbacks when a provider takes too long to respond. ### Response header (`cf-aig-step`) When using fallbacks, the response header `cf-aig-step` indicates which model successfully processed the request by returning the step number: * `cf-aig-step:0` — The first (primary) model was used successfully. * `cf-aig-step:1` — The request fell back to the second model. * `cf-aig-step:2` — The request fell back to the third model. * Subsequent steps — Each fallback increments the step number by 1. ## Request timeouts A request timeout triggers a fallback if a provider takes too long to respond. Configure the timeout by setting a `requestTimeout` property (in milliseconds) within the provider-specific `config` object. Each provider can have a different `requestTimeout` value. The timeout is based on when the first part of the response comes back. As long as the first part of the response returns within the specified timeframe — such as when streaming a response — your gateway will wait for the response. ```bash curl 'https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}' \ --header 'Content-Type: application/json' \ --data '[ { "provider": "workers-ai", "endpoint": "@cf/meta/llama-3.1-8b-instruct", "headers": { "Authorization": "Bearer {cloudflare_token}", "Content-Type": "application/json" }, "config": { "requestTimeout": 1000 }, "query": { "messages": [ { "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "What is Cloudflare?" } ] } }, { "provider": "workers-ai", "endpoint": "@cf/meta/llama-3.1-8b-instruct-fast", "headers": { "Authorization": "Bearer {cloudflare_token}", "Content-Type": "application/json" }, "query": { "messages": [ { "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "What is Cloudflare?" } ] }, "config": { "requestTimeout": 3000 }, } ]' ``` ## Request retries The Universal Endpoint supports automatic retries for failed requests, with a maximum of five retry attempts. Retries are attempted before triggering any configured fallbacks. Configure the retry settings with the following properties in the provider-specific `config`: ```ts config:{ maxAttempts?: number; retryDelay?: number; backoff?: "constant" | "linear" | "exponential"; } ``` * `maxAttempts`: Maximum number of retry attempts (up to 5). * `retryDelay`: Delay before retrying, in milliseconds (maximum of 5 seconds). * `backoff`: Backoff method — `constant`, `linear`, or `exponential`. On the final retry attempt, your gateway will wait until the request completes, regardless of how long it takes. Each provider can have different retry settings. ```bash curl 'https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}' \ --header 'Content-Type: application/json' \ --data '[ { "provider": "workers-ai", "endpoint": "@cf/meta/llama-3.1-8b-instruct", "headers": { "Authorization": "Bearer {cloudflare_token}", "Content-Type": "application/json" }, "config": { "maxAttempts": 2, "retryDelay": 1000, "backoff": "constant" }, "query": { "messages": [ { "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "What is Cloudflare?" } ] } }, { "provider": "workers-ai", "endpoint": "@cf/meta/llama-3.1-8b-instruct-fast", "headers": { "Authorization": "Bearer {cloudflare_token}", "Content-Type": "application/json" }, "query": { "messages": [ { "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "What is Cloudflare?" } ] }, "config": { "maxAttempts": 4, "retryDelay": 1000, "backoff": "exponential" }, } ]' ``` ## WebSockets API beta The Universal Endpoint can also be accessed via a [WebSockets API](https://developers.cloudflare.com/ai-gateway/usage/websockets-api/) which provides a single persistent connection, enabling continuous communication. This API supports all AI providers connected to AI Gateway, including those that do not natively support WebSockets. ### WebSockets example ```javascript import WebSocket from "ws"; const ws = new WebSocket( "wss://gateway.ai.cloudflare.com/v1/my-account-id/my-gateway/", { headers: { "cf-aig-authorization": "Bearer AI_GATEWAY_TOKEN", }, }, ); ws.send( JSON.stringify({ type: "universal.create", request: { eventId: "my-request", provider: "workers-ai", endpoint: "@cf/meta/llama-3.1-8b-instruct", headers: { Authorization: "Bearer WORKERS_AI_TOKEN", "Content-Type": "application/json", }, query: { prompt: "tell me a joke", }, }, }), ); ws.on("message", function incoming(message) { console.log(message.toString()); }); ``` ## Workers Binding example ```jsonc { "ai": { "binding": "AI", }, } ``` ```toml [ai] binding = "AI" ``` ```typescript type Env = { AI: Ai; }; export default { async fetch(request: Request, env: Env) { return env.AI.gateway("my-gateway").run({ provider: "workers-ai", endpoint: "@cf/meta/llama-3.1-8b-instruct", headers: { authorization: "Bearer my-api-token", }, query: { prompt: "tell me a joke", }, }); }, }; ``` ## Header configuration hierarchy The Universal Endpoint allows you to set fallback models or providers and customize headers for each provider or request. You can configure headers at three levels: 1. **Provider level**: Headers specific to a particular provider. 2. **Request level**: Headers included in individual requests. 3. **Gateway settings**: Default headers configured in your gateway dashboard. Since the same settings can be configured in multiple locations, AI Gateway applies a hierarchy to determine which configuration takes precedence: * **Provider-level headers** override all other configurations. * **Request-level headers** are used if no provider-level headers are set. * **Gateway-level settings** are used only if no headers are configured at the provider or request levels. This hierarchy ensures consistent behavior, prioritizing the most specific configurations. Use provider-level and request-level headers for fine-tuned control, and gateway settings for general defaults. ### Hierarchy example This example demonstrates how headers set at different levels impact caching behavior: * **Request-level header**: The `cf-aig-cache-ttl` is set to `3600` seconds, applying this caching duration to the request by default. * **Provider-level header**: For the fallback provider (OpenAI), `cf-aig-cache-ttl` is explicitly set to `0` seconds, overriding the request-level header and disabling caching for responses when OpenAI is used as the provider. This shows how provider-level headers take precedence over request-level headers, allowing for granular control of caching behavior. ```bash curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id} \ --header 'Content-Type: application/json' \ --header 'cf-aig-cache-ttl: 3600' \ --data '[ { "provider": "workers-ai", "endpoint": "@cf/meta/llama-3.1-8b-instruct", "headers": { "Authorization": "Bearer {cloudflare_token}", "Content-Type": "application/json" }, "query": { "messages": [ { "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "What is Cloudflare?" } ] } }, { "provider": "openai", "endpoint": "chat/completions", "headers": { "Authorization": "Bearer {open_ai_token}", "Content-Type": "application/json", "cf-aig-cache-ttl": "0" }, "query": { "model": "gpt-4o-mini", "stream": true, "messages": [ { "role": "user", "content": "What is Cloudflare?" } ] } } ]' ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/usage/universal/#page","headline":"Universal Endpoint (Deprecated) · Cloudflare AI Gateway docs","description":"Route requests to any AI provider through a single AI Gateway endpoint with support for fallbacks and retries.","url":"https://developers.cloudflare.com/ai-gateway/usage/universal/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-05-08","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Use provider-native web search tools through AI Gateway, or reach search-first providers like Perplexity and Parallel through their proxy endpoints. title: Web Search image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Web Search Last updated Jun 26, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/usage/web-search/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) AI Gateway proxies native web search tools from supported providers so models can answer questions about events after their training cutoff. Search runs on the upstream provider; AI Gateway applies its standard features — logging, caching, rate limiting, and guardrails — to the request. How you enable web search depends on the provider. Activation is either a tool entry on a `tools` array or a top-level flag on the request body. The table below points you to the right section. ## Supported providers | Provider | Endpoint | Activation | | --------- | ---------------------------- | ------------------------------------------------------------------------------------- | | Anthropic | POST /ai/v1/messages | tools: \[{ "type": "web\_search\_20250305", "name": "web\_search", "max\_uses": N }\] | | OpenAI | POST /ai/v1/responses | tools: \[{ "type": "web\_search\_preview" }\] | | xAI | POST /ai/v1/responses | tools: \[{ "type": "web\_search" }\] | | Alibaba | POST /ai/v1/chat/completions | top-level "enable\_search": true | For providers whose product is search itself — Perplexity and Parallel — refer to [Search-first providers](#search-first-providers). ## Anthropic web search Anthropic models expose web search through their native [web\_search\_20250305 tool ↗](https://platform.claude.com/docs/en/agents-and-tools/tool-use/web-search-tool). Add it to the `tools` array on a `POST /ai/v1/messages` request. Supported models — `anthropic/claude-haiku-4.5`, `anthropic/claude-opus-4.5`, `anthropic/claude-opus-4.6`, `anthropic/claude-opus-4.7`, `anthropic/claude-opus-4.8`, `anthropic/claude-sonnet-4.5`, `anthropic/claude-sonnet-4.6`. ```bash # Run `wrangler whoami` to get your account ID to replace $CLOUDFLARE_ACCOUNT_ID, # and `wrangler auth token` to get an auth token to replace $CLOUDFLARE_API_TOKEN. curl -X POST "https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/v1/messages" \ --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ --header "Content-Type: application/json" \ --data '{ "model": "anthropic/claude-haiku-4.5", "max_tokens": 4096, "messages": [ { "role": "user", "content": "What were the top news stories about Cloudflare this week? Summarize in three bullets." } ], "tools": [ { "type": "web_search_20250305", "name": "web_search", "max_uses": 3 } ] }' ``` Equivalent call from a Worker using the AI binding: ```js const resp = await env.AI.run( "anthropic/claude-haiku-4.5", { max_tokens: 4096, messages: [ { role: "user", content: "What were the top news stories about Cloudflare this week? Summarize in three bullets.", }, ], tools: [{ type: "web_search_20250305", name: "web_search", max_uses: 3 }], }, { gateway: { id: "default", // or use a specific gateway name }, }, ); ``` ```ts const resp = await env.AI.run( "anthropic/claude-haiku-4.5", { max_tokens: 4096, messages: [ { role: "user", content: "What were the top news stories about Cloudflare this week? Summarize in three bullets.", }, ], tools: [{ type: "web_search_20250305", name: "web_search", max_uses: 3 }], }, { gateway: { id: "default", // or use a specific gateway name }, }, ); ``` Search invocations and results appear in the response as `server_tool_use` and `web_search_tool_result` content blocks. Configurable parameters include `max_uses`, `allowed_domains`, `blocked_domains`, and `user_location` — refer to Anthropic's [web search tool documentation ↗](https://platform.claude.com/docs/en/agents-and-tools/tool-use/web-search-tool) for the full list. ## OpenAI web search OpenAI models expose web search through the [web\_search\_preview tool ↗](https://developers.openai.com/api/docs/guides/tools-web-search) on the Responses API. Use the `POST /ai/v1/responses` endpoint and add the tool to the `tools` array. Supported models — `openai/gpt-4.1`, `openai/gpt-4.1-mini`, `openai/gpt-4o`, `openai/gpt-4o-mini`, `openai/gpt-5`, `openai/gpt-5-mini`, `openai/gpt-5-nano`, `openai/gpt-5.1`, `openai/gpt-5.4`, `openai/gpt-5.4-mini`, `openai/gpt-5.4-nano`, `openai/gpt-5.4-pro`, `openai/gpt-5.5`, `openai/gpt-5.5-pro`, `openai/o3`, `openai/o4-mini`. ```bash # Run `wrangler whoami` to get your account ID to replace $CLOUDFLARE_ACCOUNT_ID, # and `wrangler auth token` to get an auth token to replace $CLOUDFLARE_API_TOKEN. curl -X POST "https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/v1/responses" \ --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ --header "Content-Type: application/json" \ --data '{ "model": "openai/gpt-4o-mini", "input": "What were the top news stories about Cloudflare this week? Summarize in three bullets.", "max_output_tokens": 4096, "tools": [ { "type": "web_search_preview" } ] }' ``` Equivalent call from a Worker using the AI binding: ```js const resp = await env.AI.run( "openai/gpt-4o-mini", { input: "What were the top news stories about Cloudflare this week? Summarize in three bullets.", max_output_tokens: 4096, tools: [{ type: "web_search_preview" }], }, { gateway: { id: "default", // or use a specific gateway name }, }, ); ``` ```ts const resp = await env.AI.run( "openai/gpt-4o-mini", { input: "What were the top news stories about Cloudflare this week? Summarize in three bullets.", max_output_tokens: 4096, tools: [{ type: "web_search_preview" }], }, { gateway: { id: "default", // or use a specific gateway name }, }, ); ``` OpenAI web search is available only on the Responses API endpoint (`POST /ai/v1/responses`). The `/ai/v1/chat/completions` endpoint does not accept the `web_search_preview` tool. Both `{ "type": "web_search_preview" }` and `{ "type": "web_search" }` are accepted on the Responses API. The examples here use `web_search_preview`. ## xAI web search xAI's multi-agent Grok model exposes web search through the [web\_search tool ↗](https://docs.x.ai/developers/tools/web-search) on the Responses API. Add `{ "type": "web_search" }` to the `tools` array on a `POST /ai/v1/responses` request. Supported models — `xai/grok-4.20-multi-agent-0309`. ```bash # Run `wrangler whoami` to get your account ID to replace $CLOUDFLARE_ACCOUNT_ID, # and `wrangler auth token` to get an auth token to replace $CLOUDFLARE_API_TOKEN. curl -X POST "https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/v1/responses" \ --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ --header "Content-Type: application/json" \ --data '{ "model": "xai/grok-4.20-multi-agent-0309", "input": "What were the top news stories about Cloudflare this week? Summarize in three bullets.", "max_turns": 4, "tools": [ { "type": "web_search" } ] }' ``` Equivalent call from a Worker using the AI binding: ```js const resp = await env.AI.run( "xai/grok-4.20-multi-agent-0309", { input: "What were the top news stories about Cloudflare this week? Summarize in three bullets.", max_turns: 4, tools: [{ type: "web_search" }], }, { gateway: { id: "default", // or use a specific gateway name }, }, ); ``` ```ts const resp = await env.AI.run( "xai/grok-4.20-multi-agent-0309", { input: "What were the top news stories about Cloudflare this week? Summarize in three bullets.", max_turns: 4, tools: [{ type: "web_search" }], }, { gateway: { id: "default", // or use a specific gateway name }, }, ); ``` `xai/grok-4.20-multi-agent-0309` is the only xAI model that accepts web search through AI Gateway. For other Grok models, refer to [Models without web search support](#models-without-web-search-support). ## Alibaba (Qwen) web search Alibaba DashScope Qwen models enable web search through a top-level [enable\_search ↗](https://www.alibabacloud.com/help/en/model-studio/qwen-search) flag on a chat completions request. Unlike Anthropic, OpenAI, and xAI, there is no `tools` entry — web search is activated by the flag alone. Supported models — `alibaba/qwen3-max`, `alibaba/qwen3.5-397b-a17b`. ```bash # Run `wrangler whoami` to get your account ID to replace $CLOUDFLARE_ACCOUNT_ID, # and `wrangler auth token` to get an auth token to replace $CLOUDFLARE_API_TOKEN. curl -X POST "https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/v1/chat/completions" \ --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ --header "Content-Type: application/json" \ --data '{ "model": "alibaba/qwen3-max", "enable_search": true, "max_tokens": 4096, "messages": [ { "role": "user", "content": "What were the top news stories about Cloudflare this week? Summarize in three bullets." } ] }' ``` Equivalent call from a Worker using the AI binding: ```js const resp = await env.AI.run( "alibaba/qwen3-max", { enable_search: true, max_tokens: 4096, messages: [ { role: "user", content: "What were the top news stories about Cloudflare this week? Summarize in three bullets.", }, ], }, { gateway: { id: "default", // or use a specific gateway name }, }, ); ``` ```ts const resp = await env.AI.run( "alibaba/qwen3-max", { enable_search: true, max_tokens: 4096, messages: [ { role: "user", content: "What were the top news stories about Cloudflare this week? Summarize in three bullets.", }, ], }, { gateway: { id: "default", // or use a specific gateway name }, }, ); ``` DashScope does not return search-grounded context as separate tool-call response blocks. It folds the fetched context into the prompt as additional input tokens — expect `prompt_tokens` to increase substantially on a successful search-grounded response. ## Search-first providers For some providers, the primary API is a search endpoint rather than a chat endpoint with a web search tool. AI Gateway exposes them through their existing provider proxy endpoints at `gateway.ai.cloudflare.com`. AI Gateway does not provide a provider-agnostic web search abstraction. Call the provider proxy directly using the patterns below. ### Perplexity Call any [Perplexity Sonar model ↗](https://docs.perplexity.ai/docs/sonar/models) through the [Perplexity provider proxy](https://developers.cloudflare.com/ai-gateway/usage/providers/perplexity/). ```bash curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/perplexity-ai/chat/completions \ --header "Authorization: Bearer $PERPLEXITY_API_TOKEN" \ --header "Content-Type: application/json" \ --data '{ "model": "sonar", "messages": [ { "role": "user", "content": "What were the top news stories about Cloudflare this week?" } ] }' ``` ### Parallel Call Parallel's Search API through the [Parallel provider proxy](https://developers.cloudflare.com/ai-gateway/usage/providers/parallel/). Refer to Parallel's [Search API documentation ↗](https://docs.parallel.ai/search/search-quickstart) for the full request schema. ```bash curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/parallel/v1beta/search \ --header "x-api-key: $PARALLEL_API_TOKEN" \ --header "Content-Type: application/json" \ --data '{ "objective": "Top news stories about Cloudflare this week.", "processor": "base", "max_results": 10 }' ``` ## Models without web search support The following models do not accept web search through AI Gateway: * **Google Gemini** — not available through the unified `web_search` tool, because Vertex's OpenAI-compatible surface does not translate it into Gemini's native `googleSearch` tool. To use Gemini grounding, pass the native `google_search` tool to the [provider-specific Vertex endpoint](https://developers.cloudflare.com/ai-gateway/usage/providers/vertex/#using-provider-specific-endpoint). * **Grok chat-completions models** — `xai/grok-4.20-0309-non-reasoning`, `xai/grok-4.20-0309-reasoning`, and `xai/grok-4.3` use the chat-completions endpoint, which does not accept the `web_search` tool. For Grok web search, refer to [xAI web search](#xai-web-search). * **DeepSeek `deepseek-v4-flash`, `deepseek-v4-pro`** — these models accept function tools only. * **MiniMax `m2.7`, `m3`** — these models accept `{ "type": "function" }` tools only. * **OpenAI `gpt-4.1-nano`, `o1-pro`, `o3-mini`** — the upstream returns `invalid_request_error` for `web_search_preview` on these models. * **OpenAI `gpt-4o-search-preview`, `gpt-4o-mini-search-preview`** — these preview models are deprecated upstream. ## Pricing and logging Web search requests are billed at the upstream provider's web-search rates and flow through [Unified Billing](https://developers.cloudflare.com/ai-gateway/features/unified-billing/) along with the rest of the model call. AI Gateway does not charge a separate web-search fee. Web search tool calls and their results are visible in AI Gateway [logs](https://developers.cloudflare.com/ai-gateway/observability/logging/) alongside the rest of the request and response. ## Related resources * [REST API](https://developers.cloudflare.com/ai-gateway/usage/rest-api/) — the four endpoints these examples target * [Workers Bindings](https://developers.cloudflare.com/ai-gateway/usage/worker-binding-methods/) — `env.AI.run` reference * [Anthropic provider](https://developers.cloudflare.com/ai-gateway/usage/providers/anthropic/) * [OpenAI provider](https://developers.cloudflare.com/ai-gateway/usage/providers/openai/) * [Grok (xAI) provider](https://developers.cloudflare.com/ai-gateway/usage/providers/grok/) * [Perplexity provider](https://developers.cloudflare.com/ai-gateway/usage/providers/perplexity/) * [Parallel provider](https://developers.cloudflare.com/ai-gateway/usage/providers/parallel/) * [Unified Billing](https://developers.cloudflare.com/ai-gateway/features/unified-billing/) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/usage/web-search/#page","headline":"Web Search · Cloudflare AI Gateway docs","description":"Use provider-native web search tools through AI Gateway, or reach search-first providers like Perplexity and Parallel through their proxy endpoints.","url":"https://developers.cloudflare.com/ai-gateway/usage/web-search/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-06-26","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"},"keywords":["AI"]} ``` --- --- description: Use persistent WebSocket connections through AI Gateway for real-time and non-realtime AI interactions. title: WebSockets API image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # WebSockets API Last updated Jun 12, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/usage/websockets-api/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) The AI Gateway WebSockets API provides a persistent connection for AI interactions, eliminating repeated handshakes and reducing latency. This API is divided into two categories: * **Realtime APIs** \- Designed for AI providers that offer low-latency, multimodal interactions over WebSockets. * **Non-Realtime APIs** \- Supports standard WebSocket communication for AI providers, including those that do not natively support WebSockets. ## When to use WebSockets WebSockets are long-lived TCP connections that enable bi-directional, real-time and non realtime communication between client and server. Unlike HTTP connections, which require repeated handshakes for each request, WebSockets maintain the connection, supporting continuous data exchange with reduced overhead. WebSockets are ideal for applications needing low-latency, real-time data, such as voice assistants. ## Key benefits * **Reduced overhead**: Avoid overhead of repeated handshakes and TLS negotiations by maintaining a single, persistent connection. * **Provider compatibility**: Works with all AI providers in AI Gateway. Even if your chosen provider does not support WebSockets, Cloudflare handles it for you, managing the requests to your preferred AI provider. ## Key differences | Feature | Realtime APIs | Non-Realtime APIs | | ----------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------ | | **Purpose** | Enables real-time, multimodal AI interactions for providers that offer dedicated WebSocket endpoints. | Supports WebSocket-based AI interactions with providers that do not natively support WebSockets. | | **Use Case** | Streaming responses for voice, video, and live interactions. | Text-based queries and responses, such as LLM requests. | | **AI Provider Support** | [Limited to providers offering real-time WebSocket APIs.](https://developers.cloudflare.com/ai-gateway/usage/websockets-api/realtime-api/#supported-providers) | [All AI providers in AI Gateway.](https://developers.cloudflare.com/ai-gateway/usage/providers/) | | **Streaming Support** | Providers natively support real-time data streaming. | AI Gateway handles streaming via WebSockets. | For details on implementation, refer to the next sections: * [Realtime WebSockets API](https://developers.cloudflare.com/ai-gateway/usage/websockets-api/realtime-api/) * [Non-Realtime WebSockets API](https://developers.cloudflare.com/ai-gateway/usage/websockets-api/non-realtime-api/) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/usage/websockets-api/#page","headline":"WebSockets API · Cloudflare AI Gateway docs","description":"Use persistent WebSocket connections through AI Gateway for real-time and non-realtime AI interactions.","url":"https://developers.cloudflare.com/ai-gateway/usage/websockets-api/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-06-12","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Establish persistent WebSocket connections for AI requests through AI Gateway without real-time streaming. title: Non-realtime WebSockets API image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Non-realtime WebSockets API Last updated May 8, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/usage/websockets-api/non-realtime-api/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) The Non-realtime WebSockets API allows you to establish persistent connections for AI requests without requiring repeated handshakes. This approach is ideal for applications that do not require real-time interactions but still benefit from reduced latency and continuous communication. ## Set up WebSockets API 1. Generate an AI Gateway token with appropriate AI Gateway Run and opt in to using an authenticated gateway. 2. Use the `wss://` protocol to initiate a WebSocket connection: ```plaintext wss://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id} ``` 3. Open a WebSocket connection authenticated with a Cloudflare token with the AI Gateway Run permission. Note Alternatively, we also support authentication via the `sec-websocket-protocol` header if you are using a browser WebSocket. ## Example request ```javascript import WebSocket from "ws"; const ws = new WebSocket( "wss://gateway.ai.cloudflare.com/v1/my-account-id/my-gateway/", { headers: { "cf-aig-authorization": "Bearer AI_GATEWAY_TOKEN", }, }, ); ws.on("open", () => { ws.send( JSON.stringify({ type: "universal.create", request: { eventId: "my-request", provider: "workers-ai", endpoint: "@cf/meta/llama-3.1-8b-instruct", headers: { Authorization: "Bearer WORKERS_AI_TOKEN", "Content-Type": "application/json", }, query: { prompt: "tell me a joke", }, }, }), ); }) ws.on("message", (message) => { console.log(message.toString()); }); ``` ## Example response ```json { "type": "universal.created", "metadata": { "cacheStatus": "MISS", "eventId": "my-request", "logId": "01JC3R94FRD97JBCBX3S0ZAXKW", "step": "0", "contentType": "application/json" }, "response": { "result": { "response": "Why was the math book sad? Because it had too many problems. Would you like to hear another one?" }, "success": true, "errors": [], "messages": [] } } ``` ## Example streaming request For streaming requests, AI Gateway sends an initial message with request metadata indicating the stream is starting: ```json { "type": "universal.created", "metadata": { "cacheStatus": "MISS", "eventId": "my-request", "logId": "01JC40RB3NGBE5XFRZGBN07572", "step": "0", "contentType": "text/event-stream" } } ``` After this initial message, all streaming chunks are relayed in real-time to the WebSocket connection as they arrive from the inference provider. Only the `eventId` field is included in the metadata for these streaming chunks. The `eventId` allows AI Gateway to include a client-defined ID with each message, even in a streaming WebSocket environment. ```json { "type": "universal.stream", "metadata": { "eventId": "my-request" }, "response": { "response": "would" } } ``` Once all chunks for a request have been streamed, AI Gateway sends a final message to signal the completion of the request. For added flexibility, this message includes all the metadata again, even though it was initially provided at the start of the streaming process. ```json { "type": "universal.done", "metadata": { "cacheStatus": "MISS", "eventId": "my-request", "logId": "01JC40RB3NGBE5XFRZGBN07572", "step": "0", "contentType": "text/event-stream" } } ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/usage/websockets-api/non-realtime-api/#page","headline":"Non-realtime WebSockets API · Cloudflare AI Gateway docs","description":"Establish persistent WebSocket connections for AI requests through AI Gateway without real-time streaming.","url":"https://developers.cloudflare.com/ai-gateway/usage/websockets-api/non-realtime-api/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-05-08","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Connect to AI providers that support real-time WebSocket interactions through AI Gateway. title: Realtime WebSockets API image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Realtime WebSockets API Last updated Apr 20, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/usage/websockets-api/realtime-api/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Some AI providers support real-time, low-latency interactions over WebSockets. AI Gateway allows seamless integration with these APIs, supporting multimodal interactions such as text, audio, and video. ## Supported Providers * [OpenAI ↗](https://platform.openai.com/docs/guides/realtime-websocket) * [Google AI Studio ↗](https://ai.google.dev/gemini-api/docs/multimodal-live) * [Cartesia ↗](https://docs.cartesia.ai/api-reference/tts/tts) * [ElevenLabs ↗](https://elevenlabs.io/docs/conversational-ai/api-reference/conversational-ai/websocket) * [Fal AI ↗](https://docs.fal.ai/model-apis/model-endpoints/websockets) * [Deepgram (Workers AI) ↗](https://developers.cloudflare.com/workers-ai/models/?authors=deepgram) ## Authentication For real-time WebSockets, authentication can be done using: * Headers (for non-browser environments) * `sec-websocket-protocol` (for browsers) Note Provider specific API Keys can also be alternatively configured on AI Gateway using our [BYOK](https://developers.cloudflare.com/ai-gateway/configuration/bring-your-own-keys) feature. You must still include the `cf-aig-authorization` header in the websocket request. ## Examples ### OpenAI ```javascript import WebSocket from "ws"; const url = "wss://gateway.ai.cloudflare.com/v1///openai?model=gpt-4o-realtime-preview-2024-12-17"; const ws = new WebSocket(url, { headers: { "cf-aig-authorization": process.env.CLOUDFLARE_API_KEY, Authorization: "Bearer " + process.env.OPENAI_API_KEY, "OpenAI-Beta": "realtime=v1", }, }); ws.on("open", () => console.log("Connected to server.")); ws.on("message", (message) => console.log(JSON.parse(message.toString()))); ws.send( JSON.stringify({ type: "response.create", response: { modalities: ["text"], instructions: "Tell me a joke" }, }), ); ``` ### Google AI Studio ```javascript const ws = new WebSocket( "wss://gateway.ai.cloudflare.com/v1///google?api_key=", ["cf-aig-authorization."], ); ws.on("open", () => console.log("Connected to server.")); ws.on("message", (message) => console.log(message.data)); ws.send( JSON.stringify({ setup: { model: "models/gemini-2.5-flash", generationConfig: { responseModalities: ["TEXT"] }, }, }), ); ``` ### Cartesia ```javascript const ws = new WebSocket( "wss://gateway.ai.cloudflare.com/v1///cartesia?cartesia_version=2024-06-10&api_key=", ["cf-aig-authorization."], ); ws.on("open", function open() { console.log("Connected to server."); }); ws.on("message", function incoming(message) { console.log(message.data); }); ws.send( JSON.stringify({ model_id: "sonic", transcript: "Hello, world! I'm generating audio on ", voice: { mode: "id", id: "a0e99841-438c-4a64-b679-ae501e7d6091" }, language: "en", context_id: "happy-monkeys-fly", output_format: { container: "raw", encoding: "pcm_s16le", sample_rate: 8000, }, add_timestamps: true, continue: true, }), ); ``` ### ElevenLabs ```javascript const ws = new WebSocket( "wss://gateway.ai.cloudflare.com/v1///elevenlabs?agent_id=", [ "xi-api-key.", "cf-aig-authorization.", ], ); ws.on("open", function open() { console.log("Connected to server."); }); ws.on("message", function incoming(message) { console.log(message.data); }); ws.send( JSON.stringify({ text: "This is a sample text ", voice_settings: { stability: 0.8, similarity_boost: 0.8 }, generation_config: { chunk_length_schedule: [120, 160, 250, 290] }, }), ); ``` ### Fal AI Fal AI supports WebSocket connections for real-time model interactions through their HTTP over WebSocket API. ```javascript const ws = new WebSocket( "wss://gateway.ai.cloudflare.com/v1///fal/fal-ai/fast-lcm-diffusion", ["fal-api-key.", "cf-aig-authorization."], ); ws.on("open", function open() { console.log("Connected to server."); }); ws.on("message", function incoming(message) { console.log(message.data); }); ws.send( JSON.stringify({ prompt: "generate an image of a cat flying an aeroplane", }), ); ``` For more information on Fal AI's WebSocket API, see their [HTTP over WebSocket documentation ↗](https://docs.fal.ai/model-apis/model-endpoints/websockets). ### Deepgram (Workers AI) Workers AI provides Deepgram models for real-time speech-to-text (STT) and text-to-speech (TTS) capabilities through WebSocket connections. #### Speech-to-Text (STT) Workers AI supports two Deepgram STT models: `@cf/deepgram/nova-3` and `@cf/deepgram/flux`. The following example demonstrates real-time audio transcription from a microphone: ```javascript import WebSocket from "ws"; import mic from "mic"; const ws = new WebSocket( "wss://gateway.ai.cloudflare.com/v1///workers-ai?model=@cf/deepgram/nova-3&encoding=linear16&sample_rate=16000&interim_results=true", { headers: { "cf-aig-authorization": process.env.CLOUDFLARE_API_KEY, }, }, ); // Configure microphone const micInstance = mic({ rate: "16000", channels: "1", debug: false, exitOnSilence: 6, }); const micInputStream = micInstance.getAudioStream(); micInputStream.on("data", (data) => { if (ws.readyState === WebSocket.OPEN) { ws.send(data); } }); micInputStream.on("error", (error) => { console.error("Microphone error:", error); }); ws.onopen = () => { console.log("Connected to WebSocket"); console.log("Starting microphone..."); micInstance.start(); }; ws.onmessage = (event) => { try { const parse = JSON.parse(event.data); if (parse.channel?.alternatives?.[0]?.transcript) { if (parse.is_final) { console.log( "Final transcript:", parse.channel.alternatives[0].transcript, ); } else { console.log( "Interim transcript:", parse.channel.alternatives[0].transcript, ); } } } catch (error) { console.error("Error parsing message:", error); } }; ws.onerror = (error) => { console.error("WebSocket error:", error); }; ws.onclose = () => { console.log("WebSocket closed"); micInstance.stop(); }; ``` #### Text-to-Speech (TTS) Workers AI supports the Deepgram `@cf/deepgram/aura-1` model for TTS. The following example demonstrates converting text input to audio: ```javascript import WebSocket from "ws"; import readline from "readline"; import Speaker from "speaker"; const ws = new WebSocket( "wss://gateway.ai.cloudflare.com/v1///workers-ai?model=@cf/deepgram/aura-1", { headers: { "cf-aig-authorization": process.env.CLOUDFLARE_API_KEY, }, }, ); // Speaker management let currentSpeaker = null; let isPlayingAudio = false; // Setup readline for text input const rl = readline.createInterface({ input: process.stdin, output: process.stdout, prompt: "Enter text to speak (or \"quit\" to exit): ", }); ws.onopen = () => { console.log("Connected to Deepgram TTS WebSocket"); rl.prompt(); }; ws.onmessage = (event) => { // Check if message is JSON (metadata, flushed, etc.) or raw audio if (event.data instanceof Buffer || event.data instanceof ArrayBuffer) { // Raw audio data - create new speaker if needed if (!currentSpeaker) { currentSpeaker = new Speaker({ channels: 1, bitDepth: 16, sampleRate: 24000, }); isPlayingAudio = true; } currentSpeaker.write(Buffer.from(event.data)); } else { try { const message = JSON.parse(event.data); switch (message.type) { case "Metadata": console.log("Model info:", message.model_name, message.model_version); break; case "Flushed": console.log("Audio complete"); // End speaker after flush to prevent buffer underflow if (currentSpeaker && isPlayingAudio) { currentSpeaker.end(); currentSpeaker = null; isPlayingAudio = false; } rl.prompt(); break; case "Cleared": console.log("Audio cleared, sequence:", message.sequence_id); break; case "Warning": console.warn("Warning:", message.description); break; } } catch (error) { // Not JSON, might be raw audio as string if (!currentSpeaker) { currentSpeaker = new Speaker({ channels: 1, bitDepth: 16, sampleRate: 24000, }); isPlayingAudio = true; } currentSpeaker.write(Buffer.from(event.data)); } } }; ws.onerror = (error) => { console.error("WebSocket error:", error); }; ws.onclose = () => { console.log("WebSocket closed"); if (currentSpeaker) { currentSpeaker.end(); } rl.close(); process.exit(0); }; // Handle user input rl.on("line", (input) => { const text = input.trim(); if (text.toLowerCase() === "quit") { // Send Close message ws.send(JSON.stringify({ type: "Close" })); ws.close(); return; } if (text.length > 0) { // Send text to TTS ws.send( JSON.stringify({ type: "Speak", text: text, }), ); // Flush to get audio immediately ws.send(JSON.stringify({ type: "Flush" })); console.log("Flushing audio"); } rl.prompt(); }); rl.on("close", () => { if (ws.readyState === WebSocket.OPEN) { ws.close(); } }); ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/usage/websockets-api/realtime-api/#page","headline":"Realtime WebSockets API · Cloudflare AI Gateway docs","description":"Connect to AI providers that support real-time WebSocket interactions through AI Gateway.","url":"https://developers.cloudflare.com/ai-gateway/usage/websockets-api/realtime-api/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-20","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Reference for the AI binding with AI Gateway. Call Workers AI and third-party models with env.AI.run(), access log IDs, and use gateway methods for feedback, logging, and URLs. title: Workers Bindings image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Workers Bindings Last updated Aug 7, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/usage/worker-binding-methods/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) The AI binding (`env.AI`) lets you call AI models and access AI Gateway features directly from your Worker. For a step-by-step setup guide, refer to [Set up Workers AI with AI Gateway](https://developers.cloudflare.com/ai-gateway/integrations/aig-workers-ai-binding/). ## Configuration Add an AI binding to your [Wrangler configuration file](https://developers.cloudflare.com/workers/wrangler/configuration/): ```jsonc { "ai": { "binding": "AI", }, } ``` ```toml [ai] binding = "AI" ``` The binding is accessible in your Worker code as `env.AI`. If you're using TypeScript, run [wrangler types](https://developers.cloudflare.com/workers/wrangler/commands/general/#types) whenever you modify your Wrangler configuration file. This generates types for the `env` object based on your bindings, as well as [runtime types](https://developers.cloudflare.com/workers/languages/typescript/). ## `env.AI.run()` Runs an inference request through AI Gateway. Accepts Workers AI models (`@cf/` prefix) and third-party models (`{author}/{model}` format). **Workers AI model:** ```js const resp = await env.AI.run( "@cf/moonshotai/kimi-k2.5", { prompt: "tell me a joke", }, { gateway: { id: "default", // or use a specific gateway name }, }, ); ``` ```ts const resp = await env.AI.run( "@cf/moonshotai/kimi-k2.5", { prompt: "tell me a joke", }, { gateway: { id: "default", // or use a specific gateway name }, }, ); ``` To use prepaid [AI Gateway credits](https://developers.cloudflare.com/ai-gateway/features/unified-billing/), set the gateway's [Workers AI billing setting](https://developers.cloudflare.com/ai-gateway/configuration/manage-gateway/#configure-workers-ai-billing) to **Unified billing** and specify that gateway in the binding request. Prepaid credits provide access to Workers AI models that otherwise require the Workers Paid plan and provide [higher rate limits for frontier models](https://developers.cloudflare.com/workers-ai/platform/limits/#frontier-models). **Third-party model:** ```js const resp = await env.AI.run( "openai/gpt-4.1-mini", { messages: [{ role: "user", content: "tell me a joke" }], }, { gateway: { id: "default", // or use a specific gateway name }, }, ); ``` ```ts const resp = await env.AI.run( "openai/gpt-4.1-mini", { messages: [{ role: "user", content: "tell me a joke" }], }, { gateway: { id: "default", // or use a specific gateway name }, }, ); ``` Third-party models require an AI Gateway and use [Unified Billing](https://developers.cloudflare.com/ai-gateway/features/unified-billing/). Cloudflare manages the provider credentials and deducts credits from your account. You do not need to supply your own API keys. Note [BYOK (Bring Your Own Keys)](https://developers.cloudflare.com/ai-gateway/configuration/bring-your-own-keys/) is not supported for third-party models called through the AI binding. To use your own provider keys, use the [provider-native endpoints](https://developers.cloudflare.com/ai-gateway/usage/providers/) instead. Browse available models in the [model catalog](https://developers.cloudflare.com/ai/models/). ### Gateway options The third argument to `env.AI.run()` accepts a `gateway` object with the following parameters: | Parameter | Type | Default | Description | | ---------- | ------- | ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | id | string | _required_ | Name of your [AI Gateway](https://developers.cloudflare.com/ai-gateway/get-started/). Must be in the same account as your Worker. Use "default" to automatically create a gateway on the first authenticated request. Refer to [Default gateway](https://developers.cloudflare.com/ai-gateway/configuration/manage-gateway/#default-gateway) for details. | | skipCache | boolean | false | Skip the [cache](https://developers.cloudflare.com/ai-gateway/features/caching/) for this request. | | cacheTtl | number | — | [Cache TTL](https://developers.cloudflare.com/ai-gateway/features/caching/) in seconds. | | cacheKey | string | — | Custom [cache key](https://developers.cloudflare.com/ai-gateway/features/caching/) for this request. | | collectLog | boolean | — | Whether to [collect logs](https://developers.cloudflare.com/ai-gateway/observability/logging/) for this request. | | metadata | object | — | [Custom metadata](https://developers.cloudflare.com/ai-gateway/observability/custom-metadata/) to attach to the log entry. | ## `env.AI.aiGatewayLogId` Returns the log ID from the most recent `env.AI.run()` request. ```typescript const myLogId = env.AI.aiGatewayLogId; ``` ## `env.AI.gateway()` Returns a gateway instance for accessing AI Gateway methods directly. ```typescript const gateway = env.AI.gateway("my-gateway"); ``` The gateway instance exposes the following methods. ### `patchLog()` Sends feedback, score, and metadata for a specific log entry. All properties in the second argument are optional. ```typescript await gateway.patchLog("my-log-id", { feedback: 1, score: 100, metadata: { user: "123", }, }); ``` **Returns:** `Promise` ### `getLog()` Retrieves details of a specific log entry. If the `AiGatewayLog` type is missing, run [wrangler types](https://developers.cloudflare.com/workers/languages/typescript/#generate-types). ```typescript const log = await gateway.getLog("my-log-id"); ``` **Returns:** `Promise` ### `getUrl()` Returns the base URL for your AI Gateway. Pass an optional provider name to get the provider-specific endpoint. ```typescript const baseUrl = await gateway.getUrl(); // https://gateway.ai.cloudflare.com/v1/my-account-id/my-gateway/ const openaiUrl = await gateway.getUrl("openai"); // https://gateway.ai.cloudflare.com/v1/my-account-id/my-gateway/openai ``` **Parameters:** Optional `provider` (string or `AIGatewayProviders` enum) **Returns:** `Promise` #### SDK integration examples **OpenAI SDK:** ```typescript import OpenAI from "openai"; const openai = new OpenAI({ apiKey: "my api key", // defaults to process.env["OPENAI_API_KEY"] baseURL: await env.AI.gateway("my-gateway").getUrl("openai"), }); ``` **Vercel AI SDK with OpenAI:** ```typescript import { createOpenAI } from "@ai-sdk/openai"; const openai = createOpenAI({ baseURL: await env.AI.gateway("my-gateway").getUrl("openai"), }); ``` **Vercel AI SDK with Anthropic:** ```typescript import { createAnthropic } from "@ai-sdk/anthropic"; const anthropic = createAnthropic({ baseURL: await env.AI.gateway("my-gateway").getUrl("anthropic"), }); ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/usage/worker-binding-methods/#page","headline":"Workers Bindings · Cloudflare AI Gateway docs","description":"Reference for the AI binding with AI Gateway. Call Workers AI and third-party models with env.AI.run(), access log IDs, and use gateway methods for feedback, logging, and URLs.","url":"https://developers.cloudflare.com/ai-gateway/usage/worker-binding-methods/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-08-07","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"},"keywords":["AI","Bindings"]} ``` --- --- description: Explore AI Gateway features including caching, rate limiting, guardrails, dynamic routing, and data loss prevention. title: Features image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Features Last updated Jun 5, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/features/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) AI Gateway provides a comprehensive set of features to help you build, deploy, and manage AI applications with confidence. From performance optimization to security and observability, these features work together to create a robust AI infrastructure. ## Core Features ### Performance & Cost Optimization [Caching](https://developers.cloudflare.com/ai-gateway/features/caching/) Serve identical requests directly from Cloudflare's global cache, reducing latency by up to 90% and significantly cutting costs by avoiding repeated API calls to AI providers. **Key benefits:** * Reduced response times for repeated queries * Lower API costs through cache hits * Configurable TTL and per-request cache control * Works across all supported AI providers Use Caching [Spend Limits](https://developers.cloudflare.com/ai-gateway/features/spend-limits/) Set cost-based budgets that track cumulative dollar spend across requests. Scope limits by model, provider, or custom metadata dimensions like user, team, or application. **Key benefits:** * Per-provider or per-model budgets * Per-user or per-team budgets using [custom metadata](https://developers.cloudflare.com/ai-gateway/observability/custom-metadata/) * Configurable time windows (daily, weekly, monthly) * Automatic request blocking when budget is exceeded Use Spend Limits [Rate Limiting](https://developers.cloudflare.com/ai-gateway/features/rate-limiting/) Control application scaling and protect against abuse with flexible rate limiting options. Set limits based on requests per time window with sliding or fixed window techniques. **Key benefits:** * Prevent API quota exhaustion * Control costs and usage patterns * Configurable per gateway or per request * Multiple rate limiting techniques available Use Rate Limiting [Dynamic Routing](https://developers.cloudflare.com/ai-gateway/features/dynamic-routing/) Create sophisticated request routing flows without code changes. Route requests based on user segments, geography, content analysis, or A/B testing requirements through a visual interface. **Key benefits:** * Visual flow-based configuration * User-based and geographic routing * A/B testing and fractional traffic splitting * Context-aware routing based on request content * Dynamic rate limiting with automatic fallbacks Use Dynamic Routing ### Security & Safety [Guardrails](https://developers.cloudflare.com/ai-gateway/features/guardrails/) Deploy AI applications safely with real-time content moderation. Automatically detect and block harmful content in both user prompts and model responses across all providers. **Key benefits:** * Consistent moderation across all AI providers * Real-time prompt and response evaluation * Configurable content categories and actions * Compliance and audit capabilities * Enhanced user safety and trust Use Guardrails [Data Loss Prevention (DLP)](https://developers.cloudflare.com/ai-gateway/features/dlp/) Protect your organization from inadvertent exposure of sensitive data through AI interactions. Scan prompts and responses for PII, financial data, and other sensitive information. **Key benefits:** * Real-time scanning of AI prompts and responses * Detection of PII, financial, healthcare, and custom data patterns * Configurable actions: flag or block sensitive content * Integration with Cloudflare's enterprise DLP solution * Compliance support for GDPR, HIPAA, and PCI DSS Use Data Loss Prevention (DLP) [Authentication](https://developers.cloudflare.com/ai-gateway/configuration/authentication/) Secure your AI Gateway with token-based authentication. Control access to your gateways and protect against unauthorized usage. **Key benefits:** * Token-based access control * Configurable per gateway * Integration with Cloudflare's security infrastructure * Audit trail for access attempts Use Authentication [Bring Your Own Keys (BYOK)](https://developers.cloudflare.com/ai-gateway/configuration/bring-your-own-keys/) Securely store and manage AI provider API keys in Cloudflare's encrypted infrastructure. Remove hardcoded keys from your applications while maintaining full control. **Key benefits:** * Encrypted key storage at rest and in transit * Centralized key management across providers * Easy key rotation without code changes * Support for 20+ AI providers * Enhanced security and compliance Use Bring Your Own Keys (BYOK) ### Observability & Analytics [Analytics](https://developers.cloudflare.com/ai-gateway/observability/analytics/) Gain deep insights into your AI application usage with comprehensive analytics. Track requests, tokens, costs, errors, and performance across all providers. **Key benefits:** * Real-time usage metrics and trends * Cost tracking and estimation across providers * Error monitoring and troubleshooting * Cache hit rates and performance insights * GraphQL API for custom dashboards Use Analytics [Logging](https://developers.cloudflare.com/ai-gateway/observability/logging/) Capture detailed logs of all AI requests and responses for debugging, compliance, and analysis. Configure log retention and export options. **Key benefits:** * Complete request/response logging * Configurable log retention policies * Export capabilities via Logpush * Custom metadata support * Compliance and audit support Use Logging [Custom Metadata](https://developers.cloudflare.com/ai-gateway/observability/custom-metadata/) Enrich your logs and analytics with custom metadata. Tag requests with user IDs, team information, or any custom data for enhanced filtering and analysis. **Key benefits:** * Enhanced request tracking and filtering * User and team-based analytics * Custom business logic integration * Improved debugging and troubleshooting Use Custom Metadata ### Advanced Configuration [Custom Costs](https://developers.cloudflare.com/ai-gateway/configuration/custom-costs/) Override default pricing with your negotiated rates or custom cost models. Apply custom costs at the request level for accurate cost tracking. **Key benefits:** * Accurate cost tracking with negotiated rates * Per-request cost customization * Better budget planning and forecasting * Support for enterprise pricing agreements Use Custom Costs ## Feature Comparison by Use Case | Use Case | Recommended Features | | -------------------------- | -------------------------------------------------- | | **Cost Optimization** | Caching, Spend Limits, Rate Limiting, Custom Costs | | **High Availability** | Fallbacks using Dynamic Routing | | **Security & Compliance** | Guardrails, DLP, Authentication, BYOK, Logging | | **Performance Monitoring** | Analytics, Logging, Custom Metadata | | **A/B Testing** | Dynamic Routing, Custom Metadata, Analytics | ## Getting Started with Features 1. **Start with the basics**: Enable [Caching](https://developers.cloudflare.com/ai-gateway/features/caching/) and [Analytics](https://developers.cloudflare.com/ai-gateway/observability/analytics/) for immediate benefits 2. **Add reliability**: Configure Fallbacks and Rate Limiting using [Dynamic routing](https://developers.cloudflare.com/ai-gateway/features/dynamic-routing/) 3. **Enhance security**: Implement [Guardrails](https://developers.cloudflare.com/ai-gateway/features/guardrails/), [DLP](https://developers.cloudflare.com/ai-gateway/features/dlp/), and [Authentication](https://developers.cloudflare.com/ai-gateway/configuration/authentication/) --- _All features work seamlessly together and across all 20+ supported AI providers. Get started with [AI Gateway](https://developers.cloudflare.com/ai-gateway/get-started/) to begin using these features in your applications._ Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"WebPage","@id":"https://developers.cloudflare.com/ai-gateway/features/#page","headline":"Features · Cloudflare AI Gateway docs","description":"Explore AI Gateway features including caching, rate limiting, guardrails, dynamic routing, and data loss prevention.","url":"https://developers.cloudflare.com/ai-gateway/features/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-06-05","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Override caching settings on a per-request basis. title: Caching image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Caching Last updated Jun 15, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/features/caching/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) AI Gateway can cache responses from your AI model providers, serving them directly from Cloudflare's cache for identical requests. ## Benefits of Using Caching * **Reduced Latency:** Serve responses faster to your users by avoiding a round trip to the origin AI provider for repeated requests. * **Cost Savings:** Minimize the number of paid requests made to your AI provider, especially for frequently accessed or non-dynamic content. * **Increased Throughput:** Offload repetitive requests from your AI provider, allowing it to handle unique requests more efficiently. Note Currently caching is supported only for text and image responses, and it applies only to identical requests. This configuration benefits use cases with limited prompt options. For example, a support bot that asks "How can I help you?" and lets the user select an answer from a limited set of options works well with the current caching configuration. We plan on adding semantic search for caching in the future to improve cache hit rates. ## Default configuration To set the default caching configuration in the dashboard: 1. Log into the [Cloudflare dashboard ↗](https://dash.cloudflare.com/) and select your account. 2. Select **AI** \> **AI Gateway**. 3. Select **Settings**. 4. Enable **Cache Responses**. 5. Change the default caching to whatever value you prefer. To set the default caching configuration using the API: 1. [Create an API token](https://developers.cloudflare.com/fundamentals/api/get-started/create-token/) with the following permissions: * `AI Gateway - Read` * `AI Gateway - Edit` 1. Get your [Account ID](https://developers.cloudflare.com/fundamentals/account/find-account-and-zone-ids/). 2. Using that API token and Account ID, send a [POST request](https://developers.cloudflare.com/api/resources/ai%5Fgateway/methods/create/) to create a new Gateway and include a value for the `cache_ttl`. This caching behavior will be uniformly applied to all requests that support caching. If you need to modify the cache settings for specific requests, you have the flexibility to override this setting on a per-request basis. To check whether a response comes from cache or not, **cf-aig-cache-status** will be designated as `HIT` or `MISS`. ## How the cache key works By default, AI Gateway constructs the cache key by concatenating the following and hashing the result with SHA-256: * **Provider** (for example, `openai`, `anthropic`) * **Endpoint** (the API path) * **Model** (for example, `gpt-4o`) * **Provider authentication header** (for example, the `Authorization` bearer token) * **Full request body** This means caching is based on **exact match** of the entire request. Any difference in the body — including messages, tools, or model parameters — will result in a separate cache entry. To override this behavior, use the [custom cache key header](#custom-cache-key-cf-aig-cache-key). ## Per-request caching While your gateway's default cache settings provide a good baseline, you might need more granular control. These situations could include data freshness, content with varying lifespans, or dynamic or personalized responses. To address these needs, AI Gateway allows you to override default cache behaviors on a per-request basis using specific HTTP headers. This gives you the precision to optimize caching for individual API calls. The following headers allow you to define this per-request cache behavior: Note The following headers have been updated to new names, though the old headers will still function. We recommend updating to the new headers to ensure future compatibility: `cf-cache-ttl` is now `cf-aig-cache-ttl` `cf-skip-cache` is now `cf-aig-skip-cache` ### Skip cache (cf-aig-skip-cache) Skip cache refers to bypassing the cache and fetching the request directly from the original provider, without utilizing any cached copy. You can use the header **cf-aig-skip-cache** to bypass the cached version of the request. As an example, when submitting a request to OpenAI, include the header in the following manner: ```bash # Run `wrangler whoami` to get your account ID to replace $CLOUDFLARE_ACCOUNT_ID, # and `wrangler auth token` to get an auth token to replace $CLOUDFLARE_API_TOKEN. curl -X POST "https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/v1/chat/completions" \ --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ --header "Content-Type: application/json" \ --header "cf-aig-skip-cache: true" \ --data '{ "model": "openai/gpt-4.1-mini", "messages": [ { "role": "user", "content": "how to build a wooden spoon in 3 short steps? give as short as answer as possible" } ] }' ``` ### Cache TTL (cf-aig-cache-ttl) Cache TTL, or Time To Live, is the duration a cached request remains valid before it expires and is refreshed from the original source. You can use **cf-aig-cache-ttl** to set the desired caching duration in seconds. The minimum TTL is 60 seconds and the maximum TTL is one month. For example, if you set a TTL of one hour, it means that a request is kept in the cache for an hour. Within that hour, an identical request will be served from the cache instead of the original API. After an hour, the cache expires and the request will go to the original API for a fresh response, and that response will repopulate the cache for the next hour. As an example, when submitting a request to OpenAI, include the header in the following manner: ```bash # Run `wrangler whoami` to get your account ID to replace $CLOUDFLARE_ACCOUNT_ID, # and `wrangler auth token` to get an auth token to replace $CLOUDFLARE_API_TOKEN. curl -X POST "https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/v1/chat/completions" \ --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ --header "Content-Type: application/json" \ --header "cf-aig-cache-ttl: 3600" \ --data '{ "model": "openai/gpt-4.1-mini", "messages": [ { "role": "user", "content": "how to build a wooden spoon in 3 short steps? give as short as answer as possible" } ] }' ``` ### Custom cache key (cf-aig-cache-key) Custom cache keys let you override the default cache key in order to precisely set the cacheability setting for any resource. To override the default cache key, you can use the header **cf-aig-cache-key**. When you use the **cf-aig-cache-key** header for the first time, you will receive a response from the provider. Subsequent requests with the same header will return the cached response. If the **cf-aig-cache-ttl** header is used, responses will be cached according to the specified Cache Time To Live. Otherwise, responses will be cached according to the cache settings in the dashboard. If caching is not enabled for the gateway, responses will be cached for 5 minutes by default. As an example, when submitting a request to OpenAI, include the header in the following manner: ```bash # Run `wrangler whoami` to get your account ID to replace $CLOUDFLARE_ACCOUNT_ID, # and `wrangler auth token` to get an auth token to replace $CLOUDFLARE_API_TOKEN. curl -X POST "https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/v1/chat/completions" \ --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ --header "Content-Type: application/json" \ --header "cf-aig-cache-key: responseA" \ --data '{ "model": "openai/gpt-4.1-mini", "messages": [ { "role": "user", "content": "how to build a wooden spoon in 3 short steps? give as short as answer as possible" } ] }' ``` AI Gateway caching behavior Cache in AI Gateway is volatile. If two identical requests are sent simultaneously, the first request may not cache in time for the second request to use it, which may result in the second request retrieving data from the original source. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/features/caching/#page","headline":"Caching · Cloudflare AI Gateway docs","description":"Override caching settings on a per-request basis.","url":"https://developers.cloudflare.com/ai-gateway/features/caching/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-06-15","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Protect sensitive data in AI Gateway prompts and responses using Cloudflare DLP detection engines. title: Data Loss Prevention (DLP) image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Data Loss Prevention (DLP) Last updated Jun 5, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/features/dlp/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Data Loss Prevention (DLP) for AI Gateway helps protect your organization from inadvertent exposure of sensitive data through AI interactions. By integrating with Cloudflare's proven DLP technology, AI Gateway can scan both incoming prompts and outgoing AI responses for sensitive information, ensuring your AI applications maintain security and compliance standards. ## How it works AI Gateway DLP leverages the same powerful detection engines used in [Cloudflare's Data Loss Prevention](https://developers.cloudflare.com/cloudflare-one/data-loss-prevention/) solution to scan AI traffic in real-time. The system analyzes both user prompts sent to AI models and responses received from AI providers, identifying sensitive data patterns and taking appropriate protective actions. ## Key benefits * **Prevent data leakage**: Stop sensitive information from being inadvertently shared with AI providers or exposed in AI responses * **Maintain compliance**: Help meet regulatory requirements like GDPR, HIPAA, and PCI DSS * **Consistent protection**: Apply the same DLP policies across all AI providers and models * **Audit visibility**: Comprehensive logging and reporting for security and compliance teams * **Zero-code integration**: Enable protection without modifying existing AI applications ## Supported AI traffic AI Gateway DLP can scan: * **User prompts** \- Content submitted to AI models, including text, code, and structured data * **AI responses** \- Output generated by AI models before being returned to users The system works with all AI providers supported by AI Gateway, providing consistent protection regardless of which models or services you use. ### Inspection scope DLP inspects the text content of request and response bodies as they pass through AI Gateway. The following details apply: * **Non-streaming requests and responses**: DLP scans the full request and response body. * **Streaming (SSE) responses**: DLP buffers the full streamed response before scanning. This means DLP-scanned streaming responses are not delivered incrementally to the client. Expect increased time-to-first-token latency when DLP response scanning is enabled on streaming requests, because the entire response must be received from the provider before DLP can evaluate it and release it to the client. * **Tool call arguments and results**: DLP scans the text content present in the message body, which includes tool call arguments and results if they appear in the JSON request or response payload. * **Base64-encoded images and file attachments**: DLP does not decode base64-encoded content or follow external URLs. Only the raw text of the request and response body is inspected. * **Multipart form data**: DLP scans the text portions of the request body. Binary data within multipart payloads is not inspected. ### Streaming behavior When DLP response scanning is enabled and a client sends a streaming request (`"stream": true`), AI Gateway buffers the complete provider response before running DLP inspection. This differs from requests without DLP, where streamed chunks are forwarded to the client as they arrive. Because of this buffering: * **Time-to-first-token latency increases** proportionally to the full response generation time. * **Request-only DLP scanning** (where the **Check** setting is set to **Request**) does not buffer the response and has no impact on streaming latency. * If you need low-latency streaming for certain requests while still using DLP on the same gateway, consider setting the DLP policy **Check** to **Request** only, or use separate gateways for latency-sensitive and DLP-scanned traffic. ### Interaction with caching DLP scanning runs after a cache miss, when AI Gateway forwards the request to the provider and receives a response. The following table describes how each DLP outcome affects caching: | DLP outcome | Response cached | Behavior | | ------------------ | --------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Pass (no findings) | Yes | The response is cached normally according to your gateway cache settings. | | Flag | Yes | The response is cached normally. DLP findings are attached to the cf-aig-dlp response header and recorded in logs, but the original response is returned to the client. | | Block | No | The provider response is discarded and replaced with a DLP error response (status 400). | **Cache hits skip DLP scanning.** When a subsequent identical request matches a cached response, AI Gateway serves it directly from cache without re-running DLP. This is safe because only responses that already passed DLP (or were flagged, not blocked) are cached. However, if you update your DLP policies after a response has been cached, the cached response is not re-evaluated. It continues to be served until the cache TTL expires. If you need DLP policy changes to take effect immediately, you can bypass the cache for new requests using the `cf-aig-skip-cache` header. For more information, refer to [Caching](https://developers.cloudflare.com/ai-gateway/features/caching/). ### Per-request DLP controls DLP policies are configured at the gateway level and apply uniformly to all requests passing through that gateway. There is no per-request header to select specific DLP profiles or to bypass DLP scanning for individual requests. If you need different DLP policies for different use cases (for example, per-tenant policy variance in a multi-tenant application), the recommended approach is to create separate gateways with different DLP configurations and route requests to the appropriate gateway based on your application logic. ## Integration with Cloudflare DLP AI Gateway DLP uses the same [detection profiles](https://developers.cloudflare.com/cloudflare-one/data-loss-prevention/dlp-profiles/) as Cloudflare One's DLP solution. Profiles are shared account-level objects, so you can reuse existing predefined or custom profiles across both [Gateway HTTP policies](https://developers.cloudflare.com/cloudflare-one/data-loss-prevention/dlp-policies/) and AI Gateway DLP policies. Key differences from Cloudflare One Gateway DLP: * **No Gateway proxy or TLS decryption required** \- AI Gateway inspects traffic directly as an AI proxy, so you do not need to set up [Gateway HTTP filtering](https://developers.cloudflare.com/cloudflare-one/traffic-policies/get-started/http/) or [TLS decryption](https://developers.cloudflare.com/cloudflare-one/traffic-policies/http-policies/tls-decryption/). * **Separate policy management** \- DLP policies for AI Gateway are configured per gateway in the AI Gateway dashboard, not in Cloudflare One traffic policies. * **Separate logs** \- DLP events for AI Gateway appear in [AI Gateway logs](https://developers.cloudflare.com/ai-gateway/observability/logging/), not in Cloudflare One HTTP request logs. * **Shared profiles** \- DLP detection profiles (predefined and custom) are shared across both products. Changes to a profile apply everywhere it is used. For more information about Cloudflare's DLP capabilities, refer to the [Data Loss Prevention documentation](https://developers.cloudflare.com/cloudflare-one/data-loss-prevention/). ## Getting started To enable DLP for your AI Gateway: 1. [Set up DLP policies](https://developers.cloudflare.com/ai-gateway/features/dlp/set-up-dlp/) for your AI Gateway 2. Configure detection profiles and response actions 3. Monitor DLP events through the Cloudflare dashboard ## Related resources * [Set up DLP for AI Gateway](https://developers.cloudflare.com/ai-gateway/features/dlp/set-up-dlp/) * [Cloudflare Data Loss Prevention](https://developers.cloudflare.com/cloudflare-one/data-loss-prevention/) * [AI Gateway Security Features](https://developers.cloudflare.com/ai-gateway/features/guardrails/) * [DLP Detection Profiles](https://developers.cloudflare.com/cloudflare-one/data-loss-prevention/dlp-profiles/) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/features/dlp/#page","headline":"Data Loss Prevention (DLP) · Cloudflare AI Gateway docs","description":"Protect sensitive data in AI Gateway prompts and responses using Cloudflare DLP detection engines.","url":"https://developers.cloudflare.com/ai-gateway/features/dlp/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-06-05","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Enable and configure DLP policies on your AI Gateway to scan prompts and responses for sensitive data. title: Set up Data Loss Prevention (DLP) image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Set up Data Loss Prevention (DLP) Last updated Apr 20, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/features/dlp/set-up-dlp/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Add Data Loss Prevention (DLP) to any AI Gateway to start scanning AI prompts and responses for sensitive data. ## Prerequisites * An existing [AI Gateway](https://developers.cloudflare.com/ai-gateway/get-started/) ## Enable DLP for AI Gateway 1. Log into the [Cloudflare dashboard ↗](https://dash.cloudflare.com/) and select your account. 2. Go to **AI** \> **AI Gateway**. 3. Select a gateway where you want to enable DLP. 4. Go to the **Firewall** tab. 5. Toggle **Data Loss Prevention (DLP)** to **On**. ## Add DLP policies After enabling DLP, you can create policies to define how sensitive data should be handled: 1. Under the DLP section, click **Add Policy**. 2. Configure the following fields for each policy: * **Policy ID**: Enter a unique name for this policy (e.g., "Block-PII-Requests") * **DLP Profiles**: Select the DLP profiles to check against. AI requests/responses will be checked against each of the selected profiles. Available profiles include: * **Financial Information** \- Credit cards, bank accounts, routing numbers * **Personal Identifiable Information (PII)** \- Names, addresses, phone numbers * **Government Identifiers** \- SSNs, passport numbers, driver's licenses * **Healthcare Information** \- Medical record numbers, patient data * **Custom Profiles** \- Organization-specific data patterns Note DLP profiles can be created and managed in the [Zero Trust DLP dashboard](https://developers.cloudflare.com/cloudflare-one/data-loss-prevention/dlp-profiles/). * **Action**: Choose the action to take when any of the selected profiles match: * **Flag** \- Record the detection for audit purposes without blocking * **Block** \- Prevent the request/response from proceeding * **Check**: Select what to scan: * **Request** \- Scan user prompts sent to AI providers * **Response** \- Scan AI model responses before returning to users * **Both** \- Scan both requests and responses 3. Click **Save** to save your policy configuration. ## Manage DLP policies You can create multiple DLP policies with different configurations: * **Add multiple policies**: Click **Add Policy** to create additional policies with different profile combinations or actions * **Enable/disable policies**: Use the toggle next to each policy to individually enable or disable them without deleting the configuration * **Edit policies**: Click on any existing policy to modify its settings * **Save changes**: Always click **Save** after making any changes to apply them ## Test your configuration After configuring DLP settings: 1. Make a test AI request through your gateway that contains sample sensitive data. 2. Check the **AI Gateway Logs** to verify DLP scanning is working. 3. Review the detection results and adjust profiles or actions as needed. ## Monitor DLP events ### Viewing DLP logs in AI Gateway DLP events are integrated into your AI Gateway logs. When a DLP policy matches, the log entry includes details about the match alongside standard log fields like provider, model, tokens, and cost. 1. Go to **AI** \> **AI Gateway** \> your gateway > **Logs**. 2. Select any log entry to view detailed information. For requests where DLP policies were triggered, the log entry includes additional DLP fields: | Field | Description | | -------------------- | --------------------------------------------------------------------- | | DLP Action | The action taken by the DLP policy: FLAG or BLOCK | | DLP Policies Matched | The IDs of the DLP policies that matched | | DLP Profiles Matched | The IDs of the DLP profiles that triggered within each matched policy | | DLP Entries Matched | The specific detection entry IDs that matched within each profile | | DLP Check | Whether the match occurred in the REQUEST, RESPONSE, or both | ### DLP fields in the Logs API When you retrieve logs through the [Logs API](https://developers.cloudflare.com/api/resources/ai%5Fgateway/subresources/logs/methods/list/), log entries for requests where DLP policies matched include DLP-specific fields in the response. These fields contain the same match data surfaced in the dashboard and in the `cf-aig-dlp` response header, including the action taken, matched policy IDs, matched profile IDs, and entry IDs. For more information on log fields, refer to the [Logging documentation](https://developers.cloudflare.com/ai-gateway/observability/logging/). ### Filter DLP events To view only DLP-related requests: 1. On the **Logs** tab, select **Add Filter**. 2. Select **DLP Action** from the filter options. 3. Choose to filter by: * **FLAG** \- Show only requests where sensitive data was flagged * **BLOCK** \- Show only requests that were blocked due to DLP policies ## Error handling When DLP policies are triggered, your application will receive additional information through response headers and error codes. ### DLP response header When a request matches DLP policies (whether flagged or blocked), an additional `cf-aig-dlp` header is returned containing detailed information about the match: #### Header schema ```json { "findings": [ { "profile": { "context": {}, "entry_ids": ["string"], "profile_id": "string" }, "policy_ids": ["string"], "check": "REQUEST" | "RESPONSE" } ], "action": "BLOCK" | "FLAG" } ``` #### Example header value ```json { "findings": [ { "profile": { "context": {}, "entry_ids": [ "a1b2c3d4-e5f6-7890-abcd-ef1234567890", "f7e8d9c0-b1a2-3456-789a-bcdef0123456" ], "profile_id": "12345678-90ab-cdef-1234-567890abcdef" }, "policy_ids": ["block_financial_data"], "check": "REQUEST" } ], "action": "BLOCK" } ``` Use this header to programmatically detect which DLP profiles and entries were matched, which policies triggered, and whether the match occurred in the request or response. ### Error codes for blocked requests When DLP blocks a request, your application will receive structured error responses: * **Request blocked by DLP** * `"code": 2029` * `"message": "Request content blocked due to DLP policy violations"` * **Response blocked by DLP** * `"code": 2030` * `"message": "Response content blocked due to DLP policy violations"` Handle these errors in your application: ```js try { const res = await env.AI.run('@cf/meta/llama-3.1-8b-instruct', { prompt: userInput }, { gateway: {id: 'your-gateway-id'} }) return Response.json(res) } catch (e) { if ((e as Error).message.includes('2029')) { return new Response('Request contains sensitive data and cannot be processed.') } if ((e as Error).message.includes('2030')) { return new Response('AI response was blocked due to sensitive content.') } return new Response('AI request failed') } ``` ## Best practices * **Start with flagging**: Begin with "Flag" actions to understand what data is being detected before implementing blocking * **Tune confidence levels**: Adjust detection sensitivity based on your false positive tolerance * **Use appropriate profiles**: Select DLP profiles that match your data protection requirements * **Monitor regularly**: Review DLP events to ensure policies are working as expected * **Test thoroughly**: Validate DLP behavior with sample sensitive data before production deployment ## Troubleshooting For general AI Gateway troubleshooting, refer to [Troubleshooting](https://developers.cloudflare.com/ai-gateway/reference/troubleshooting/). ### DLP not triggering * Verify DLP toggle is enabled for your gateway * Ensure selected DLP profiles are appropriate for your test data * Confirm confidence levels aren't set too high ### Unexpected blocking * Review DLP logs to see which profiles triggered * Consider lowering confidence levels for problematic profiles * Test with different sample data to understand detection patterns * Adjust profile selections if needed For additional support with DLP configuration, refer to the [Cloudflare Data Loss Prevention documentation](https://developers.cloudflare.com/cloudflare-one/data-loss-prevention/) or contact your Cloudflare support team. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/features/dlp/set-up-dlp/#page","headline":"Set up Data Loss Prevention (DLP) · Cloudflare AI Gateway docs","description":"Enable and configure DLP policies on your AI Gateway to scan prompts and responses for sensitive data.","url":"https://developers.cloudflare.com/ai-gateway/features/dlp/set-up-dlp/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-20","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Route AI Gateway requests based on conditions, quotas, and fallbacks using a visual interface or JSON configuration. title: Dynamic routing image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Dynamic routing Last updated Aug 7, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/features/dynamic-routing/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) ## Introduction Dynamic routing enables you to create request routing flows through a **visual interface** or a **JSON-based configuration**. Instead of hard-coding a single model, with Dynamic Routing you compose a small flow that evaluates conditions, enforces quotas, and chooses models with fallbacks. You can iterate without touching application code—publish a new route version and you’re done. With dynamic routing, you can easily implement advanced use cases such as: * Directing different segments (paid/not-paid user) to different models * Restricting each user/project/team with budget/rate limits * A/B and gradual rollouts while making it accessible to both developers and non-technical team members. ![Dynamic Routing Overview](https://developers.cloudflare.com/cdn-cgi/image/onerror=redirect,width=1814,height=1642,format=webp/_astro/dynamic-routing.BtwkWywo.png) ## Core Concepts * **Route**: A named, versioned flow (for example, dynamic/support) that you can use as instead of the model name in your requests. * **Nodes** * **Start**: Entry point for the route. * **Conditional**: If/Else branch based on expressions that reference request body, headers, or metadata (for example, user\_plan == "paid"). * **Percentage**: Routes requests probabilistically across multiple outputs, useful for A/B testing and gradual rollouts. * **Model**: Calls a provider/model with the request parameters * **Rate Limit**: Enforces number of requests quotas (per your key, per period) and switches to fallback when exceeded. * **Budget Limit**: Enforces cost quotas (per your key, per period) and switches to fallback when exceeded. * **End**: Terminates the flow and returns the final model response. * **Metadata**: Arbitrary key-value context attached to the request (for example, userId, orgId, plan). You can pass this from your app so rules can reference it. * **Versions**: Each change produces a new draft. Deploy to make it live with instant rollback. ## Getting Started Caution Ensure your gateway has [authentication](https://developers.cloudflare.com/ai-gateway/configuration/authentication/) turned on, and you have your upstream providers keys stored with [BYOK](https://developers.cloudflare.com/ai-gateway/configuration/bring-your-own-keys/). 1. Create a route. * Go to **(Select your gateway)** \> **Dynamic Routes** \> **Add Route**, and name it (for example, `support`). * Open **Editor**. 2. Define conditionals, limits and other settings. * You can use [Custom Metadata](https://developers.cloudflare.com/ai-gateway/observability/custom-metadata/) in your conditionals. 3. Configure model nodes. * Example: * Node A: Provider OpenAI, Model `o4-mini-high` * Node B: Provider OpenAI, Model `gpt-4.1` 4. Save a version. * Click **Save** to save the state. You can always roll back to earlier versions from **Versions**. * Deploy the version to make it live. 5. Call the route from your code. * Use the [OpenAI compatible](https://developers.cloudflare.com/ai-gateway/usage/chat-completion/) endpoint (`/compat/chat/completions`), and use the route name in place of the model, for example, `dynamic/support`. See [Using a dynamic route](https://developers.cloudflare.com/ai-gateway/features/dynamic-routing/usage/) for examples. Note The OpenAI-compatible endpoint is marked **Deprecated** for standard single-model chat completions, but it remains the required way to call dynamic routes. Dynamic routing is not currently available on the [REST API](https://developers.cloudflare.com/ai-gateway/usage/rest-api/). Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"WebPage","@id":"https://developers.cloudflare.com/ai-gateway/features/dynamic-routing/#page","headline":"Dynamic routing · Cloudflare AI Gateway docs","description":"Route AI Gateway requests based on conditions, quotas, and fallbacks using a visual interface or JSON configuration.","url":"https://developers.cloudflare.com/ai-gateway/features/dynamic-routing/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-08-07","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Define AI Gateway dynamic routing flows using the REST API and JSON element structure. title: JSON Configuration image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # JSON Configuration Last updated Apr 29, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/features/dynamic-routing/json-configuration/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Instead of using the **dashboard editor UI** to define the route graph, you can do it using the REST API. Routes are internally represented using a simple JSON structure: ```json { "id": "", "name": "", "elements": [] } ``` ## Supported elements Dynamic routing supports several types of elements that you can combine to create sophisticated routing flows. Each element has specific inputs, outputs, and configuration options. ### Start Element Marks the beginning of a route. Every route must start with a Start element. * **Inputs**: None * **Outputs**: * `next`: Forwards the unchanged request to the next element ```json { "id": "", "type": "start", "outputs": { "next": { "elementId": "" } } } ``` ### Conditional Element (If/Else) Evaluates a condition based on request parameters and routes the request accordingly. * **Inputs**: Request * **Outputs**: * `true`: Forwards request to provided element if condition evaluates to true * `false`: Forwards request to provided element if condition evaluates to false `conditions` supports MongoDB-like operators such as `$eq`, `$ne`, `$in`, `$and`, and `$or`. ```json { "id": "", "type": "conditional", "properties": { "conditions": { "metadata.plan": { "$eq": "free" } } }, "outputs": { "true": { "elementId": "" }, "false": { "elementId": "" } } } ``` ### Percentage Split Routes requests probabilistically across multiple outputs, useful for A/B testing and gradual rollouts. * **Inputs**: Request * **Outputs**: Up to 5 named percentage outputs * Each output key (for example, `"10%"`) is the probability for that branch, and the keys must sum to 100% ```json { "id": "", "type": "percentage", "outputs": { "10%": { "elementId": "" }, "40%": { "elementId": "" }, "50%": { "elementId": "" } } } ``` ### Rate/Budget Limit Apply limits based on request metadata. Supports both count-based and cost-based limits. * **Inputs**: Request * **Outputs**: * `success`: Forwards request to provided element if request is not rate limited * `fallback`: Optional output for rate-limited requests (route terminates if not provided) **Properties**: * `limitType`: "count" or "cost" * `key`: Request field to use for rate limiting (e.g. "metadata.user\_id") * `limit`: Maximum allowed requests/cost * `window`: Time window in seconds ```json { "id": "", "type": "rate", "properties": { "limitType": "count", "key": "metadata.user_id", "limit": 100, "window": 3600 }, "outputs": { "success": { "elementId": "node_model_workers_ai" }, "fallback": { "elementId": "node_model_openai_mini" } } } ``` ### Model Executes inference using a specified model and provider with configurable timeout and retry settings. * **Inputs**: Request * **Outputs**: * `success`: Forwards request to provided element if model successfully starts streaming a response * `fallback`: Optional output if model fails after all retries or times out **Properties**: * `provider`: AI provider (e.g. "openai", "anthropic") * `model`: Specific model name * `timeout`: Request timeout in milliseconds * `retries`: Number of retry attempts ```json { "id": "", "type": "model", "properties": { "provider": "openai", "model": "gpt-4o-mini", "timeout": 60000, "retries": 4 }, "outputs": { "success": { "elementId": "" }, "fallback": { "elementId": "" } } } ``` ### End element Marks the end of a route. Returns the last successful model response, or an error if no model response was generated. * **Inputs**: Request * **Outputs**: None (provide an empty `outputs` object) ```json { "id": "", "type": "end", "outputs": {} } ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/features/dynamic-routing/json-configuration/#page","headline":"JSON Configuration · Cloudflare AI Gateway docs","description":"Define AI Gateway dynamic routing flows using the REST API and JSON element structure.","url":"https://developers.cloudflare.com/ai-gateway/features/dynamic-routing/json-configuration/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-29","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Send requests through an AI Gateway dynamic route using the OpenAI SDK, a direct HTTP request, or the Workers AI binding. title: Using a dynamic route image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Using a dynamic route Last updated Aug 7, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/features/dynamic-routing/usage/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Caution Ensure your gateway has [authentication](https://developers.cloudflare.com/ai-gateway/configuration/authentication/) turned on and you have your upstream providers keys stored with [BYOK](https://developers.cloudflare.com/ai-gateway/configuration/bring-your-own-keys/). ## Examples ### OpenAI SDK ```js import OpenAI from "openai"; const cloudflareToken = "CF_AIG_TOKEN"; const accountId = "{account_id}"; const gatewayId = "{gateway_id}"; const baseURL = `https://gateway.ai.cloudflare.com/v1/${accountId}/${gatewayId}/compat`; const openai = new OpenAI({ apiKey: cloudflareToken, baseURL, }); try { const model = "dynamic/"; const messages = [{ role: "user", content: "What is a neuron?" }]; const chatCompletion = await openai.chat.completions.create({ model, messages, }); const response = chatCompletion.choices[0].message; console.log(response); } catch (e) { console.error(e); } ``` ### Fetch ```bash curl -X POST https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat/chat/completions \ --header 'cf-aig-authorization: Bearer {CF_AIG_TOKEN}' \ --header 'Content-Type: application/json' \ --data '{ "model": "dynamic/", "messages": [ { "role": "user", "content": "What is Cloudflare?" } ] }' ``` ### Workers ```ts export interface Env { AI: Ai; } export default { async fetch(request: Request, env: Env) { const response = await env.AI.gateway("default").run({ provider: "compat", endpoint: "chat/completions", headers: {}, query: { model: "dynamic/", messages: [ { role: "user", content: "What is Cloudflare?", }, ], }, }); return Response(response); }, }; ``` ## Response Metadata The response from a dynamic route is the same as the response from a model. There is additional metadata used to notify the model and provider used, you can check the following headers * `cf-aig-model` \- The model used * `cf-aig-provider` \- The slug of provider used Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/features/dynamic-routing/usage/#page","headline":"Using a dynamic route · Cloudflare AI Gateway docs","description":"Send requests through an AI Gateway dynamic route using the OpenAI SDK, a direct HTTP request, or the Workers AI binding.","url":"https://developers.cloudflare.com/ai-gateway/features/dynamic-routing/usage/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-08-07","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Evaluate AI Gateway prompts and responses for harmful content and enforce safety policies across providers. title: Guardrails image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Guardrails Last updated Jun 5, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/features/guardrails/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Guardrails help you deploy AI applications safely by intercepting and evaluating both user prompts and model responses for harmful content. Acting as a proxy between your application and [model providers](https://developers.cloudflare.com/ai-gateway/usage/providers/) (such as OpenAI, Anthropic, DeepSeek, and others), AI Gateway's Guardrails ensure a consistent and secure experience across your entire AI ecosystem. Guardrails proactively monitor interactions between users and AI models, giving you: * **Consistent moderation**: Uniform moderation layer that works across models and providers. * **Enhanced safety and user trust**: Proactively protect users from harmful or inappropriate interactions. * **Flexibility and control over allowed content**: Specify which categories to monitor and choose between flagging or outright blocking. * **Auditing and compliance capabilities**: Receive updates on evolving regulatory requirements with logs of user prompts, model responses, and enforced guardrails. ## Video demo ## How Guardrails work AI Gateway inspects all interactions in real time by evaluating content against predefined safety parameters. Guardrails work by: 1. Intercepting interactions: AI Gateway proxies requests and responses, sitting between the user and the AI model. 2. Inspecting content: * User prompts: AI Gateway checks prompts against safety parameters (for example, violence, hate, or sexual content). Based on your settings, prompts can be flagged or blocked before reaching the model. * Model responses: Once processed, the AI model response is inspected. If hazardous content is detected, it can be flagged or blocked before being delivered to the user. 3. Applying actions: Depending on your configuration, flagged content is logged for review, while blocked content is prevented from proceeding. ## Related resource * [Cloudflare Blog: Keep AI interactions secure and risk-free with Guardrails in AI Gateway ↗](https://blog.cloudflare.com/guardrails-in-ai-gateway/) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"WebPage","@id":"https://developers.cloudflare.com/ai-gateway/features/guardrails/#page","headline":"Guardrails · Cloudflare AI Gateway docs","description":"Evaluate AI Gateway prompts and responses for harmful content and enforce safety policies across providers.","url":"https://developers.cloudflare.com/ai-gateway/features/guardrails/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-06-05","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"},"keywords":["AI"]} ``` --- --- description: Enable and configure AI Gateway Guardrails to flag or block harmful content in prompts and responses. title: Set up Guardrails image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Set up Guardrails Last updated Apr 20, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/features/guardrails/set-up-guardrail/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Add Guardrails to any gateway to start evaluating and potentially modifying responses. 1. Log into the [Cloudflare dashboard ↗](https://dash.cloudflare.com/) and select your account. 2. Go to **AI** \> **AI Gateway**. 3. Select a gateway. 4. Go to **Guardrails**. 5. Switch the toggle to **On**. 6. To customize categories, select **Change** \> **Configure specific categories**. 7. Update your choices for how Guardrails works on specific prompts or responses (**Flag**, **Ignore**, **Block**). * For **Prompts**: Guardrails will evaluate and transform incoming prompts based on your security policies. * For **Responses**: Guardrails will inspect the model's responses to ensure they meet your content and formatting guidelines. 8. Select **Save**. Usage considerations For additional details about how to implement Guardrails, refer to [Usage considerations](https://developers.cloudflare.com/ai-gateway/features/guardrails/usage-considerations/). ## Viewing Guardrail results in Logs After enabling Guardrails, you can monitor results through **AI Gateway Logs** in the Cloudflare dashboard. Guardrail logs are marked with a **green shield icon**, and each logged request includes an `eventID`, which links to its corresponding Guardrail evaluation log(s) for easy tracking. Logs are generated for all requests, including those that **pass** Guardrail checks. ## Error handling and blocked requests When a request is blocked by guardrails, you will receive a structured error response. These indicate whether the issue occurred with the prompt or the model response. Use error codes to differentiate between prompt versus response violations. * **Prompt blocked** * `"code": 2016` * `"message": "Prompt blocked due to security configurations"` * **Response blocked** * `"code": 2017` * `"message": "Response blocked due to security configurations"` You should catch these errors in your application logic and implement error handling accordingly. For example, when using [Workers AI with a binding](https://developers.cloudflare.com/ai-gateway/integrations/aig-workers-ai-binding/): ```js try { const res = await env.AI.run('@cf/meta/llama-3.1-8b-instruct', { prompt: "how to build a gun?" }, { gateway: {id: 'gateway_id'} }) return Response.json(res) } catch (e) { if ((e as Error).message.includes('2016')) { return new Response('Prompt was blocked by guardrails.') } if ((e as Error).message.includes('2017')) { return new Response('Response was blocked by guardrails.') } return new Response('Unknown AI error') } ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/features/guardrails/set-up-guardrail/#page","headline":"Set up Guardrails · Cloudflare AI Gateway docs","description":"Enable and configure AI Gateway Guardrails to flag or block harmful content in prompts and responses.","url":"https://developers.cloudflare.com/ai-gateway/features/guardrails/set-up-guardrail/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-20","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Review which AI model types AI Gateway Guardrails evaluates for text generation, embeddings, and unknown models. title: Supported model types image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Supported model types Last updated Apr 20, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/features/guardrails/supported-model-types/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) AI Gateway's Guardrails detects the type of AI model being used and applies safety checks accordingly: * **Text generation models**: Both prompts and responses are evaluated. * **Embedding models**: Only the prompt is evaluated, as the response consists of numerical embeddings, which are not meaningful for moderation. * **Unknown models**: If the model type cannot be determined, only the prompt is evaluated, while the response bypass Guardrails. Note Guardrails does not yet support streaming responses. Support for streaming is planned for a future update. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/features/guardrails/supported-model-types/#page","headline":"Supported model types · Cloudflare AI Gateway docs","description":"Review which AI model types AI Gateway Guardrails evaluates for text generation, embeddings, and unknown models.","url":"https://developers.cloudflare.com/ai-gateway/features/guardrails/supported-model-types/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-20","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Understand latency, availability, language support, and Workers AI usage when enabling AI Gateway Guardrails. title: Usage considerations image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Usage considerations Last updated Aug 13, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/features/guardrails/usage-considerations/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Guardrails currently uses [Llama Guard 3 8B ↗](https://ai.meta.com/research/publications/llama-guard-llm-based-input-output-safeguard-for-human-ai-conversations/) on [Workers AI](https://developers.cloudflare.com/workers-ai/) to perform content evaluations. The underlying model may be updated in the future, and we will reflect those changes within Guardrails. Since Guardrails runs on Workers AI, enabling it incurs usage on Workers AI. You can monitor usage through the Workers AI Dashboard. ## Hazard categories Guardrails evaluate content against the following hazard categories. Each category is identified by a code that appears in your Guardrail configuration and in AI Gateway Logs. You can independently set each category to **Flag**, **Ignore**, or **Block** for prompts and responses. Guardrails evaluate categories `S1` through `S13`, a subset of the [Llama Guard 3 ↗ ↗](https://ai.meta.com/research/publications/llama-guard-llm-based-input-output-safeguard-for-human-ai-conversations/) hazard categories, using the [@cf/meta/llama-guard-3-8b](https://developers.cloudflare.com/workers-ai/models/llama-guard-3-8b/) model on Workers AI. The Llama Guard 3 category `S14` (Code interpreter abuse) is not evaluated by Guardrails. Category `P1` is prompt injection, evaluated separately by the `@cf/meta/prompt-guard-2-86m` model. These codes also appear in the `guardrails` property of the AI Gateway REST API, where you configure each category's action programmatically. See the [create](https://developers.cloudflare.com/api/resources/ai%5Fgateway/methods/create/) and [update](https://developers.cloudflare.com/api/resources/ai%5Fgateway/methods/update/) methods. | Code | Category | | ---- | ------------------------- | | S1 | Violent Crimes | | S2 | Non-Violent Crimes | | S3 | Sex-Related Crimes | | S4 | Child Sexual Exploitation | | S5 | Defamation | | S6 | Specialized Advice | | S7 | Privacy | | S8 | Intellectual Property | | S9 | Indiscriminate Weapons | | S10 | Hate | | S11 | Suicide & Self-Harm | | S12 | Sexual Content | | S13 | Elections | | P1 | Prompt Injection | ## Additional considerations * **Model availability**: If at least one hazard category is set to `block`, but AI Gateway is unable to receive a response from Workers AI, the request will be blocked. Conversely, if a hazard category is set to `flag` and AI Gateway cannot obtain a response from Workers AI, the request will proceed without evaluation. This approach prioritizes availability, allowing requests to continue even when content evaluation is not possible. * **Latency impact**: Enabling Guardrails introduces additional latency to requests. Typically, evaluations using Llama Guard 3 8B on Workers AI add approximately 500 milliseconds per request. However, larger requests may experience increased latency, though this increase is not linear. Consider this when balancing safety and performance. * **Handling long content**: When evaluating long prompts or responses, Guardrails automatically segments the content into smaller chunks, processing each through separate Guardrail requests. This approach ensures comprehensive moderation but may result in increased latency for longer inputs. * **Supported languages**: Llama Guard 3.3 8B supports content safety classification in the following languages: English, French, German, Hindi, Italian, Portuguese, Spanish, and Thai. * **Streaming support**: Streaming is not supported when using Guardrails. Note Llama Guard is provided as-is without any representations, warranties, or guarantees. Any rules or examples contained in blogs, developer docs, or other reference materials are provided for informational purposes only. You acknowledge and understand that you are responsible for the results and outcomes of your use of AI Gateway. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/features/guardrails/usage-considerations/#page","headline":"Usage considerations · Cloudflare AI Gateway docs","description":"Understand latency, availability, language support, and Workers AI usage when enabling AI Gateway Guardrails.","url":"https://developers.cloudflare.com/ai-gateway/features/guardrails/usage-considerations/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-08-13","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Control traffic to your AI Gateway with fixed or sliding rate limits to prevent excessive costs and suspicious activity. title: Rate limiting image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Rate limiting Last updated Jun 5, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/features/rate-limiting/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Rate limiting controls the traffic that reaches your application, which prevents expensive bills and suspicious activity. ## Parameters You can define rate limits as the number of requests that get sent in a specific time frame. For example, you can limit your application to 100 requests per 60 seconds. You can also select if you would like a **fixed** or **sliding** rate limiting technique. With rate limiting, we allow a certain number of requests within a window of time. For example, if it is a fixed rate, the window is based on time, so there would be no more than `x` requests in a ten minute window. If it is a sliding rate, there would be no more than `x` requests in the last ten minutes. To illustrate this, let us say you had a limit of ten requests per ten minutes, starting at 12:00\. So the fixed window is 12:00-12:10, 12:10-12:20, and so on. If you sent ten requests at 12:09 and ten requests at 12:11, all 20 requests would be successful in a fixed window strategy. However, they would fail in a sliding window strategy since there were more than ten requests in the last ten minutes. ## Handling rate limits When your requests exceed the allowed rate, you will encounter rate limiting. This means the server will respond with a `429 Too Many Requests` status code and your request will not be processed. ## Default configuration To set the default rate limiting configuration in the dashboard: 1. Log into the [Cloudflare dashboard ↗](https://dash.cloudflare.com/) and select your account. 2. Go to **AI** \> **AI Gateway**. 3. Go to **Settings**. 4. Enable **Rate-limiting**. 5. Adjust the rate, time period, and rate limiting method as desired. To set the default rate limiting configuration using the API: 1. [Create an API token](https://developers.cloudflare.com/fundamentals/api/get-started/create-token/) with the following permissions: * `AI Gateway - Read` * `AI Gateway - Edit` 1. Get your [Account ID](https://developers.cloudflare.com/fundamentals/account/find-account-and-zone-ids/). 2. Using that API token and Account ID, send a [POST request](https://developers.cloudflare.com/api/resources/ai%5Fgateway/methods/create/) to create a new Gateway and include a value for the `rate_limiting_interval`, `rate_limiting_limit`, and `rate_limiting_technique`. This rate limiting behavior will be uniformly applied to all requests for that gateway. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/features/rate-limiting/#page","headline":"Rate limiting · Cloudflare AI Gateway docs","description":"Control traffic to your AI Gateway with fixed or sliding rate limits to prevent excessive costs and suspicious activity.","url":"https://developers.cloudflare.com/ai-gateway/features/rate-limiting/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-06-05","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Set cost-based budgets on your AI Gateway to control spending by model, provider, or custom metadata dimensions like user or team. title: Spend limits image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Spend limits Last updated Aug 17, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/features/spend-limits/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Spend limits let you set cost-based budgets on your AI Gateway. When cumulative spend reaches the limit within a time window, AI Gateway blocks further requests with a `429` response until the window resets. Unlike [rate limiting](https://developers.cloudflare.com/ai-gateway/features/rate-limiting/), which caps the number of requests, spend limits track actual dollar cost per request based on model pricing. You can scope limits to any combination of model, provider, or [custom metadata](https://developers.cloudflare.com/ai-gateway/observability/custom-metadata/) dimensions like user ID, team, or application. Spend limits apply to both [Unified Billing](https://developers.cloudflare.com/ai-gateway/features/unified-billing/) requests and [BYOK](https://developers.cloudflare.com/ai-gateway/configuration/bring-your-own-keys/) requests for models with known pricing. ![Spend limits rules configured on a gateway](https://developers.cloudflare.com/cdn-cgi/image/onerror=redirect,width=2364,height=728,format=webp/_astro/spend-limits-rules.p6zy0Vea.png) ## How it works Each spend limit rule defines a budget (in dollars) over a rolling or fixed time window. AI Gateway calculates the cost of each request based on token usage and model pricing, then tracks cumulative spend against the limit in real time. Before sending a request to the provider, AI Gateway evaluates all applicable spend limit rules at once. If any individual rule is over budget, the request is blocked with a `429` response. Spend limits are eventually consistent. The current request's cost is recorded after completion, so a burst of concurrent requests can briefly exceed the limit before enforcement catches up. ## Scoping with dimensions Each rule can be scoped by one or more dimensions: * **Limit by provider** — the provider used for the request. * **Limit by model** — the model used for the request. * **Limit by metadata** — a [custom metadata](https://developers.cloudflare.com/ai-gateway/observability/custom-metadata/) key you attach to requests. Enter the metadata key name (for example, `agent_id` or `environment`). Each dimension can be configured in one of two modes: | Mode | Behavior | Example | | ------------------- | ----------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- | | **Split by value** | Each distinct value gets its own independent budget bucket. | For example, if you pass in agent\_id, splitting by agent\_id gives every agent its own budget. | | **Filter by value** | The rule applies only when the dimension equals a specific value. | For example, if you pass in agent\_id, filtering agent\_id to agent\_42 limits only that agent's requests. | If a dimension is not configured on a rule, all values share one budget bucket. For example, a rule without a `provider` dimension tracks spend across all providers together. ### Dimension examples Given a request with model `openai/gpt-5.5` and an `agent_id` metadata value of `agent_42`: | Scenario | Dimensions | Budget bucket | | -------------------------- | ------------------------------------------------------------ | ---------------------------------------------- | | Global budget for everyone | None | One shared bucket | | Per-agent budget | agent\_id metadata: split by value | Separate bucket per agent | | Per-provider, per-agent | agent\_id metadata: split by value, provider: split by value | Separate bucket per agent+provider combination | | Specific model only | model: filter by value openai/gpt-5.5 | Only applies to openai/gpt-5.5 requests | | Per-agent, per-model | agent\_id metadata: split by value, model: split by value | Separate bucket per agent+model combination | ## Configure spend limits Spend limits are configured on the gateway via the dashboard or the API. You can define up to 20 rules per gateway. ![Add spend limit rule form](https://developers.cloudflare.com/cdn-cgi/image/onerror=redirect,width=1350,height=1316,format=webp/_astro/spend-limits-add-rule.BnBR5VIn.png) To scope spend limits by custom dimensions like user ID or team, attach [custom metadata](https://developers.cloudflare.com/ai-gateway/observability/custom-metadata/) to your requests. ### Set spend limits by user You can give every user their own budget by scoping a rule to a user identifier. How you get that identifier depends on how your gateway is authenticated. #### With Cloudflare Access If your gateway is protected by [Cloudflare Access](https://developers.cloudflare.com/ai-gateway/configuration/cloudflare-access/), AI Gateway automatically adds the authenticated Access user ID to each request as the reserved [cf.user\_id](https://developers.cloudflare.com/ai-gateway/observability/custom-metadata/#reserved-metadata) metadata key. You do not need to pass user IDs from your client application. To set a per-user budget: 1. In the [Cloudflare dashboard ↗](https://dash.cloudflare.com/), go to **AI** \> **AI Gateway** and select your gateway. 2. Go to the spend limits settings and add a rule. 3. Under **Limit by metadata**, select **Add metadata dimension** and enter `cf.user_id` as the key. 4. Set the dimension to **Split by value**. 5. Set the budget amount and time window, then save. Each authenticated Access user now gets an independent budget. To instead limit a single user, set the dimension to **Filter by value** and enter that user's Access JWT `sub` claim. Note `cf.user_id` is only present on requests that arrive through an Access-protected [custom domain](https://developers.cloudflare.com/ai-gateway/configuration/custom-domains/) with a valid Access user subject. Service-token requests do not include `cf.user_id`. #### Without Cloudflare Access If your gateway is not behind Access, pass your own user identifier as [custom metadata](https://developers.cloudflare.com/ai-gateway/observability/custom-metadata/) (for example, a `user_id` key). Then, under **Limit by metadata**, add a dimension with the key `user_id` and set it to **Split by value**. ## Behavior when a limit is reached When a spend limit is exceeded, AI Gateway returns a `429 Too Many Requests` response. You have two options: * **Block requests** (default) - The request is rejected until the budget window resets. * **Fall back to a cheaper model** \- Create a [Dynamic Route](https://developers.cloudflare.com/ai-gateway/features/dynamic-routing/) with a primary model and a fallback (for example, `anthropic/claude-opus-4.7` with a fallback to `@cf/moonshotai/kimi-k2.6`). Then set a spend limit on the primary model using this feature. When the primary model's budget is exceeded, AI Gateway automatically routes requests to the fallback model instead of blocking them. ## Monitoring spend You can track your spend per model, provider, or any custom metadata attribute on the [Analytics dashboard](https://developers.cloudflare.com/ai-gateway/observability/analytics/). Use this to understand usage patterns and set informed budgets. ## Limitations * Cost tracking is a best-effort estimation based on token counts and model pricing. Refer to your provider's dashboard for exact billing amounts. * A maximum of 20 spend limit rules can be configured per gateway. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/features/spend-limits/#page","headline":"Spend limits · Cloudflare AI Gateway docs","description":"Set cost-based budgets on your AI Gateway to control spending by model, provider, or custom metadata dimensions like user or team.","url":"https://developers.cloudflare.com/ai-gateway/features/spend-limits/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-08-17","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Use the Cloudflare billing to pay for and authenticate your inference requests. title: Unified Billing image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Unified Billing Last updated Aug 7, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/features/unified-billing/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Unified Billing allows users to call Workers AI and connect to various AI providers (such as OpenAI, Anthropic, and Google AI Studio) and receive a single Cloudflare bill. To use Unified Billing, you must purchase and load credits into your Cloudflare account in the Cloudflare dashboard, which you can then spend with AI Gateway. A 5% fee is applied to all credits purchased through Unified Billing. For example, a $100 credit purchase will result in a $105 charge. Inference pricing from providers is passed through with no markup — you pay the same per-token rates as you would directly with the provider. Caution In rare instances, your credit balance may go negative. If this happens, Cloudflare will charge the payment method on file for the outstanding amount. Charges occur at the beginning of each month for the previous month. ## Pre-requisites * Ensure your Cloudflare account has [sufficient credits loaded](#load-credits). * Ensure you have [authenticated](https://developers.cloudflare.com/ai-gateway/configuration/authentication/) your AI Gateway. * To use credits for Workers AI, set your gateway's **Workers AI Billing** setting to **Unified billing**. ## Load credits To load credits for AI Gateway: 1. In the Cloudflare dashboard, go to the **AI Gateway** page. [Go to **AI Gateway** ↗](https://dash.cloudflare.com/?to=/:account/ai/ai-gateway) The **Credits Available** card on the top right shows how many AI gateway credits you have on your account currently. 2. In **Credits Available**, select **Manage**. 3. If your account does not have an available payment method, AI Gateway will prompt you to add a payment method to purchase credits. Add a payment method. 4. Select **Top-up credits**. 5. Add the amount of credits you want to purchase, then select **Confirm and pay**. ### Auto-top up You can configure AI Gateway to automatically replenish your credits when they fall below a certain threshold. To configure auto top-up: 1. In the Cloudflare dashboard, go to the **AI Gateway** page. [Go to **AI Gateway** ↗](https://dash.cloudflare.com/?to=/:account/ai/ai-gateway) 2. In **Credits Available**, select **Manage**. 3. Select **Setup auto top-up credits**. 4. Choose a threshold and a recharge amount for auto top-up. When your balance falls below the set threshold, AI Gateway will automatically apply the auto top-up amount to your account. ## Credential precedence When a request reaches AI Gateway, credentials are resolved in this order: 1. **Provider key on the request** — if the request carries provider authentication (for example, an `Authorization` header), AI Gateway forwards it to the provider unchanged. BYOK and Unified Billing are not consulted. 2. **BYOK (stored key)** — if no provider key is on the request and the gateway has a [stored key](https://developers.cloudflare.com/ai-gateway/configuration/bring-your-own-keys/) for the provider under the `default` alias, that key is used. 3. **Unified Billing** — if neither of the above applies, the request is served with Cloudflare-managed credentials and billed against your Cloudflare credit balance. Note On requests routed through Unified Billing endpoints (for example, `env.AI.run()` or `/ai/v1/chat/completions`), only the BYOK key stored under the `default` alias prevents fall-through to Unified Billing. Keys stored under other aliases are not consulted on this path — a request will fall through to Unified Billing even if you have a key stored under, for example, `production` or `testing`. The `cf-aig-byok-alias` header selects a non-default alias only on [direct provider-passthrough](https://developers.cloudflare.com/ai-gateway/usage/providers/) requests. ## Use Unified Billing Unified Billing works in two ways: through the AI binding or through the HTTP API. Both deduct credits from your account automatically without requiring provider API keys. To use credits for Workers AI, [configure the gateway's Workers AI billing setting](https://developers.cloudflare.com/ai-gateway/configuration/manage-gateway/#configure-workers-ai-billing) as **Unified billing**. Workers AI requests routed through that gateway deduct from your prepaid credit balance in real time. In the AI binding, include the gateway ID in the third argument to `env.AI.run()`. For REST API requests, include the `cf-aig-gateway-id` header. Prepaid credits provide access to Workers AI models that otherwise require the Workers Paid plan and provide [higher rate limits for frontier models](https://developers.cloudflare.com/workers-ai/platform/limits/#frontier-models). ### AI binding Call any model listed in the [model catalog](https://developers.cloudflare.com/ai/models/) using `env.AI.run()`. This includes both Workers AI models and third-party models from providers like OpenAI, Anthropic, and Google. ```typescript const resp = await env.AI.run( "openai/gpt-4.1-mini", { messages: [{ role: "user", content: "What is Cloudflare?" }], }, { gateway: { id: "my-gateway" }, }, ); ``` Refer to the [binding reference](https://developers.cloudflare.com/ai-gateway/usage/worker-binding-methods/) for the full API surface. ### HTTP API Call a supported provider through the AI Gateway REST API without passing a provider API key. #### REST API Use the Cloudflare API to call third-party models. Pass your Cloudflare API token in the `Authorization` header: ```bash # Run `wrangler whoami` to get your account ID to replace $CLOUDFLARE_ACCOUNT_ID, # and `wrangler auth token` to get an auth token to replace $CLOUDFLARE_API_TOKEN. curl -X POST "https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/v1/chat/completions" \ --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ --header "Content-Type: application/json" \ --data '{ "model": "openai/gpt-4.1-mini", "messages": [{"role": "user", "content": "What is Cloudflare?"}] }' ``` Refer to [REST API](https://developers.cloudflare.com/ai-gateway/usage/rest-api/) for more details on all available endpoints. #### AI Gateway provider-native endpoints You can also call providers directly through [provider-native endpoints](https://developers.cloudflare.com/ai-gateway/usage/providers/) using the `cf-aig-authorization` header to authenticate: The HTTP API supports the following providers: * [OpenAI](https://developers.cloudflare.com/ai-gateway/usage/providers/openai/) * [Anthropic](https://developers.cloudflare.com/ai-gateway/usage/providers/anthropic/) * [Google AI Studio](https://developers.cloudflare.com/ai-gateway/usage/providers/google-ai-studio/) * [Google Vertex AI](https://developers.cloudflare.com/ai-gateway/usage/providers/vertex/) * [xAI](https://developers.cloudflare.com/ai-gateway/usage/providers/grok/) * [Groq](https://developers.cloudflare.com/ai-gateway/usage/providers/groq/) ### Spend limits Set [spend limit rules](https://developers.cloudflare.com/ai-gateway/features/spend-limits/) on individual gateways to cap spend, scoped by model, provider, or custom metadata dimensions like user or team. ### Zero Data Retention (ZDR) Zero Data Retention (ZDR) routes Unified Billing traffic through provider endpoints that do not retain prompts or responses. Enable it with the gateway-level `zdr` setting, which maps to ZDR-capable upstream provider configurations. This setting only applies to Unified Billing requests that use Cloudflare-managed credentials. It does not apply to BYOK or other AI Gateway requests. ZDR does not control AI Gateway logging. To disable request/response logging in AI Gateway, update the logging settings separately in [Logging](https://developers.cloudflare.com/ai-gateway/observability/logging/). ZDR is currently supported for: * [OpenAI](https://developers.cloudflare.com/ai-gateway/usage/providers/openai/) * [Anthropic](https://developers.cloudflare.com/ai-gateway/usage/providers/anthropic/) If ZDR is enabled for a provider that does not support it, AI Gateway falls back to the standard (non-ZDR) Unified Billing configuration. #### Default configuration To set ZDR as the default for Unified Billing in the dashboard: 1. Log into the [Cloudflare dashboard ↗](https://dash.cloudflare.com/) and select your account. 2. Go to **AI** \> **AI Gateway**. 3. Select your gateway. 4. Go to **Settings** and toggle **Zero Data Retention (ZDR)**. To set ZDR as the default for Unified Billing using the API: 1. [Create an API token](https://developers.cloudflare.com/fundamentals/api/get-started/create-token/) with the following permissions: * `AI Gateway - Read` * `AI Gateway - Edit` 2. Get your [Account ID](https://developers.cloudflare.com/fundamentals/account/find-account-and-zone-ids/). 3. Send a [PUT request](https://developers.cloudflare.com/api/resources/ai%5Fgateway/methods/update/) to update the gateway and include `zdr: true` or `zdr: false` in the request body. #### Per-request override (`cf-aig-zdr`) Use the `cf-aig-zdr` header to override the gateway default for a single Unified Billing request. Set it to `true` to force ZDR, or `false` to disable ZDR for the request. ```bash # Run `wrangler whoami` to get your account ID to replace $CLOUDFLARE_ACCOUNT_ID, # and `wrangler auth token` to get an auth token to replace $CLOUDFLARE_API_TOKEN. curl -X POST "https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/v1/chat/completions" \ --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ --header "Content-Type: application/json" \ --header "cf-aig-zdr: true" \ --data '{ "model": "openai/gpt-4.1-mini", "messages": [ { "role": "user", "content": "Explain Zero Data Retention." } ] }' ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/features/unified-billing/#page","headline":"Unified Billing · Cloudflare AI Gateway docs","description":"Use the Cloudflare billing to pay for and authenticate your inference requests.","url":"https://developers.cloudflare.com/ai-gateway/features/unified-billing/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-08-07","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Connect AI Gateway with Workers bindings, Vercel AI SDK, and other platforms. title: Integrations image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Integrations Last updated Apr 20, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/integrations/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/integrations/#page","headline":"Integrations · Cloudflare AI Gateway docs","description":"Connect AI Gateway with Workers bindings, Vercel AI SDK, and other platforms.","url":"https://developers.cloudflare.com/ai-gateway/integrations/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-20","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Create stateful AI agents with persistent memory, real-time WebSocket connections, and scheduled tasks using the Cloudflare Agents SDK. title: Build Agents on Cloudflare image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/agents/llms.txt > Use this file to discover all available pages before exploring further. # Build Agents on Cloudflare Last updated Jun 24, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/agents/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Build and host Agents on Cloudflare, connect chat, voice, email, Slack, and webhooks to a durable agent runtime with Browser, Sandbox, AI Search, MCP, Payments, and other MCP tools. When you host agents on Cloudflare, each agent session has a durable identity, local SQL storage, real-time connections, scheduled work, and recoverable execution. Deploy once and Cloudflare runs your agents across its global network, scaling to tens of millions of instances. No infrastructure to manage, no sessions to reconstruct, no state to externalize. [Chat](https://developers.cloudflare.com/agents/communication-channels/chat/)[Email](https://developers.cloudflare.com/agents/communication-channels/email/)[Voice](https://developers.cloudflare.com/agents/communication-channels/voice/)[Slack](https://developers.cloudflare.com/agents/communication-channels/slack/)[Webhook](https://developers.cloudflare.com/agents/communication-channels/webhooks/) Agent harness Controls planning, tool use, and response flow. [Project Think](https://developers.cloudflare.com/agents/harnesses/think/)[Build-your-own agent](https://developers.cloudflare.com/agents/runtime/agents-api/) Agents SDK runtime Durable identity, state, connections, scheduling, and recovery. [Agent class](https://developers.cloudflare.com/agents/runtime/agents-api/) [State](https://developers.cloudflare.com/agents/runtime/lifecycle/state/)[Sessions](https://developers.cloudflare.com/agents/runtime/lifecycle/sessions/)[Routing](https://developers.cloudflare.com/agents/runtime/communication/routing/)[WebSockets](https://developers.cloudflare.com/agents/runtime/communication/websockets/)[Scheduling](https://developers.cloudflare.com/agents/runtime/execution/schedule-tasks/)[Fibers](https://developers.cloudflare.com/agents/runtime/execution/durable-execution/) [Sandbox](https://developers.cloudflare.com/agents/tools/sandbox/)[MCP](https://developers.cloudflare.com/agents/tools/mcp/)[Browser](https://developers.cloudflare.com/agents/tools/browser/)[AI Search](https://developers.cloudflare.com/agents/tools/ai-search/)[Payments](https://developers.cloudflare.com/agents/tools/payments/) [ObservabilityLogs · metrics · traces](https://developers.cloudflare.com/agents/runtime/operations/observability/) Agents on Cloudflare are composed from four parts: * **Communication channels** define how users and systems reach your agent, such as [chat](https://developers.cloudflare.com/agents/communication-channels/chat/), [voice](https://developers.cloudflare.com/agents/communication-channels/voice/), [email](https://developers.cloudflare.com/agents/communication-channels/email/), [Slack](https://developers.cloudflare.com/agents/communication-channels/slack/), [webhooks](https://developers.cloudflare.com/agents/communication-channels/webhooks/), and other event sources. * **The agent harness** defines the loop: how the agent calls models, selects tools, handles tool results, streams responses, and decides whether to continue. Use [Project Think](https://developers.cloudflare.com/agents/harnesses/think/) for an opinionated harness, or build your own loop directly on the [Agents SDK runtime](https://developers.cloudflare.com/agents/runtime/agents-api/). * **The Agents SDK runtime** provides durable infrastructure: the [Agent class](https://developers.cloudflare.com/agents/runtime/lifecycle/agent-class/), [state](https://developers.cloudflare.com/agents/runtime/lifecycle/state/), [sessions](https://developers.cloudflare.com/agents/runtime/lifecycle/sessions/), [routing](https://developers.cloudflare.com/agents/runtime/communication/routing/), [WebSockets](https://developers.cloudflare.com/agents/runtime/communication/websockets/), [scheduling](https://developers.cloudflare.com/agents/runtime/execution/schedule-tasks/), [fibers](https://developers.cloudflare.com/agents/runtime/execution/durable-execution/), and [observability](https://developers.cloudflare.com/agents/runtime/operations/observability/). * **Tools** give the agent capabilities: [browser automation](https://developers.cloudflare.com/agents/tools/browser/), [sandboxed code execution](https://developers.cloudflare.com/agents/tools/sandbox/), [AI Search](https://developers.cloudflare.com/agents/tools/ai-search/), [MCP tools](https://developers.cloudflare.com/agents/tools/mcp/), and [payments](https://developers.cloudflare.com/agents/tools/payments/). [Code Mode](https://developers.cloudflare.com/agents/tools/codemode/) lets models discover and orchestrate multiple tools by writing code. ### Get started Three commands to a running agent. No API keys required — the starter uses [Workers AI](https://developers.cloudflare.com/workers-ai/) by default. ```sh npx create-cloudflare@latest --template cloudflare/agents-starter cd agents-starter && npm install npm run dev ``` The starter includes streaming AI chat, server-side and client-side tools, human-in-the-loop approval, and task scheduling — a foundation you can build on or tear apart. You can also swap in [OpenAI, Anthropic, Google Gemini, or any other provider](https://developers.cloudflare.com/agents/runtime/operations/using-ai-models/). ### Example agents [Chat agent](https://developers.cloudflare.com/agents/examples/chat-agent/) Build a streaming AI chat agent with tools and human-in-the-loop approvals. [Slack agent](https://developers.cloudflare.com/agents/examples/slack-agent/) Build an agent that responds to Slack messages, mentions, and commands. [Voice agent](https://developers.cloudflare.com/agents/examples/voice-agent/) Build a real-time voice agent with speech-to-text and text-to-speech. [Browser agent](https://developers.cloudflare.com/agents/examples/browser-agent/) Build an agent that can inspect pages, capture screenshots, and use browser tools. [Email agent](https://developers.cloudflare.com/agents/examples/email-agent/) Build an agent that sends, receives, routes, and replies to email. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"WebPage","@id":"https://developers.cloudflare.com/agents/#page","headline":"Agents · Cloudflare Agents docs","description":"Create stateful AI agents with persistent memory, real-time WebSocket connections, and scheduled tasks using the Cloudflare Agents SDK.","url":"https://developers.cloudflare.com/agents/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-06-24","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"},"keywords":["AI"]} ``` --- --- description: This guide will walk you through setting up and deploying a Workers AI project. You will use Workers, an AI Gateway binding, and a large language model (LLM) to deploy your first AI-powered application on the Cloudflare global network. title: Set up Workers AI with AI Gateway image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Set up Workers AI with AI Gateway Last updated Jun 12, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/integrations/aig-workers-ai-binding/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) This guide will walk you through setting up and deploying a Workers AI project. You will use [Workers](https://developers.cloudflare.com/workers/), an AI Gateway binding, and a large language model (LLM), to deploy your first AI-powered application on the Cloudflare global network. ## Prerequisites 1. Sign up for a [Cloudflare account ↗](https://dash.cloudflare.com/sign-up/workers-and-pages). 2. Install [Node.js ↗](https://docs.npmjs.com/downloading-and-installing-node-js-and-npm). Node.js version manager Use a Node version manager like [Volta ↗](https://volta.sh/) or [nvm ↗](https://github.com/nvm-sh/nvm) to avoid permission issues and change Node.js versions. [Wrangler](https://developers.cloudflare.com/workers/wrangler/install-and-update/), discussed later in this guide, requires a Node version of `16.17.0` or later. ## 1\. Create a Worker Project You will create a new Worker project using the create-Cloudflare CLI (C3). C3 is a command-line tool designed to help you set up and deploy new applications to Cloudflare. Create a new project named `hello-ai` by running: npmyarnpnpm ``` npm create cloudflare@latest -- hello-ai ``` ``` yarn create cloudflare hello-ai ``` ``` pnpm create cloudflare@latest hello-ai ``` Running `npm create cloudflare@latest` will prompt you to install the create-cloudflare package and lead you through setup. C3 will also install [Wrangler](https://developers.cloudflare.com/workers/wrangler/), the Cloudflare Developer Platform CLI. For setup, select the following options: * For _What would you like to start with?_, choose `Hello World example`. * For _Which template would you like to use?_, choose `Worker only`. * For _Which language do you want to use?_, choose `TypeScript`. * For _Do you want to use git for version control?_, choose `Yes`. * For _Do you want to deploy your application?_, choose `No` (we will be making some changes before deploying). This will create a new `hello-ai` directory. Your new `hello-ai` directory will include: * A "Hello World" Worker at `src/index.ts`. * A [Wrangler configuration file](https://developers.cloudflare.com/workers/wrangler/configuration/) Go to your application directory: ```bash cd hello-ai ``` ## 2\. Connect your Worker to Workers AI You must create an AI binding for your Worker to connect to Workers AI. Bindings allow your Workers to interact with resources, like Workers AI, on the Cloudflare Developer Platform. To bind Workers AI to your Worker, add the following to the end of your [Wrangler configuration file](https://developers.cloudflare.com/workers/wrangler/configuration/): ```jsonc { "ai": { "binding": "AI", }, } ``` ```toml [ai] binding = "AI" ``` Your binding is [available in your Worker code](https://developers.cloudflare.com/workers/reference/migrate-to-module-workers/#bindings-in-es-modules-format) on [env.AI](https://developers.cloudflare.com/workers/runtime-apis/handlers/fetch/). You can use `"default"` as the gateway ID in the next step. AI Gateway automatically creates a default gateway on the first authenticated request. Alternatively, you can [create a gateway manually](https://developers.cloudflare.com/ai-gateway/get-started/) and use its ID. ## 3\. Run an inference task containing AI Gateway in your Worker You are now ready to run an inference task in your Worker. In this case, you will use an LLM, [llama-3.1-8b-instruct-fast](https://developers.cloudflare.com/workers-ai/models/llama-3.1-8b-instruct-fast/), to answer a question. Update the `index.ts` file in your `hello-ai` application directory with the following code: ```typescript export interface Env { // If you set another name in the [Wrangler configuration file](/workers/wrangler/configuration/) as the value for 'binding', // replace "AI" with the variable name you defined. AI: Ai; } export default { async fetch(request, env): Promise { // Specify the gateway label and other options here const response = await env.AI.run( "@cf/meta/llama-3.1-8b-instruct-fast", { prompt: "What is the origin of the phrase Hello, World", }, { gateway: { id: "default", // Uses the default gateway, or replace with your gateway ID skipCache: true, // Optional: Skip cache if needed }, }, ); // Return the AI response as a JSON object return new Response(JSON.stringify(response), { headers: { "Content-Type": "application/json" }, }); }, } satisfies ExportedHandler; ``` Up to this point, you have created an AI binding for your Worker and configured your Worker to be able to execute the Llama 3.1 model. You can now test your project locally before you deploy globally. ## 4\. Develop locally with Wrangler While in your project directory, test Workers AI locally by running [wrangler dev](https://developers.cloudflare.com/workers/wrangler/commands/general/#dev): ```bash npx wrangler dev ``` Workers AI local development usage charges Using Workers AI always accesses your Cloudflare account in order to run AI models and will incur usage charges even in local development. You will be prompted to log in after you run `wrangler dev`. When you run `npx wrangler dev`, Wrangler will give you a URL (most likely `localhost:8787`) to review your Worker. After you go to the URL Wrangler provides, you will see a message that resembles the following example: ```json { "response": "A fascinating question!\n\nThe phrase \"Hello, World!\" originates from a simple computer program written in the early days of programming. It is often attributed to Brian Kernighan, a Canadian computer scientist and a pioneer in the field of computer programming.\n\nIn the early 1970s, Kernighan, along with his colleague Dennis Ritchie, were working on the C programming language. They wanted to create a simple program that would output a message to the screen to demonstrate the basic structure of a program. They chose the phrase \"Hello, World!\" because it was a simple and recognizable message that would illustrate how a program could print text to the screen.\n\nThe exact code was written in the 5th edition of Kernighan and Ritchie's book \"The C Programming Language,\" published in 1988. The code, literally known as \"Hello, World!\" is as follows:\n\n main()\n {\n printf(\"Hello, World!\");\n }\n\nThis code is still often used as a starting point for learning programming languages, as it demonstrates how to output a simple message to the console.\n\nThe phrase \"Hello, World!\" has since become a catch-all phrase to indicate the start of a new program or a small test program, and is widely used in computer science and programming education.\n\nSincerely, I'm glad I could help clarify the origin of this iconic phrase for you!" } ``` ## 5\. Deploy your AI Worker Before deploying your AI Worker globally, log in with your Cloudflare account by running: ```bash npx wrangler login ``` You will be directed to a web page asking you to log in to the Cloudflare dashboard. After you have logged in, you will be asked if Wrangler can make changes to your Cloudflare account. Scroll down and select **Allow** to continue. Finally, deploy your Worker to make your project accessible on the Internet. To deploy your Worker, run: ```bash npx wrangler deploy ``` Once deployed, your Worker will be available at a URL like: ```bash https://hello-ai..workers.dev ``` Your Worker will be deployed to your custom [workers.dev](https://developers.cloudflare.com/workers/configuration/routing/workers-dev/) subdomain. You can now visit the URL to run your AI Worker. By completing this tutorial, you have created a Worker, connected it to Workers AI through an AI Gateway binding, and successfully ran an inference task using the Llama 3.1 model. ## Next steps * [Workers bindings](https://developers.cloudflare.com/ai-gateway/usage/worker-binding-methods/) — Call third-party models, access gateway methods, and integrate with AI SDKs. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/integrations/aig-workers-ai-binding/#page","headline":"Set up Workers AI with AI Gateway · Cloudflare AI Gateway docs","description":"This guide will walk you through setting up and deploying a Workers AI project. You will use Workers, an AI Gateway binding, and a large language model (LLM) to deploy your first AI-powered application on the Cloudflare global network.","url":"https://developers.cloudflare.com/ai-gateway/integrations/aig-workers-ai-binding/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-06-12","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Route Claude Code, Claude Desktop, GitHub Copilot CLI, OpenAI Codex, and Pi through AI Gateway for observability, caching, rate limiting, and cost tracking. title: Coding agents image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Coding agents Last updated Jul 2, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/integrations/coding-agents/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Coding agents send model requests to a provider on your behalf. By pointing the agent at AI Gateway instead of the provider, you observe and control that traffic without changing how you work. ## Why route a coding agent through AI Gateway Routing a coding agent through AI Gateway gives you: * **Observability** — view every request, token count, and latency in the dashboard. * **Caching** — return [cached responses](https://developers.cloudflare.com/ai-gateway/features/caching/) for repeated prompts. * **Rate limiting** — cap request volume with [rate limiting](https://developers.cloudflare.com/ai-gateway/features/rate-limiting/). * **Cost tracking** — attribute spend across sessions and models. * **Data Loss Prevention** — scan prompts and responses for secrets, credentials, and other sensitive data with [DLP](https://developers.cloudflare.com/ai-gateway/features/dlp/). ## Set up your agent Follow the setup guide for your coding agent: * [Claude Code](https://developers.cloudflare.com/ai-gateway/integrations/coding-agents/claude-code/) * [Claude Desktop](https://developers.cloudflare.com/ai-gateway/integrations/coding-agents/claude-desktop/) * [GitHub Copilot CLI](https://developers.cloudflare.com/ai-gateway/integrations/coding-agents/github-copilot-cli/) * [OpenAI Codex](https://developers.cloudflare.com/ai-gateway/integrations/coding-agents/openai-codex/) * [Pi](https://developers.cloudflare.com/ai-gateway/integrations/coding-agents/pi/) ## Protect sensitive code with DLP Coding agents routinely send source code, configuration files, and snippets to model providers. That traffic can include API keys, customer data, or other sensitive material. Because AI Gateway sits between the agent and the provider, you can inspect and control it without changing the agent. [Data Loss Prevention (DLP)](https://developers.cloudflare.com/ai-gateway/features/dlp/) scans request and response bodies against [detection profiles](https://developers.cloudflare.com/cloudflare-one/data-loss-prevention/dlp-profiles/) and either flags or blocks matches. Use it to catch secrets, credentials, or regulated data leaving (or returning to) the agent. Note Many coding agents stream responses by default. When DLP response scanning is enabled, AI Gateway buffers the full provider response before returning it, which increases time-to-first-token. If you need low-latency streaming, set the DLP policy **Check** to **Request** only, or use a separate gateway for latency-sensitive traffic. Refer to [streaming behavior](https://developers.cloudflare.com/ai-gateway/features/dlp/#streaming-behavior). ## Verify it works After you configure a tool, confirm that traffic reaches AI Gateway. 1. Send a prompt from the coding agent. 2. In the Cloudflare dashboard, go to the **AI Gateway** page. [Go to **AI Gateway** ↗](https://dash.cloudflare.com/?to=/:account/ai/ai-gateway) 3. Select your gateway, then select **Logs**. Confirm that the request appears with its model, token count, and latency. For more information on logs, refer to [Logging](https://developers.cloudflare.com/ai-gateway/observability/logging/). Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"WebPage","@id":"https://developers.cloudflare.com/ai-gateway/integrations/coding-agents/#page","headline":"Coding agents · Cloudflare AI Gateway docs","description":"Route Claude Code, Claude Desktop, GitHub Copilot CLI, OpenAI Codex, and Pi through AI Gateway for observability, caching, rate limiting, and cost tracking.","url":"https://developers.cloudflare.com/ai-gateway/integrations/coding-agents/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-07-02","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Route Claude Code through AI Gateway using your Cloudflare gateway token. title: Claude Code image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Claude Code Last updated Aug 17, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/integrations/coding-agents/claude-code/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) By pointing [Claude Code ↗](https://docs.anthropic.com/en/docs/claude-code/overview) at AI Gateway instead of a provider directly, you get observability, caching, rate limiting, and centralized credentials for Anthropic, Amazon Bedrock, or Google Vertex AI, without changing how you invoke `claude`. Claude Code reads its endpoint and credentials from environment variables. If your gateway is protected by Cloudflare Access, refer to [Use with Cloudflare Access](#use-with-cloudflare-access). This configuration sends requests to AI Gateway's [Anthropic endpoint](https://developers.cloudflare.com/ai-gateway/usage/providers/anthropic/), authenticated with your Cloudflare gateway token. The Anthropic endpoint exposes the same `/v1/messages` API that Claude Code expects. When AI Gateway supplies the Anthropic credentials for you — using either an Anthropic API key you [store as a provider key (BYOK)](https://developers.cloudflare.com/ai-gateway/configuration/bring-your-own-keys/) or [Unified Billing](https://developers.cloudflare.com/ai-gateway/features/unified-billing/) credits — the `ANTHROPIC_API_KEY` that Claude Code requires can be any placeholder value. ## Prerequisites Before you start, you need: * An [authenticated gateway](https://developers.cloudflare.com/ai-gateway/configuration/authentication/) and its [gateway token](https://developers.cloudflare.com/ai-gateway/configuration/authentication/#setting-up-authenticated-gateway-using-the-dashboard). The gateway token must have `Run` permissions. * Your Cloudflare account ID. To find it, refer to [Find your account and zone IDs](https://developers.cloudflare.com/fundamentals/account/find-account-and-zone-ids/). * Credentials for the provider you route to: * **Anthropic**: Either [Unified Billing](https://developers.cloudflare.com/ai-gateway/features/unified-billing/) credits loaded on your Cloudflare account (Cloudflare bills you), or your own Anthropic API key (Anthropic bills you). You can provide your own key either by storing it in AI Gateway as a [provider key (BYOK)](https://developers.cloudflare.com/ai-gateway/configuration/bring-your-own-keys/) or by passing it directly in `ANTHROPIC_API_KEY`. * **Amazon Bedrock** or **Google Vertex AI**: your provider credentials stored in AI Gateway as a [provider key (BYOK)](https://developers.cloudflare.com/ai-gateway/configuration/bring-your-own-keys/). * [Claude Code ↗](https://docs.anthropic.com/en/docs/claude-code/setup) installed and updated to the latest version. Note The `cf-aig-authorization` header is what authenticates your request to AI Gateway. When AI Gateway already holds the Anthropic credentials — through [Unified Billing](https://developers.cloudflare.com/ai-gateway/features/unified-billing/) or a [stored provider key (BYOK)](https://developers.cloudflare.com/ai-gateway/configuration/bring-your-own-keys/) — the `ANTHROPIC_API_KEY` value is ignored. Claude Code still requires the variable to be set, so you can set it to any value (this example reuses the gateway token). Otherwise, set `ANTHROPIC_API_KEY` to your own Anthropic API key and AI Gateway forwards it to Anthropic. AI Gateway provides observability, caching, and rate limiting for the traffic either way. For details on where Claude Code reads credentials from, refer to [Anthropic's authentication documentation ↗](https://docs.anthropic.com/en/docs/claude-code/iam#credential-management). 1. Set the base URL to your gateway's Anthropic endpoint and send your gateway token in the `cf-aig-authorization` header. Set `ANTHROPIC_API_KEY` to the same token, since Claude Code requires the variable to be set. The following commands set these as shell environment variables for the current session. To persist them, add them to your shell profile (for example, `~/.zshrc` or `~/.bashrc`) or to Claude Code's [settings.json ↗](https://docs.anthropic.com/en/docs/claude-code/settings#settings-files) under the `env` key. Replace ``, ``, and `` with your values. ```bash export ANTHROPIC_BASE_URL="https://gateway.ai.cloudflare.com/v1///anthropic" export ANTHROPIC_API_KEY="" export ANTHROPIC_CUSTOM_HEADERS="cf-aig-authorization: Bearer " ``` ```powershell $env:ANTHROPIC_BASE_URL = "https://gateway.ai.cloudflare.com/v1///anthropic" $env:ANTHROPIC_API_KEY = "" $env:ANTHROPIC_CUSTOM_HEADERS = "cf-aig-authorization: Bearer " ``` 2. Start Claude Code and send a prompt. Requests now route through AI Gateway. ```bash claude ``` ## Use Amazon Bedrock To run Claude models through [Amazon Bedrock](https://developers.cloudflare.com/ai-gateway/usage/providers/bedrock/) instead, point Claude Code at your gateway's Amazon Bedrock endpoint. AI Gateway authenticates to Bedrock with the AWS credentials you [store as a provider key](https://developers.cloudflare.com/ai-gateway/configuration/bring-your-own-keys/), so you can skip Claude Code's own AWS authentication. 1. Replace ``, ``, `` (for example, `us-east-1`), and `` with your values. ```bash export CLAUDE_CODE_USE_BEDROCK="1" export ANTHROPIC_BEDROCK_BASE_URL="https://gateway.ai.cloudflare.com/v1///aws-bedrock/bedrock-runtime//" export CLAUDE_CODE_SKIP_BEDROCK_AUTH="1" export ANTHROPIC_CUSTOM_HEADERS="cf-aig-authorization: Bearer " ``` ```powershell $env:CLAUDE_CODE_USE_BEDROCK = "1" $env:ANTHROPIC_BEDROCK_BASE_URL = "https://gateway.ai.cloudflare.com/v1///aws-bedrock/bedrock-runtime//" $env:CLAUDE_CODE_SKIP_BEDROCK_AUTH = "1" $env:ANTHROPIC_CUSTOM_HEADERS = "cf-aig-authorization: Bearer " ``` 2. Start Claude Code and send a prompt. Requests now route through AI Gateway to Amazon Bedrock. ```bash claude ``` ## Use Google Vertex AI To run Claude models through [Google Vertex AI](https://developers.cloudflare.com/ai-gateway/usage/providers/vertex/) instead, point Claude Code at your gateway's Google Vertex AI endpoint. AI Gateway authenticates to Vertex AI with the Google Cloud credentials you [store as a provider key](https://developers.cloudflare.com/ai-gateway/configuration/bring-your-own-keys/), so you can skip Claude Code's own Vertex authentication. 1. Replace ``, ``, ``, `` (for example, `us-east5`), and `` with your values. ```bash export CLAUDE_CODE_USE_VERTEX="1" export ANTHROPIC_VERTEX_BASE_URL="https://gateway.ai.cloudflare.com/v1///google-vertex-ai/v1" export ANTHROPIC_VERTEX_PROJECT_ID="" export CLOUD_ML_REGION="" export CLAUDE_CODE_SKIP_VERTEX_AUTH="1" export ANTHROPIC_CUSTOM_HEADERS="cf-aig-authorization: Bearer " ``` ```powershell $env:CLAUDE_CODE_USE_VERTEX = "1" $env:ANTHROPIC_VERTEX_BASE_URL = "https://gateway.ai.cloudflare.com/v1///google-vertex-ai/v1" $env:ANTHROPIC_VERTEX_PROJECT_ID = "" $env:CLOUD_ML_REGION = "" $env:CLAUDE_CODE_SKIP_VERTEX_AUTH = "1" $env:ANTHROPIC_CUSTOM_HEADERS = "cf-aig-authorization: Bearer " ``` 2. Start Claude Code and send a prompt. Requests now route through AI Gateway to Google Vertex AI. ```bash claude ``` ## Use with Cloudflare Access If your gateway is protected by [Cloudflare Access](https://developers.cloudflare.com/ai-gateway/configuration/cloudflare-access/), Claude Code can authenticate with a short-lived Access token instead of a gateway token. Point `ANTHROPIC_BASE_URL` at your [custom domain](https://developers.cloudflare.com/ai-gateway/configuration/custom-domains/) and use Claude Code's `apiKeyHelper` to fetch the token with [cloudflared](https://developers.cloudflare.com/cloudflare-one/access-controls/authenticate-agents/#make-requests-with-cloudflared-access-curl). Claude Code sends the token as the API key, and Access verifies it at the edge. Add the following to Claude Code's [settings.json ↗](https://docs.anthropic.com/en/docs/claude-code/settings#settings-files), replacing `ai-gateway.example.com` with your custom domain: ```json { "apiKeyHelper": "cloudflared access login --no-verbose https://ai-gateway.example.com", "env": { "ANTHROPIC_BASE_URL": "https://ai-gateway.example.com/anthropic" } } ``` The first request opens your identity provider's login flow. After you authenticate, requests route through AI Gateway with your Access identity attached as [cf.user\_id](https://developers.cloudflare.com/ai-gateway/observability/custom-metadata/#reserved-metadata). To confirm traffic reaches AI Gateway, refer to [Verify it works](https://developers.cloudflare.com/ai-gateway/integrations/coding-agents/#verify-it-works). Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/integrations/coding-agents/claude-code/#page","headline":"Claude Code · Cloudflare AI Gateway docs","description":"Route Claude Code through AI Gateway using your Cloudflare gateway token.","url":"https://developers.cloudflare.com/ai-gateway/integrations/coding-agents/claude-code/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-08-17","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Route Claude Desktop through AI Gateway using third-party inference settings and your Cloudflare gateway token. title: Claude Desktop image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Claude Desktop Last updated Aug 17, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/integrations/coding-agents/claude-desktop/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) By pointing [Claude Desktop ↗](https://claude.ai/download) at AI Gateway instead of Anthropic directly, you get observability, caching, and centralized credentials for your Anthropic requests, without changing how you use Claude Desktop. Claude Desktop can send third-party inference requests to a custom gateway; this configuration sends those requests to AI Gateway's [Anthropic endpoint](https://developers.cloudflare.com/ai-gateway/usage/providers/anthropic/), authenticated with your Cloudflare gateway token. AI Gateway can supply the Anthropic credentials for you through [Unified Billing](https://developers.cloudflare.com/ai-gateway/features/unified-billing/) or a [stored provider key (BYOK)](https://developers.cloudflare.com/ai-gateway/configuration/bring-your-own-keys/). ## Prerequisites Before you start, you need: * An [authenticated gateway](https://developers.cloudflare.com/ai-gateway/configuration/authentication/) and its [gateway token](https://developers.cloudflare.com/ai-gateway/configuration/authentication/#setting-up-authenticated-gateway-using-the-dashboard). The gateway token must have `Run` permissions. * Your Cloudflare account ID. To find it, refer to [Find your account and zone IDs](https://developers.cloudflare.com/fundamentals/account/find-account-and-zone-ids/). * Credentials for Anthropic requests. Use either [Unified Billing](https://developers.cloudflare.com/ai-gateway/features/unified-billing/) credits or an Anthropic API key stored in AI Gateway as a [provider key (BYOK)](https://developers.cloudflare.com/ai-gateway/configuration/bring-your-own-keys/). * Claude Desktop installed and updated to the latest version. Note For the simplest setup, set Claude Desktop's gateway API key to your Cloudflare gateway token. If you set the gateway API key to your own Anthropic API key instead, add `cf-aig-authorization: Bearer ` as a custom inference header so AI Gateway can authenticate the request. 1. In Claude Desktop, select **Help** \> **Troubleshooting** \> **Enable Developer Mode**. 2. Select **Developer** \> **Configure Third-Party Inference**. 3. In **Connection**, set the connection type to _Gateway_. 4. In **Gateway credentials**, set **Credential kind** to _Static API key_. 5. Set **Gateway API key** to your Cloudflare gateway token. Replace `` with your gateway token. ```txt ``` 6. Set **Gateway auth scheme** to _Bearer_. 7. Set the gateway base URL to your gateway's Anthropic endpoint. Replace `` and `` with your values. ```txt https://gateway.ai.cloudflare.com/v1///anthropic ``` 8. In **Models**, add the Claude model you want to use. | Field | Value | | ------------ | ----------------- | | Model ID | claude-sonnet-4-5 | | Display name | Claude Sonnet 4.5 | | Tier alias | sonnet | 9. In **Gateway credentials**, select **Test connection**. 10. Start a Claude Desktop conversation. Requests now route through AI Gateway. To confirm traffic reaches AI Gateway, refer to [Verify it works](https://developers.cloudflare.com/ai-gateway/integrations/coding-agents/#verify-it-works). Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/integrations/coding-agents/claude-desktop/#page","headline":"Claude Desktop · Cloudflare AI Gateway docs","description":"Route Claude Desktop through AI Gateway using third-party inference settings and your Cloudflare gateway token.","url":"https://developers.cloudflare.com/ai-gateway/integrations/coding-agents/claude-desktop/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-08-17","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Route GitHub Copilot CLI through AI Gateway using the REST API and Unified Billing. title: GitHub Copilot CLI image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # GitHub Copilot CLI Last updated Jul 2, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/integrations/coding-agents/github-copilot-cli/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) [GitHub Copilot CLI ↗](https://docs.github.com/en/copilot/concepts/agents/about-copilot-cli) supports bring-your-own-key (BYOK) model providers configured through environment variables. Route it through AI Gateway's [REST API](https://developers.cloudflare.com/ai-gateway/usage/rest-api/), an OpenAI-compatible `/chat/completions` endpoint authenticated with a Cloudflare API token. Third-party models are billed through [Unified Billing](https://developers.cloudflare.com/ai-gateway/features/unified-billing/), so no provider API keys are needed in your environment. Alternatively, you can store your own provider API keys in AI Gateway with [BYOK (Store Keys)](https://developers.cloudflare.com/ai-gateway/configuration/bring-your-own-keys/) and use the same Cloudflare API token to authenticate — AI Gateway resolves the stored key on each request. Unlike [Claude Code](https://developers.cloudflare.com/ai-gateway/integrations/coding-agents/claude-code/), GitHub Copilot CLI authenticates the model provider with a single `Authorization` header and cannot send custom request headers. This is why the configuration uses the REST API — it accepts a Cloudflare API token in the standard `Authorization` header — rather than the gateway token and `cf-aig-authorization` header flow used for Claude Code. Because Copilot CLI cannot set the `cf-aig-gateway-id` header either, requests route through your account's [default gateway](https://developers.cloudflare.com/ai-gateway/usage/rest-api/#specify-a-gateway). ## Prerequisites Before you start, you need: * GitHub Copilot CLI installed. To install it, refer to [Installing GitHub Copilot CLI ↗](https://docs.github.com/en/copilot/how-tos/set-up/install-copilot-cli). * A [Cloudflare API token](https://developers.cloudflare.com/fundamentals/api/get-started/create-token/) with `AI Gateway` permission. * [Credits loaded](https://developers.cloudflare.com/ai-gateway/features/unified-billing/#load-credits) on your account for third-party models. * A model that supports tool calling and streaming. For best results, use a model with a context window of at least 128k tokens. 1. Set the provider environment variables. GitHub Copilot CLI reads these on startup and appends `/chat/completions` to the base URL. The commands set these variables for the current session. To persist them, add them to your shell profile (for example, `~/.zshrc` or `~/.bashrc`). Replace `` with your Cloudflare account ID and `` with your Cloudflare API token. Set `COPILOT_MODEL` to any supported model in `provider/model` format. ```bash export COPILOT_PROVIDER_TYPE="openai" export COPILOT_PROVIDER_BASE_URL="https://api.cloudflare.com/client/v4/accounts//ai/v1" export COPILOT_PROVIDER_API_KEY="" export COPILOT_MODEL="openai/gpt-4.1" ``` ```powershell $env:COPILOT_PROVIDER_TYPE = "openai" $env:COPILOT_PROVIDER_BASE_URL = "https://api.cloudflare.com/client/v4/accounts//ai/v1" $env:COPILOT_PROVIDER_API_KEY = "" $env:COPILOT_MODEL = "openai/gpt-4.1" ``` 2. Start GitHub Copilot CLI and send a prompt. Requests now route through AI Gateway. ```bash copilot ``` Note GitHub Copilot CLI keeps a built-in catalog of known models and their token limits. If your selected model is not in the catalog, Copilot CLI prints a warning and falls back to default token limits. You can ignore the warning, or set the limits explicitly to match your model: ```bash export COPILOT_PROVIDER_MAX_PROMPT_TOKENS="200000" export COPILOT_PROVIDER_MAX_OUTPUT_TOKENS="32000" ``` To confirm traffic reaches AI Gateway, refer to [Verify it works](https://developers.cloudflare.com/ai-gateway/integrations/coding-agents/#verify-it-works). Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/integrations/coding-agents/github-copilot-cli/#page","headline":"GitHub Copilot CLI · Cloudflare AI Gateway docs","description":"Route GitHub Copilot CLI through AI Gateway using the REST API and Unified Billing.","url":"https://developers.cloudflare.com/ai-gateway/integrations/coding-agents/github-copilot-cli/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-07-02","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Route OpenAI Codex through AI Gateway using a custom model provider that points at the OpenAI endpoint. title: OpenAI Codex image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # OpenAI Codex Last updated Aug 5, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/integrations/coding-agents/openai-codex/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) [OpenAI Codex ↗](https://developers.openai.com/codex/) is a coding agent you run in your terminal. It supports [custom model providers ↗](https://developers.openai.com/codex/config-advanced#custom-model-providers) defined in `config.toml`. This configuration adds a provider that points at AI Gateway's [OpenAI endpoint](https://developers.cloudflare.com/ai-gateway/usage/providers/openai/), so Codex sends its requests through AI Gateway. AI Gateway authenticates the model provider for you through [Unified Billing](https://developers.cloudflare.com/ai-gateway/features/unified-billing/), so you pass a Cloudflare API token instead of an OpenAI API key. If your gateway is protected by Cloudflare Access, refer to [Use with Cloudflare Access](#use-with-cloudflare-access). Note Codex custom providers only support the OpenAI Responses API (`wire_api = "responses"`). This means you can only use OpenAI models that support the Responses API, such as `gpt-5.5`. Models from other providers (for example, Anthropic or Google) do not use the OpenAI Responses request format, so they do not work with Codex through this configuration. ## Prerequisites Before you start, you need: * Your Cloudflare account ID. To find it, refer to [Find your account and zone IDs](https://developers.cloudflare.com/fundamentals/account/find-account-and-zone-ids/). * An AI Gateway. You can use your account's `default` gateway or [create a gateway](https://developers.cloudflare.com/ai-gateway/get-started/) and use its slug. * A [Cloudflare API token](https://developers.cloudflare.com/fundamentals/api/get-started/create-token/) with `AI Gateway` permission. * [Credits loaded](https://developers.cloudflare.com/ai-gateway/features/unified-billing/#load-credits) on your account for third-party models. * [Codex ↗](https://developers.openai.com/codex/cli/) installed and updated to the latest version. 1. Create a Codex [profile ↗](https://developers.openai.com/codex/config-advanced#profiles) file at `~/.codex/cloudflare-aig.config.toml`. The profile defines a custom model provider that points at your gateway's OpenAI endpoint and reads your Cloudflare API token from an environment variable. Replace `` and `` with your values. You can use `default` for the gateway to route through your account's default gateway, or change it to another gateway slug. ```toml model_provider = "cloudflare-ai-gateway" model = "gpt-5.5" model_reasoning_effort = "medium" [model_providers.cloudflare-ai-gateway] name = "Cloudflare AI Gateway" # Run `wrangler whoami` to get your account ID, then replace . # Use `default` for to route through your account's default gateway. base_url = "https://gateway.ai.cloudflare.com/v1///openai" env_key = "CLOUDFLARE_API_KEY" wire_api = "responses" ``` Note Codex does not expand environment variables inside `base_url`, so the account ID and gateway slug must be literal values. Only `CLOUDFLARE_API_KEY` is read from the environment. 2. Set your Cloudflare API token as the `CLOUDFLARE_API_KEY` environment variable. The following commands set it for the current session. To persist it, add it to your shell profile (for example, `~/.zshrc` or `~/.bashrc`). Replace `` with your value. ```bash # Run `wrangler auth token` to get an auth token. export CLOUDFLARE_API_KEY="" ``` ```powershell # Run `wrangler auth token` to get an auth token. $env:CLOUDFLARE_API_KEY = "" ``` 3. Start Codex with the profile and send a prompt. Requests now route through AI Gateway. The `cloudflare-aig` profile name matches the `cloudflare-aig.config.toml` file you created. ```bash codex --profile cloudflare-aig ``` ## Use with Cloudflare Access If your gateway is protected by [Cloudflare Access](https://developers.cloudflare.com/ai-gateway/configuration/cloudflare-access/), Codex can authenticate with a short-lived Access token instead of a Cloudflare API token. Point the provider at your [custom domain](https://developers.cloudflare.com/ai-gateway/configuration/custom-domains/), and configure the provider's `auth` command to fetch the token with [cloudflared](https://developers.cloudflare.com/cloudflare-one/access-controls/authenticate-agents/#make-requests-with-cloudflared-access-curl) instead of passing a token through an environment variable. Update the same `~/.codex/cloudflare-aig.config.toml` profile from the Unified Billing setup so the provider points at your custom domain and uses `cloudflared` for authentication. The `cloudflare-aig` in `codex --profile cloudflare-aig` refers to this file's name. Replace `ai-gateway.example.com` with your custom domain. ```toml model_provider = "cloudflare-ai-gateway" model = "gpt-5.5" model_reasoning_effort = "medium" [model_providers.cloudflare-ai-gateway] name = "Cloudflare AI Gateway" base_url = "https://ai-gateway.example.com/openai" wire_api = "responses" [model_providers.cloudflare-ai-gateway.auth] command = "cloudflared" args = ["access", "login", "--no-verbose", "https://ai-gateway.example.com"] timeout_ms = 30000 refresh_interval_ms = 0 ``` Compared to the Unified Billing setup in the previous section, the custom domain replaces the account ID and gateway ID in `base_url`, and the `auth` block replaces `env_key`. Start Codex with the profile: ```bash codex --profile cloudflare-aig ``` The first request opens your identity provider's login flow. After you authenticate, requests route through AI Gateway with your Access identity attached as [cf.user\_id](https://developers.cloudflare.com/ai-gateway/observability/custom-metadata/#reserved-metadata). To confirm traffic reaches AI Gateway, refer to [Verify it works](https://developers.cloudflare.com/ai-gateway/integrations/coding-agents/#verify-it-works). Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/integrations/coding-agents/openai-codex/#page","headline":"OpenAI Codex · Cloudflare AI Gateway docs","description":"Route OpenAI Codex through AI Gateway using a custom model provider that points at the OpenAI endpoint.","url":"https://developers.cloudflare.com/ai-gateway/integrations/coding-agents/openai-codex/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-08-05","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Route the Pi coding agent through AI Gateway using its built-in Cloudflare AI Gateway provider. title: Pi image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Pi Last updated Jul 2, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/integrations/coding-agents/pi/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) [Pi ↗](https://pi.dev) is a coding agent you run in your terminal. It has built-in support for AI Gateway, so instead of setting a base URL you select the `cloudflare-ai-gateway` provider and point Pi at your gateway. Pi builds the gateway endpoint from your account ID and gateway slug and routes requests through it. ## Prerequisites Before you start, you need: * An [authenticated gateway](https://developers.cloudflare.com/ai-gateway/configuration/authentication/) and its [gateway token](https://developers.cloudflare.com/ai-gateway/configuration/authentication/#setting-up-authenticated-gateway-using-the-dashboard). The gateway token must have `Run` permissions. * Your Cloudflare account ID. To find it, refer to [Find your account and zone IDs](https://developers.cloudflare.com/fundamentals/account/find-account-and-zone-ids/). * Pi installed and updated to the latest version. Note The token you give Pi is your gateway token, not a model provider key. To pay for model usage, enable [Unified Billing](https://developers.cloudflare.com/ai-gateway/features/unified-billing/) or store provider keys in AI Gateway with [BYOK (Store Keys)](https://developers.cloudflare.com/ai-gateway/configuration/bring-your-own-keys/). Either way, AI Gateway handles the provider authentication for you. 1. Set your gateway token, account ID, and gateway slug as environment variables. The following commands set them for the current session. To persist them, add them to your shell profile (for example, `~/.zshrc` or `~/.bashrc`). Replace `` and `` with your values. You can leave `CLOUDFLARE_GATEWAY_ID` as `default` to route through your account's default gateway, or change it to another gateway slug. ```bash # Run `wrangler auth token` to get an auth token. export CLOUDFLARE_API_KEY="" # Run `wrangler whoami` to get your account ID. export CLOUDFLARE_ACCOUNT_ID="" # Use `default` to route through your account's default gateway. export CLOUDFLARE_GATEWAY_ID="default" ``` ```powershell # Run `wrangler auth token` to get an auth token. $env:CLOUDFLARE_API_KEY = "" # Run `wrangler whoami` to get your account ID. $env:CLOUDFLARE_ACCOUNT_ID = "" # Use `default` to route through your account's default gateway. $env:CLOUDFLARE_GATEWAY_ID = "default" ``` Alternatively, leave out `CLOUDFLARE_API_KEY` and run `/login` inside Pi to store the token instead. 2. Start a session against a model. Requests now route through AI Gateway. ```bash pi --provider cloudflare-ai-gateway --model "claude-sonnet-4-6" ``` To confirm traffic reaches AI Gateway, refer to [Verify it works](https://developers.cloudflare.com/ai-gateway/integrations/coding-agents/#verify-it-works). Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/integrations/coding-agents/pi/#page","headline":"Pi · Cloudflare AI Gateway docs","description":"Route the Pi coding agent through AI Gateway using its built-in Cloudflare AI Gateway provider.","url":"https://developers.cloudflare.com/ai-gateway/integrations/coding-agents/pi/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-07-02","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Route Vercel AI SDK requests through AI Gateway using the ai-gateway-provider package. title: Vercel AI SDK image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Vercel AI SDK Last updated Apr 20, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/integrations/vercel-ai-sdk/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) The [Vercel AI SDK ↗](https://sdk.vercel.ai/) is a TypeScript library for building AI applications. The SDK supports many different AI providers, tools for streaming completions, and more. To use Cloudflare AI Gateway with Vercel AI SDK, you will need to use the `ai-gateway-provider` package. ## Installation ```bash npm install ai-gateway-provider ``` ## Examples Make a request to ![]() OpenAI Unified API with Stored Key (BYOK) ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createUnified } from 'ai-gateway-provider/providers/unified'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const unified = createUnified(); const { text } = await generateText({ model: aigateway(unified('openai/gpt-5.2')), prompt: 'What is Cloudflare?', }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createUnified } from 'ai-gateway-provider/providers/unified'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const unified = createUnified(); const { text } = await generateText({ model: aigateway(unified('anthropic/claude-4-5-sonnet')), prompt: 'What is Cloudflare?', }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createUnified } from 'ai-gateway-provider/providers/unified'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const unified = createUnified(); const { text } = await generateText({ model: aigateway(unified('google/gemini-2.5-pro')), prompt: 'What is Cloudflare?', }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createUnified } from 'ai-gateway-provider/providers/unified'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const unified = createUnified(); const { text } = await generateText({ model: aigateway(unified('grok/grok-4')), prompt: 'What is Cloudflare?', }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createUnified } from 'ai-gateway-provider/providers/unified'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const unified = createUnified(); const { text } = await generateText({ model: aigateway(unified('dynamic/customer-support')), prompt: 'What is Cloudflare?', }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createUnified } from 'ai-gateway-provider/providers/unified'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const unified = createUnified(); const { text } = await generateText({ model: aigateway(unified('workers-ai/@cf/meta/llama-3.3-70b-instruct-fp8-fast')), prompt: 'What is Cloudflare?', }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createUnified } from 'ai-gateway-provider/providers/unified'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const unified = createUnified({ apiKey: '{API_KEY}' }); const { text } = await generateText({ model: aigateway(unified('openai/gpt-5.2')), prompt: 'What is Cloudflare?', }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createUnified } from 'ai-gateway-provider/providers/unified'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const unified = createUnified({ apiKey: '{API_KEY}' }); const { text } = await generateText({ model: aigateway(unified('anthropic/claude-4-5-sonnet')), prompt: 'What is Cloudflare?', }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createUnified } from 'ai-gateway-provider/providers/unified'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const unified = createUnified({ apiKey: '{API_KEY}' }); const { text } = await generateText({ model: aigateway(unified('google/gemini-2.5-pro')), prompt: 'What is Cloudflare?', }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createUnified } from 'ai-gateway-provider/providers/unified'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const unified = createUnified({ apiKey: '{API_KEY}' }); const { text } = await generateText({ model: aigateway(unified('grok/grok-4')), prompt: 'What is Cloudflare?', }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createUnified } from 'ai-gateway-provider/providers/unified'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const unified = createUnified({ apiKey: '{API_KEY}' }); const { text } = await generateText({ model: aigateway(unified('dynamic/customer-support')), prompt: 'What is Cloudflare?', }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createUnified } from 'ai-gateway-provider/providers/unified'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const unified = createUnified({ apiKey: '{API_KEY}' }); const { text } = await generateText({ model: aigateway(unified('workers-ai/@cf/meta/llama-3.3-70b-instruct-fp8-fast')), prompt: 'What is Cloudflare?', }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createOpenAI } from 'ai-gateway-provider/providers/openai'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const openai = createOpenAI(); const { text } = await generateText({ model: aigateway(openai.chat('gpt-5.2')), prompt: 'What is Cloudflare?', }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createAnthropic } from 'ai-gateway-provider/providers/anthropic'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const anthropic = createAnthropic(); const { text } = await generateText({ model: aigateway(anthropic('claude-4-5-sonnet')), prompt: 'What is Cloudflare?', }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createGoogle } from 'ai-gateway-provider/providers/google'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const google = createGoogle(); const { text } = await generateText({ model: aigateway(google('gemini-2.5-pro')), prompt: 'What is Cloudflare?', }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createXai } from 'ai-gateway-provider/providers/xai'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const xai = createXai(); const { text } = await generateText({ model: aigateway(xai('grok-4')), prompt: 'What is Cloudflare?', }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createUnified } from 'ai-gateway-provider/providers/unified'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const unified = createUnified(); const { text } = await generateText({ model: aigateway(unified('customer-support')), prompt: 'What is Cloudflare?', }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createUnified } from 'ai-gateway-provider/providers/unified'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const unified = createUnified(); const { text } = await generateText({ model: aigateway(unified('@cf/meta/llama-3.3-70b-instruct-fp8-fast')), prompt: 'What is Cloudflare?', }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createOpenAI } from 'ai-gateway-provider/providers/openai'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const openai = createOpenAI({ apiKey: '{API_KEY}' }); const { text } = await generateText({ model: aigateway(openai.chat('gpt-5.2')), prompt: 'What is Cloudflare?', }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createAnthropic } from 'ai-gateway-provider/providers/anthropic'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const anthropic = createAnthropic({ apiKey: '{API_KEY}' }); const { text } = await generateText({ model: aigateway(anthropic('claude-4-5-sonnet')), prompt: 'What is Cloudflare?', }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createGoogle } from 'ai-gateway-provider/providers/google'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const google = createGoogle({ apiKey: '{API_KEY}' }); const { text } = await generateText({ model: aigateway(google('gemini-2.5-pro')), prompt: 'What is Cloudflare?', }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createXai } from 'ai-gateway-provider/providers/xai'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const xai = createXai({ apiKey: '{API_KEY}' }); const { text } = await generateText({ model: aigateway(xai('grok-4')), prompt: 'What is Cloudflare?', }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createUnified } from 'ai-gateway-provider/providers/unified'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const unified = createUnified({ apiKey: '{API_KEY}' }); const { text } = await generateText({ model: aigateway(unified('customer-support')), prompt: 'What is Cloudflare?', }); ``` ```javascript import { createAiGateway } from 'ai-gateway-provider'; import { createUnified } from 'ai-gateway-provider/providers/unified'; import { generateText } from "ai"; const aigateway = createAiGateway({ accountId: "{CLOUDFLARE_ACCOUNT_ID}", gateway: '{GATEWAY_NAME}', apiKey: '{CF_AIG_TOKEN}', }); const unified = createUnified({ apiKey: '{API_KEY}' }); const { text } = await generateText({ model: aigateway(unified('@cf/meta/llama-3.3-70b-instruct-fp8-fast')), prompt: 'What is Cloudflare?', }); ``` ### AI binding with third-party models If you are already using the [workers-ai-provider ↗](https://www.npmjs.com/package/workers-ai-provider) package, you can route requests through AI Gateway to call third-party models without needing separate provider SDKs. Pass a `gateway` option with your gateway ID to `createWorkersAI`: ```js import { createWorkersAI } from "workers-ai-provider"; import { streamText } from "ai"; export default { async fetch(request, env) { const workersai = createWorkersAI({ binding: env.AI, gateway: { id: "my-gateway" }, }); const result = streamText({ model: workersai("openai/gpt-4o"), messages: [{ role: "user", content: "Write a short story" }], }); return result.toTextStreamResponse(); }, }; ``` ```ts import { createWorkersAI } from "workers-ai-provider"; import { streamText } from "ai"; export default { async fetch(request, env) { const workersai = createWorkersAI({ binding: env.AI, gateway: { id: "my-gateway" }, }); const result = streamText({ model: workersai("openai/gpt-4o"), messages: [{ role: "user", content: "Write a short story" }], }); return result.toTextStreamResponse(); }, } satisfies ExportedHandler; ``` This works with any [supported provider and model](https://developers.cloudflare.com/ai/models/) available through AI Gateway. ### Fallback Providers To specify model or provider fallbacks to handle request failures and ensure reliability, you can pass an array of models to the `model` option. ```js const { text } = await generateText({ model: aigateway([openai.chat("gpt-5.1"), anthropic("claude-sonnet-4-5")]), prompt: "Write a vegetarian lasagna recipe for 4 people.", }); ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/integrations/vercel-ai-sdk/#page","headline":"Vercel AI SDK · Cloudflare AI Gateway docs","description":"Route Vercel AI SDK requests through AI Gateway using the ai-gateway-provider package.","url":"https://developers.cloudflare.com/ai-gateway/integrations/vercel-ai-sdk/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-20","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Step-by-step AI Gateway tutorials for deploying Workers, connecting providers, and building AI applications. title: Tutorials image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Tutorials Last updated May 19, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/tutorials/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) View tutorials to help you get started with AI Gateway. | Name | Last Updated | Difficulty | | --------------------------------------------------------------------------------------------------------------------------------- | ------------ | ---------- | | [Create your first AI Gateway using Workers AI](https://developers.cloudflare.com/ai-gateway/tutorials/create-first-aig-workers/) | 2 years ago | Beginner | | [Set up Workers AI with AI Gateway](https://developers.cloudflare.com/ai-gateway/integrations/aig-workers-ai-binding/) | | Beginner | | [Use Pruna P-video through AI Gateway](https://developers.cloudflare.com/ai-gateway/tutorials/pruna-p-video/) | | Beginner | Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"WebPage","@id":"https://developers.cloudflare.com/ai-gateway/tutorials/#page","headline":"Tutorials · Cloudflare AI Gateway docs","description":"Step-by-step AI Gateway tutorials for deploying Workers, connecting providers, and building AI applications.","url":"https://developers.cloudflare.com/ai-gateway/tutorials/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-05-19","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: This tutorial guides you through creating your first AI Gateway using Workers AI on the Cloudflare dashboard. title: Create your first AI Gateway using Workers AI image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Create your first AI Gateway using Workers AI Last updated Jun 15, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/tutorials/create-first-aig-workers/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) This tutorial guides you through creating your first AI Gateway using Workers AI on the Cloudflare dashboard. The intended audience is beginners who are new to AI Gateway and Workers AI. Creating an AI Gateway enables the user to efficiently manage and secure AI requests, allowing them to utilize AI models for tasks such as content generation, data processing, or predictive analysis with enhanced control and performance. ## Sign up and log in 1. **Sign up**: If you do not have a Cloudflare account, [sign up ↗](https://cloudflare.com/sign-up). 2. **Log in**: Access the Cloudflare dashboard by logging in to the [Cloudflare dashboard ↗](https://dash.cloudflare.com/login). ## Create gateway Then, create a new AI Gateway. [Go to **AI Gateway** ↗](https://dash.cloudflare.com/?to=/:account/ai/ai-gateway) 1. Log into the [Cloudflare dashboard ↗](https://dash.cloudflare.com/) and select your account. 2. Go to **AI** \> **AI Gateway**. 3. Select **Create Gateway**. 4. Enter your **Gateway name**. Note: Gateway name has a 64 character limit. 5. In **Workers AI Billing**, choose how Workers AI requests through this gateway are billed: * **Standard billing** charges your Cloudflare account at the end of each billing cycle. * **Unified billing** deducts from your prepaid AI Gateway credit balance in real time. 6. Select **Create**. To set up an AI Gateway using the API: 1. [Create an API token](https://developers.cloudflare.com/fundamentals/api/get-started/create-token/) with the following permissions: * `AI Gateway - Read` * `AI Gateway - Edit` 2. Get your [Account ID](https://developers.cloudflare.com/fundamentals/account/find-account-and-zone-ids/). 3. Using that API token and Account ID, send a [POST request](https://developers.cloudflare.com/api/resources/ai%5Fgateway/methods/create/) to the Cloudflare API. ## Connect Your AI Provider 1. In the AI Gateway section, select the gateway you created. 2. Select **Workers AI** as your provider to set up an endpoint specific to Workers AI. You will receive an endpoint URL for sending requests. ## Send your first request 1. Go to **AI** \> **Workers AI** in the Cloudflare dashboard. 2. Select **Use REST API** and follow the steps to create and copy the API token and Account ID. 3. Send a request using the [REST API](https://developers.cloudflare.com/ai-gateway/usage/rest-api/). Replace `$CLOUDFLARE_ACCOUNT_ID` and `$CLOUDFLARE_API_TOKEN` with your actual account ID and API token: ```bash # Run `wrangler whoami` to get your account ID to replace $CLOUDFLARE_ACCOUNT_ID, # and `wrangler auth token` to get an auth token to replace $CLOUDFLARE_API_TOKEN. curl -X POST "https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/v1/chat/completions" \ --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ --header "cf-aig-gateway-id: default" \ --header "Content-Type: application/json" \ --data '{ "model": "@cf/moonshotai/kimi-k2.6", "messages": [{"role": "user", "content": "What is Cloudflare?"}] }' ``` The expected output would be similar to : ```bash {"result":{"response":"I'd be happy to explain what Cloudflare is.\n\nCloudflare is a cloud-based service that provides a range of features to help protect and improve the performance, security, and reliability of websites, applications, and other online services. Think of it as a shield for your online presence!\n\nHere are some of the key things Cloudflare does:\n\n1. **Content Delivery Network (CDN)**: Cloudflare has a network of servers all over the world. When you visit a website that uses Cloudflare, your request is sent to the nearest server, which caches a copy of the website's content. This reduces the time it takes for the content to load, making your browsing experience faster.\n2. **DDoS Protection**: Cloudflare protects against Distributed Denial-of-Service (DDoS) attacks. This happens when a website is overwhelmed with traffic from multiple sources to make it unavailable. Cloudflare filters out this traffic, ensuring your site remains accessible.\n3. **Firewall**: Cloudflare acts as an additional layer of security, filtering out malicious traffic and hacking attempts, such as SQL injection or cross-site scripting (XSS) attacks.\n4. **SSL Encryption**: Cloudflare offers free SSL encryption, which secure sensitive information (like passwords, credit card numbers, and browsing data) with an HTTPS connection (the \"S\" stands for Secure).\n5. **Bot Protection**: Cloudflare has an AI-driven system that identifies and blocks bots trying to exploit vulnerabilities or scrape your content.\n6. **Analytics**: Cloudflare provides insights into website traffic, helping you understand your audience and make informed decisions.\n7. **Cybersecurity**: Cloudflare offers advanced security features, such as intrusion protection, DNS filtering, and Web Application Firewall (WAF) protection.\n\nOverall, Cloudflare helps protect against cyber threats, improves website performance, and enhances security for online businesses, bloggers, and individuals who need to establish a strong online presence.\n\nWould you like to know more about a specific aspect of Cloudflare?"},"success":true,"errors":[],"messages":[]}% ``` ## View Analytics Monitor your AI Gateway to view usage metrics. 1. Go to **AI** \> **AI Gateway** in the dashboard. 2. Select your gateway to view metrics such as request counts, token usage, caching efficiency, errors, and estimated costs. You can also turn on additional configurations like logging and rate limiting. ## Optional - Next steps To build more with Workers, refer to [Tutorials](https://developers.cloudflare.com/workers/tutorials/). If you have any questions, need assistance, or would like to share your project, join the Cloudflare Developer community on [Discord ↗](https://discord.cloudflare.com) to connect with other developers and the Cloudflare team. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/tutorials/create-first-aig-workers/#page","headline":"Create your first AI Gateway using Workers AI · Cloudflare AI Gateway docs","description":"This tutorial guides you through creating your first AI Gateway using Workers AI on the Cloudflare dashboard.","url":"https://developers.cloudflare.com/ai-gateway/tutorials/create-first-aig-workers/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-06-15","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Learn how to call prunaai/p-video on Replicate through AI Gateway title: Use Pruna P-video through AI Gateway image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Use Pruna P-video through AI Gateway Last updated Apr 20, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/tutorials/pruna-p-video/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) This tutorial shows how to call the [Pruna's P-video ↗](https://replicate.com/prunaai/p-video) model on [Replicate](https://developers.cloudflare.com/ai-gateway/usage/providers/replicate/) through AI Gateway. ## Prerequisites * A [Cloudflare account ↗](https://cloudflare.com/sign-up) * A [Replicate account ↗](https://replicate.com/) with an API token ## 1\. Get a Replicate API token 1. Go to [replicate.com ↗](https://replicate.com/) and sign up for an account. 2. Once logged in, go to [replicate.com/settings/api-tokens ↗](https://replicate.com/account/api-tokens). 3. Select **Create token** and give it a name. 4. Copy the token and store it somewhere safe. ## 2\. Create an AI Gateway [Go to **AI Gateway** ↗](https://dash.cloudflare.com/?to=/:account/ai/ai-gateway) 1. Log into the [Cloudflare dashboard ↗](https://dash.cloudflare.com/) and select your account. 2. Go to **AI** \> **AI Gateway**. 3. Select **Create Gateway**. 4. Enter your **Gateway name**. Note: Gateway name has a 64 character limit. 5. In **Workers AI Billing**, choose how Workers AI requests through this gateway are billed: * **Standard billing** charges your Cloudflare account at the end of each billing cycle. * **Unified billing** deducts from your prepaid AI Gateway credit balance in real time. 6. Select **Create**. To set up an AI Gateway using the API: 1. [Create an API token](https://developers.cloudflare.com/fundamentals/api/get-started/create-token/) with the following permissions: * `AI Gateway - Read` * `AI Gateway - Edit` 2. Get your [Account ID](https://developers.cloudflare.com/fundamentals/account/find-account-and-zone-ids/). 3. Using that API token and Account ID, send a [POST request](https://developers.cloudflare.com/api/resources/ai%5Fgateway/methods/create/) to the Cloudflare API. Note your **Account ID** and **Gateway name** for use in later steps. To add authentication to your gateway, refer to [Authenticated Gateway](https://developers.cloudflare.com/ai-gateway/configuration/authentication/). ## 3\. Construct the gateway URL Replace the standard Replicate API base URL with the AI Gateway URL: ```txt # Instead of: https://api.replicate.com/v1 # Use: https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/replicate ``` For example, if your account ID is `abc123` and your gateway is `my-gateway`: ```txt https://gateway.ai.cloudflare.com/v1/abc123/my-gateway/replicate ``` ## 4\. Generate a video P-video predictions generally complete within 30 seconds. Because this is under Replicate's 60-second synchronous limit, you can use the `Prefer: wait` header to send a request and get the result in a single call: ```bash curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/replicate/predictions \ --header "Authorization: Bearer {replicate_api_token}" \ --header "cf-aig-authorization: Bearer {cloudflare_api_token}" \ --header "Content-Type: application/json" \ --header "Prefer: wait" \ --data '{ "version": "prunaai/p-video", "input": { "prompt": "A cat walking through a field of flowers in slow motion", "duration": 5, "aspect_ratio": "16:9", "resolution": "720p", "fps": 24 } }' ``` * `Authorization` — your Replicate API token (authenticates with Replicate). * `cf-aig-authorization` — your Cloudflare API token (for authenticated gateways). * `Prefer: wait` — blocks until the prediction completes instead of returning immediately. For a full list of available input parameters, check out the [prunaai/p-video model page ↗](https://replicate.com/prunaai/p-video) on Replicate. When the prediction completes, the response includes the `output` field with a URL to the generated video file. ## 5\. (Optional) Use async polling for longer requests If your request may exceed 60 seconds (for example, with longer durations or higher resolutions), use async mode instead. Send the request without the `Prefer: wait` header: ```bash curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/replicate/predictions \ --header "Authorization: Bearer {replicate_api_token}" \ --header "cf-aig-authorization: Bearer {cloudflare_api_token}" \ --header "Content-Type: application/json" \ --data '{ "version": "prunaai/p-video", "input": { "prompt": "A cat walking through a field of flowers in slow motion", "duration": 5, "aspect_ratio": "16:9", "resolution": "720p", "fps": 24 } }' ``` The response includes a prediction `id`: ```json { "id": "xyz789...", "status": "starting", "urls": { "get": "https://api.replicate.com/v1/predictions/xyz789...", "cancel": "https://api.replicate.com/v1/predictions/xyz789.../cancel" } } ``` Poll the prediction status until it completes: ```bash curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/replicate/predictions/{prediction_id} \ --header "Authorization: Bearer {replicate_api_token}" \ --header "cf-aig-authorization: Bearer {cloudflare_api_token}" ``` Keep polling until `status` is `succeeded` (or `failed`). When complete, the `output` field contains a URL to the generated video file. ## Next steps From here you can: * Use [logging](https://developers.cloudflare.com/ai-gateway/observability/logging/) to monitor requests and debug issues. * Set up [rate limiting](https://developers.cloudflare.com/ai-gateway/features/rate-limiting/) to control usage. * Use other models on Replicate or our other [supported providers](https://developers.cloudflare.com/ai-gateway/usage/providers/) through AI Gateway. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/tutorials/pruna-p-video/#page","headline":"Use Pruna P-video through AI Gateway · Cloudflare AI Gateway docs","description":"Learn how to call prunaai/p-video on Replicate through AI Gateway","url":"https://developers.cloudflare.com/ai-gateway/tutorials/pruna-p-video/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-20","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"},"keywords":["AI"]} ``` --- --- description: Track the latest updates, new features, and fixes for AI Gateway. title: Changelog image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Changelog Last updated Jun 5, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/changelog/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) [Subscribe to RSS](https://developers.cloudflare.com/changelog/rss/ai-gateway.xml) ## 2026-08-07 **Workers AI and AI Gateway unify model access and billing** Workers AI and AI Gateway now provide a unified path for accessing models and managing inference traffic. Use the same AI binding and REST API to call models hosted on Workers AI or by supported third-party providers, with AI Gateway providing observability, logging, caching, security, and billing controls. #### Unified entrypoints and observability The [AI binding](https://developers.cloudflare.com/ai-gateway/usage/worker-binding-methods/) supports both Workers AI and third-party models through `env.AI.run()`. The [REST API](https://developers.cloudflare.com/ai-gateway/usage/rest-api/) provides shared `/ai/` endpoints with Cloudflare authentication across providers. Route a Workers AI request through AI Gateway by specifying a gateway ID. Use `default` to automatically create a gateway on the first authenticated request, or specify an existing gateway to separate applications and workloads: ```js const response = await env.AI.run( "@cf/zai-org/glm-5.2", { messages: [{ role: "user", content: "What is the capital of France?" }], }, { gateway: { id: "default" }, }, ); ``` ```ts const response = await env.AI.run( "@cf/zai-org/glm-5.2", { messages: [{ role: "user", content: "What is the capital of France?" }], }, { gateway: { id: "default" }, }, ); ``` Requests routed through AI Gateway can be logged and included in analytics for request volume, errors, latency, token usage, and costs. You can also configure controls such as caching, rate limiting, and request retries on the gateway. #### Unified billing and higher rate limits You can now use prepaid [AI Gateway credits](https://developers.cloudflare.com/ai-gateway/features/unified-billing/) to pay for Workers AI inference. This provides one credit balance for Workers AI and supported third-party model providers. To use credits for Workers AI, set the gateway's [Workers AI billing setting](https://developers.cloudflare.com/ai-gateway/configuration/manage-gateway/#configure-workers-ai-billing) to **Unified billing**. Workers AI requests routed through that gateway deduct from your credit balance in real time. Prepaid credits also provide access to the following Workers AI frontier models without requiring the Workers Paid plan. Each frontier Workers AI model has a rate limit of 50 requests per minute per account, per model when billed with AI Gateway credits, compared to 20 requests per minute through standard Workers AI billing: * [@cf/moonshotai/kimi-k2.6](https://developers.cloudflare.com/workers-ai/models/kimi-k2.6/) * [@cf/moonshotai/kimi-k2.7-code](https://developers.cloudflare.com/workers-ai/models/kimi-k2.7-code/) * [@cf/zai-org/glm-5.2](https://developers.cloudflare.com/workers-ai/models/glm-5.2/) These limits are designed for typical agentic and coding workloads, where requests to frontier models can take longer to complete. For details, refer to [Workers AI limits](https://developers.cloudflare.com/workers-ai/platform/limits/), [Workers AI pricing](https://developers.cloudflare.com/workers-ai/platform/pricing/), [Unified Billing](https://developers.cloudflare.com/ai-gateway/features/unified-billing/), and the [AI Gateway model catalog](https://developers.cloudflare.com/ai/models/). ## 2026-08-05 **Track AI spend and catch anomalous usage with User Insights** AI Gateway now includes User Insights, a dashboard that gives you two things at once: clear visibility into how much your organization spends on AI, and a security signal that surfaces users whose usage suddenly looks abnormal. It works on the traffic already flowing through your gateway, so there is no additional setup. On the spend side, User Insights shows organization-wide totals for cost, requests, tokens, and adoption, and lets you drill into an individual user to see their spend, top models and providers, cache hit rate, and more. To attribute usage to individual users, add a user identifier with custom metadata or put your gateway behind Cloudflare Access. On the security side, User Insights baselines each user's normal usage from their 95th percentile (p95) session cost over the last 30 days, then flags sessions that exceed both that baseline and an organization-level threshold. A sudden jump above a user's own pattern is often the first sign of a compromised credential or a misbehaving agent, so you can investigate before it shows up on your bill. User Insights is available to all AI Gateway customers at no additional cost. ## 2026-08-05 **Identity-aware controls are now available in AI Gateway** AI Gateway now integrates with Cloudflare Access, giving you two new capabilities: * **Protect your gateway endpoint.** Put your AI Gateway behind Access so you can set policies that control who is allowed to call a specific gateway's endpoint. * **Identity-aware controls.** When traffic reaches AI Gateway through an Access-protected custom domain, AI Gateway can use the authenticated user's Access identity in logs, analytics, routing, and spend controls. With identity-aware controls, you can set spend limits by authenticated user, control which gateways different users can access, filter logs by user, and build policies without passing user IDs from the client application. AI Gateway adds the verified Access user ID to request metadata as `cf.user_id`. For setup instructions, refer to [Cloudflare Access](https://developers.cloudflare.com/ai-gateway/configuration/cloudflare-access/). ## 2026-06-12 **View the user agent of requests in AI Gateway logs** AI Gateway logs now capture the user agent of the client that made each request, making it easier to identify which SDK, library, or application sent the traffic flowing through your gateway. For example, you can tell apart requests coming from `openai-python` versus a custom application or a Cloudflare Worker. The user agent appears alongside the other details in each log entry, and you can filter logs by user agent (equals, does not equal, or contains) in the dashboard. For more information, refer to [Logging](https://developers.cloudflare.com/ai-gateway/observability/logging/). ## 2026-06-05 **Control AI costs with spend limits** AI Gateway now supports spend limits — cost-based budgets that track cumulative dollar spend and block requests when the budget is exceeded. Unlike rate limiting, which caps the number of requests, spend limits track actual cost based on token usage and model pricing. You can scope limits by model, provider, or custom metadata dimensions. For example, give each user a $200/day budget, cap total gateway spend at $10,000/day, or limit a specific model to $50/day per user. Each rule uses a configurable time window with fixed or sliding enforcement. Spend limits work with both [Unified Billing](https://developers.cloudflare.com/ai-gateway/features/unified-billing/) and [BYOK](https://developers.cloudflare.com/ai-gateway/configuration/bring-your-own-keys/) requests for models with known pricing. For more details, refer to the [Spend limits documentation](https://developers.cloudflare.com/ai-gateway/features/spend-limits/). ## 2026-05-21 **Call any AI model through AI Gateway's new REST API** AI Gateway now uses the AI REST API on `api.cloudflare.com`. You can call any model — whether from OpenAI, Anthropic, Google, or hosted on Workers AI — through one unified API, using the same endpoints and authentication regardless of provider. Four endpoints are available: * `POST /ai/run` — universal endpoint for all models and modalities * `POST /ai/v1/chat/completions` — OpenAI SDK compatible * `POST /ai/v1/responses` — OpenAI Responses API compatible * `POST /ai/v1/messages` — Anthropic SDK compatible ```bash curl -X POST "https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/v1/chat/completions" \ --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ --header "Content-Type: application/json" \ --data '{ "model": "openai/gpt-5.5", "messages": [{"role": "user", "content": "What is Cloudflare?"}] }' ``` All AI Gateway features — logging, caching, rate limiting, and guardrails — are applied automatically. Third-party models are billed through [Unified Billing](https://developers.cloudflare.com/ai-gateway/features/unified-billing/), so you do not need to manage separate provider API keys. Third-party model requests are routed through your account's default gateway, which is created automatically on first use. To route requests through a specific gateway, add the `cf-aig-gateway-id` header. If you are already calling Workers AI models through the existing REST API, that path (`/ai/run/@cf/{model}`) continues to work. To call Workers AI models through AI Gateway, use the `@cf/` model prefix (for example, `@cf/moonshotai/kimi-k2.6`) and include the `cf-aig-gateway-id` header to specify which gateway to route through. For more details and examples, refer to the [REST API documentation](https://developers.cloudflare.com/ai-gateway/usage/rest-api/). ## 2026-04-02 **Automatically retry on upstream provider failures on AI Gateway** AI Gateway now supports automatic retries at the gateway level. When an upstream provider returns an error, your gateway retries the request based on the retry policy you configure, without requiring any client-side changes. You can configure the retry count (up to 5 attempts), the delay between retries (from 100ms to 5 seconds), and the backoff strategy (Constant, Linear, or Exponential). These defaults apply to all requests through the gateway, and per-request headers can override them. ![Retry Requests settings in the AI Gateway dashboard](https://developers.cloudflare.com/cdn-cgi/image/onerror=redirect,width=2344,height=502,format=webp/_astro/auto-retry-changelog.DoCXZnDy.png) This is particularly useful when you do not control the client making the request and cannot implement retry logic on the caller side. For more complex failover scenarios — such as failing across different providers — use [Dynamic Routing](https://developers.cloudflare.com/ai-gateway/features/dynamic-routing/). For more information, refer to [Manage gateways](https://developers.cloudflare.com/ai-gateway/configuration/manage-gateway/#retry-requests). ## 2026-03-17 **Log AI Gateway request metadata without storing payloads** AI Gateway now supports the `cf-aig-collect-log-payload` header, which controls whether request and response bodies are stored in logs. By default, this header is set to `true` and payloads are stored alongside metadata. Set this header to `false` to skip payload storage while still logging metadata such as token counts, model, provider, status code, cost, and duration. This is useful when you need usage metrics but do not want to persist sensitive prompt or response data. ```bash curl https://gateway.ai.cloudflare.com/v1/$ACCOUNT_ID/$GATEWAY_ID/openai/chat/completions \ --header "Authorization: Bearer $TOKEN" \ --header 'Content-Type: application/json' \ --header 'cf-aig-collect-log-payload: false' \ --data '{ "model": "gpt-4o-mini", "messages": [ { "role": "user", "content": "What is the email address and phone number of user123?" } ] }' ``` For more information, refer to [Logging](https://developers.cloudflare.com/ai-gateway/observability/logging/#collect-log-payload-cf-aig-collect-log-payload). ## 2026-03-02 **Get started with AI Gateway automatically** You can now start using AI Gateway with a single API call — no setup required. Use `default` as your gateway ID, and AI Gateway creates one for you automatically on the first request. To try it out, [create an API token](https://developers.cloudflare.com/fundamentals/api/get-started/create-token/) with `AI Gateway - Read`, `AI Gateway - Edit`, and `Workers AI - Read` permissions, then run: ```bash curl -X POST https://gateway.ai.cloudflare.com/v1/$CLOUDFLARE_ACCOUNT_ID/default/compat/chat/completions \ --header "cf-aig-authorization: Bearer $CLOUDFLARE_API_TOKEN" \ --header 'Content-Type: application/json' \ --data '{ "model": "workers-ai/@cf/meta/llama-3.3-70b-instruct-fp8-fast", "messages": [ { "role": "user", "content": "What is Cloudflare?" } ] }' ``` AI Gateway gives you logging, caching, rate limiting, and access to multiple AI providers through a single endpoint. For more information, refer to [Get started](https://developers.cloudflare.com/ai-gateway/get-started/). ## 2026-02-19 **AI dashboard experience improvements** [Workers AI](https://developers.cloudflare.com/workers-ai/) and [AI Gateway](https://developers.cloudflare.com/ai-gateway/) have received a series of dashboard improvements to help you get started faster and manage your AI workloads more easily. **Navigation and discoverability** AI now has its own top-level section in the Cloudflare dashboard sidebar, so you can find AI features without digging through menus. ![AI sidebar navigation in the Cloudflare dashboard](https://developers.cloudflare.com/cdn-cgi/image/onerror=redirect,width=2328,height=1140,format=webp/_astro/sidebar-navigation.BQNFBmAk.png) _The new top-level AI section in the dashboard sidebar._ **Onboarding and getting started** [Getting started](https://developers.cloudflare.com/ai-gateway/get-started/) with AI Gateway is now simpler. When you create your first gateway, we now show your gateway's OpenAI-compatible endpoint and step-by-step guidance to help you configure it. The Playground also includes helpful prompts, and usage pages have clear next steps if you have not made any requests yet. ![AI Gateway onboarding flow](https://developers.cloudflare.com/cdn-cgi/image/onerror=redirect,width=2400,height=1232,format=webp/_astro/onboarding-flow.DZ7aMcHa.png) _The first-run setup experience for new gateways._ We've also combined the previously separate code example sections into one view with dropdown selectors for API type, provider, SDK, and authentication method so you can now customize the exact code snippet you need from one place. **Dynamic Routing** * The [route builder](https://developers.cloudflare.com/ai-gateway/features/dynamic-routing/) is now more performant and responsive. * You can now copy route names to your clipboard with a single click. * Code examples use the [Universal Endpoint](https://developers.cloudflare.com/ai-gateway/usage/universal/) format, making it easier to integrate routes into your application. **Observability and analytics** * Small monetary values now display correctly in [cost analytics](https://developers.cloudflare.com/ai-gateway/observability/costs/) charts, so you can accurately track spending at any scale. **Accessibility** * Improvements to keyboard navigation within the AI Gateway, specifically when exploring usage by [provider](https://developers.cloudflare.com/ai-gateway/usage/providers/). * Improvements to sorting and filtering components on the [Workers AI](https://developers.cloudflare.com/workers-ai/models/) models page. For more information, refer to the [AI Gateway documentation](https://developers.cloudflare.com/ai-gateway/). ## 2025-08-25 **Manage and deploy your AI provider keys through Bring Your Own Key (BYOK) with AI Gateway, now powered by Cloudflare Secrets Store** Cloudflare Secrets Store is now integrated with AI Gateway, allowing you to store, manage, and deploy your AI provider keys in a secure and seamless configuration through [Bring Your Own Key ↗](https://developers.cloudflare.com/ai-gateway/configuration/bring-your-own-keys/). Instead of passing your AI provider keys directly in every request header, you can centrally manage each key with Secrets Store and deploy in your gateway configuration using only a reference, rather than passing the value in plain text. You can now create a secret directly from your AI Gateway [in the dashboard ↗](http://dash.cloudflare.com/?to=/:account/ai-gateway) by navigating into your gateway -> **Provider Keys** \-> **Add**. ![Import repo or choose template](https://developers.cloudflare.com/cdn-cgi/image/onerror=redirect,width=2410,height=1842,format=webp/_astro/add-secret-ai-gateway.B-SIPr6s.png) You can also create your secret with the newly available **ai\_gateway** scope via [wrangler ↗](https://developers.cloudflare.com/workers/wrangler/commands/), the [Secrets Store dashboard ↗](http://dash.cloudflare.com/?to=/:account/secrets-store), or the [API ↗](https://developers.cloudflare.com/api/resources/secrets%5Fstore/). Then, pass the key in the request header using its Secrets Store reference: ```bash curl -X POST https://gateway.ai.cloudflare.com/v1//my-gateway/anthropic/v1/messages \ --header 'cf-aig-authorization: ANTHROPIC_KEY_1 \ --header 'anthropic-version: 2023-06-01' \ --header 'Content-Type: application/json' \ --data '{"model": "claude-3-opus-20240229", "messages": [{"role": "user", "content": "What is Cloudflare?"}]}' ``` Or, using Javascript: ```plaintext import Anthropic from '@anthropic-ai/sdk'; const anthropic = new Anthropic({ apiKey: "ANTHROPIC_KEY_1", baseURL: "https://gateway.ai.cloudflare.com/v1//my-gateway/anthropic", }); const message = await anthropic.messages.create({ model: 'claude-3-opus-20240229', messages: [{role: "user", content: "What is Cloudflare?"}], max_tokens: 1024 }); ``` For more information, check out the [blog ↗](https://blog.cloudflare.com/ai-gateway-aug-2025-refresh)! ## 2025-06-03 **AI Gateway adds OpenAI compatible endpoint** Users can now use an [OpenAI Compatible endpoint](https://developers.cloudflare.com/ai-gateway/usage/chat-completion/) in AI Gateway to easily switch between providers, while keeping the exact same request and response formats. We're launching now with the chat completions endpoint, with the embeddings endpoint coming up next. To get started, use the OpenAI compatible chat completions endpoint URL with your own account id and gateway id and switch between providers by changing the `model` and `apiKey` parameters. ```js import OpenAI from "openai"; const client = new OpenAI({ apiKey: "YOUR_PROVIDER_API_KEY", // Provider API key baseURL: "https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat", }); const response = await client.chat.completions.create({ model: "google-ai-studio/gemini-2.0-flash", messages: [{ role: "user", content: "What is Cloudflare?" }], }); console.log(response.choices[0].message.content); ``` Additionally, the [OpenAI Compatible endpoint](https://developers.cloudflare.com/ai-gateway/usage/chat-completion/) can be combined with our [Universal Endpoint](https://developers.cloudflare.com/ai-gateway/usage/universal/) to add fallbacks across multiple providers. That means AI Gateway will return every response in the same standardized format, no extra parsing logic required! Learn more in the [OpenAI Compatibility](https://developers.cloudflare.com/ai-gateway/usage/chat-completion/) documentation. ## 2025-03-21 **AI Gateway launches Realtime WebSockets API** We are excited to announce that [AI Gateway](https://developers.cloudflare.com/ai-gateway/) now supports real-time AI interactions with the new [Realtime WebSockets API](https://developers.cloudflare.com/ai-gateway/usage/websockets-api/realtime-api/). This new capability allows developers to establish persistent, low-latency connections between their applications and AI models, enabling natural, real-time conversational AI experiences, including speech-to-speech interactions. The Realtime WebSockets API works with the [OpenAI Realtime API ↗](https://platform.openai.com/docs/guides/realtime#connect-with-websockets), [Google Gemini Live API ↗](https://ai.google.dev/gemini-api/docs/multimodal-live), and supports real-time text and speech interactions with models from [Cartesia ↗](https://docs.cartesia.ai/api-reference/tts/tts), and [ElevenLabs ↗](https://elevenlabs.io/docs/conversational-ai/api-reference/conversational-ai/websocket). Here's how you can connect AI Gateway to [OpenAI's Realtime API ↗](https://platform.openai.com/docs/guides/realtime#connect-with-websockets) using WebSockets: ```javascript import WebSocket from "ws"; const url = "wss://gateway.ai.cloudflare.com/v1///openai?model=gpt-4o-realtime-preview-2024-12-17"; const ws = new WebSocket(url, { headers: { "cf-aig-authorization": process.env.CLOUDFLARE_API_KEY, Authorization: "Bearer " + process.env.OPENAI_API_KEY, "OpenAI-Beta": "realtime=v1", }, }); ws.on("open", () => console.log("Connected to server.")); ws.on("message", (message) => console.log(JSON.parse(message.toString()))); ws.send( JSON.stringify({ type: "response.create", response: { modalities: ["text"], instructions: "Tell me a joke" }, }), ); ``` Get started by checking out the [Realtime WebSockets API](https://developers.cloudflare.com/ai-gateway/usage/websockets-api/realtime-api/) documentation. ## 2025-02-26 **Introducing Guardrails in AI Gateway** [AI Gateway](https://developers.cloudflare.com/ai-gateway/) now includes [Guardrails](https://developers.cloudflare.com/ai-gateway/features/guardrails/), to help you monitor your AI apps for harmful or inappropriate content and deploy safely. Within the AI Gateway settings, you can configure: * **Guardrails**: Enable or disable content moderation as needed. * **Evaluation scope**: Select whether to moderate user prompts, model responses, or both. * **Hazard categories**: Specify which categories to monitor and determine whether detected inappropriate content should be blocked or flagged. ![Guardrails in AI Gateway](https://developers.cloudflare.com/cdn-cgi/image/onerror=redirect,width=2524,height=444,format=webp/_astro/Guardrails.BTNc0qeC.png) Learn more in the [blog ↗](https://blog.cloudflare.com/guardrails-in-ai-gateway/) or our [documentation](https://developers.cloudflare.com/ai-gateway/features/guardrails/). ## 2025-02-06 **Request timeouts and retries with AI Gateway** AI Gateway adds additional ways to handle requests - [Request Timeouts](https://developers.cloudflare.com/ai-gateway/configuration/request-handling/#request-timeouts) and [Request Retries](https://developers.cloudflare.com/ai-gateway/configuration/request-handling/#request-retries), making it easier to keep your applications responsive and reliable. Timeouts and retries can be used on both the [Universal Endpoint](https://developers.cloudflare.com/ai-gateway/usage/universal/) or directly to a [supported provider](https://developers.cloudflare.com/ai-gateway/usage/providers/). **Request timeouts**A [request timeout](https://developers.cloudflare.com/ai-gateway/configuration/request-handling/#request-timeouts) allows you to trigger [fallbacks](https://developers.cloudflare.com/ai-gateway/configuration/fallbacks/) or a retry if a provider takes too long to respond. To set a request timeout directly to a provider, add a `cf-aig-request-timeout` header. ```bash curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/workers-ai/@cf/meta/llama-3.1-8b-instruct \ --header 'Authorization: Bearer {cf_api_token}' \ --header 'Content-Type: application/json' \ --header 'cf-aig-request-timeout: 5000' --data '{"prompt": "What is Cloudflare?"}' ``` **Request retries**A [request retry](https://developers.cloudflare.com/ai-gateway/configuration/request-handling/#request-retries) automatically retries failed requests, so you can recover from temporary issues without intervening. To set up request retries directly to a provider, add the following headers: * cf-aig-max-attempts (number) * cf-aig-retry-delay (number) * cf-aig-backoff ("constant" | "linear" | "exponential) ## 2025-02-05 **AI Gateway adds Cerebras, ElevenLabs, and Cartesia as new providers** [AI Gateway](https://developers.cloudflare.com/ai-gateway/) has added three new providers: [Cartesia](https://developers.cloudflare.com/ai-gateway/usage/providers/cartesia/), [Cerebras](https://developers.cloudflare.com/ai-gateway/usage/providers/cerebras/), and [ElevenLabs](https://developers.cloudflare.com/ai-gateway/usage/providers/elevenlabs/), giving you more even more options for providers you can use through AI Gateway. Here's a brief overview of each: * [Cartesia](https://developers.cloudflare.com/ai-gateway/usage/providers/cartesia/) provides text-to-speech models that produce natural-sounding speech with low latency. * [Cerebras](https://developers.cloudflare.com/ai-gateway/usage/providers/cerebras/) delivers low-latency AI inference to Meta's Llama 3.1 8B and Llama 3.3 70B models. * [ElevenLabs](https://developers.cloudflare.com/ai-gateway/usage/providers/elevenlabs/) offers text-to-speech models with human-like voices in 32 languages. ![Example of Cerebras log in AI Gateway](https://developers.cloudflare.com/cdn-cgi/image/onerror=redirect,width=2278,height=1020,format=webp/_astro/cerebras2.qHYP0ZnF.png) To get started with AI Gateway, just update the base URL. Here's how you can send a request to [Cerebras](https://developers.cloudflare.com/ai-gateway/usage/providers/cerebras/) using cURL: ```bash curl -X POST https://gateway.ai.cloudflare.com/v1/ACCOUNT_TAG/GATEWAY/cerebras/chat/completions \ --header 'content-type: application/json' \ --header 'Authorization: Bearer CEREBRAS_TOKEN' \ --data '{ "model": "llama-3.3-70b", "messages": [ { "role": "user", "content": "What is Cloudflare?" } ] }' ``` ## 2025-01-30 **AI Gateway Introduces New Worker Binding Methods** We have released new [Workers bindings API methods](https://developers.cloudflare.com/ai-gateway/usage/worker-binding-methods/), allowing you to connect Workers applications to AI Gateway directly. These methods simplify how Workers calls AI services behind your AI Gateway configurations, removing the need to use the REST API and manually authenticate. To add an AI binding to your Worker, include the following in your [Wrangler configuration file](https://developers.cloudflare.com/workers/wrangler/configuration/): ![Add an AI binding to your Worker.](https://developers.cloudflare.com/cdn-cgi/image/onerror=redirect,width=754,height=135,format=webp/_astro/add-binding.BoYTiyon.png) With the new AI Gateway binding methods, you can now: * Send feedback and update metadata with `patchLog`. * Retrieve detailed log information using `getLog`. * Execute [universal requests](https://developers.cloudflare.com/ai-gateway/usage/universal/) to any AI Gateway provider with `run`. For example, to send feedback and update metadata using `patchLog`: ![Send feedback and update metadata using patchLog:](https://developers.cloudflare.com/cdn-cgi/image/onerror=redirect,width=736,height=235,format=webp/_astro/send-feedback.BGRzKmd9.png) ## 2025-01-02 **AI Gateway adds DeepSeek as a Provider** [**AI Gateway**](https://developers.cloudflare.com/ai-gateway/) now supports [**DeepSeek**](https://developers.cloudflare.com/ai-gateway/usage/providers/deepseek/), including their cutting-edge DeepSeek-V3 model. With this addition, you have even more flexibility to manage and optimize your AI workloads using AI Gateway. Whether you're leveraging DeepSeek or other providers, like OpenAI, Anthropic, or [Workers AI](https://developers.cloudflare.com/workers-ai/), AI Gateway empowers you to: * **Monitor**: Gain actionable insights with analytics and logs. * **Control**: Implement caching, rate limiting, and fallbacks. * **Optimize**: Improve performance with feedback and evaluations. ![AI Gateway adds DeepSeek as a provider](https://developers.cloudflare.com/cdn-cgi/image/onerror=redirect,width=1600,height=131,format=webp/_astro/deepseek.hirkr3rv.png) To get started, simply update the base URL of your DeepSeek API calls to route through AI Gateway. Here's how you can send a request using cURL: ```bash curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/deepseek/chat/completions \ --header 'content-type: application/json' \ --header 'Authorization: Bearer DEEPSEEK_TOKEN' \ --data '{ "model": "deepseek-chat", "messages": [ { "role": "user", "content": "What is Cloudflare?" } ] }' ``` For detailed setup instructions, see our [DeepSeek provider documentation](https://developers.cloudflare.com/ai-gateway/usage/providers/deepseek/). Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"BlogPosting","@id":"https://developers.cloudflare.com/ai-gateway/changelog/#page","headline":"Changelog · Cloudflare AI Gateway docs","description":"Track the latest updates, new features, and fixes for AI Gateway.","url":"https://developers.cloudflare.com/ai-gateway/changelog/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-06-05","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Reference all supported AI Gateway headers for configuring, customizing, and managing API requests. title: Header Glossary image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Header Glossary Last updated May 8, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/glossary/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) AI Gateway supports a variety of headers to help you configure, customize, and manage your API requests. This page provides a complete list of all supported headers, along with a short description | Term | Definition | | ---------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | cf-aig-backoff | Header to customize the backoff type for [request retries](https://developers.cloudflare.com/ai-gateway/configuration/request-handling/#request-retries) of a request. | | cf-aig-cache-key | The [cf-aig-cache-key-aig-cache-key](https://developers.cloudflare.com/ai-gateway/features/caching/#custom-cache-key-cf-aig-cache-key) let you override the default cache key in order to precisely set the cacheability setting for any resource. | | cf-aig-cache-status | [Status indicator for caching](https://developers.cloudflare.com/ai-gateway/features/caching/#default-configuration), showing if a request was served from cache. | | cf-aig-cache-ttl | Specifies the [cache time-to-live for responses](https://developers.cloudflare.com/ai-gateway/features/caching/#cache-ttl-cf-aig-cache-ttl). | | cf-aig-collect-log | The [cf-aig-collect-log](https://developers.cloudflare.com/ai-gateway/observability/logging/#collect-logs-cf-aig-collect-log) header allows you to bypass the default log setting for the gateway. | | cf-aig-custom-cost | Allows the [customization of request cost](https://developers.cloudflare.com/ai-gateway/configuration/custom-costs/#custom-cost) to reflect user-defined parameters. | | cf-aig-dlp | A response header returned when a [DLP policy](https://developers.cloudflare.com/ai-gateway/features/dlp/set-up-dlp/#dlp-response-header) matches a request or response. Contains JSON with the action taken (Flag or Block), matched policy IDs, matched profile IDs, and detection entry IDs. | | cf-aig-event-id | [cf-aig-event-id](https://developers.cloudflare.com/ai-gateway/evaluations/add-human-feedback-api/#3-retrieve-the-cf-aig-log-id) is a unique identifier for an event, used to trace specific events through the system. | | cf-aig-log-id | The [cf-aig-log-id](https://developers.cloudflare.com/ai-gateway/evaluations/add-human-feedback-api/#3-retrieve-the-cf-aig-log-id) is a unique identifier for the specific log entry to which you want to add feedback. | | cf-aig-max-attempts | Header to customize the number of max attempts for [request retries](https://developers.cloudflare.com/ai-gateway/configuration/request-handling/#request-retries) of a request. | | cf-aig-metadata | [Custom metadata](https://developers.cloudflare.com/ai-gateway/configuration/custom-metadata/)allows you to tag requests with user IDs or other identifiers, enabling better tracking and analysis of your requests. | | cf-aig-request-timeout | Header to set a [request timeout](https://developers.cloudflare.com/ai-gateway/configuration/request-handling/#request-timeouts) (measured in milliseconds). If the provider does not respond within this time, the request returns an error. | | cf-aig-retry-delay | Header to customize the retry delay for [request retries](https://developers.cloudflare.com/ai-gateway/configuration/request-handling/#request-retries) of a request. | | cf-aig-skip-cache | Header to [bypass caching for a specific request](https://developers.cloudflare.com/ai-gateway/features/caching/#skip-cache-cf-aig-skip-cache). | | cf-aig-step | The cf-aig-step response header identifies which step in a request flow successfully processed the request, useful for tracking and debugging. | | cf-cache-ttl | Deprecated: This header is replaced by cf-aig-cache-ttl. It specifies cache time-to-live. | | cf-skip-cache | Deprecated: This header is replaced by cf-aig-skip-cache. It bypasses caching for a specific request. | ## Configuration hierarchy Settings in AI Gateway can be configured at two levels: **Request** and **Gateway**. Since the same settings can be configured in multiple locations, the following hierarchy determines which value is applied: 1. **Request-level headers**: Headers included in individual requests take precedence over gateway-level settings. 2. **Gateway-level settings**: Act as the default if no headers are set at the request level. This hierarchy ensures consistent behavior, prioritizing the most specific configurations. Use request-level headers for fine-tuned control, and gateway settings for general defaults. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/glossary/#page","headline":"Header Glossary · Cloudflare AI Gateway docs","description":"Reference all supported AI Gateway headers for configuring, customizing, and managing API requests.","url":"https://developers.cloudflare.com/ai-gateway/glossary/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-05-08","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- ## List Gateways **get** `/accounts/{account_id}/ai-gateway/gateways` Lists all AI Gateway evaluator types configured for the account. ### Path Parameters - `account_id: string` ### Query Parameters - `page: optional number` - `per_page: optional number` - `search: optional string` Search by id ### Returns - `result: array of object { id, cache_invalidate_on_update, cache_ttl, 24 more }` - `id: string` gateway id - `cache_invalidate_on_update: boolean` - `cache_ttl: number` - `collect_logs: boolean` - `created_at: string` - `modified_at: string` - `rate_limiting_interval: number` - `rate_limiting_limit: number` - `authentication: optional boolean` - `dlp: optional object { action, enabled, profiles } or object { enabled, policies }` - `object { action, enabled, profiles }` - `action: "BLOCK" or "FLAG"` - `"BLOCK"` - `"FLAG"` - `enabled: boolean` - `profiles: array of string` - `object { enabled, policies }` - `enabled: boolean` - `policies: array of object { id, action, check, 2 more }` - `id: string` - `action: "FLAG" or "BLOCK"` - `"FLAG"` - `"BLOCK"` - `check: array of "REQUEST" or "RESPONSE"` - `"REQUEST"` - `"RESPONSE"` - `enabled: boolean` - `profiles: array of string` - `guardrails: optional object { prompt, response }` - `prompt: object { P1, S1, S10, 11 more }` - `P1: optional "FLAG" or "BLOCK"` - `"FLAG"` - `"BLOCK"` - `S1: optional "FLAG" or "BLOCK"` - `"FLAG"` - `"BLOCK"` - `S10: optional "FLAG" or "BLOCK"` - `"FLAG"` - `"BLOCK"` - `S11: optional "FLAG" or "BLOCK"` - `"FLAG"` - `"BLOCK"` - `S12: optional "FLAG" or "BLOCK"` - `"FLAG"` - `"BLOCK"` - `S13: optional "FLAG" or "BLOCK"` - `"FLAG"` - `"BLOCK"` - `S2: optional "FLAG" or "BLOCK"` - `"FLAG"` - `"BLOCK"` - `S3: optional "FLAG" or "BLOCK"` - `"FLAG"` - `"BLOCK"` - `S4: optional "FLAG" or "BLOCK"` - `"FLAG"` - `"BLOCK"` - `S5: optional "FLAG" or "BLOCK"` - `"FLAG"` - `"BLOCK"` - `S6: optional "FLAG" or "BLOCK"` - `"FLAG"` - `"BLOCK"` - `S7: optional "FLAG" or "BLOCK"` - `"FLAG"` - `"BLOCK"` - `S8: optional "FLAG" or "BLOCK"` - `"FLAG"` - `"BLOCK"` - `S9: optional "FLAG" or "BLOCK"` - `"FLAG"` - `"BLOCK"` - `response: object { P1, S1, S10, 11 more }` - `P1: optional "FLAG" or "BLOCK"` - `"FLAG"` - `"BLOCK"` - `S1: optional "FLAG" or "BLOCK"` - `"FLAG"` - `"BLOCK"` - `S10: optional "FLAG" or "BLOCK"` - `"FLAG"` - `"BLOCK"` - `S11: optional "FLAG" or "BLOCK"` - `"FLAG"` - `"BLOCK"` - `S12: optional "FLAG" or "BLOCK"` - `"FLAG"` - `"BLOCK"` - `S13: optional "FLAG" or "BLOCK"` - `"FLAG"` - `"BLOCK"` - `S2: optional "FLAG" or "BLOCK"` - `"FLAG"` - `"BLOCK"` - `S3: optional "FLAG" or "BLOCK"` - `"FLAG"` - `"BLOCK"` - `S4: optional "FLAG" or "BLOCK"` - `"FLAG"` - `"BLOCK"` - `S5: optional "FLAG" or "BLOCK"` - `"FLAG"` - `"BLOCK"` - `S6: optional "FLAG" or "BLOCK"` - `"FLAG"` - `"BLOCK"` - `S7: optional "FLAG" or "BLOCK"` - `"FLAG"` - `"BLOCK"` - `S8: optional "FLAG" or "BLOCK"` - `"FLAG"` - `"BLOCK"` - `S9: optional "FLAG" or "BLOCK"` - `"FLAG"` - `"BLOCK"` - `is_default: optional boolean` - `log_classification: optional boolean` - `log_management: optional number` - `log_management_strategy: optional "STOP_INSERTING" or "DELETE_OLDEST"` - `"STOP_INSERTING"` - `"DELETE_OLDEST"` - `logpush: optional boolean` - `logpush_public_key: optional string` - `otel: optional array of object { headers, url, authorization, content_type }` - `headers: map[string]` - `url: string` - `authorization: optional string` - `content_type: optional "json" or "protobuf"` - `"json"` - `"protobuf"` - `rate_limiting_technique: optional "fixed" or "sliding"` - `"fixed"` - `"sliding"` - `retry_backoff: optional "constant" or "linear" or "exponential"` Backoff strategy for retry delays - `"constant"` - `"linear"` - `"exponential"` - `retry_delay: optional number` Delay between retry attempts in milliseconds (0-5000) - `retry_max_attempts: optional number` Maximum number of retry attempts for failed requests (1-5) - `spend_limits: optional object { enabled, rules }` - `enabled: optional boolean` - `rules: optional array of object { limit, limitType, window, 6 more }` - `limit: number` - `limitType: "cost"` - `"cost"` - `window: number` - `id: optional string` - `enabled: optional boolean` - `metadata: optional map[object { mode } or object { mode, values } ]` - `Mode object { mode }` - `mode: "partition"` - `"partition"` - `object { mode, values }` - `mode: "filter"` - `"filter"` - `values: array of string` - `model: optional object { mode, values }` - `mode: "filter"` - `"filter"` - `values: array of string` - `provider: optional object { mode, values }` - `mode: "filter"` - `"filter"` - `values: array of string` - `technique: optional "fixed" or "sliding"` - `"fixed"` - `"sliding"` - `store_id: optional string` - `stripe: optional object { authorization, usage_events }` - `authorization: string` - `usage_events: array of object { payload }` - `payload: string` - `workers_ai_billing_mode: optional "postpaid" or "unified"` Controls how Workers AI inference calls routed through this gateway are billed. 'postpaid' bills the account directly through Workers AI; 'unified' deducts credits via AI Gateway using neuron-based pricing and delegates billing to AI Gateway. - `"postpaid"` - `"unified"` - `zdr: optional boolean` - `success: boolean` ### Example ```http curl https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/ai-gateway/gateways \ -H "Authorization: Bearer $CLOUDFLARE_API_TOKEN" ``` #### Response ```json { "result": [ { "id": "my-gateway", "cache_invalidate_on_update": true, "cache_ttl": 0, "collect_logs": true, "created_at": "2019-12-27T18:11:19.117Z", "modified_at": "2019-12-27T18:11:19.117Z", "rate_limiting_interval": 0, "rate_limiting_limit": 0, "authentication": true, "dlp": { "action": "BLOCK", "enabled": true, "profiles": [ "string" ] }, "guardrails": { "prompt": { "P1": "FLAG", "S1": "FLAG", "S10": "FLAG", "S11": "FLAG", "S12": "FLAG", "S13": "FLAG", "S2": "FLAG", "S3": "FLAG", "S4": "FLAG", "S5": "FLAG", "S6": "FLAG", "S7": "FLAG", "S8": "FLAG", "S9": "FLAG" }, "response": { "P1": "FLAG", "S1": "FLAG", "S10": "FLAG", "S11": "FLAG", "S12": "FLAG", "S13": "FLAG", "S2": "FLAG", "S3": "FLAG", "S4": "FLAG", "S5": "FLAG", "S6": "FLAG", "S7": "FLAG", "S8": "FLAG", "S9": "FLAG" } }, "is_default": true, "log_classification": true, "log_management": 10000, "log_management_strategy": "STOP_INSERTING", "logpush": true, "logpush_public_key": "xxxxxxxxxxxxxxxx", "otel": [ { "headers": { "foo": "string" }, "url": "https://example.com", "authorization": "authorization", "content_type": "json" } ], "rate_limiting_technique": "fixed", "retry_backoff": "constant", "retry_delay": 0, "retry_max_attempts": 1, "spend_limits": { "enabled": true, "rules": [ { "limit": 1, "limitType": "cost", "window": 1, "id": "x", "enabled": true, "metadata": { "foo": { "mode": "partition" } }, "model": { "mode": "filter", "values": [ "string" ] }, "provider": { "mode": "filter", "values": [ "string" ] }, "technique": "fixed" } ] }, "store_id": "store_id", "stripe": { "authorization": "authorization", "usage_events": [ { "payload": "payload" } ] }, "workers_ai_billing_mode": "postpaid", "zdr": true } ], "success": true } ``` --- --- description: Assess AI Gateway application performance with datasets, human feedback, and evaluation metrics. title: Evaluations image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Evaluations Last updated Jul 28, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/evaluations/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Deprecated Evaluations are deprecated and no longer supported for new accounts. Understanding your application's performance is essential for optimization. Developers often have different priorities, and finding the optimal solution involves balancing key factors such as cost, latency, and accuracy. Some prioritize low-latency responses, while others focus on accuracy or cost-efficiency. AI Gateway's Evaluations provide the data needed to make informed decisions on how to optimize your AI application. Whether it is adjusting the model, provider, or prompt, this feature delivers insights into key metrics around performance, speed, and cost. It empowers developers to better understand their application's behavior, ensuring improved accuracy, reliability, and customer satisfaction. Evaluations use datasets which are collections of logs stored for analysis. You can create datasets by applying filters in the Logs tab, which help narrow down specific logs for evaluation. Our first step toward comprehensive AI evaluations starts with human feedback (currently in open beta). We will continue to build and expand AI Gateway with additional evaluators. [Learn how to set up an evaluation](https://developers.cloudflare.com/ai-gateway/evaluations/set-up-evaluations/) including creating datasets, selecting evaluators, and running the evaluation process. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"WebPage","@id":"https://developers.cloudflare.com/ai-gateway/evaluations/#page","headline":"Evaluations · Cloudflare AI Gateway docs","description":"Assess AI Gateway application performance with datasets, human feedback, and evaluation metrics.","url":"https://developers.cloudflare.com/ai-gateway/evaluations/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-07-28","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Annotate AI Gateway logs with thumbs-up or thumbs-down feedback in the Cloudflare dashboard. title: Add Human Feedback using Dashboard image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Add Human Feedback using Dashboard Last updated Apr 20, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/evaluations/add-human-feedback/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Human feedback is a valuable metric to assess the performance of your AI models. By incorporating human feedback, you can gain deeper insights into how the model's responses are perceived and how well it performs from a user-centric perspective. This feedback can then be used in evaluations to calculate performance metrics, driving optimization and ultimately enhancing the reliability, accuracy, and efficiency of your AI application. Human feedback measures the performance of your dataset based on direct human input. The metric is calculated as the percentage of positive feedback (thumbs up) given on logs, which are annotated in the Logs tab of the Cloudflare dashboard. This feedback helps refine model performance by considering real-world evaluations of its output. This tutorial will guide you through the process of adding human feedback to your evaluations in AI Gateway using the Cloudflare dashboard. On the next guide, you can [learn how to add human feedback via the API](https://developers.cloudflare.com/ai-gateway/evaluations/add-human-feedback-api/). ## 1\. Log in to the dashboard In the Cloudflare dashboard, go to the **AI Gateway** page. [Go to **AI Gateway** ↗](https://dash.cloudflare.com/?to=/:account/ai/ai-gateway) ## 2\. Access the Logs tab 1. Go to **Logs**. 2. The Logs tab displays all logs associated with your datasets. These logs show key information, including: * Timestamp: When the interaction occurred. * Status: Whether the request was successful, cached, or failed. * Model: The model used in the request. * Tokens: The number of tokens consumed by the response. * Cost: The cost based on token usage. * Duration: The time taken to complete the response. * Feedback: Where you can provide human feedback on each log. ## 3\. Provide human feedback 1. Click on the log entry you want to review. This expands the log, allowing you to see more detailed information. 2. In the expanded log, you can view additional details such as: * The user prompt. * The model response. * HTTP response details. * Endpoint information. 3. You will see two icons: * Thumbs up: Indicates positive feedback. * Thumbs down: Indicates negative feedback. 4. Click either the thumbs up or thumbs down icon based on how you rate the model response for that particular log entry. ## 4\. Evaluate human feedback After providing feedback on your logs, it becomes a part of the evaluation process. When you run an evaluation (as outlined in the [Set Up Evaluations](https://developers.cloudflare.com/ai-gateway/evaluations/set-up-evaluations/) guide), the human feedback metric will be calculated based on the percentage of logs that received thumbs-up feedback. Note You need to select human feedback as an evaluator to receive its metrics. ## 5\. Review results After running the evaluation, review the results on the Evaluations tab. You will be able to see the performance of the model based on cost, speed, and now human feedback, represented as the percentage of positive feedback (thumbs up). The human feedback score is displayed as a percentage, showing the distribution of positively rated responses from the database. For more information on running evaluations, refer to the documentation [Set Up Evaluations](https://developers.cloudflare.com/ai-gateway/evaluations/set-up-evaluations/). Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/evaluations/add-human-feedback/#page","headline":"Add Human Feedback using Dashboard · Cloudflare AI Gateway docs","description":"Annotate AI Gateway logs with thumbs-up or thumbs-down feedback in the Cloudflare dashboard.","url":"https://developers.cloudflare.com/ai-gateway/evaluations/add-human-feedback/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-20","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Submit human feedback on AI Gateway request logs using the Cloudflare API. title: Add Human Feedback using API image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Add Human Feedback using API Last updated Apr 20, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/evaluations/add-human-feedback-api/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) This guide will walk you through the steps of adding human feedback to an AI Gateway request using the Cloudflare API. You will learn how to retrieve the relevant request logs, and submit feedback using the API. If you prefer to add human feedback via the dashboard, refer to [Add Human Feedback](https://developers.cloudflare.com/ai-gateway/evaluations/add-human-feedback/). ## 1\. Create an API Token 1. [Create an API token](https://developers.cloudflare.com/fundamentals/api/get-started/create-token/) with the following permissions: * `AI Gateway - Read` * `AI Gateway - Edit` 1. Get your [Account ID](https://developers.cloudflare.com/fundamentals/account/find-account-and-zone-ids/). 2. Using that API token and Account ID, send a [POST request](https://developers.cloudflare.com/api/resources/ai%5Fgateway/methods/create/) to the Cloudflare API. ## 2\. Retrieve the `cf-aig-log-id` The `cf-aig-log-id` is a unique identifier for the specific log entry to which you want to add feedback. Below are two methods to obtain this identifier. ### Method 1: Locate the `cf-aig-log-id` in the request response This method allows you to directly find the `cf-aig-log-id` within the header of the response returned by the AI Gateway. This is the most straightforward approach if you have access to the original API response. The steps below outline how to do this. 1. **Make a Request to the AI Gateway**: This could be a request your application sends to the AI Gateway. Once the request is made, the response will contain various pieces of metadata. 2. **Check the Response Headers**: The response will include a header named `cf-aig-log-id`. This is the identifier you will need to submit feedback. In the example below, the `cf-aig-log-id` is `01JADMCQQQBWH3NXZ5GCRN98DP`. ```json { "status": "success", "headers": { "cf-aig-log-id": "01JADMCQQQBWH3NXZ5GCRN98DP" }, "data": { "response": "Sample response data" } } ``` ### Method 2: Retrieve the `cf-aig-log-id` via API (GET request) If you do not have the `cf-aig-log-id` in the response body or you need to access it after the fact, you are able to retrieve it by querying the logs using the [Cloudflare API](https://developers.cloudflare.com/api/resources/ai%5Fgateway/subresources/logs/methods/list/). Send a `GET` request to get a list of logs and then find a specific ID Required API token permissions At least one of the following [token permissions](https://developers.cloudflare.com/fundamentals/api/reference/permissions/) is required: * `AI Gateway Write` * `AI Gateway Read` ```bash curl "https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/ai-gateway/gateways/$GATEWAY_ID/logs" \ --request GET \ --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" ``` ```json { "result": [ { "id": "01JADMCQQQBWH3NXZ5GCRN98DP", "cached": true, "created_at": "2019-08-24T14:15:22Z", "custom_cost": true, "duration": 0, "id": "string", "metadata": "string", "model": "string", "model_type": "string", "path": "string", "provider": "string", "request_content_type": "string", "request_type": "string", "response_content_type": "string", "status_code": 0, "step": 0, "success": true, "tokens_in": 0, "tokens_out": 0 } ] } ``` ### Method 3: Retrieve the `cf-aig-log-id` via a binding You can also retrieve the `cf-aig-log-id` using a binding, which streamlines the process. Here's how to retrieve the log ID directly: ```js const resp = await env.AI.run( "@cf/meta/llama-3-8b-instruct", { prompt: "tell me a joke", }, { gateway: { id: "my_gateway_id", }, }, ); const myLogId = env.AI.aiGatewayLogId; ``` Note: The `aiGatewayLogId` property, will only hold the last inference call log id. ## 3\. Submit feedback via PATCH request Once you have both the API token and the `cf-aig-log-id`, you can send a PATCH request to submit feedback. Required API token permissions At least one of the following [token permissions](https://developers.cloudflare.com/fundamentals/api/reference/permissions/) is required: * `AI Gateway Write` ```bash curl "https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/ai-gateway/gateways/$GATEWAY_ID/logs/$ID" \ --request PATCH \ --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ --json '{ "feedback": 1 }' ``` If you had negative feedback, adjust the body of the request to be `-1`. ```json { "feedback": -1 } ``` ## 4\. Verify the feedback submission You can verify the feedback submission in two ways: * **Through the [Cloudflare dashboard ↗](https://dash.cloudflare.com)**: check the updated feedback on the AI Gateway interface. * **Through the API**: Send another GET request to retrieve the updated log entry and confirm the feedback has been recorded. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/evaluations/add-human-feedback-api/#page","headline":"Add Human Feedback using API · Cloudflare AI Gateway docs","description":"Submit human feedback on AI Gateway request logs using the Cloudflare API.","url":"https://developers.cloudflare.com/ai-gateway/evaluations/add-human-feedback-api/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-20","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Provide human feedback on AI Gateway evaluations programmatically using Worker bindings. title: Add human feedback using Worker Bindings image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Add human feedback using Worker Bindings Last updated Jun 12, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/evaluations/add-human-feedback-bindings/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) This guide explains how to provide human feedback for AI Gateway evaluations using Worker bindings. ## 1\. Run an AI Evaluation Start by sending a prompt to the AI model through your AI Gateway. ```javascript const resp = await env.AI.run( "@cf/meta/llama-3.1-8b-instruct", { prompt: "tell me a joke", }, { gateway: { id: "my-gateway", }, }, ); const myLogId = env.AI.aiGatewayLogId; ``` Let the user interact with or evaluate the AI response. This interaction will inform the feedback you send back to the AI Gateway. ## 2\. Send Human Feedback Use the [patchLog()](https://developers.cloudflare.com/ai-gateway/usage/worker-binding-methods/#patchlog) method to provide feedback for the AI evaluation. ```javascript await env.AI.gateway("my-gateway").patchLog(myLogId, { feedback: 1, // all fields are optional; set values that fit your use case score: 100, metadata: { user: "123", // Optional metadata to provide additional context }, }); ``` ## Feedback parameters explanation * `feedback`: is either `-1` for negative or `1` to positive, `0` is considered not evaluated. * `score`: A number between 0 and 100. * `metadata`: An object containing additional contextual information. ### patchLog: Send Feedback The `patchLog` method allows you to send feedback, score, and metadata for a specific log ID. All object properties are optional, so you can include any combination of the parameters: ```javascript gateway.patchLog("my-log-id", { feedback: 1, score: 100, metadata: { user: "123", }, }); ``` Returns: `Promise` (Make sure to `await` the request.) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/evaluations/add-human-feedback-bindings/#page","headline":"Add human feedback using Worker Bindings · Cloudflare AI Gateway docs","description":"Provide human feedback on AI Gateway evaluations programmatically using Worker bindings.","url":"https://developers.cloudflare.com/ai-gateway/evaluations/add-human-feedback-bindings/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-06-12","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Create datasets, select evaluators, and run evaluations for your AI Gateway logs. title: Set up Evaluations image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Set up Evaluations Last updated Jul 28, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/evaluations/set-up-evaluations/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Deprecated Evaluations are deprecated and no longer supported for new accounts. This guide walks you through the process of setting up an evaluation in AI Gateway. These steps are done in the [Cloudflare dashboard ↗](https://dash.cloudflare.com/). ## 1\. Select or create a dataset Datasets are collections of logs stored for analysis that can be used in an evaluation. You can create datasets by applying filters in the Logs tab. Datasets will update automatically based on the set filters. ### Set up a dataset from the Logs tab 1. Apply filters to narrow down your logs. Filter options include provider, number of tokens, request status, and more. 2. Select **Create Dataset** to store the filtered logs for future analysis. You can manage datasets by selecting **Manage datasets** from the Logs tab. Note Please keep in mind that datasets currently use `AND` joins, so there can only be one item per filter (for example, one model or one provider). Future updates will allow more flexibility in dataset creation. ### List of available filters | Filter category | Filter options | Filter by description | | --------------- | ------------------------------------------------------------ | ----------------------------------------- | | Status | error, status | error type or status. | | Cache | cached, not cached | based on whether they were cached or not. | | Provider | specific providers | the selected AI provider. | | AI Models | specific models | the selected AI model. | | Cost | less than, greater than | cost, specifying a threshold. | | Request type | Workers AI Binding, WebSockets | the type of request. | | Tokens | Total tokens, Tokens In, Tokens Out | token count (less than or greater than). | | Duration | less than, greater than | request duration. | | Feedback | equals, does not equal (thumbs up, thumbs down, no feedback) | feedback type. | | Metadata Key | equals, does not equal | specific metadata keys. | | Metadata Value | equals, does not equal | specific metadata values. | | Log ID | equals, does not equal | a specific Log ID. | | Event ID | equals, does not equal | a specific Event ID. | ## 2\. Select evaluators After creating a dataset, choose the evaluation parameters: * Cost: Calculates the average cost of inference requests within the dataset (only for requests with [cost data](https://developers.cloudflare.com/ai-gateway/observability/costs/)). * Speed: Calculates the average duration of inference requests within the dataset. * Performance: * Human feedback: measures performance based on human feedback, calculated by the % of thumbs up on the logs, annotated from the Logs tab. Note Additional evaluators will be introduced in future updates to expand performance analysis capabilities. ## 3\. Name, review, and run the evaluation 1. Create a unique name for your evaluation to reference it in the dashboard. 2. Review the selected dataset and evaluators. 3. Select **Run** to start the process. ## 4\. Review and analyze results Evaluation results will appear in the Evaluations tab. The results show the status of the evaluation (for example, in progress, completed, or error). Metrics for the selected evaluators will be displayed, excluding any logs with missing fields. You will also see the number of logs used to calculate each metric. While datasets automatically update based on filters, evaluations do not. You will have to create a new evaluation if you want to evaluate new logs. Use these insights to optimize based on your application's priorities. Based on the results, you may choose to: * Change the model or [provider](https://developers.cloudflare.com/ai-gateway/usage/providers/) * Adjust your prompts * Explore further optimizations, such as setting up [Retrieval Augmented Generation (RAG)](https://developers.cloudflare.com/reference-architecture/diagrams/ai/ai-rag/) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/evaluations/set-up-evaluations/#page","headline":"Set up Evaluations · Cloudflare AI Gateway docs","description":"Create datasets, select evaluators, and run evaluations for your AI Gateway logs.","url":"https://developers.cloudflare.com/ai-gateway/evaluations/set-up-evaluations/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-07-28","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Explore reference architectures and design guides that incorporate AI Gateway into your infrastructure. title: Architectures image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Architectures Last updated Apr 20, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/demos/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Learn how you can use AI Gateway within your existing architecture. ## Reference architectures Explore the following reference architectures that use AI Gateway: [**Fullstack applications**A practical example of how these services come together in a real fullstack application architecture.](https://developers.cloudflare.com/reference-architecture/diagrams/serverless/fullstack-application/) [**Multi-vendor AI observability and control**By shifting features such as rate limiting, caching, and error handling to the proxy layer, organizations can apply unified configurations across services and inference service providers.](https://developers.cloudflare.com/reference-architecture/diagrams/ai/ai-multivendor-observability-control/) [**AI Vibe Coding Platform**Cloudflare's low-latency, fully serverless compute platform, Workers offers powerful capabilities to enable A/B testing using a server-side implementation.](https://developers.cloudflare.com/reference-architecture/diagrams/ai/ai-vibe-coding-platform/) [**Enterprise AI agent workspace**Reference architecture for building governed, stateful enterprise AI agent workspaces on Cloudflare.](https://developers.cloudflare.com/reference-architecture/diagrams/ai/enterprise-ai-agent-workspace/) [**Enterprise AI Vibe Coding Platform**Reference architecture for building a governed enterprise AI vibe coding platform on Cloudflare.](https://developers.cloudflare.com/reference-architecture/diagrams/ai/enterprise-ai-vibe-coding-platform/) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"WebPage","@id":"https://developers.cloudflare.com/ai-gateway/demos/#page","headline":"Architectures · Cloudflare AI Gateway docs","description":"Explore reference architectures and design guides that incorporate AI Gateway into your infrastructure.","url":"https://developers.cloudflare.com/ai-gateway/demos/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-20","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Add security by requiring a valid authorization token for each request. title: Authenticated Gateway image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Authenticated Gateway Last updated Jun 17, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/configuration/authentication/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) AI Gateway requires a valid Cloudflare API token for each request. This prevents unauthorized access and protects against invalid requests that can inflate log storage usage. When using the [REST API](https://developers.cloudflare.com/ai-gateway/usage/rest-api/), pass your Cloudflare API token in the standard `Authorization` header. When using [provider-native endpoints](https://developers.cloudflare.com/ai-gateway/usage/providers/) at `gateway.ai.cloudflare.com`, use the `cf-aig-authorization` header instead. Note The `cf-aig-authorization` header is used with the `gateway.ai.cloudflare.com` endpoints, which continue to work. For new integrations, we recommend using the [REST API](https://developers.cloudflare.com/ai-gateway/usage/rest-api/) at `api.cloudflare.com`, which uses the standard `Authorization` header. ## Setting up Authenticated Gateway using the dashboard 1. Go to the Settings for the specific gateway you want to enable authentication for. 2. Select **Create authentication token** to generate a custom token with the required `Run` permissions. Be sure to securely save this token, as it will not be displayed again. 3. Include the API token in each request: * If using the REST API (`/ai/run`), include your Cloudflare API token in the standard `Authorization` header. * If using [provider-native endpoints](https://developers.cloudflare.com/ai-gateway/usage/providers/) at `gateway.ai.cloudflare.com`, use the `cf-aig-authorization` header. 4. Return to the settings page and toggle on Authenticated Gateway. AI Gateway API tokens are account-scoped The `AI Gateway Read`, `Run`, and `Edit` permissions cannot be restricted to a single gateway — unlike R2, which supports per-bucket scoping. Any token with `AI Gateway Run` can send requests through every gateway in the account, including any configured with stored provider keys through [Bring Your Own Keys (BYOK)](https://developers.cloudflare.com/ai-gateway/configuration/bring-your-own-keys/), consuming those credentials. For isolation between gateways or tenants, use separate Cloudflare accounts or a Worker-side [AI Gateway binding](https://developers.cloudflare.com/ai-gateway/integrations/aig-workers-ai-binding/) rather than relying on token scope. ## Example requests ```bash # Run `wrangler whoami` to get your account ID to replace $CLOUDFLARE_ACCOUNT_ID, # and `wrangler auth token` to get an auth token to replace $CLOUDFLARE_API_TOKEN. curl -X POST "https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/v1/chat/completions" \ --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ --header "Content-Type: application/json" \ --data '{"model": "openai/gpt-4.1-mini", "messages": [{"role": "user", "content": "What is Cloudflare?"}]}' ``` Using the OpenAI SDK: ```javascript import OpenAI from "openai"; const openai = new OpenAI({ apiKey: CLOUDFLARE_API_TOKEN, baseURL: `https://api.cloudflare.com/client/v4/accounts/${ACCOUNT_ID}/ai/v1`, }); const response = await openai.chat.completions.create({ model: "openai/gpt-4.1-mini", messages: [{ role: "user", content: "What is Cloudflare?" }], }); ``` Using the Vercel AI SDK: ```javascript import { createOpenAI } from "@ai-sdk/openai"; const openai = createOpenAI({ apiKey: CLOUDFLARE_API_TOKEN, baseURL: `https://api.cloudflare.com/client/v4/accounts/${ACCOUNT_ID}/ai/v1`, }); ``` ## Expected behavior Note When an AI Gateway is accessed from a Cloudflare Worker using a **binding**, the `cf-aig-authorization` header does not need to be manually included. Requests made through bindings are **pre-authenticated** within the associated Cloudflare account. The following table outlines gateway behavior based on the authentication settings and header status: | Authentication Setting | Header Info | Gateway State | Response | | ---------------------- | -------------- | ----------------------- | ------------------------------------------ | | On | Header present | Authenticated gateway | Request succeeds | | On | No header | Error | Request fails due to missing authorization | | Off | Header present | Unauthenticated gateway | Request succeeds | | Off | No header | Unauthenticated gateway | Request succeeds | Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/configuration/authentication/#page","headline":"Authenticated Gateway · Cloudflare AI Gateway docs","description":"Add security by requiring a valid authorization token for each request.","url":"https://developers.cloudflare.com/ai-gateway/configuration/authentication/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-06-17","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Securely store AI provider API keys in AI Gateway and reference them in your gateway configuration. title: BYOK (Store Keys) image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # BYOK (Store Keys) Last updated Jul 31, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/configuration/bring-your-own-keys/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) ## Introduction Bring your own keys (BYOK) is a feature in Cloudflare AI Gateway that allows you to securely store your AI provider API keys directly in the Cloudflare dashboard. Instead of including API keys in every request to your AI models, you can configure them once in the dashboard, and reference them in your gateway configuration. The keys are stored securely with [Secrets Store](https://developers.cloudflare.com/secrets-store/) and allows for: * Secure storage and limit exposure * Easier key rotation * Rate limit, budget limit and other restrictions with [Dynamic Routes](https://developers.cloudflare.com/ai-gateway/features/dynamic-routing/) ## Setting up BYOK ### Prerequisites * Ensure your gateway is [authenticated](https://developers.cloudflare.com/ai-gateway/configuration/authentication/). * Ensure you have appropriate [permissions](https://developers.cloudflare.com/secrets-store/access-control/) to create and deploy secrets on Secrets Store. ### Configure API keys You can configure BYOK from the dashboard or by using the API. #### Dashboard When you add a provider key from the dashboard, AI Gateway creates and names the Secrets Store secret automatically. 1. Log into the [Cloudflare dashboard ↗](https://dash.cloudflare.com/) and select your account. 2. Go to **AI** \> **AI Gateway**. 3. Select your gateway or create a new one. 4. Go to the **Provider Keys** section. 5. Click **Add API Key**. 6. Select your AI provider from the dropdown. 7. Enter your API key and optionally provide a description. 8. Click **Save**. #### API If you use the API to configure BYOK, create the Secrets Store secret before you create the provider configuration. Name the secret with this format: ```txt {gateway_id}_{provider_slug}_{alias} ``` For example, for gateway `my-gateway`, provider `anthropic`, and alias `default`, create the Secrets Store secret as: ```txt my-gateway_anthropic_default ``` Then create the provider configuration with the same `provider_slug` and `alias` values. The `secret_id` returned by Secrets Store is not used by AI Gateway for runtime lookup, so API-created secrets must follow the naming convention. ### Update your applications Once you've configured your API keys in the dashboard: 1. **Remove API keys from your code**: Delete any hardcoded API keys or environment variables. 2. **Update request headers**: Remove provider authorization headers from your requests. Note that you still need to pass `cf-aig-authorization`. 3. **Test your integration**: Verify that requests work without including API keys. ## Example With BYOK enabled, your workflow changes from: 1. **Traditional approach**: Include API key in every request header ```bash curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/openai/chat/completions \ -H 'cf-aig-authorization: Bearer {CF_AIG_TOKEN}' \ -H "Authorization: Bearer YOUR_OPENAI_API_KEY" \ -H "Content-Type: application/json" \ -d '{"model": "gpt-4", "messages": [...]}' ``` 2. **BYOK approach**: Configure key once in dashboard, make requests without exposing keys ```bash curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/openai/chat/completions \ -H 'cf-aig-authorization: Bearer {CF_AIG_TOKEN}' \ -H "Content-Type: application/json" \ -d '{"model": "gpt-4", "messages": [...]}' ``` ## Managing API keys ### Viewing configured keys In the AI Gateway dashboard, you can: * View all configured API keys by provider * See when each key was last used * Check the status of each key (active, expired, invalid) ### Rotating keys To rotate an API key: 1. Generate a new API key from your AI provider 2. In the Cloudflare dashboard, edit the existing key entry 3. Replace the old key with the new one 4. Save the changes Your applications will immediately start using the new key without any code changes or downtime. ### Revoking access To remove an API key: 1. In the AI Gateway dashboard, find the key you want to remove 2. Click the **Delete** button 3. Confirm the deletion Impact of key deletion Deleting an API key will immediately stop all requests that depend on it. Make sure to update your applications or configure alternative keys before deletion. ## Multiple keys per provider AI Gateway supports storing multiple API keys for the same provider. This allows you to: * Use different keys for different use cases (for example, development vs production) * Gradually migrate between keys during rotation ### Key aliases Each API key can be assigned an alias to identify it. When you add a key, you can specify a custom alias, or the system will use `default` as the alias. When making requests, AI Gateway uses the key with the `default` alias by default. To use a different key, include the `cf-aig-byok-alias` header with the alias of the key you want to use. Note The `cf-aig-byok-alias` header applies to [direct provider-passthrough](https://developers.cloudflare.com/ai-gateway/usage/providers/) requests. On requests routed through [Unified Billing](https://developers.cloudflare.com/ai-gateway/features/unified-billing/) endpoints (for example, `env.AI.run()` or `/ai/v1/chat/completions`), only the `default` alias is consulted — if the `default` key is missing, the request falls through to Unified Billing. See [Credential precedence](https://developers.cloudflare.com/ai-gateway/features/unified-billing/#credential-precedence) for the full order. ### Example: Using a specific key alias If you have multiple OpenAI keys configured with different aliases (for example, `default`, `production`, and `testing`), you can specify which one to use: ```bash # Uses the key with alias "default" (no header needed) curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/openai/chat/completions \ -H 'cf-aig-authorization: Bearer {CF_AIG_TOKEN}' \ -H "Content-Type: application/json" \ -d '{"model": "gpt-4", "messages": [...]}' ``` ```bash # Uses the key with alias "production" curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/openai/chat/completions \ -H 'cf-aig-authorization: Bearer {CF_AIG_TOKEN}' \ -H 'cf-aig-byok-alias: production' \ -H "Content-Type: application/json" \ -d '{"model": "gpt-4", "messages": [...]}' ``` ```bash # Uses the key with alias "testing" curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/openai/chat/completions \ -H 'cf-aig-authorization: Bearer {CF_AIG_TOKEN}' \ -H 'cf-aig-byok-alias: testing' \ -H "Content-Type: application/json" \ -d '{"model": "gpt-4", "messages": [...]}' ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"WebPage","@id":"https://developers.cloudflare.com/ai-gateway/configuration/bring-your-own-keys/#page","headline":"BYOK (Store Keys) · Cloudflare AI Gateway docs","description":"Securely store AI provider API keys in AI Gateway and reference them in your gateway configuration.","url":"https://developers.cloudflare.com/ai-gateway/configuration/bring-your-own-keys/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-07-31","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Add identity-aware controls to AI Gateway so users authenticate with your identity provider before they can call your gateway. title: Cloudflare Access image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Cloudflare Access Last updated Aug 5, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/configuration/cloudflare-access/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Protect your gateway with [Cloudflare Access](https://developers.cloudflare.com/cloudflare-one/access-controls/applications/http-apps/) so users authenticate with your identity provider before they can send requests. Putting AI Gateway behind Access gives you identity-aware control over your AI traffic: you decide who can reach the gateway, tie each request to a verified user, and govern usage per user without building your own authentication layer or passing user IDs from the client application. To put AI Gateway behind Access, you must first [set a custom domain](https://developers.cloudflare.com/ai-gateway/configuration/custom-domains/) on your gateway and have [Cloudflare Access](https://developers.cloudflare.com/cloudflare-one/access-controls/applications/http-apps/) enabled on your account. ## How it works When a request to a custom domain includes a valid Cloudflare Access JWT, AI Gateway accepts the Access JWT as the request credential. The client does not need to send an AI Gateway token for that request. AI Gateway also adds the verified Access user ID to request metadata as [cf.user\_id](https://developers.cloudflare.com/ai-gateway/observability/custom-metadata/#reserved-metadata). This value is the Access JWT `sub` claim, not the user's email address. You can then filter logs, analytics, and spend by the authenticated user. Before AI Gateway forwards the request to the upstream provider, it removes Cloudflare-only credentials such as the Access JWT and AI Gateway authorization headers. Once a custom domain is protected by Access, every request to that domain must pass an Access policy. Requests that only include an AI Gateway token, without a valid Access token, are blocked by Access before they reach the gateway. Update existing integrations to authenticate through Access, or keep sending gateway-token traffic to the default `gateway.ai.cloudflare.com` endpoint, which is not protected by Access. ## Set up Access on a gateway 1. [Set up a custom domain](https://developers.cloudflare.com/ai-gateway/configuration/custom-domains/) for the gateway you want to protect. 2. In the [Cloudflare dashboard ↗](https://dash.cloudflare.com/), go to **AI** \> **AI Gateway**. 3. Select the gateway you configured with a custom domain. 4. Go to the **Access** tab and set up Cloudflare Access on the gateway. 5. Add Access policies that define which users can call the gateway. Setting up Access from the **Access** tab configures the Access application for you, so coding agents and other non-browser clients can authenticate by sending the Access token as a bearer token. After setup, users can make requests to the custom domain after authenticating through Access. Requests with a valid Access user subject include `cf.user_id` in AI Gateway metadata. ## Make a request After the user authenticates to Access, send requests to the custom domain without the account ID or gateway ID in the path: ```bash curl -X POST "https://ai.example.com/openai/v1/chat/completions" \ --header "Content-Type: application/json" \ --data '{ "model": "gpt-4.1-mini", "messages": [ { "role": "user", "content": "What is Cloudflare?" } ] }' ``` If you call the custom domain from a non-browser client, include the Access token using the header or cookie format supported by Cloudflare Access. For example, [cloudflared access curl](https://developers.cloudflare.com/cloudflare-one/access-controls/authenticate-agents/#make-requests-with-cloudflared-access-curl) can send the Access token for command-line requests. For coding agents, refer to the per-agent setup under [Coding agents](https://developers.cloudflare.com/ai-gateway/integrations/coding-agents/) — for example, [Claude Code](https://developers.cloudflare.com/ai-gateway/integrations/coding-agents/claude-code/#use-with-cloudflare-access) and [OpenAI Codex](https://developers.cloudflare.com/ai-gateway/integrations/coding-agents/openai-codex/#use-with-cloudflare-access). ## Limitations * `cf.user_id` is only added when AI Gateway receives a valid Access JWT with a non-empty user subject. * Service-token requests do not include `cf.user_id` because they do not represent an individual Access user. * You may not supply metadata keys that begin with `cf.`. These keys are reserved and are not saved. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/configuration/cloudflare-access/#page","headline":"Cloudflare Access · Cloudflare AI Gateway docs","description":"Add identity-aware controls to AI Gateway so users authenticate with your identity provider before they can call your gateway.","url":"https://developers.cloudflare.com/ai-gateway/configuration/cloudflare-access/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-08-05","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Override default or public model costs on a per-request basis. title: Custom costs image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Custom costs Last updated Apr 20, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/configuration/custom-costs/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) AI Gateway allows you to set custom costs at the request level. By using this feature, the cost metrics can accurately reflect your unique pricing, overriding the default or public model costs. Note Custom costs will only apply to requests that pass tokens in their response. Requests without token information will not have costs calculated. ## Custom cost To add custom costs to your API requests, use the `cf-aig-custom-cost` header. This header enables you to specify the cost per token for both input (tokens sent) and output (tokens received). * **per\_token\_in**: The negotiated input token cost (per token). * **per\_token\_out**: The negotiated output token cost (per token). There is no limit to the number of decimal places you can include, ensuring precise cost calculations, regardless of how small the values are. Custom costs will appear in the logs with an underline, making it easy to identify when custom pricing has been applied. In this example, if you have a negotiated price of $1 per million input tokens and $2 per million output tokens, include the `cf-aig-custom-cost` header as shown below. ```bash curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/openai/chat/completions \ --header "Authorization: Bearer $TOKEN" \ --header 'Content-Type: application/json' \ --header 'cf-aig-custom-cost: {"per_token_in":0.000001,"per_token_out":0.000002}' \ --data ' { "model": "gpt-4o-mini", "messages": [ { "role": "user", "content": "When is Cloudflare’s Birthday Week?" } ] }' ``` Note If a response is served from cache (cache hit), the cost is always `0`, even if you specified a custom cost. Custom costs only apply when the request reaches the model provider. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/configuration/custom-costs/#page","headline":"Custom costs · Cloudflare AI Gateway docs","description":"Override default or public model costs on a per-request basis.","url":"https://developers.cloudflare.com/ai-gateway/configuration/custom-costs/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-20","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Send AI Gateway requests through a hostname that you own, such as ai.example.com. title: Custom domains image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Custom domains Last updated Aug 5, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/configuration/custom-domains/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Custom domains let you send AI Gateway requests through a hostname that you own, such as `ai.example.com`, instead of the default `gateway.ai.cloudflare.com` endpoint. The hostname identifies your account and gateway, so you can omit the account ID and gateway ID from request URLs. Requests go directly to AI Gateway provider-native routes and OpenAI-compatible `compat` routes. Custom domains are also the foundation for [identity-aware controls with Cloudflare Access](https://developers.cloudflare.com/ai-gateway/configuration/cloudflare-access/), which let users authenticate with your identity provider before they can call your gateway. Note Custom domains work with the AI Gateway endpoint (`gateway.ai.cloudflare.com`). They do not work with the [Cloudflare REST API](https://developers.cloudflare.com/api/) (`api.cloudflare.com`). ## How it works For example, if your custom domain is `ai.example.com`, an OpenAI provider request uses: ```txt https://ai.example.com/openai/v1/chat/completions ``` Instead of: ```txt https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/openai/v1/chat/completions ``` The same applies to the OpenAI-compatible endpoint. For example, `https://ai.example.com/compat/chat/completions` routes through the same gateway using the `compat` route. ## Set up a custom domain in the dashboard To add a custom domain: 1. In the [Cloudflare dashboard ↗](https://dash.cloudflare.com/), go to **AI** \> **AI Gateway**. 2. Select the gateway you want to configure. 3. Go to the **Domains** tab. 4. Select **Add Domain** and enter the hostname you want to use. Optionally, choose a subdomain. 5. AI Gateway automatically creates the DNS record in the dashboard. After the domain is set up, you can send requests to it. To require users to authenticate before they can call the gateway, [set up Cloudflare Access on the domain](https://developers.cloudflare.com/ai-gateway/configuration/cloudflare-access/). ## Set up a custom domain via API You can also create and manage custom domains through the Cloudflare API. Unlike the dashboard, the API does not create the DNS record for you. After you create the custom domain, use the returned `cname_target` to create a proxied CNAME record for your hostname. ### Create a custom domain ```bash curl "https://api.cloudflare.com/client/v4/accounts/%7Baccount_id%7D/ai-gateway/gateways/%7Bgateway_id%7D/custom-domains" \ --request POST \ --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ --json '{ "domain": "ai.example.com", "zone_id": "395566f4c8a5b9d2a5fc0d457b007c44" }' ``` The response includes the custom domain status and the CNAME target: ```json { "success": true, "result": { "hostname": "ai.example.com", "gateway_id": "my-gateway", "status": "pending_dcv", "cname_target": "", "created_at": 1782925200000, "modified_at": 1782925200000 } } ``` ### Create the DNS record Create a proxied CNAME record in the zone that owns your custom domain. Set `content` to the `cname_target` returned when you created the custom domain. ```bash curl "https://api.cloudflare.com/client/v4/zones/%7Bzone_id%7D/dns_records" \ --request POST \ --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ --json '{ "type": "CNAME", "name": "ai", "content": "", "proxied": true }' ``` The custom domain remains in `pending_dcv` until domain control validation completes. ### List custom domains ```bash curl "https://api.cloudflare.com/client/v4/accounts/%7Baccount_id%7D/ai-gateway/gateways/%7Bgateway_id%7D/custom-domains" \ --request GET \ --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" ``` ### Get a custom domain ```bash curl "https://api.cloudflare.com/client/v4/accounts/%7Baccount_id%7D/ai-gateway/gateways/%7Bgateway_id%7D/custom-domains/%7Bhostname%7D" \ --request GET \ --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" ``` ### Delete a custom domain ```bash curl "https://api.cloudflare.com/client/v4/accounts/%7Baccount_id%7D/ai-gateway/gateways/%7Bgateway_id%7D/custom-domains/%7Bhostname%7D" \ --request DELETE \ --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" ``` ## Example request Send requests to the custom domain without the account ID or gateway ID in the path: ```bash curl -X POST "https://ai.example.com/openai/v1/chat/completions" \ --header "Content-Type: application/json" \ --data '{ "model": "gpt-4.1-mini", "messages": [ { "role": "user", "content": "What is Cloudflare?" } ] }' ``` If the domain is protected by Cloudflare Access, the request must also include a valid Access token. For details, refer to [Cloudflare Access](https://developers.cloudflare.com/ai-gateway/configuration/cloudflare-access/). ## Limitations Custom domains are not yet supported on the newer AI Gateway endpoints served through the [Cloudflare REST API](https://developers.cloudflare.com/api/) (`api.cloudflare.com`). Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/configuration/custom-domains/#page","headline":"Custom domains · Cloudflare AI Gateway docs","description":"Send AI Gateway requests through a hostname that you own, such as ai.example.com.","url":"https://developers.cloudflare.com/ai-gateway/configuration/custom-domains/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-08-05","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Create and manage custom AI providers for your account. title: Custom Providers image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Custom Providers Last updated Jun 15, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/configuration/custom-providers/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) ## Overview Custom Providers allow you to integrate AI providers that are not natively supported by AI Gateway. This feature enables you to use AI Gateway's observability, caching, rate limiting, and other features with any AI provider that has an HTTPS API endpoint. ## Use cases * **Internal AI models**: Connect to your organization's self-hosted AI models * **Regional providers**: Integrate with AI providers specific to your region * **Specialized models**: Use domain-specific AI services not available through standard providers * **Custom endpoints**: Route requests to your own AI infrastructure ## Before you begin ### Prerequisites * An active Cloudflare account with AI Gateway access * A valid API key from your custom AI provider * The HTTPS base URL for your provider's API ### Authentication The API endpoints for creating, reading, updating, or deleting custom providers require authentication. You need to create a Cloudflare API token with the appropriate permissions. To create an API token: 1. Go to the [Cloudflare dashboard API tokens page ↗](https://dash.cloudflare.com/?to=:account/api-tokens) 2. Click **Create Token** 3. Select **Custom Token** and add the following permissions: * `AI Gateway - Edit` 4. Click **Continue to summary** and then **Create Token** 5. Copy the token - you'll use it in the `Authorization: Bearer $CLOUDFLARE_API_TOKEN` header ## Create a custom provider To create a new custom provider using the API: 1. Get your [Account ID](https://developers.cloudflare.com/fundamentals/account/find-account-and-zone-ids/) and Account Tag. 2. Send a `POST` request to create a new custom provider: ```bash # Run `wrangler whoami` to get your account ID to replace $CLOUDFLARE_ACCOUNT_ID, # and `wrangler auth token` to get an auth token to replace $CLOUDFLARE_API_TOKEN. curl -X POST "https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/ai-gateway/custom-providers" \ -H "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ -H "Content-Type: application/json" \ -d '{ "name": "My Custom Provider", "slug": "some-provider", "base_url": "https://api.myprovider.com", "description": "Custom AI provider for internal models", "enable": true }' ``` **Required fields:** * `name` (string): Display name for your provider * `slug` (string): Unique identifier (alphanumeric with hyphens). Must be unique within your account. * `base_url` (string): HTTPS URL for your provider's API endpoint. Must start with `https://`. **Optional fields:** * `description` (string): Description of the provider * `link` (string): URL to provider documentation * `enable` (boolean): Whether the provider is active (default: `false`) * `beta` (boolean): Mark as beta feature (default: `false`) * `curl_example` (string): Example cURL command for using the provider * `js_example` (string): Example JavaScript code for using the provider **Response:** ```json { "success": true, "result": { "id": "550e8400-e29b-41d4-a716-446655440000", "account_id": "abc123def456", "account_tag": "my-account", "name": "My Custom Provider", "slug": "some-provider", "base_url": "https://api.myprovider.com", "description": "Custom AI provider for internal models", "enable": true, "beta": false, "logo": "Base64 encoded SVG logo", "link": null, "curl_example": null, "js_example": null, "created_at": 1700000000, "modified_at": 1700000000 } } ``` Auto-generated logo A default SVG logo is automatically generated for each custom provider. The logo is returned as a base64-encoded string. To create a new custom provider using the dashboard: 1. Log in to the [Cloudflare dashboard ↗](https://dash.cloudflare.com) and select your account. 2. Go to [**Compute & AI** \> **AI Gateway** \> **Custom Providers** ↗](https://dash.cloudflare.com/?to=/:account/ai/ai-gateway/custom-providers). 3. Select **Add Custom Provider**. 4. Enter the following information: * **Provider Name**: Display name for your provider * **Provider Slug**: Unique identifier (alphanumeric with hyphens) * **Base URL**: HTTPS URL for your provider's API endpoint (e.g., `https://api.myprovider.com/v1`) 5. Select **Save** to create your custom provider. ## List custom providers Retrieve all custom providers with optional filtering and pagination: ```bash # Run `wrangler whoami` to get your account ID to replace $CLOUDFLARE_ACCOUNT_ID, # and `wrangler auth token` to get an auth token to replace $CLOUDFLARE_API_TOKEN. curl "https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/ai-gateway/custom-providers" \ -H "Authorization: Bearer $CLOUDFLARE_API_TOKEN" ``` **Query parameters:** * `page` (number): Page number (default: `1`) * `per_page` (number): Items per page (default: `20`, max: `100`) * `enable` (boolean): Filter by enabled status * `beta` (boolean): Filter by beta status * `search` (string): Search in id, name, or slug fields * `order_by` (string): Sort field and direction (default: `"name ASC"`) **Examples:** List only enabled providers: ```bash # Run `wrangler whoami` to get your account ID to replace $CLOUDFLARE_ACCOUNT_ID, # and `wrangler auth token` to get an auth token to replace $CLOUDFLARE_API_TOKEN. curl "https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/ai-gateway/custom-providers?enable=true" \ -H "Authorization: Bearer $CLOUDFLARE_API_TOKEN" ``` Search for specific providers: ```bash # Run `wrangler whoami` to get your account ID to replace $CLOUDFLARE_ACCOUNT_ID, # and `wrangler auth token` to get an auth token to replace $CLOUDFLARE_API_TOKEN. curl "https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/ai-gateway/custom-providers?search=custom" \ -H "Authorization: Bearer $CLOUDFLARE_API_TOKEN" ``` **Response:** ```json { "success": true, "result": [ { "id": "550e8400-e29b-41d4-a716-446655440000", "name": "My Custom Provider", "slug": "some-provider", "base_url": "https://api.myprovider.com", "enable": true, "created_at": 1700000000, "modified_at": 1700000000 } ], "result_info": { "page": 1, "per_page": 20, "total_count": 1, "total_pages": 1 } } ``` To view all your custom providers: 1. Log in to the [Cloudflare dashboard ↗](https://dash.cloudflare.com) and select your account. 2. Go to [**Compute & AI** \> **AI Gateway** \> **Custom Providers** ↗](https://dash.cloudflare.com/?to=/:account/ai/ai-gateway/custom-providers). 3. You will see a list of all your custom providers with their names, slugs, base URLs, and status. ## Get a specific custom provider Retrieve details for a specific custom provider by its ID: ```bash # Run `wrangler whoami` to get your account ID to replace $CLOUDFLARE_ACCOUNT_ID, # and `wrangler auth token` to get an auth token to replace $CLOUDFLARE_API_TOKEN. curl "https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/ai-gateway/custom-providers/{provider_id}" \ -H "Authorization: Bearer $CLOUDFLARE_API_TOKEN" ``` **Response:** ```json { "success": true, "result": { "id": "550e8400-e29b-41d4-a716-446655440000", "account_id": "abc123def456", "account_tag": "my-account", "name": "My Custom Provider", "slug": "some-provider", "base_url": "https://api.myprovider.com", "description": "Custom AI provider for internal models", "enable": true, "beta": false, "logo": "Base64 encoded SVG logo", "link": "https://docs.myprovider.com", "curl_example": "curl -X POST https://api.myprovider.com/v1/chat ...", "js_example": "fetch('https://api.myprovider.com/v1/chat', {...})", "created_at": 1700000000, "modified_at": 1700000000 } } ``` ## Update a custom provider Update an existing custom provider. All fields are optional - only include the fields you want to change: ```bash # Run `wrangler whoami` to get your account ID to replace $CLOUDFLARE_ACCOUNT_ID, # and `wrangler auth token` to get an auth token to replace $CLOUDFLARE_API_TOKEN. curl -X PATCH "https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/ai-gateway/custom-providers/{provider_id}" \ -H "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ -H "Content-Type: application/json" \ -d '{ "name": "Updated Provider Name", "enable": true, "description": "Updated description" }' ``` **Updatable fields:** * `name` (string): Provider display name * `slug` (string): Provider identifier * `base_url` (string): API endpoint URL (must be HTTPS) * `description` (string): Provider description * `link` (string): Documentation URL * `enable` (boolean): Active status * `beta` (boolean): Beta flag * `curl_example` (string): Example cURL command * `js_example` (string): Example JavaScript code **Examples:** Enable a provider: ```bash # Run `wrangler whoami` to get your account ID to replace $CLOUDFLARE_ACCOUNT_ID, # and `wrangler auth token` to get an auth token to replace $CLOUDFLARE_API_TOKEN. curl -X PATCH "https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/ai-gateway/custom-providers/{provider_id}" \ -H "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ -H "Content-Type: application/json" \ -d '{"enable": true}' ``` Update provider URL: ```bash # Run `wrangler whoami` to get your account ID to replace $CLOUDFLARE_ACCOUNT_ID, # and `wrangler auth token` to get an auth token to replace $CLOUDFLARE_API_TOKEN. curl -X PATCH "https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/ai-gateway/custom-providers/{provider_id}" \ -H "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ -H "Content-Type: application/json" \ -d '{"base_url": "https://api.newprovider.com"}' ``` Cache invalidation Updates to custom providers automatically invalidate any cached entries related to that provider. To update an existing custom provider: 1. Log in to the [Cloudflare dashboard ↗](https://dash.cloudflare.com) and select your account. 2. Go to [**Compute & AI** \> **AI Gateway** \> **Custom Providers** ↗](https://dash.cloudflare.com/?to=/:account/ai/ai-gateway/custom-providers). 3. Find the custom provider you want to update and select **Edit**. 4. Update the fields you want to change (name, slug, base URL, etc.). 5. Select **Save** to apply your changes. ## Delete a custom provider Delete a custom provider: ```bash # Run `wrangler whoami` to get your account ID to replace $CLOUDFLARE_ACCOUNT_ID, # and `wrangler auth token` to get an auth token to replace $CLOUDFLARE_API_TOKEN. curl -X DELETE "https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/ai-gateway/custom-providers/{provider_id}" \ -H "Authorization: Bearer $CLOUDFLARE_API_TOKEN" ``` **Response:** ```json { "success": true, "result": { "id": "550e8400-e29b-41d4-a716-446655440000", "name": "My Custom Provider", "slug": "some-provider" } } ``` Impact of deletion Deleting a custom provider will immediately stop all requests routed through it. Ensure you have updated your applications before deleting a provider. Cache entries related to the provider will also be invalidated. To delete a custom provider: 1. Log in to the [Cloudflare dashboard ↗](https://dash.cloudflare.com) and select your account. 2. Go to [**Compute & AI** \> **AI Gateway** \> **Custom Providers** ↗](https://dash.cloudflare.com/?to=/:account/ai/ai-gateway/custom-providers). 3. Find the custom provider you want to delete and select **Delete**. 4. Confirm the deletion when prompted. Impact of deletion Deleting a custom provider will immediately stop all requests routed through it. Ensure you have updated your applications before deleting a provider. ## Using custom providers with AI Gateway Once you've created a custom provider, you can route requests through AI Gateway using one of two approaches: the **Unified API** or the **provider-specific endpoint**. When referencing your custom provider with either approach, you must prefix the slug with `custom-`. Custom provider prefix All custom provider slugs must be prefixed with `custom-` when making requests through AI Gateway. For example, if your provider slug is `some-provider`, you must use `custom-some-provider` in your requests. ### How URL routing works When AI Gateway receives a request for a custom provider, it constructs the upstream URL by combining the provider's configured `base_url` with the path that comes after `custom-{slug}/` in the gateway URL. **The `base_url` field should contain only the root domain** (or domain with a fixed prefix) of the provider's API. Any API-specific path segments (like `/v1/chat/completions`) go in the request URL, not in `base_url`. The formula is: ```plaintext Gateway URL: https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/custom-{slug}/{provider-path} Upstream URL: {base_url}/{provider-path} ``` Everything after `custom-{slug}/` in your request URL is appended directly to the `base_url` to form the final upstream URL. This means `{provider-path}` can include multiple path segments, query parameters, or any path structure your provider requires. ### Choosing between Unified API and provider-specific endpoint | | Unified API (/compat) | Provider-specific endpoint | | ------------------------------- | ----------------------------------------------- | ----------------------------------------- | | **Best for** | Providers with OpenAI-compatible APIs | Providers with any API structure | | **Request format** | Must follow the OpenAI /chat/completions schema | Uses the provider's native request format | | **Path control** | Fixed to /compat/chat/completions | Full control over the upstream path | | **How to specify the provider** | model field: custom-{slug}/{model-name} | URL path: /custom-{slug}/{path} | Use the **Unified API** when your custom provider accepts the OpenAI-compatible `/chat/completions` request format. This is the simplest option and works well with OpenAI SDKs. Use the **provider-specific endpoint** when your custom provider uses a non-standard API path or request format. This gives you full control over both the URL path and the request body sent to the upstream provider. ### Via Unified API The Unified API sends requests to the provider's chat completions endpoint using the OpenAI-compatible format. Specify the model using the format `custom-{slug}/{model-name}`. ```bash # Run `wrangler auth token` to get an auth token to replace $CF_AIG_TOKEN for use with the API. curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat/chat/completions \ -H "Authorization: Bearer $PROVIDER_API_KEY" \ -H "cf-aig-authorization: Bearer $CF_AIG_TOKEN" \ -H "Content-Type: application/json" \ -d '{ "model": "custom-some-provider/model-name", "messages": [{"role": "user", "content": "Hello!"}] }' ``` ### Via provider-specific endpoint The provider-specific endpoint gives you full control over the upstream path. Everything after `custom-{slug}/` in the URL is appended to the `base_url`. ```bash # Run `wrangler auth token` to get an auth token to replace $CF_AIG_TOKEN for use with the API. curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/custom-some-provider/v1/chat/completions \ -H "Authorization: Bearer $PROVIDER_API_KEY" \ -H "cf-aig-authorization: Bearer $CF_AIG_TOKEN" \ -H "Content-Type: application/json" \ -d '{ "model": "model-name", "messages": [{"role": "user", "content": "Hello!"}] }' ``` If `base_url` is `https://api.myprovider.com`, this request is proxied to: `https://api.myprovider.com/v1/chat/completions` ### Examples The following examples show how to configure `base_url` and construct request URLs for different types of providers. #### Example 1: OpenAI-compatible provider (standard `/v1/` path) Many providers follow the OpenAI convention of hosting their API at `{domain}/v1/chat/completions`. **Configuration:** * `slug`: `my-openai-compat` * `base_url`: `https://api.example-provider.com` **Provider-specific endpoint:** ```bash curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/custom-my-openai-compat/v1/chat/completions \ -H "Authorization: Bearer $PROVIDER_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "example-model", "messages": [{"role": "user", "content": "Hello!"}] }' ``` **URL mapping:** | Component | Value | | ------------- | ------------------------------------------------------------------------------------------------------------ | | Gateway URL | https://gateway.ai.cloudflare.com/v1/{account\_id}/{gateway\_id}/custom-my-openai-compat/v1/chat/completions | | base\_url | https://api.example-provider.com | | Provider path | /v1/chat/completions | | Upstream URL | https://api.example-provider.com/v1/chat/completions | Since this provider is OpenAI-compatible, you could also use the Unified API: ```bash curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat/chat/completions \ -H "Authorization: Bearer $PROVIDER_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "custom-my-openai-compat/example-model", "messages": [{"role": "user", "content": "Hello!"}] }' ``` #### Example 2: Provider with a non-standard API path Some providers use API paths that don't follow the `/v1/` convention. For example, a provider whose chat endpoint is at `https://api.custom-ai.com/api/coding/paas/v4/chat/completions`. **Configuration:** * `slug`: `custom-ai` * `base_url`: `https://api.custom-ai.com` **Provider-specific endpoint:** ```bash curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/custom-custom-ai/api/coding/paas/v4/chat/completions \ -H "Authorization: Bearer $PROVIDER_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "custom-ai-model", "messages": [{"role": "user", "content": "Hello!"}] }' ``` **URL mapping:** | Component | Value | | ------------- | --------------------------------------------------------------------------------------------------------------------- | | Gateway URL | https://gateway.ai.cloudflare.com/v1/{account\_id}/{gateway\_id}/custom-custom-ai/api/coding/paas/v4/chat/completions | | base\_url | https://api.custom-ai.com | | Provider path | /api/coding/paas/v4/chat/completions | | Upstream URL | https://api.custom-ai.com/api/coding/paas/v4/chat/completions | Note For providers with non-standard paths, you must use the provider-specific endpoint. The Unified API only supports the `/chat/completions` path and cannot route to custom API paths. #### Example 3: Self-hosted model with a path prefix If you host your own model behind a reverse proxy or on a platform that adds a path prefix, include only the fixed prefix portion in `base_url` if all your endpoints share it. Otherwise, keep `base_url` as just the domain. **Configuration (domain-only `base_url`):** * `slug`: `internal-llm` * `base_url`: `https://ml.internal.example.com` **Provider-specific endpoint:** ```bash curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/custom-internal-llm/serving/models/my-model:predict \ -H "Authorization: Bearer $INTERNAL_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "instances": [{"prompt": "Summarize the following text:"}] }' ``` **URL mapping:** | Component | Value | | ------------- | -------------------------------------------------------------------------------------------------------------------- | | Gateway URL | https://gateway.ai.cloudflare.com/v1/{account\_id}/{gateway\_id}/custom-internal-llm/serving/models/my-model:predict | | base\_url | https://ml.internal.example.com | | Provider path | /serving/models/my-model:predict | | Upstream URL | https://ml.internal.example.com/serving/models/my-model:predict | #### Example 4: Provider using OpenAI SDK with a custom base URL When using the OpenAI SDK to connect to a custom provider through AI Gateway, set the SDK's `base_url` to the gateway's provider-specific endpoint path (up to and including the API version prefix that your provider expects). **Configuration:** * `slug`: `alt-provider` * `base_url`: `https://api.alt-provider.com` **Python (OpenAI SDK):** ```python from openai import OpenAI client = OpenAI( api_key="your-provider-api-key", base_url="https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/custom-alt-provider/v1", default_headers={ "cf-aig-authorization": "Bearer {cf_aig_token}", }, ) # The SDK appends /chat/completions to the base_url automatically. # Final upstream URL: https://api.alt-provider.com/v1/chat/completions response = client.chat.completions.create( model="alt-model-v2", messages=[{"role": "user", "content": "Hello!"}], ) ``` **URL mapping:** | Component | Value | | ------------------ | -------------------------------------------------------------------------------------------------------- | | SDK base\_url | https://gateway.ai.cloudflare.com/v1/{account\_id}/{gateway\_id}/custom-alt-provider/v1 | | SDK appends | /chat/completions | | Full gateway URL | https://gateway.ai.cloudflare.com/v1/{account\_id}/{gateway\_id}/custom-alt-provider/v1/chat/completions | | Provider base\_url | https://api.alt-provider.com | | Provider path | /v1/chat/completions | | Upstream URL | https://api.alt-provider.com/v1/chat/completions | ## Common errors ### 409 Conflict - Duplicate slug ```json { "success": false, "errors": [ { "code": 1003, "message": "A custom provider with this slug already exists", "path": ["body", "slug"] } ] } ``` Each custom provider slug must be unique within your account. Choose a different slug or update the existing provider. ### 404 Not Found ```json { "success": false, "errors": [ { "code": 1004, "message": "Custom Provider not found" } ] } ``` The specified provider ID does not exist or you don't have access to it. Verify the provider ID and your authentication credentials. ### 400 Bad Request - Invalid base\_url ```json { "success": false, "errors": [ { "code": 1002, "message": "base_url must be a valid HTTPS URL starting with https://", "path": ["body", "base_url"] } ] } ``` The `base_url` field must be a valid HTTPS URL. HTTP URLs are not supported for security reasons. ### 404 when making requests to a custom provider If you receive a 404 from the upstream provider, the most common cause is an incorrect path mapping. Verify that: 1. Your `base_url` is set to the provider's **root domain** (for example, `https://api.provider.com`) rather than including API path segments. 2. Your request URL includes the **full API path** after `custom-{slug}/`. For example, if the upstream endpoint is `https://api.provider.com/api/v2/chat`, your gateway URL should end in `/custom-{slug}/api/v2/chat`. 3. There is no duplicate or missing path segment. A common mistake is including `/v1` in both `base_url` and the request path, resulting in the upstream receiving `/v1/v1/chat/completions`. ## Best practices 1. **Use descriptive slugs**: Choose slugs that clearly identify the provider (e.g., `internal-gpt`, `regional-ai`) 2. **Document your integrations**: Use the `curl_example` and `js_example` fields to provide usage examples 3. **Enable gradually**: Test with `enable: false` before making the provider active 4. **Monitor usage**: Use AI Gateway's analytics to track requests to your custom providers 5. **Secure your endpoints**: Ensure your custom provider's base URL implements proper authentication and authorization 6. **Use BYOK**: Store provider API keys securely using [BYOK](https://developers.cloudflare.com/ai-gateway/configuration/bring-your-own-keys/) instead of including them in every request ## Limitations * Custom providers are account-specific and not shared across Cloudflare accounts * The `base_url` must use HTTPS (HTTP is not supported) * Provider slugs must be unique within each account * Cache and rate limiting settings apply globally to the provider, not per-model ## Related resources * [Get started with AI Gateway](https://developers.cloudflare.com/ai-gateway/get-started/) * [Configure authentication](https://developers.cloudflare.com/ai-gateway/configuration/authentication/) * [BYOK (Store Keys)](https://developers.cloudflare.com/ai-gateway/configuration/bring-your-own-keys/) * [Dynamic routing](https://developers.cloudflare.com/ai-gateway/features/dynamic-routing/) * [Caching](https://developers.cloudflare.com/ai-gateway/features/caching/) * [Rate limiting](https://developers.cloudflare.com/ai-gateway/features/rate-limiting/) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/configuration/custom-providers/#page","headline":"Custom Providers · Cloudflare AI Gateway docs","description":"Create and manage custom AI providers for your account.","url":"https://developers.cloudflare.com/ai-gateway/configuration/custom-providers/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-06-15","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Specify model or provider fallbacks in AI Gateway to handle request failures and ensure reliability. title: Fallbacks image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Fallbacks Last updated Apr 20, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/configuration/fallbacks/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Specify model or provider fallbacks with your [Universal endpoint](https://developers.cloudflare.com/ai-gateway/usage/universal/) to handle request failures and ensure reliability. Cloudflare can trigger your fallback provider in response to [request errors](#request-failures) or [predetermined request timeouts](https://developers.cloudflare.com/ai-gateway/configuration/request-handling#request-timeouts). The [response header cf-aig-step](#response-headercf-aig-step) indicates which step successfully processed the request. ## Request failures By default, Cloudflare triggers your fallback if a model request returns an error. ### Example In the following example, a request first goes to the [Workers AI](https://developers.cloudflare.com/workers-ai/) Inference API. If the request fails, it falls back to OpenAI. The response header `cf-aig-step` indicates which provider successfully processed the request. 1. Sends a request to Workers AI Inference API. 2. If that request fails, proceeds to OpenAI. graph TD A[AI Gateway] --> B[Request to Workers AI Inference API] B -->|Success| C[Return Response] B -->|Failure| D[Request to OpenAI API] D --> E[Return Response] You can add as many fallbacks as you need, just by adding another object in the array. ```bash curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id} \ --header 'Content-Type: application/json' \ --data '[ { "provider": "workers-ai", "endpoint": "@cf/meta/llama-3.1-8b-instruct", "headers": { "Authorization": "Bearer {cloudflare_token}", "Content-Type": "application/json" }, "query": { "messages": [ { "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "What is Cloudflare?" } ] } }, { "provider": "openai", "endpoint": "chat/completions", "headers": { "Authorization": "Bearer {open_ai_token}", "Content-Type": "application/json" }, "query": { "model": "gpt-4o-mini", "stream": true, "messages": [ { "role": "user", "content": "What is Cloudflare?" } ] } } ]' ``` ## Response header(cf-aig-step) When using the [Universal endpoint](https://developers.cloudflare.com/ai-gateway/usage/universal/) with fallbacks, the response header `cf-aig-step` indicates which model successfully processed the request by returning the step number. This header provides visibility into whether a fallback was triggered and which model ultimately processed the response. * `cf-aig-step:0` – The first (primary) model was used successfully. * `cf-aig-step:1` – The request fell back to the second model. * `cf-aig-step:2` – The request fell back to the third model. * Subsequent steps – Each fallback increments the step number by 1. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/configuration/fallbacks/#page","headline":"Fallbacks · Cloudflare AI Gateway docs","description":"Specify model or provider fallbacks in AI Gateway to handle request failures and ensure reliability.","url":"https://developers.cloudflare.com/ai-gateway/configuration/fallbacks/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-20","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Create, edit, and delete AI Gateway instances using the dashboard or API. title: Manage gateways image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Manage gateways Last updated Aug 7, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/configuration/manage-gateway/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) You have several different options for managing an AI Gateway. ## Create gateway ### Default gateway AI Gateway can automatically create a gateway for you. If you omit the gateway ID from your request entirely, AI Gateway defaults to using `default` as the gateway ID. When no gateway named `default` exists in your account, AI Gateway creates it on the first authenticated request. This means you can start sending requests without creating a gateway first — AI Gateway handles gateway creation for you. The request that triggers auto-creation must be authenticated. When using the [REST API](https://developers.cloudflare.com/ai-gateway/usage/rest-api/), the standard `Authorization` header is sufficient. When using [provider-native endpoints](https://developers.cloudflare.com/ai-gateway/usage/providers/) at `gateway.ai.cloudflare.com`, include a valid `cf-aig-authorization` header. For Workers AI bindings, the account identity from the binding is used instead of a header. The auto-created default gateway uses the following settings: | Setting | Default value | | ------------------ | ---------------- | | Authentication | On | | Log collection | On | | Caching | Off (TTL of 0) | | Rate limiting | Off | | Workers AI billing | Standard billing | After creation, you can edit the default gateway settings like any other gateway. If you delete the default gateway, sending a new authenticated request to the `default` gateway ID auto-creates it again. Note Auto-creation only applies to the gateway ID `default`. Using any other gateway ID requires creating the gateway first. ### Create a gateway manually [Go to **AI Gateway** ↗](https://dash.cloudflare.com/?to=/:account/ai/ai-gateway) 1. Log into the [Cloudflare dashboard ↗](https://dash.cloudflare.com/) and select your account. 2. Go to **AI** \> **AI Gateway**. 3. Select **Create Gateway**. 4. Enter your **Gateway name**. Note: Gateway name has a 64 character limit. 5. In **Workers AI Billing**, choose how Workers AI requests through this gateway are billed: * **Standard billing** charges your Cloudflare account at the end of each billing cycle. * **Unified billing** deducts from your prepaid AI Gateway credit balance in real time. 6. Select **Create**. To set up an AI Gateway using the API: 1. [Create an API token](https://developers.cloudflare.com/fundamentals/api/get-started/create-token/) with the following permissions: * `AI Gateway - Read` * `AI Gateway - Edit` 2. Get your [Account ID](https://developers.cloudflare.com/fundamentals/account/find-account-and-zone-ids/). 3. Using that API token and Account ID, send a [POST request](https://developers.cloudflare.com/api/resources/ai%5Fgateway/methods/create/) to the Cloudflare API. ## Edit gateway To edit an AI Gateway in the dashboard: 1. Log into the [Cloudflare dashboard ↗](https://dash.cloudflare.com/) and select your account. 2. Go to **AI** \> **AI Gateway**. 3. Select your gateway. 4. Go to **Settings** and update as needed. To edit an AI Gateway, send a [PUT request](https://developers.cloudflare.com/api/resources/ai%5Fgateway/methods/update/) to the Cloudflare API. Note For more details about what settings are available for editing, refer to [Configuration](https://developers.cloudflare.com/ai-gateway/configuration/). ### Configure Workers AI billing By default, Workers AI requests use **Standard billing**, which charges your Cloudflare account at the end of each billing cycle. To use prepaid AI Gateway credits for Workers AI requests: 1. [Load credits](https://developers.cloudflare.com/ai-gateway/features/unified-billing/#load-credits) into your Cloudflare account. 2. In the Cloudflare dashboard, go to **AI** \> **AI Gateway** and select your gateway. 3. Go to **Settings** and find **Workers AI Billing**. 4. Select **Unified billing**. 5. Select **Save**. Workers AI requests routed through this gateway will deduct from your AI Gateway credit balance in real time. ## Retry requests You can configure your gateway to automatically retry failed requests to upstream providers. This is useful when you do not control the client and cannot implement client-side retries or backoff logic. To configure retry settings: 1. Log in to the [Cloudflare dashboard ↗](https://dash.cloudflare.com/) and select your account. 2. Go to **AI** \> **AI Gateway** and select your gateway. 3. Go to **Settings** and find the **Retry Requests** section. 4. Turn on the toggle to turn on automatic retries. 5. Configure the following settings: * **Retry count** — the maximum number of retry attempts (up to 5). * **Delay** — the base delay between retries. Available values: 100ms, 500ms, 1 second, 2 seconds, 3 seconds, or 5 seconds. * **Backoff** — the backoff strategy for subsequent retries: Constant, Linear, or Exponential. 6. Select **Save**. ![Retry Requests settings in the AI Gateway dashboard](https://developers.cloudflare.com/cdn-cgi/image/onerror=redirect,width=2344,height=636,format=webp/_astro/auto-retry-settings.UcvmkohL.png) These gateway-level defaults apply to all requests routed through the gateway. Per-request headers can override these defaults — refer to [Request handling](https://developers.cloudflare.com/ai-gateway/configuration/request-handling/#request-retries) for details. For more complex failover scenarios where you need to fail across different providers, refer to [Dynamic Routing](https://developers.cloudflare.com/ai-gateway/features/dynamic-routing/). ## Delete gateway Deleting your gateway is permanent and can not be undone. To delete an AI Gateway in the dashboard: 1. Log into the [Cloudflare dashboard ↗](https://dash.cloudflare.com/) and select your account. 2. Go to **AI** \> **AI Gateway**. 3. Select your gateway from the list of available options. 4. Go to **Settings**. 5. For **Delete Gateway**, select **Delete** (and confirm your deletion). To delete an AI Gateway, send a [DELETE request](https://developers.cloudflare.com/api/resources/ai%5Fgateway/methods/delete/) to the Cloudflare API. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/configuration/manage-gateway/#page","headline":"Manage gateways · Cloudflare AI Gateway docs","description":"Create, edit, and delete AI Gateway instances using the dashboard or API.","url":"https://developers.cloudflare.com/ai-gateway/configuration/manage-gateway/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-08-07","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Configure AI Gateway request timeouts and retries for reliable AI provider interactions. title: Request handling image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Request handling Last updated Jun 15, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/configuration/request-handling/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Note [Dynamic Routing](https://developers.cloudflare.com/ai-gateway/features/dynamic-routing/) also offers timeouts and retries per model, along with conditional routing, rate limiting, and budget limiting through a visual interface. This page documents request-handling configuration available through per-request `cf-aig-*` headers that work with any provider endpoint. You can also configure retries at the [gateway level](https://developers.cloudflare.com/ai-gateway/configuration/manage-gateway/#retry-requests). Your AI gateway supports different strategies for handling requests to providers, which allows you to manage AI interactions effectively and ensure your applications remain responsive and reliable. ## Request timeouts A request timeout allows you to return an error or trigger a retry if a provider takes too long to respond. These timeouts help: * Improve user experience, by preventing users from waiting too long for a response * Proactively handle errors, by detecting unresponsive providers A timeout is set in milliseconds. The timeout is based on when the first part of the response comes back. As long as the first part of the response returns within the specified timeframe — such as when streaming a response — your gateway will wait for the response. ### Configuration For a provider-specific endpoint, configure the timeout value by adding a `cf-aig-request-timeout` header. ```bash # Run `wrangler whoami` to get your account ID to replace $CLOUDFLARE_ACCOUNT_ID, # and `wrangler auth token` to get an auth token to replace $CLOUDFLARE_API_TOKEN. curl -X POST "https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/v1/chat/completions" \ --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ --header "Content-Type: application/json" \ --header "cf-aig-request-timeout: 5000" \ --data '{ "model": "openai/gpt-4.1-mini", "messages": [{"role": "user", "content": "What is Cloudflare?"}] }' ``` --- ## Request retries AI Gateway supports automatic retries for failed requests, with a maximum of five retry attempts. This feature improves your application's resiliency, ensuring you can recover from temporary issues without manual intervention. With request retries, you can adjust a combination of three properties: * Number of attempts (maximum of 5 tries) * How long before retrying (in milliseconds, maximum of 5 seconds) * Backoff method (constant, linear, or exponential) On the final retry attempt, your gateway will wait until the request completes, regardless of how long it takes. ### Configuration For a provider-specific endpoint, configure the retry settings by adding different header values: * `cf-aig-max-attempts` (number) * `cf-aig-retry-delay` (number) * `cf-aig-backoff` ("constant" | "linear" | "exponential) Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/configuration/request-handling/#page","headline":"Request handling · Cloudflare AI Gateway docs","description":"Configure AI Gateway request timeouts and retries for reliable AI provider interactions.","url":"https://developers.cloudflare.com/ai-gateway/configuration/request-handling/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-06-15","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: View AI Gateway metrics for requests, tokens, caching, errors, and costs in the dashboard or via GraphQL. title: Analytics image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Analytics Last updated Apr 20, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/observability/analytics/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Your AI Gateway dashboard shows metrics on requests, tokens, caching, errors, and cost. You can filter these metrics by time. These analytics help you understand traffic patterns, token consumption, and potential issues across AI providers. You can view the following analytics: * **Requests**: Track the total number of requests processed by AI Gateway. * **Token Usage**: Analyze token consumption across requests, giving insight into usage patterns. * **Costs**: Gain visibility into the costs associated with using different AI providers, allowing you to track spending, manage budgets, and optimize resources. * **Errors**: Monitor the number of errors across the gateway, helping to identify and troubleshoot issues. * **Cached Responses**: View the percentage of responses served from cache, which can help reduce costs and improve speed. ## View analytics To view analytics in the dashboard: 1. Log into the [Cloudflare dashboard ↗](https://dash.cloudflare.com) and select your account. 2. Go to **AI** \> **AI Gateway**. 3. Make sure you have your gateway selected. You can use GraphQL to query your usage data outside of the AI Gateway dashboard. See the example query below. You will need to use your Cloudflare token when making the request, and change `{account_id}` to match your account tag. ```bash curl https://api.cloudflare.com/client/v4/graphql \ --header 'Authorization: Bearer TOKEN \ --header 'Content-Type: application/json' \ --data '{ "query": "query{\n viewer {\n accounts(filter: { accountTag: \"{account_id}\" }) {\n requests: aiGatewayRequestsAdaptiveGroups(\n limit: $limit\n filter: { datetimeHour_geq: $start, datetimeHour_leq: $end }\n orderBy: [datetimeMinute_ASC]\n ) {\n count,\n dimensions {\n model,\n provider,\n gateway,\n ts: datetimeMinute\n }\n \n }\n \n }\n }\n}", "variables": { "limit": 1000, "start": "2023-09-01T10:00:00.000Z", "end": "2023-09-30T10:00:00.000Z", "orderBy": "date_ASC" } }' ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/observability/analytics/#page","headline":"Analytics · Cloudflare AI Gateway docs","description":"View AI Gateway metrics for requests, tokens, caching, errors, and costs in the dashboard or via GraphQL.","url":"https://developers.cloudflare.com/ai-gateway/observability/analytics/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-20","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Track and estimate token-based costs across AI providers using AI Gateway cost metrics. title: Costs image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Costs Last updated Apr 20, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/observability/costs/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Cost metrics are only available for endpoints where the models return token data and the model name in their responses. ## Track costs across AI providers AI Gateway makes it easier to monitor and estimate token based costs across all your AI providers. This can help you: * Understand and compare usage costs between providers. * Monitor trends and estimate spend using consistent metrics. * Apply custom pricing logic to match negotiated rates. Note The cost metric is an **estimation** based on the number of tokens sent and received in requests. While this metric can help you monitor and predict cost trends, refer to your provider's dashboard for the most **accurate** cost details. Caution Providers may introduce new models or change their pricing. If you notice outdated cost data or are using a model not yet supported by our cost tracking, please [submit a request ↗](https://forms.gle/8kRa73wRnvq7bxL48) ## Custom costs AI Gateway allows users to set custom costs when operating under special pricing agreements or negotiated rates. Custom costs can be applied at the request level, and when applied, they will override the default or public model costs. For more information on configuration of custom costs, please visit the [Custom Costs](https://developers.cloudflare.com/ai-gateway/configuration/custom-costs/) configuration page. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/observability/costs/#page","headline":"Costs · Cloudflare AI Gateway docs","description":"Track and estimate token-based costs across AI providers using AI Gateway cost metrics.","url":"https://developers.cloudflare.com/ai-gateway/observability/costs/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-20","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Tag AI Gateway requests with custom metadata such as user IDs to improve log filtering and analysis. title: Custom metadata image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Custom metadata Last updated Aug 5, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/observability/custom-metadata/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Custom metadata in AI Gateway allows you to tag requests with user IDs or other identifiers, enabling better tracking and analysis of your requests. Metadata values can be strings, numbers, or booleans, and will appear in your logs, making it easy to search and filter through your data. ## Key Features * **Custom Tagging**: Add user IDs, team names, test indicators, and other relevant information to your requests. * **Enhanced Logging**: Metadata appears in your logs, allowing for detailed inspection and troubleshooting. * **Search and Filter**: Use metadata to efficiently search and filter through logged requests. Note AI Gateway allows you to pass up to five custom metadata entries per request. If more than five entries are provided, only the first five will be saved; additional entries will be ignored. Ensure your custom metadata is limited to five entries to avoid unprocessed or lost data. ## Supported Metadata Types * String * Number * Boolean Note Objects are not supported as metadata values. ## Reserved metadata Metadata keys that begin with `cf.` are reserved for metadata added by Cloudflare. Do not send your own `cf.*` metadata keys. AI Gateway removes customer-supplied `cf.*` keys before saving request metadata. When a request reaches AI Gateway through a custom domain protected by [Cloudflare Access](https://developers.cloudflare.com/ai-gateway/configuration/cloudflare-access/), AI Gateway adds the authenticated Access user ID to request metadata as `cf.user_id`. This value is the verified Access JWT `sub` claim, not the user's email address. AI Gateway guarantees that `cf.user_id` is saved when a valid Access user ID is present. If the request already has five custom metadata entries, AI Gateway may remove the last custom entry so `cf.user_id` can be saved. Service-token requests and requests without a user subject do not receive `cf.user_id` metadata. ## Implementations ### Using cURL To include custom metadata in your request using cURL: ```bash # Run `wrangler whoami` to get your account ID to replace $CLOUDFLARE_ACCOUNT_ID, # and `wrangler auth token` to get an auth token to replace $CLOUDFLARE_API_TOKEN. curl -X POST "https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/v1/chat/completions" \ --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ --header "Content-Type: application/json" \ --header 'cf-aig-metadata: {"team": "AI", "user": 12345, "test":true}' \ --data '{"model": "openai/gpt-4.1", "messages": [{"role": "user", "content": "What should I eat for lunch?"}]}' ``` ### Using SDK To include custom metadata in your request using the OpenAI SDK: ```js import OpenAI from "openai"; export default { async fetch(request, env, ctx) { const openai = new OpenAI({ apiKey: env.CLOUDFLARE_API_TOKEN, baseURL: `https://api.cloudflare.com/client/v4/accounts/${env.CLOUDFLARE_ACCOUNT_ID}/ai/v1`, }); try { const chatCompletion = await openai.chat.completions.create( { model: "openai/gpt-4.1", messages: [{ role: "user", content: "What should I eat for lunch?" }], max_tokens: 50, }, { headers: { "cf-aig-metadata": JSON.stringify({ user: "JaneDoe", team: 12345, test: true, }), }, }, ); const response = chatCompletion.choices[0].message; return new Response(JSON.stringify(response)); } catch (e) { console.log(e); return new Response(e); } }, }; ``` ```ts import OpenAI from "openai"; export default { async fetch(request, env, ctx) { const openai = new OpenAI({ apiKey: env.CLOUDFLARE_API_TOKEN, baseURL: `https://api.cloudflare.com/client/v4/accounts/${env.CLOUDFLARE_ACCOUNT_ID}/ai/v1`, }); try { const chatCompletion = await openai.chat.completions.create( { model: "openai/gpt-4.1", messages: [{ role: "user", content: "What should I eat for lunch?" }], max_tokens: 50, }, { headers: { "cf-aig-metadata": JSON.stringify({ user: "JaneDoe", team: 12345, test: true, }), }, }, ); const response = chatCompletion.choices[0].message; return new Response(JSON.stringify(response)); } catch (e) { console.log(e); return new Response(e); } }, }; ``` ### Using Binding To include custom metadata in your request using [Bindings](https://developers.cloudflare.com/workers/runtime-apis/bindings/): ```javascript export default { async fetch(request, env, ctx) { const aiResp = await env.AI.run( "@cf/mistral/mistral-7b-instruct-v0.1", { prompt: "What should I eat for lunch?" }, { gateway: { id: "gateway_id", metadata: { team: "AI", user: 12345, test: true }, }, }, ); return new Response(aiResp); }, }; ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/observability/custom-metadata/#page","headline":"Custom metadata · Cloudflare AI Gateway docs","description":"Tag AI Gateway requests with custom metadata such as user IDs to improve log filtering and analysis.","url":"https://developers.cloudflare.com/ai-gateway/observability/custom-metadata/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-08-05","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Store and inspect AI Gateway request logs including prompts, responses, tokens, costs, and DLP actions. title: Logging image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Logging Last updated Jun 15, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/observability/logging/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) Logging is a fundamental building block for application development. Logs provide insights during the early stages of development and are often critical to understanding issues occurring in production. Your AI Gateway dashboard shows logs of individual requests, including the user prompt, model response, provider, timestamp, request status, token usage, cost, duration, and the user agent of the client that made the request. When [DLP](https://developers.cloudflare.com/ai-gateway/features/dlp/) policies are configured, logs for requests that trigger a DLP match also include the DLP action taken (Flag or Block), matched policy IDs, matched profile IDs, and the specific detection entries that were triggered. These logs persist, giving you the flexibility to store them for your preferred duration and do more with valuable request data. Each gateway has a storage limit based on your plan. You can customize this limit per gateway in your gateway settings. If your storage limit is reached, new logs will stop being saved. To continue saving logs, you must delete older logs to free up space for new logs. To learn more about your plan limits, refer to [Limits](https://developers.cloudflare.com/ai-gateway/reference/limits/). We recommend using an authenticated gateway when storing logs to prevent unauthorized access and protects against invalid requests that can inflate log storage usage and make it harder to find the data you need. Learn more about setting up an [authenticated gateway](https://developers.cloudflare.com/ai-gateway/configuration/authentication/). ## Default configuration Logs, which include metrics as well as request and response data, are enabled by default for each gateway. This logging behavior will be uniformly applied to all requests in the gateway. If you are concerned about privacy or compliance and want to turn log collection off, you can go to settings and opt out of logs. If you need to modify the log settings for specific requests, you can override this setting on a per-request basis. To change the default log configuration in the dashboard: 1. In the Cloudflare dashboard, go to the **AI Gateway** page. [Go to **AI Gateway** ↗](https://dash.cloudflare.com/?to=/:account/ai/ai-gateway) 2. Select **Settings**. 3. Change the **Logs** setting to your preference. ## Per-request logging To override the default logging behavior set in the settings tab, you can define headers on a per-request basis. ### Collect logs (`cf-aig-collect-log`) The `cf-aig-collect-log` header allows you to bypass the default log setting for the gateway. If the gateway is configured to save logs, the header will exclude the log for that specific request. Conversely, if logging is disabled at the gateway level, this header will save the log for that request. In the example below, we use `cf-aig-collect-log` to bypass the default setting to avoid saving the log. ```bash # Run `wrangler whoami` to get your account ID to replace $CLOUDFLARE_ACCOUNT_ID, # and `wrangler auth token` to get an auth token to replace $CLOUDFLARE_API_TOKEN. curl -X POST "https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/v1/chat/completions" \ --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ --header "Content-Type: application/json" \ --header "cf-aig-collect-log: false" \ --data '{ "model": "openai/gpt-4.1-mini", "messages": [ { "role": "user", "content": "What is the email address and phone number of user123?" } ] }' ``` ### Collect log payload (`cf-aig-collect-log-payload`) The `cf-aig-collect-log-payload` header allows you to control whether the raw request and response bodies (payloads) are stored for a given request. Unlike `cf-aig-collect-log`, which controls the entire log entry, this header only affects payload storage — metadata such as token counts, model, provider, status code, cost, and duration will still be logged. This is useful when you want to maintain visibility into usage metrics and request metadata without persisting sensitive prompt or completion data. | Header value | Behavior | | ------------ | ---------------------------------------------------------------------- | | true | Request and response payloads are stored. | | false | Payload storage is skipped. Metadata-only log entries are still saved. | In the example below, we use `cf-aig-collect-log-payload` to skip storing the request and response bodies while keeping the metadata log. ```bash # Run `wrangler whoami` to get your account ID to replace $CLOUDFLARE_ACCOUNT_ID, # and `wrangler auth token` to get an auth token to replace $CLOUDFLARE_API_TOKEN. curl -X POST "https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/v1/chat/completions" \ --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ --header "Content-Type: application/json" \ --header "cf-aig-collect-log-payload: false" \ --data '{ "model": "openai/gpt-4.1-mini", "messages": [ { "role": "user", "content": "What is the email address and phone number of user123?" } ] }' ``` Note If `cf-aig-collect-log` is set to `false`, the entire log entry (including metadata) is skipped regardless of the `cf-aig-collect-log-payload` value. Use `cf-aig-collect-log-payload: false` on its own if you only want to suppress payload storage while retaining metadata logs. ## DLP fields in logs When [Data Loss Prevention (DLP)](https://developers.cloudflare.com/ai-gateway/features/dlp/) policies are enabled on a gateway, log entries for requests that trigger a DLP policy match include additional fields: | Field | Description | | -------------------- | --------------------------------------------------------------------- | | DLP Action | The action taken by the DLP policy: FLAG or BLOCK | | DLP Policies Matched | The IDs of the DLP policies that matched | | DLP Profiles Matched | The IDs of the DLP profiles that triggered within each matched policy | | DLP Entries Matched | The specific detection entry IDs that matched within each profile | | DLP Check | Whether the match occurred in the REQUEST, RESPONSE, or both | These fields are available both in the dashboard log viewer and through the [Logs API](https://developers.cloudflare.com/api/resources/ai%5Fgateway/subresources/logs/methods/list/). You can filter logs by **DLP Action** in the dashboard to view only flagged or blocked requests. For more details on DLP monitoring, refer to [Monitor DLP events](https://developers.cloudflare.com/ai-gateway/features/dlp/set-up-dlp/#monitor-dlp-events). ## Managing log storage To manage your log storage effectively, you can: * Set Storage Limits: Configure a limit on the number of logs stored per gateway in your gateway settings to ensure you only pay for what you need. * Enable Automatic Log Deletion: Activate the Automatic Log Deletion feature in your gateway settings to automatically delete the oldest logs once the storage limit for your account is reached. This ensures new logs are always saved without manual intervention. ## How to delete logs To manage your log storage effectively and ensure continuous logging, you can delete logs using the following methods: ### Automatic Log Deletion ​To maintain continuous logging within your gateway's storage constraints, enable Automatic Log Deletion in your Gateway settings. This feature automatically deletes the oldest logs once the storage limit for your account is reached, ensuring new logs are saved without manual intervention. ### Manual deletion To manually delete logs through the dashboard, navigate to the Logs tab in the dashboard. Use the available filters such as status, cache, provider, cost, or any other options in the dropdown to refine the logs you wish to delete. Once filtered, select Delete logs to complete the action. See full list of available filters and their descriptions below: | Filter category | Filter options | Filter by description | | --------------- | ------------------------------------------------------------ | --------------------------------------------------- | | Status | error, status | error type or status. | | Cache | cached, not cached | based on whether they were cached or not. | | Provider | specific providers | the selected AI provider. | | AI Models | specific models | the selected AI model. | | Cost | less than, greater than | cost, specifying a threshold. | | Request type | Workers AI Binding, WebSockets | the type of request. | | Tokens | Total tokens, Tokens In, Tokens Out | token count (less than or greater than). | | Duration | less than, greater than | request duration. | | Feedback | equals, does not equal (thumbs up, thumbs down, no feedback) | feedback type. | | Metadata Key | equals, does not equal | specific metadata keys. | | Metadata Value | equals, does not equal | specific metadata values. | | Log ID | equals, does not equal | a specific Log ID. | | Event ID | equals, does not equal | a specific Event ID. | | DLP Action | FLAG, BLOCK | the DLP action taken on the request. | | User Agent | equals, does not equal, contains | the user agent of the client that made the request. | ### API deletion You can programmatically delete logs using the AI Gateway API. For more comprehensive information on the `DELETE` logs endpoint, check out the [Cloudflare API documentation](https://developers.cloudflare.com/api/resources/ai%5Fgateway/subresources/logs/methods/delete/). Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/observability/logging/#page","headline":"Logging · Cloudflare AI Gateway docs","description":"Store and inspect AI Gateway request logs including prompts, responses, tokens, costs, and DLP actions.","url":"https://developers.cloudflare.com/ai-gateway/observability/logging/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-06-15","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Export encrypted AI Gateway logs to external storage using Workers Logpush. title: Workers Logpush image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Workers Logpush Last updated Jul 28, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/observability/logging/logpush/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) AI Gateway allows you to securely export logs to an external storage location, where you can decrypt and process them. You can toggle Workers Logpush on and off in the [Cloudflare dashboard ↗](https://dash.cloudflare.com) settings. This product is available on the Workers Paid plan. For pricing information, refer to [Pricing](https://developers.cloudflare.com/ai-gateway/reference/pricing). This guide explains how to set up Workers Logpush for AI Gateway, generate an RSA key pair for encryption, and decrypt the logs once they are received. You can store up to 10 million logs per gateway. If your limit is reached, new logs will stop being saved and will not be exported through Workers Logpush. To continue saving and exporting logs, you must delete older logs to free up space for new logs. Workers Logpush has a limit of 4 jobs and a maximum request size of 1 MB per log. Note To export logs using Workers Logpush, you must have logs turned on for the gateway. Need a higher limit? To request an increase to a limit, complete the [Limit Increase Request Form ↗](https://forms.gle/cuXu1QnQCrSNkkaS8). If the limit can be increased, Cloudflare will contact you with next steps. ## How logs are encrypted We employ a hybrid encryption model efficiency and security. Initially, an AES key is generated for each log. This AES key is what actually encrypts the bulk of your data, chosen for its speed and security in handling large datasets efficiently. Now, for securely sharing this AES key, we use RSA encryption. Here's what happens: the AES key, although lightweight, needs to be transmitted securely to the recipient. We encrypt this key with the recipient's RSA public key. This step leverages RSA's strength in secure key distribution, ensuring that only someone with the corresponding RSA private key can decrypt and use the AES key. Once encrypted, both the AES-encrypted data and the RSA-encrypted AES key are sent together. Upon arrival, the recipient's system uses the RSA private key to decrypt the AES key. With the AES key now accessible, it is straightforward to decrypt the main data payload. This method combines the best of both worlds: the efficiency of AES for data encryption with the secure key exchange capabilities of RSA, ensuring data integrity, confidentiality, and performance are all optimally maintained throughout the data lifecycle. ## Setting up Workers Logpush To configure Workers Logpush for AI Gateway, follow these steps: ## 1\. Generate an RSA key pair locally You need to generate a key pair to encrypt and decrypt the logs. This script will output your RSA privateKey and publicKey. Keep the private key secure, as it will be used to decrypt the logs. Below is a sample script to generate the keys using Node.js and OpenSSL. ```js const crypto = require("crypto"); const { privateKey, publicKey } = crypto.generateKeyPairSync("rsa", { modulusLength: 4096, publicKeyEncoding: { type: "spki", format: "pem", }, privateKeyEncoding: { type: "pkcs8", format: "pem", }, }); console.log(publicKey); console.log(privateKey); ``` Run the script by executing the below code on your terminal. Replace `file name` with the name of your JavaScript file. ```bash node {file name} ``` 1. Generate private key: Use the following command to generate a RSA private key: ```bash openssl genpkey -algorithm RSA -out private_key.pem -pkeyopt rsa_keygen_bits:4096 ``` 2. Generate public key: After generating the private key, you can extract the corresponding public key using: ```bash openssl rsa -pubout -in private_key.pem -out public_key.pem ``` ## 2\. Upload public key to gateway settings Once you have generated the key pair, upload the public key to your AI Gateway settings. This key will be used to encrypt your logs. In order to enable Workers Logpush, you will need logs enabled for that gateway. ## 3\. Set up Logpush Uploading your public key enables Workers Logpush for the gateway, but logs will not be exported until you also create and enable a Logpush job that specifies where to send them. Both steps are required. To create the Logpush job, choose your destination and follow the steps in the [Enable destinations](https://developers.cloudflare.com/logs/logpush/logpush-job/enable-destinations/) documentation. For example, to export logs to Cloudflare R2, refer to [Enable Cloudflare R2](https://developers.cloudflare.com/logs/logpush/logpush-job/enable-destinations/r2/). When configuring the job, select the AI Gateway dataset. ## 4\. Receive encrypted logs After configuring Workers Logpush, logs will be sent encrypted using the public key you uploaded. To access the data, you will need to decrypt it using your private key. The logs will be sent to the object storage provider that you have selected. ## 5\. Decrypt logs To decrypt the encrypted log bodies and metadata from AI Gateway, you can use the following Node.js script or OpenSSL: To decrypt the encrypted log bodies and metadata from AI Gateway, download the logs to a folder, in this case its named `my_log.log.gz`. Then copy this JavaScript file into the same folder and place your private key in the top variable. ```js const privateKeyStr = `-----BEGIN RSA PRIVATE KEY----- .... -----END RSA PRIVATE KEY-----`; const crypto = require("crypto"); const privateKey = crypto.createPrivateKey(privateKeyStr); const fs = require("fs"); const zlib = require("zlib"); const readline = require("readline"); async function importAESGCMKey(keyBuffer) { try { // Ensure the key length is valid for AES if ([128, 192, 256].includes(256)) { return await crypto.webcrypto.subtle.importKey( "raw", keyBuffer, { name: "AES-GCM", length: 256, }, true, // Whether the key is extractable (true in this case to allow for export later if needed) ["encrypt", "decrypt"], // Use for encryption and decryption ); } else { throw new Error("Invalid AES key length. Must be 128, 12, or 256 bits."); } } catch (error) { console.error("Failed to import key:", error); throw error; } } async function decryptData(encryptedData, aesKey, iv) { const decryptedData = await crypto.subtle.decrypt( { name: "AES-GCM", iv: iv }, aesKey, encryptedData, ); return new TextDecoder().decode(decryptedData); } async function decryptBase64(privateKey, data) { if (data.key === undefined) { return data; } const aesKeyBuf = crypto.privateDecrypt( { key: privateKey, oaepHash: "SHA256", }, Buffer.from(data.key, "base64"), ); const aesKey = await importAESGCMKey(aesKeyBuf); const decryptedData = await decryptData( Buffer.from(data.data, "base64"), aesKey, Buffer.from(data.iv, "base64"), ); return decryptedData.toString(); } async function run() { let lineReader = readline.createInterface({ input: fs.createReadStream("my_log.log.gz").pipe(zlib.createGunzip()), }); lineReader.on("line", async (line) => { line = JSON.parse(line); const { Metadata, RequestBody, ResponseBody, ...remaining } = line; console.log({ ...remaining, Metadata: await decryptBase64(privateKey, Metadata), RequestBody: await decryptBase64(privateKey, RequestBody), ResponseBody: await decryptBase64(privateKey, ResponseBody), }); console.log("--"); }); } run(); ``` Run the script by executing the below code on your terminal. Replace `file name` with the name of your JavaScript file. ```bash node {file name} ``` The script reads the encrypted log file `(my_log.log.gz)`, decrypts the metadata, request body, and response body, and prints the decrypted data. Ensure you replace the `privateKey` variable with your actual private RSA key that you generated in step 1. 1. Decrypt the encrypted log file using the private key. Assuming that the logs were encrypted with the public key (for example `public_key.pem`), you can use the private key (`private_key.pem`) to decrypt the log file. For example, if the encrypted logs are in a file named `encrypted_logs.bin`, you can decrypt it like this: ```bash openssl rsautl -decrypt -inkey private_key.pem -in encrypted_logs.bin -out decrypted_logs.txt ``` * `-decrypt` tells OpenSSL that we want to decrypt the file. * `-inkey private_key.pem` specifies the private key that will be used to decrypt the logs. * `-in encrypted_logs.bin` is the encrypted log file. * `-out decrypted_logs.txt`decrypted logs will be saved into this file. 1. View the decrypted logs Once decrypted, you can view the logs by simply running: ```bash cat decrypted_logs.txt ``` This command will output the decrypted logs to the terminal. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/observability/logging/logpush/#page","headline":"Workers Logpush · Cloudflare AI Gateway docs","description":"Export encrypted AI Gateway logs to external storage using Workers Logpush.","url":"https://developers.cloudflare.com/ai-gateway/observability/logging/logpush/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-07-28","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Export AI Gateway trace spans to OpenTelemetry-compatible backends for distributed tracing and performance monitoring. title: OpenTelemetry image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # OpenTelemetry Last updated Jun 1, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/observability/otel-integration/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) AI Gateway supports exporting traces to OpenTelemetry-compatible backends, enabling you to monitor and analyze AI request performance alongside your existing observability infrastructure. ## Overview The OpenTelemetry (OTEL) integration automatically exports trace spans for AI requests processed through your gateway. These spans include detailed information about: * Request model and provider * Token usage (input and output) * Request prompts and completions * Cost estimates * Custom metadata This integration follows the [OpenTelemetry specification ↗](https://opentelemetry.io/docs/specs/otel/) for distributed tracing and uses the OTLP (OpenTelemetry Protocol) format, supporting both JSON and protobuf encoding. ## Configuration To enable OpenTelemetry tracing for your gateway, configure one or more OTEL exporters in your gateway settings. Each exporter accepts: * **URL** (required): The endpoint URL of your OTEL collector * **Headers** (optional): Additional custom headers to include in export requests. If your collector requires authentication, pass it here (for example, `Authorization: Bearer `). * **Authorization** (optional): A reference to a secret in [Secrets Store](https://developers.cloudflare.com/secrets-store/) containing your collector's authorization header value. When set, AI Gateway resolves the secret at runtime and sends it as the `Authorization` header on export requests. For most use cases, passing authentication via **Headers** is simpler. * **Content type** (optional): The export format — `json` (default) or `protobuf`. ### Configuration via Dashboard 1. Navigate to your AI Gateway in the Cloudflare dashboard. 2. Go to **Settings** tab. 3. Add an OTEL exporter with your collector endpoint URL. 4. If your collector requires authentication, add an `Authorization` header in the **Headers** field with your token value. ## Exported Span Attributes AI Gateway exports spans with the following attributes following the [Semantic Conventions for Gen AI ↗](https://opentelemetry.io/docs/specs/semconv/gen-ai/): ### Standard Attributes | Attribute | Type | Description | | ---------------------------- | ------ | ----------------------------------------------- | | gen\_ai.request.model | string | The AI model used for the request | | gen\_ai.model.provider | string | The AI provider (e.g., openai, anthropic) | | gen\_ai.usage.input\_tokens | int | Number of input tokens consumed | | gen\_ai.usage.output\_tokens | int | Number of output tokens generated | | gen\_ai.prompt\_json | string | JSON-encoded prompt/messages sent to the model | | gen\_ai.completion\_json | string | JSON-encoded completion/response from the model | | gen\_ai.usage.cost | double | Estimated cost of the request | ### Custom Metadata Any custom metadata added to your requests via the `cf-aig-metadata` header will also be included as span attributes. This allows you to correlate traces with user IDs, team names, or other business context. ```bash curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/openai/chat/completions \ --header 'Authorization: Bearer {api_token}' \ --header 'Content-Type: application/json' \ --header 'cf-aig-metadata: {"user_id": "user123", "team": "engineering"}' \ --data '{ "model": "gpt-4o", "messages": [{"role": "user", "content": "Hello!"}] }' ``` The above request will include `user_id` and `team` as additional span attributes in the exported trace. Note Custom metadata attributes that start with `gen_ai.` are reserved for standard GenAI semantic conventions and will not be added as custom attributes. ## Trace Context Propagation AI Gateway supports trace context propagation, allowing you to link AI Gateway spans with your application's traces. You can provide trace context using custom headers: * `cf-aig-otel-trace-id` (optional): A 32-character hex string to use as the trace ID * `cf-aig-otel-parent-span-id` (optional): A 16-character hex string to use as the parent span ID ```bash curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/openai/chat/completions \ --header 'cf-aig-otel-trace-id: a1b2c3d4e5f6g7h8i9j0k1l2m3n4o5p6' \ --header 'cf-aig-otel-parent-span-id: a1b2c3d4e5f6g7h8' \ --header 'Authorization: Bearer {api_token}' \ --header 'Content-Type: application/json' \ --data '{ "model": "gpt-4o", "messages": [{"role": "user", "content": "Hello!"}] }' ``` When these headers are provided, the AI Gateway span will use them to link with your existing trace. If not provided, AI Gateway will generate a new trace ID automatically. ## Common OTEL Backends AI Gateway's OTEL integration works with any OpenTelemetry-compatible backend, including: * [Honeycomb ↗](https://www.honeycomb.io/) * [Braintrust ↗](https://www.braintrust.dev/docs/integrations/sdk-integrations/opentelemetry) * [Langfuse ↗](https://langfuse.com/integrations/native/opentelemetry) Note AI Gateway supports both OTLP/JSON and OTLP/protobuf export formats. Use the **Content type** setting to choose the format your collector expects. Refer to your observability platform's documentation for the correct OTLP endpoint URL and authentication requirements. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/observability/otel-integration/#page","headline":"OpenTelemetry · Cloudflare AI Gateway docs","description":"Export AI Gateway trace spans to OpenTelemetry-compatible backends for distributed tracing and performance monitoring.","url":"https://developers.cloudflare.com/ai-gateway/observability/otel-integration/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-06-01","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Track organization-wide AI spend, attribute usage to identities, and detect anomalous sessions in AI Gateway. title: User Insights image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # User Insights Last updated Aug 4, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/observability/user-insights/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) The User Insights dashboard shows how much your organization spends on AI, which identities are responsible for that spend, and which users deviate from their typical usage. It uses the traffic already flowing through your gateway, so there is no additional setup. ## Attribute usage to identities User Insights is available to all AI Gateway customers at no additional cost and works on any traffic through your gateway. Without an identity or custom metadata on your requests, all usage is grouped under a single anonymous identifier, and User Insights cannot distinguish between individual users. To attribute usage to individual users, add a user identifier with [custom metadata](https://developers.cloudflare.com/ai-gateway/observability/custom-metadata/), or put your gateway behind [Cloudflare Access](https://developers.cloudflare.com/cloudflare-one/access-controls/). With Access, each authenticated request carries a verified identity you can filter spend and analytics by. ## Key metrics At the top of the User Insights page, you can view the following organization-wide metrics for the selected time range: * **Active users**: Identities with gateway usage. * **Total requests**: Gateway requests in this range. * **Adoption rate**: IdP identities with at least one request. * **Tokens per active user**: Median over this time range. * **Median spend / active user**: Observed spend per attributed identity. * **Top 10% request activity**: Share of attributed requests made by the most active users. * **Users to review**: Users whose cost is at least 2x the median spend. * **Identity coverage**: Share of requests attributed to users. ## Anomaly detection User Insights baselines each user's normal usage and flags sessions that fall outside it, which can indicate a compromised credential or a misbehaving agent. Baselines are calculated per session, not per request. For each user, User Insights uses the 95th percentile (p95) session cost over the last 30 days. The baseline is rolling and updates as usage changes. A session is flagged when it exceeds both of the following thresholds: * **Relative**: More than 2x the user's own p95 session cost. * **Absolute**: Above the organization-level p99 session cost across all users. Both thresholds must be met. This avoids flagging small spikes from low-usage users and routine high-cost sessions from heavy users. Flagged users appear in a filtered view with the sessions that triggered the flag and their cost. User Insights does not block requests. ## User view Select a user to see their usage in detail: * **Spend**: Total observed spend for the user in this range. * **Requests**: Total gateway requests made by the user. * **Tokens**: Total tokens consumed by the user. * **Gateway cached requests**: Number of requests served from cache. * **Errored requests**: Number of requests that returned an error. * **Cache hit rate**: Share of requests served from cache. * **Sessions**: Approximate session count from request metadata. * **Top model**: The model the user sent the most requests to. * **Top provider**: The provider the user sent the most requests to. * **Last seen**: Most recent activity, from the daily spend trend. * **Active days**: Number of days the user sent traffic in this range. * **Identity coverage**: Share of the user's requests attributed to an identity. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/observability/user-insights/#page","headline":"User Insights · Cloudflare AI Gateway docs","description":"Track organization-wide AI spend, attribute usage to identities, and detect anomalous sessions in AI Gateway.","url":"https://developers.cloudflare.com/ai-gateway/observability/user-insights/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-08-04","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: View audit log entries for AI Gateway configuration changes such as gateway creation, deletion, and updates. title: Audit logs image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Audit logs Last updated Apr 20, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/reference/audit-logs/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) [Audit logs](https://developers.cloudflare.com/fundamentals/account/account-security/review-audit-logs/) provide a comprehensive summary of changes made within your Cloudflare account, including those made to gateways in AI Gateway. This functionality is available on all plan types, free of charge, and is enabled by default. ## Viewing Audit Logs To view audit logs for AI Gateway, in the Cloudflare dashboard, go to the **Audit logs** page. [Go to **Audit logs** ↗](https://dash.cloudflare.com/?to=/:account/audit-log) For more information on how to access and use audit logs, refer to [review audit logs documentation](https://developers.cloudflare.com/fundamentals/account/account-security/review-audit-logs/). ## Logged Operations The following configuration actions are logged: | Operation | Description | | --------------- | -------------------------------- | | gateway created | Creation of a new gateway. | | gateway deleted | Deletion of an existing gateway. | | gateway updated | Edit of an existing gateway. | ## Example Log Entry Below is an example of an audit log entry showing the creation of a new gateway: ```json { "action": { "info": "gateway created", "result": true, "type": "create" }, "actor": { "email": "", "id": "3f7b730e625b975bc1231234cfbec091", "ip": "fe32:43ed:12b5:526::1d2:13", "type": "user" }, "id": "5eaeb6be-1234-406a-87ab-1971adc1234c", "interface": "UI", "metadata": {}, "newValue": "", "newValueJson": { "cache_invalidate_on_update": false, "cache_ttl": 0, "collect_logs": true, "id": "test", "rate_limiting_interval": 0, "rate_limiting_limit": 0, "rate_limiting_technique": "fixed" }, "oldValue": "", "oldValueJson": {}, "owner": { "id": "1234d848c0b9e484dfc37ec392b5fa8a" }, "resource": { "id": "89303df8-1234-4cfa-a0f8-0bd848e831ca", "type": "ai_gateway.gateway" }, "when": "2024-07-17T14:06:11.425Z" } ``` Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/reference/audit-logs/#page","headline":"Audit logs · Cloudflare AI Gateway docs","description":"View audit log entries for AI Gateway configuration changes such as gateway creation, deletion, and updates.","url":"https://developers.cloudflare.com/ai-gateway/reference/audit-logs/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-20","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Review AI Gateway limits for gateways, log storage, cache size, metadata entries, and Logpush jobs. title: Limits image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Limits Last updated May 27, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/reference/limits/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) The following limits apply to gateway configurations, logs, and related features in Cloudflare's platform. ## Gateway and log limits | Feature | Limit | | ------------------------------------------------------------------------------------------------------ | ----------------------------------------- | | [Cacheable request size](https://developers.cloudflare.com/ai-gateway/features/caching/) | 25 MB per request | | [Cache TTL](https://developers.cloudflare.com/ai-gateway/features/caching/#cache-ttl-cf-aig-cache-ttl) | 1 month | | [Custom metadata](https://developers.cloudflare.com/ai-gateway/observability/custom-metadata/) | 5 entries per request | | [Datasets](https://developers.cloudflare.com/ai-gateway/evaluations/set-up-evaluations/) | 10 per gateway | | Gateways free plan | 10 per account | | Gateways paid plan | 20 per account | | Gateway name length | 64 characters | | Log storage rate limit | 500 logs per second per gateway | | [Unified Billing](https://developers.cloudflare.com/ai-gateway/features/unified-billing/) request rate | 200 requests per 60 seconds per gateway 4 | | Logs stored [paid plan](https://developers.cloudflare.com/ai-gateway/reference/pricing/) | 10 million per gateway 1 | | Logs stored [free plan](https://developers.cloudflare.com/ai-gateway/reference/pricing/) | 100,000 per account 2 | | [Log size stored](https://developers.cloudflare.com/ai-gateway/observability/logging/) | 10 MB per log 3 | | [Logpush jobs](https://developers.cloudflare.com/ai-gateway/observability/logging/logpush/) | 4 per account | | [Logpush size limit](https://developers.cloudflare.com/ai-gateway/observability/logging/logpush/) | 1MB per log | 1 When you reach the log storage limit for a gateway, you can configure your gateway to either automatically delete the oldest logs to make room for new ones, or stop saving new logs. You can also use [Logpush](https://developers.cloudflare.com/ai-gateway/observability/logging/logpush/) to export logs to external storage. Refer to [Automatic log deletion](https://developers.cloudflare.com/ai-gateway/observability/logging/#automatic-log-deletion)for more details. 2 On the free plan, the log storage limit applies to total logs across all gateways in your account. Same auto-delete or stop-saving behavior as 1. 3 Logs larger than 10 MB will not be stored. 4 This rate limit applies to requests that use Cloudflare-managed credentials through [Unified Billing](https://developers.cloudflare.com/ai-gateway/features/unified-billing/). When the limit is exceeded, AI Gateway returns a `429` error. This limit does not apply to requests that use your own provider keys through [bring your own keys (BYOK)](https://developers.cloudflare.com/ai-gateway/configuration/bring-your-own-keys/). ## DLP limits [DLP](https://developers.cloudflare.com/ai-gateway/features/dlp/) for AI Gateway uses shared [Cloudflare One DLP profiles](https://developers.cloudflare.com/cloudflare-one/data-loss-prevention/dlp-profiles/). The following limits apply to DLP profiles and detection entries at the account level: | Feature | Limit | | ---------------------------------------- | --------- | | Custom entries | 25 | | Exact Data Match cells per spreadsheet | 100,000 | | Custom Wordlist keywords per spreadsheet | 200 | | Custom Wordlist keywords per account | 1,000 | | Dataset cells per account | 1,000,000 | DLP profiles are shared with Cloudflare One and are not coupled to individual gateways. You can apply the same DLP profiles across multiple gateways without additional profile limits. There is no separate limit on the number of DLP policies per gateway. Need a higher limit? To request an increase to a limit, complete the [Limit Increase Request Form ↗](https://forms.gle/cuXu1QnQCrSNkkaS8). If the limit can be increased, Cloudflare will contact you with next steps. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/reference/limits/#page","headline":"Limits · Cloudflare AI Gateway docs","description":"Review AI Gateway limits for gateways, log storage, cache size, metadata entries, and Logpush jobs.","url":"https://developers.cloudflare.com/ai-gateway/reference/limits/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-05-27","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Review AI Gateway pricing, including free core features, persistent log storage limits, and premium add-ons. title: Pricing image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Pricing Last updated May 19, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/reference/pricing/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) AI Gateway is available to use on all plans. AI Gateway's core features available today are offered for free, and all it takes is a Cloudflare account and one line of code to [get started](https://developers.cloudflare.com/ai-gateway/get-started/). Core features include: dashboard analytics, caching, and rate limiting. We will continue to build and expand AI Gateway. Some new features may be additional core features that will be free while others may be part of a premium plan. We will announce these as they become available. You can monitor your usage in the AI Gateway dashboard. ## Persistent logs Persistent logs are available on all plans. Log storage limits vary by plan. ### Log storage limits | Plan | Log storage limit | | ------------ | -------------------------------------- | | Workers Free | 100,000 logs total across all gateways | | Workers Paid | 10,000,000 logs per gateway | For more details on log storage behavior and automatic log deletion, refer to [Limits](https://developers.cloudflare.com/ai-gateway/reference/limits/) and [Logging](https://developers.cloudflare.com/ai-gateway/observability/logging/#automatic-log-deletion). ## Data Loss Prevention (DLP) DLP scanning in AI Gateway is free on all plans. Accounts without a Zero Trust subscription have access to two predefined [DLP profiles](https://developers.cloudflare.com/cloudflare-one/data-loss-prevention/dlp-profiles/): Financial Information and Social / Insurance / National Identifier Numbers. DLP profiles are shared at the account level with [Cloudflare One](https://developers.cloudflare.com/cloudflare-one/data-loss-prevention/). If your account has a Zero Trust subscription that includes DLP, the full set of profiles — including all predefined profiles, custom profiles, integration profiles, DLP datasets, and OCR — is automatically available in AI Gateway. ## Guardrails [Guardrails](https://developers.cloudflare.com/ai-gateway/features/guardrails/) evaluates prompts and responses using [@cf/meta/llama-guard-3-8b](https://developers.cloudflare.com/workers-ai/models/llama-guard-3-8b/) on Workers AI. Usage is billed as [Workers AI](https://developers.cloudflare.com/workers-ai/platform/pricing/) token-based inference — cost scales with the length of the prompts and responses being evaluated. ## Unified Billing A 5% fee is applied to all credits purchased through [Unified Billing](https://developers.cloudflare.com/ai-gateway/features/unified-billing/). For example, a $100 credit purchase will result in a $105 charge. Inference pricing from providers is passed through with no markup — you pay the same per-token rates as you would directly with the provider. ## Logpush Logpush is only available on the Workers Paid plan. | | Paid plan | | -------- | ---------------------------------- | | Requests | 10 million / month, +$0.05/million | ## Pricing notes Prices subject to change. If you are an Enterprise customer, reach out to your account team to confirm pricing details. Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/reference/pricing/#page","headline":"Pricing · Cloudflare AI Gateway docs","description":"Review AI Gateway pricing, including free core features, persistent log storage limits, and premium add-ons.","url":"https://developers.cloudflare.com/ai-gateway/reference/pricing/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-05-19","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ``` --- --- description: Resolve common AI Gateway issues including authentication errors, missing logs, and provider connectivity problems. title: Troubleshooting image: https://developers.cloudflare.com/og-docs.png --- [Skip to content](#main-content) > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/ai-gateway/llms.txt > Use this file to discover all available pages before exploring further. # Troubleshooting Last updated Apr 20, 2026|Copy as Markdown|[View as Markdown](https://developers.cloudflare.com/ai-gateway/reference/troubleshooting/index.md)|[Agent setup](https://developers.cloudflare.com/agent-setup/) This page covers common issues when using AI Gateway. For provider-specific troubleshooting, refer to the relevant provider documentation. ## Authentication errors ### 401 or unauthenticated errors If you receive authentication errors from your AI provider, AI Gateway did not pass valid credentials upstream. Check the following: 1. **Verify header placement**: Make sure your Cloudflare token is in `cf-aig-authorization`, not `Authorization`. The `Authorization` header is reserved for provider credentials. 2. **Check your configuration based on endpoint type**: * **Provider-specific endpoints**: Confirm your request URL includes the provider path (for example, `/google-vertex-ai/` or `/openai/`). AI Gateway uses this to identify the provider and apply the correct stored credentials. * **Unified `/compat/chat/completions` endpoint**: Confirm your `model` name starts with the provider prefix (for example, `google-vertex-ai/google/gemini-2.5-flash` or `openai/gpt-4o`). AI Gateway uses this prefix to route the request and select the correct stored credentials. 3. **Verify BYOK key selection**: If you have multiple keys configured for a provider, ensure either: * You are using the key with alias `default`, or * You include the `cf-aig-byok-alias` header with the correct alias name 4. **Verify BYOK configuration**: If using BYOK, confirm in the dashboard that your credentials were saved correctly. For provider-specific authentication issues: * [Google Vertex AI troubleshooting](https://developers.cloudflare.com/ai-gateway/usage/providers/vertex/#troubleshooting) ## DLP issues For troubleshooting Data Loss Prevention issues such as DLP not triggering or unexpected blocking, refer to [DLP troubleshooting](https://developers.cloudflare.com/ai-gateway/features/dlp/set-up-dlp/#troubleshooting). ## Request failures ### Requests timing out * Check if the upstream provider is experiencing issues * Consider implementing [dynamic routing](https://developers.cloudflare.com/ai-gateway/features/dynamic-routing/) with fallbacks for transient failures * Review your [rate limiting](https://developers.cloudflare.com/ai-gateway/features/rate-limiting/) configuration ### Requests returning errors from the provider * Verify your API key or credentials are valid with the provider directly * Check the provider's status page for outages * Review [AI Gateway logs](https://developers.cloudflare.com/ai-gateway/observability/logging/) for detailed error information ## Caching issues ### Requests not being cached * Verify [caching is enabled](https://developers.cloudflare.com/ai-gateway/features/caching/) for your gateway * Check that the request method and content type are cacheable * Streaming responses are not cached by default ### Unexpected cache hits or misses * Review your cache TTL settings * Check if you have request headers that are [bypassing the cache](https://developers.cloudflare.com/ai-gateway/features/caching/#skip-cache-cf-aig-skip-cache) or setting a [custom cache key](https://developers.cloudflare.com/ai-gateway/features/caching/#custom-cache-key-cf-aig-cache-key). Was this helpful? YesNo ## On this page [![](https://developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://developers.cloudflare.com/) ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-gateway/reference/troubleshooting/#page","headline":"Troubleshooting · Cloudflare AI Gateway docs","description":"Resolve common AI Gateway issues including authentication errors, missing logs, and provider connectivity problems.","url":"https://developers.cloudflare.com/ai-gateway/reference/troubleshooting/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-04-20","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}} ```