AI models are only as good as the context you give them. Models are trained and then frozen at a point in time, so they only know about information that existed before their knowledge cutoff date. This makes it difficult to work with models on recent events, changing APIs, or fast-moving news.
Without web search, an agent that needs live information usually guesses the URL of a page and fetches it directly. When the guess is wrong, the request returns a 404 Not Found and the agent has to try again.
Web Search API gives your agents a better option. Like a person starting with a search engine, your agent sends a query and receives a list of relevant, up-to-date results. You can inject those results into the model's context so its response is grounded in live information.
When you send a search request, Web Search API:
- Routes the request through the AI Gateway you specify.
- Forwards the query to the search provider you choose — Ceramic.ai, Exa, or Linkup. If you do not set a provider, Web Search API uses Ceramic.ai as default.
- Normalizes the provider's response into a consistent format, with a URL, title, and optional description, image, favicon, and last-modified date for each result.
- Records the request in your AI Gateway logs and bills it to your account.
Because every provider returns the same result format, you can switch providers by changing a single parameter.
Web Search API is built on AI Gateway, which acts as the control plane for your AI applications. Every search request goes through a gateway, which gives you:
- Observability — Search requests appear in your gateway's logs and analytics alongside your model inference requests.
- Unified Billing — Searches draw down from your AI Gateway credit balance. You pay each provider's list API price, with no additional markup. For rates, refer to Providers.
- Bring your own key (BYOK) — If you already have an account with a search provider, you can store your provider API key on your gateway. The provider bills you directly.
- Access control — Control which providers your gateway can use and who can send requests.
Cloudflare believes that crawlers should be honest and transparent, and should respect the rules and preferences that site owners set. Site owners should have meaningful visibility into and control over how their content is used.
Every Web Search API provider has committed to meet the following standards:
- Verified bot compliance — The crawler the provider uses must meet Cloudflare's published requirements for verified bots. This includes identifying the crawler and respecting
robots.txt. - Source attribution — Every search result must include a link to the location of the crawled content.
When you use Web Search API, your agents consume search results from operators that give site owners transparency, control, and visibility over how their content is crawled.