Query rewriting
Query rewriting is an optional step in the AI Search pipeline that improves retrieval quality for follow-up queries. It applies to both Search and Chat Completions requests.
The wording of a user's question may not match how your documents are written. Query rewriting helps bridge this gap by:
- Rephrasing informal or vague queries into precise, information-dense terms
- Adding synonyms or related keywords
- Removing filler words or irrelevant details
- Resolving follow-up queries that reference previous messages (for example, "tell me more about that" becomes a specific query based on conversation history)
This leads to more relevant search matches which improves the accuracy of results and generated responses.
Query rewriting requires the messages format and does not apply when using the query format. The first message is always used as-is. For follow-up queries, AI Search sends the conversation history, the latest user message, and the query rewrite system prompt to the configured LLM. The rewritten query is then embedded and used to perform the search.
First message: What is Cloudflare Workers?
(used as-is, no rewriting)
Follow-up message: How do I deploy one?
Rewritten query: deploy Cloudflare Worker getting started
The follow-up "How do I deploy one?" is vague on its own. Query rewriting uses the conversation context to understand "one" refers to a Cloudflare Worker. How the query is rewritten depends on your query rewrite system prompt.
Enabling query rewriting adds an extra LLM call to the query pipeline, which may increase latency.