Get started

Cloudflare Queues is a flexible messaging queue that allows you to queue messages for asynchronous processing. By following this guide, you will create your first queue, a Worker to publish messages to that queue, and a consumer Worker to consume messages from that queue.

Prerequisites

To use Queues, you will need:

Sign up for a Cloudflare account ↗.
Install Node.js ↗.

Node.js version manager

Use a Node version manager like Volta ↗ or nvm ↗ to avoid permission issues and change Node.js versions. Wrangler, discussed later in this guide, requires a Node version of 16.17.0 or later.

1. Create a Worker project

You will access your queue from a Worker, the producer Worker. You must create at least one producer Worker to publish messages onto your queue. If you are using R2 Bucket Event Notifications, then you do not need a producer Worker.

To create a producer Worker, run:

npm create cloudflare@latest -- producer-worker

yarn create cloudflare@latest producer-worker

pnpm create cloudflare@latest producer-worker

For setup, select the following options:

For What would you like to start with?, choose Hello World example.
For Which template would you like to use?, choose Hello World Worker.
For Which language do you want to use?, choose TypeScript.
For Do you want to use git for version control?, choose Yes.
For Do you want to deploy your application?, choose No (we will be making some changes before deploying).

This will create a new directory, which will include both a src/index.ts Worker script, and a wrangler.toml configuration file. After you create your Worker, you will create a Queue to access.

Move into the newly created directory:

cd producer-worker

2. Create a queue

To use queues, you need to create at least one queue to publish messages to and consume messages from.

To create a queue, run:

npx wrangler queues create <MY-QUEUE-NAME>

Choose a name that is descriptive and relates to the types of messages you intend to use this queue for. Descriptive queue names look like: debug-logs, user-clickstream-data, or password-reset-prod.

Queue names must be 1 to 63 characters long. Queue names cannot contain special characters outside dashes (-), and must start and end with a letter or number.

You cannot change your queue name after you have set it. After you create your queue, you will set up your producer Worker to access it.

3. Set up your producer Worker

To expose your queue to the code inside your Worker, you need to connect your queue to your Worker by creating a binding. Bindings allow your Worker to access resources, such as Queues, on the Cloudflare developer platform.

To create a binding, open your newly generated wrangler.toml configuration file and add the following:

[[queues.producers]]
 queue = "MY-QUEUE-NAME"
 binding = "MY_QUEUE"

Replace MY-QUEUE-NAME with the name of the queue you created in step 3. Next, replace MY_QUEUE with the name you want for your binding. The binding must be a valid JavaScript variable name. This is the variable you will use to reference this queue in your Worker.

Write your producer Worker

You will now configure your producer Worker to create messages to publish to your queue. Your producer Worker will:

Take a request it receives from the browser.
Transform the request to JSON format.
Write the request directly to your queue.

In your Worker project directory, open the src folder and add the following to your index.ts file:

export default {
  async fetch(request, env, ctx): Promise<Response> {
    let log = {
      url: request.url,
      method: request.method,
      headers: Object.fromEntries(request.headers),
    };
    await env.<MY_QUEUE>.send(log);
    return new Response('Success!');
  },
} satisfies ExportedHandler<Env>;

Replace MY_QUEUE with the name you have set for your binding from your wrangler.toml.

Also add the queue to Env interface in index.ts.

export interface Env {
   <MY_QUEUE>: Queue<any>;
}

If this write fails, your Worker will return an error (raise an exception). If this write works, it will return Success back with a HTTP 200 status code to the browser.

In a production application, you would likely use a try...catch ↗ statement to catch the exception and handle it directly (for example, return a custom error or even retry).

Publish your producer Worker

With your wrangler.toml file and index.ts file configured, you are ready to publish your producer Worker. To publish your producer Worker, run:

npx wrangler deploy

You should see output that resembles the below, with a *.workers.dev URL by default.

Uploaded <YOUR-WORKER-NAME> (0.76 sec)
Published <YOUR-WORKER-NAME> (0.29 sec)
  https://<YOUR-WORKER-NAME>.<YOUR-ACCOUNT>.workers.dev

Copy your *.workers.dev subdomain and paste it into a new browser tab. Refresh the page a few times to start publishing requests to your queue. Your browser should return the Success response after writing the request to the queue each time.

You have built a queue and a producer Worker to publish messages to the queue. You will now create a consumer Worker to consume the messages published to your queue. Without a consumer Worker, the messages will stay on the queue until they expire, which defaults to four (4) days.

4. Create your consumer Worker

A consumer Worker receives messages from your queue. When the consumer Worker receives your queue’s messages, it can write them to another source, such as a logging console or storage objects.

In this guide, you will create a consumer Worker and use it to log and inspect the messages with wrangler tail. You will create your consumer Worker in the same Worker project that you created your producer Worker.

To create a consumer Worker, open your index.ts file and add the following queue handler to your existing fetch handler:

export default {
  async fetch(request, env, ctx): Promise<Response> {
    let log = {
      url: request.url,
      method: request.method,
      headers: Object.fromEntries(request.headers),
    };
    await env.<MY_QUEUE>.send(log);
    return new Response('Success!');
  },
  async queue(batch, env): Promise<void> {
    let messages = JSON.stringify(batch.messages);
    console.log(`consumed from our queue: ${messages}`);
  },
} satisfies ExportedHandler<Env>;

Replace MY_QUEUE with the name you have set for your binding from your wrangler.toml.

Every time messages are published to the queue, your consumer Worker’s queue handler (async queue) is called and it is passed one or more messages.

In this example, your consumer Worker transforms the queue’s JSON formatted message into a string and logs that output. In a real world application, your consumer Worker can be configured to write messages to object storage (such as R2), write to a database (like D1), further process messages before calling an external API (such as an email API) or a data warehouse with your legacy cloud provider.

When performing asynchronous tasks from within your consumer handler, use waitUntil() to ensure the response of the function is handled. Other asynchronous methods are not supported within the scope of this method.

Connect the consumer Worker to your queue

After you have configured your consumer Worker, you are ready to connect it to your queue.

Each queue can only have one consumer Worker connected to it. If you try to connect multiple consumers to the same queue, you will encounter an error when attempting to publish that Worker.

To connect your queue to your consumer Worker, open your wrangler.toml file and add this to the bottom:

[[queues.consumers]]
 queue = "<MY-QUEUE-NAME>"
 # Required: this should match the name of the queue you created in step 3.
 # If you misspell the name, you will receive an error when attempting to publish your Worker.
 max_batch_size = 10 # optional: defaults to 10
 max_batch_timeout = 5 # optional: defaults to 5 seconds

Replace MY-QUEUE-NAME with the queue you created in step 3.

In your consumer Worker, you are using queues to auto batch messages using the max_batch_size option and the max_batch_timeout option. The consumer Worker will receive messages in batches of 10 or every 5 seconds, whichever happens first.

max_batch_size (defaults to 10) helps to reduce the amount of times your consumer Worker needs to be called. Instead of being called for every message, it will only be called after 10 messages have entered the queue.

max_batch_timeout (defaults to 5 seconds) helps to reduce wait time. If the producer Worker is not sending up to 10 messages to the queue for the consumer Worker to be called, the consumer Worker will be called every 5 seconds to receive messages that are waiting in the queue.

Publish your consumer Worker

With your wrangler.toml file and index.ts file configured, publish your consumer Worker by running:

npx wrangler deploy

5. Read messages from your queue

After you set up consumer Worker, you can read messages from the queue.

Run wrangler tail to start waiting for our consumer to log the messages it receives:

npx wrangler tail

With wrangler tail running, open the Worker URL you opened in step 4.

You should receive a Success message in your browser window.

If you receive a Success message, refresh the URL a few times to generate messages and push them onto the queue.

With wrangler tail running, your consumer Worker will start logging the requests generated by refreshing.

If you refresh less than 10 times, it may take a few seconds for the messages to appear because batch timeout is configured for 10 seconds. After 10 seconds, messages should arrive in your terminal.

If you get errors when you refresh, check that the queue name you created in step 3 and the queue you referenced in your wrangler.toml file is the same. You should ensure that your producer Worker is returning Success and is not returning an error.

By completing this guide, you have now created a queue, a producer Worker that publishes messages to that queue, and a consumer Worker that consumes those messages from it.

Learn more about Cloudflare Workers and the applications you can build on Cloudflare.

Cloudflare Dashboard Discord Community Learning Center Support Portal

Cookie Settings