Pull consumers

Last updated Jul 1, 2026 A pull-based consumer allows you to pull from a queue over HTTP from any environment and/or programming language outside of Cloudflare Workers. A pull-based consumer can be useful when your message consumption rate is limited by upstream infrastructure or long-running tasks.

How to choose between push or pull consumer

Deciding whether to configure a push-based consumer or a pull-based consumer will depend on how you are using your queues, as well as the configuration of infrastructure upstream from your queue consumer.

  • Starting with a push-based consumer is the easiest way to get started and consume from a queue. A push-based consumer runs on Workers, and by default, will automatically scale up and consume messages as they are written to the queue.
  • Use a pull-based consumer if you need to consume messages from existing infrastructure outside of Cloudflare Workers, and/or where you need to carefully control how fast messages are consumed. A pull-based consumer must explicitly make a call to pull (and then acknowledge) messages from the queue, only when it is ready to do so.

You can remove and attach a new consumer on a queue at any time, allowing you to change from a pull-based to a push-based consumer if your requirements change.

Retrieve an API bearer token

To configure a pull-based consumer, create an API token with both the queues#read and queues#write permissions. A consumer must be able to write to a queue to acknowledge messages.

To configure a pull-based consumer and receive messages from a queue, you need to:

  1. Enable HTTP pull for the queue.
  2. Create a valid authentication token for the HTTP client.
  3. Pull message batches from the queue.
  4. Acknowledge and/or retry messages within a batch.

1. Enable HTTP pull

You can enable HTTP pull or change a queue from push-based to pull-based via the wrangler CLI or via the Cloudflare dashboard ↗︎. Enabling HTTP pull from a Wrangler configuration file is no longer supported.

Note

If you have specified type = "http_pull" in your Wrangler configuration file, remove and redeploy. Your Worker will retain access to the HTTP pull endpoint, and HTTP pull will remain enabled on your queue.

wrangler CLI

You can enable a pull-based consumer on any existing queue by using the wrangler queues consumer http sub-commands and providing a queue name.

npx wrangler queues consumer http add $QUEUE-NAME

If you have an existing push-based consumer, you will need to remove that first. wrangler will return an error if you attempt to call consumer http add on a queue with an existing consumer configuration:

wrangler queues consumer worker remove $QUEUE-NAME $SCRIPT_NAME

Note

If you remove the Worker consumer with wrangler but do not delete the [[queues.consumer]] configuration from your Wrangler configuration file, subsequent deployments of your Worker will fail when they attempt to add a conflicting consumer configuration.

Ensure you remove the consumer configuration first.

2. Consumer authentication

HTTP Pull consumers require an API token with the com.cloudflare.api.account.queues_read and com.cloudflare.api.account.queues_write permissions.

Both read and write are required as a pull-based consumer needs to write to the queue state to acknowledge the messages it receives. Consuming messages mutates the queue.

API tokens are presented as Bearer tokens in the Authorization header of a HTTP request in the format Authorization: Bearer $YOUR_TOKEN_HERE. The following example shows how to pass an API token using the curl HTTP client:

curl "https://api.cloudflare.com/client/v4/accounts/${CF_ACCOUNT_ID}/queues/${QUEUE_ID}/messages/pull" \
--header "Authorization: Bearer ${QUEUES_TOKEN}" \
--header "Content-Type: application/json" \
--data '{ "visibility_timeout_ms": 10000, "batch_size": 2 }'

You may authenticate and run multiple concurrent pull-based consumers against a single queue.

Create API tokens

To create an API token:

  1. Log in to the Cloudflare dashboard ↗︎.
  2. Go to My Profile > API Tokens ↗︎.
  3. Select Create Token.
  4. Scroll to the bottom of the page and select Create Custom Token.
  5. Give the token a name. For example, queue-pull-token.
  6. Under the Permissions section, choose Account and then Queues. Ensure you have selected Edit (read+write).
  7. (Optional) Select All accounts (default) or a specific account to scope the token to.
  8. Select Continue to summary and then Create token.

You will need to note the token down: it will only be displayed once.

3. Pull messages

To pull a message, make a HTTP POST request to the Queues REST API with a JSON-encoded body that optionally specifies a visibility_timeout and a batch_size, or an empty JSON object ({}):

index.jsjs

// POST /accounts/${CF_ACCOUNT_ID}/queues/${QUEUE_ID}/messages/pull with the timeout & batch size
let resp = await fetch(
    `https://api.cloudflare.com/client/v4/accounts/${CF_ACCOUNT_ID}/queues/${QUEUE_ID}/messages/pull`,
    {
        method: "POST",
        headers: {
            "content-type": "application/json",
            authorization: `Bearer ${QUEUES_API_TOKEN}`,
        },
        // Optional - you can provide an empty object '{}' and the defaults will apply.
        body: JSON.stringify({ visibility_timeout_ms: 6000, batch_size: 50 }),
    },
);

index.tsts

// POST /accounts/${CF_ACCOUNT_ID}/queues/${QUEUE_ID}/messages/pull with the timeout & batch size
let resp = await fetch(
    `https://api.cloudflare.com/client/v4/accounts/${CF_ACCOUNT_ID}/queues/${QUEUE_ID}/messages/pull`,
    {
        method: "POST",
        headers: {
            "content-type": "application/json",
            authorization: `Bearer ${QUEUES_API_TOKEN}`,
        },
        // Optional - you can provide an empty object '{}' and the defaults will apply.
        body: JSON.stringify({ visibility_timeout_ms: 6000, batch_size: 50 }),
    },
);

import json
from workers import fetch

# POST /accounts/${CF_ACCOUNT_ID}/queues/${QUEUE_ID}/messages/pull with the timeout & batch size

resp = await fetch(
    f"https://api.cloudflare.com/client/v4/accounts/{CF_ACCOUNT_ID}/queues/{QUEUE_ID}/messages/pull",
    method="POST",
    headers={
        "content-type": "application/json",
        "authorization": f"Bearer {QUEUES_API_TOKEN}",
    }, # Optional - you can provide an empty object '{}' and the defaults will apply.
    body=json.dumps({"visibility_timeout_ms": 6000, "batch_size": 50}),
)

This will return an array of messages (up to the specified batch_size) in the below format:

{
    "success": true,
    "errors": [],
    "messages": [],
    "result": {
        "message_backlog_count": 10,
        "messages": [
            {
                "body": "hello",
                "id": "1ad27d24c83de78953da635dc2ea208f",
                "timestamp_ms": 1689615013586,
                "attempts": 2,
                "metadata": {
                    "CF-sourceMessageSource": "dash",
                    "CF-Content-Type": "json"
                },
                "lease_id": "eyJhbGciOiJkaXIiLCJlbmMiOiJBMjU2Q0JDLUhTNTEyIn0..NXmbr8h6tnKLsxJ_AuexHQ.cDt8oBb_XTSoKUkVKRD_Jshz3PFXGIyu7H1psTO5UwI.smxSvQ8Ue3-ymfkV6cHp5Va7cyUFPIHuxFJA07i17sc"
            },
            {
                "body": "world",
                "id": "95494c37bb89ba8987af80b5966b71a7",
                "timestamp_ms": 1689615013586,
                "attempts": 2,
                "metadata": {
                    "CF-sourceMessageSource": "dash",
                    "CF-Content-Type": "json"
                },
                "lease_id": "eyJhbGciOiJkaXIiLCJlbmMiOiJBMjU2Q0JDLUhTNTEyIn0..QXPgHfzETsxYQ1Vd-H0hNA.mFALS3lyouNtgJmGSkTzEo_imlur95EkSiH7fIRIn2U.PlwBk14CY_EWtzYB-_5CR1k30bGuPFPUx1Nk5WIipFU"
            }
        ]
    }
}

Pull consumers follow a "short polling" approach: if there are messages available to be delivered, Queues will return a response immediately with messages up to the configured batch_size. If there are no messages to deliver, Queues will return an empty response. Queues does not hold an open connection (often referred to as "long polling") if there are no messages to deliver.

Note

The pull and ack endpoints use the new /queues/queue_id/messages/{action} API format, as defined in the Queues API documentation.

The undocumented /queues/queue_id/{action} endpoints are not supported and will be deprecated as of June 30th, 2024.

Each message object has five fields:

  1. body - this may be base64 encoded based on the content-type the message was published as.
  2. id - a unique, read-only ephemeral identifier for the message.
  3. timestamp_ms - when the message was published to the queue in milliseconds since the Unix epoch ↗︎. This allows you to determine how old a message is by subtracting it from the current timestamp.
  4. attempts - how many times the message has been attempted to be delivered in full. When this reaches the value of max_retries, the message will not be re-delivered and will be deleted from the queue permanently.
  5. lease_id - the encoded lease ID of the message. The lease_id is used to explicitly acknowledge or retry the message.