Terraform (3)

Terraform (3)

Last updated Oct 1, 2026 This example shows how to configure Basin Pipelines and Basin Catalog with Terraform using the Cloudflare provider ↗︎ (v5.19.0+).

Legacy product names

Terraform resource and API permission group names for Basin Catalog still use legacy R2 Data Catalog identifiers, such as cloudflare_r2_data_catalog and Workers R2 Data Catalog Write. These names will be updated in the coming weeks. Until then, use the identifiers shown in this guide.

The configuration creates a complete data pipeline: an R2 bucket with the data catalog enabled, a scoped API token for the sink, and the stream, sink, and pipeline resources that ingest JSON data into an Apache Iceberg ↗︎ table.

Prerequisites

Terraform CLI ↗︎ >= 1.0
A Cloudflare account with R2 and Basin Pipelines enabled
An API token scoped to your account with the following permissions:
Basin Pipelines - Edit
Workers R2 Storage - Edit
Basin Catalog - Edit
Account API Tokens - Edit

For general information on using Terraform with Cloudflare, refer to the Terraform documentation.

Terraform resources

This example uses the following Cloudflare Terraform resources:

Resource	Description
`cloudflare_r2_bucket` ↗︎	Creates an R2 bucket to store pipeline data
`cloudflare_r2_data_catalog` ↗︎	Enables the Basin Catalog on a bucket
`cloudflare_pipeline_stream` ↗︎	Creates a stream that receives events via HTTP or Worker bindings
`cloudflare_pipeline_sink` ↗︎	Creates a sink that writes data to Basin Catalog or R2
`cloudflare_pipeline` ↗︎	Creates a pipeline with SQL that connects a stream to a sink
`cloudflare_account_token` ↗︎	Creates a scoped API token for sink authentication

End-to-end example

With terraform ↗︎ installed, create a directory and the following files.

1. Define variables and provider

Create variables.tf:

terraform {
  required_providers {
    cloudflare = {
      source  = "cloudflare/cloudflare"
      version = "~> 5.19"
    }
  }
}

provider "cloudflare" {
  api_token = var.cloudflare_api_token
}

variable "cloudflare_api_token" {
  type      = string
  sensitive = true
}

variable "cloudflare_account_id" {
  type = string
}

2. Create the pipeline resources

Create main.tf:

# --- R2 bucket and Data Catalog ---

resource "cloudflare_r2_bucket" "pipeline_bucket" {
  account_id = var.cloudflare_account_id
  name       = "my-pipeline-bucket"
}

resource "cloudflare_r2_data_catalog" "pipeline_catalog" {
  account_id  = var.cloudflare_account_id
  bucket_name = cloudflare_r2_bucket.pipeline_bucket.name
}

# --- Scoped API token for the sink ---

data "cloudflare_account_api_token_permission_groups_list" "r2_bucket_item_write" {
  account_id = var.cloudflare_account_id
  name       = "Workers R2 Storage Bucket Item Write"
}

data "cloudflare_account_api_token_permission_groups_list" "r2_data_catalog_write" {
  account_id = var.cloudflare_account_id
  name       = "Workers R2 Data Catalog Write"
}

resource "cloudflare_account_token" "sink_token" {
  name       = "pipeline-sink-token"
  account_id = var.cloudflare_account_id

  policies = [{
    effect = "allow"
    permission_groups = [
      { id = data.cloudflare_account_api_token_permission_groups_list.r2_bucket_item_write.result[0].id },
      { id = data.cloudflare_account_api_token_permission_groups_list.r2_data_catalog_write.result[0].id },
    ]
    resources = jsonencode({
      "com.cloudflare.api.account.${var.cloudflare_account_id}" = "*"
    })
  }]
}

# --- Stream ---

resource "cloudflare_pipeline_stream" "my_stream" {
  account_id = var.cloudflare_account_id
  name       = "my_stream"
  format = {
    type = "json"
  }
  schema = {
    fields = [{
      name     = "value"
      type     = "json"
      required = true
    }]
  }
  http = {
    enabled        = true
    authentication = false
    cors           = {}
  }
  worker_binding = {
    enabled = false
  }
}

# --- Sink (Basin Catalog) ---

resource "cloudflare_pipeline_sink" "my_sink" {
  account_id = var.cloudflare_account_id
  name       = "my_sink"
  type       = "r2_data_catalog"
  format = {
    type = "parquet"
  }
  schema = {
    fields = []
  }
  config = {
    account_id = var.cloudflare_account_id
    bucket     = cloudflare_r2_bucket.pipeline_bucket.name
    table_name = cloudflare_r2_data_catalog.pipeline_catalog.name
    token      = cloudflare_account_token.sink_token.value
  }
}

# --- Pipeline ---

resource "cloudflare_pipeline" "my_pipeline" {
  account_id = var.cloudflare_account_id
  name       = "my_pipeline"
  sql        = "INSERT INTO ${cloudflare_pipeline_sink.my_sink.name} SELECT * FROM ${cloudflare_pipeline_stream.my_stream.name}"
}

Use an R2 sink instead of Basin Catalog

</summary>

To write raw Parquet or JSON files to R2 instead of Iceberg tables, replace the sink resource with an R2 sink. This requires R2 S3-compatible credentials instead of a catalog token.

Add variables for S3 credentials to <code>variables.tf</code>:

variable "r2_access_key_id" {
  type      = string
  sensitive = true
}

variable "r2_access_key_secret" {
  type      = string
  sensitive = true
}

Replace the sink resource in <code>main.tf</code>:

resource "cloudflare_pipeline_sink" "my_sink" {
  account_id = var.cloudflare_account_id
  name       = "my_sink"
  type       = "r2"
  format = {
    type = "json"
  }
  schema = {
    fields = []
  }
  config = {
    account_id = var.cloudflare_account_id
    bucket     = cloudflare_r2_bucket.pipeline_bucket.name
    credentials = {
      access_key_id     = var.r2_access_key_id
      secret_access_key = var.r2_access_key_secret
    }
  }
}

When using an R2 sink, you can remove the <code>cloudflare_r2_data_catalog</code>, <code>cloudflare_account_token</code>, and the two <code>cloudflare_account_api_token_permission_groups_list</code> data sources from your configuration.

</details>

3. Define outputs

Create outputs.tf:

output "pipeline_id" {
  value = cloudflare_pipeline.my_pipeline.id
}

output "pipeline_status" {
  value = cloudflare_pipeline.my_pipeline.status
}

output "stream_endpoint" {
  value = cloudflare_pipeline_stream.my_stream.endpoint
}

output "sink_id" {
  value = cloudflare_pipeline_sink.my_sink.id
}

4. Deploy

Set your environment variables:

export TF_VAR_cloudflare_api_token="<YOUR_API_TOKEN>"
export TF_VAR_cloudflare_account_id="<YOUR_ACCOUNT_ID>"

You can then use terraform plan to view the changes and terraform apply to apply them:

terraform init
terraform plan
terraform apply

After the apply completes, Terraform outputs the stream endpoint URL. Use it to send data to your pipeline:

curl -X POST https://<STREAM_ENDPOINT> \
  -H "Content-Type: application/json" \
  -d '[{"value": {"event": "page_view", "user_id": "user_123"}}]'

Clean up

To remove all resources created by this configuration:

terraform destroy

Was this helpful?

YesNo

{"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/basin-pipelines/reference/terraform/#page","headline":"Terraform","description":"Configure Basin Pipelines and Basin Catalog with Terraform using the Cloudflare provider.","url":"https://developers.cloudflare.com/basin-pipelines/reference/terraform/","inLanguage":"en","image":"https://developers.cloudflare.com/basin-pipelines/reference/terraform/og.png?v=0fa71271ab3b77ff","dateModified":"2026-10-01","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"},"keywords":["Terraform"]}

description: Wrangler commands for managing Basin Pipelines resources. title: Wrangler commands image: https://developers.cloudflare.com/basin-pipelines/reference/wrangler-commands/og.png?v=56104d764447ca64

Documentation Index
Fetch the complete documentation index at: https://developers.cloudflare.com/basin-pipelines/llms.txt
Use this file to discover all available pages before exploring further.

Terraform (3)

Terraform (3)

Prerequisites

Terraform resources

End-to-end example

1. Define variables and provider

2. Create the pipeline resources

3. Define outputs

4. Deploy

Clean up

On this page