Metrics and analytics (2)

Last updated Oct 1, 2026 Basin Pipelines exposes metrics which allow you to measure data ingested, processed, and delivered to sinks.

The metrics displayed in the Cloudflare dashboard ↗︎ are queried from Cloudflare's GraphQL Analytics API. You can access the metrics programmatically via GraphQL or HTTP client.

Metrics

Operator metrics

Basin Pipelines exports the below metrics within the pipelinesOperatorAdaptiveGroups dataset. These metrics track data read and processed by pipeline operators.

Metric GraphQL Field Name Description
Bytes In bytesIn Total number of bytes read by the pipeline (filter by streamId_neq: "" to get data read from streams)
Records In recordsIn Total number of records read by the pipeline (filter by streamId_neq: "" to get data read from streams)
Decode Errors decodeErrors Number of messages that could not be deserialized in the stream schema

For a detailed breakdown of why events were dropped (including specific error types like missing_field, type_mismatch, parse_failure, and null_value), refer to User error metrics.

The pipelinesOperatorAdaptiveGroups dataset provides the following dimensions for filtering and grouping queries:

  • pipelineId - ID of the pipeline
  • streamId - ID of the source stream
  • datetime - Timestamp of the operation
  • date - Timestamp of the operation, truncated to the start of a day
  • datetimeHour - Timestamp of the operation, truncated to the start of an hour

Sink metrics

Basin Pipelines exports the below metrics within the pipelinesSinkAdaptiveGroups dataset. These metrics track data delivery to sinks.

Metric GraphQL Field Name Description
Bytes Written bytesWritten Total number of bytes written to the sink, after compression
Records Written recordsWritten Total number of records written to the sink
Files Written filesWritten Number of files written to the sink
Row Groups Written rowGroupsWritten Number of row groups written (for Parquet files)
Uncompressed Bytes Written uncompressedBytesWritten Total number of bytes written before compression

The pipelinesSinkAdaptiveGroups dataset provides the following dimensions for filtering and grouping queries:

  • pipelineId - ID of the pipeline
  • sinkId - ID of the destination sink
  • datetime - Timestamp of the operation
  • date - Timestamp of the operation, truncated to the start of a day
  • datetimeHour - Timestamp of the operation, truncated to the start of an hour

User error metrics

Basin Pipelines tracks events that are dropped during processing due to deserialization errors. When a structured stream receives events that do not match its defined schema, those events are accepted during ingestion but dropped during processing. The pipelinesUserErrorsAdaptiveGroups dataset provides visibility into these dropped events, telling you which events were dropped and why. You can explore the full schema of this dataset using GraphQL introspection.

Metric GraphQL Field Name Description
Count count Number of events that failed validation

The pipelinesUserErrorsAdaptiveGroups dataset provides the following dimensions for filtering and grouping queries:

  • pipelineId - ID of the pipeline
  • errorFamily - Category of the error (for example, deserialization)
  • errorType - Specific error type within the family
  • date - Date of the error, truncated to start of day
  • datetime - Timestamp of the error
  • datetimeHour - Timestamp of the error, truncated to the start of an hour
  • datetimeMinute - Timestamp of the error, truncated to the start of a minute

Known error types

Error family Error type Description
deserialization missing_field A required field defined in the stream schema was not present in the event
deserialization type_mismatch A field value did not match the expected type in the schema (for example, string sent where number expected)
deserialization parse_failure The event could not be parsed as valid JSON, or a field value could not be parsed into the expected type
deserialization null_value A required field was present but had a null value

Note

To prevent incorrect data from being ingested in the first place, consider using typed pipeline bindings to catch schema violations at compile time.

View metrics and errors in the dashboard

Per-pipeline analytics are available in the Cloudflare dashboard. To view current and historical metrics for a pipeline:

  1. Log in to the Cloudflare dashboard ↗︎ and select your account.
  2. Go to Basin Pipelines > Pipelines.
  3. Select a pipeline.
  4. Go to the Metrics tab to view its metrics or Errors tab to view dropped events.

You can optionally select a time window to query. This defaults to the last 24 hours.

Query via the GraphQL API

You can programmatically query analytics for your pipelines via the GraphQL Analytics API. This API queries the same datasets as the Cloudflare dashboard and supports GraphQL introspection.

Basin Pipelines GraphQL datasets require an accountTag filter with your Cloudflare account ID.

Measure operator metrics over time period

This query returns the total bytes and records read by a pipeline from streams, along with any decode errors.

query PipelineOperatorMetrics(
    $accountTag: String!
    $pipelineId: String!
    $datetimeStart: Time!
    $datetimeEnd: Time!
) {
    viewer {
        accounts(filter: { accountTag: $accountTag }) {
            pipelinesOperatorAdaptiveGroups(
                limit: 10000
                filter: {
                    pipelineId: $pipelineId
                    streamId_neq: ""
                    datetime_geq: $datetimeStart
                    datetime_leq: $datetimeEnd
                }
            ) {
                sum {
                    bytesIn
                    recordsIn
                    decodeErrors
                }
            }
        }
    }
}

Measure sink delivery metrics

This query returns detailed metrics about data written to a specific sink, including file and compression statistics.

query PipelineSinkMetrics(
    $accountTag: String!
    $pipelineId: String!
    $sinkId: String!
    $datetimeStart: Time!
    $datetimeEnd: Time!
) {
    viewer {
        accounts(filter: { accountTag: $accountTag }) {
            pipelinesSinkAdaptiveGroups(
                limit: 10000
                filter: {
                    pipelineId: $pipelineId
                    sinkId: $sinkId
                    datetime_geq: $datetimeStart
                    datetime_leq: $datetimeEnd
                }
            ) {
                sum {
                    bytesWritten
                    recordsWritten
                    filesWritten
                    rowGroupsWritten
                    uncompressedBytesWritten
                }
            }
        }
    }
}

Query dropped event errors

This query returns a summary of events that were dropped due to schema validation failures, grouped by error type and ordered by frequency.

query GetPipelineUserErrors(
    $accountTag: String!
    $pipelineId: String!
    $datetimeStart: Time!
    $datetimeEnd: Time!
) {
    viewer {
        accounts(filter: { accountTag: $accountTag }) {
            pipelinesUserErrorsAdaptiveGroups(
                limit: 100
                filter: {
                    pipelineId: $pipelineId
                    datetime_geq: $datetimeStart
                    datetime_leq: $datetimeEnd
                }
                orderBy: [count_DESC]
            ) {
                count
                dimensions {
                    date
                    errorFamily
                    errorType
                }
            }
        }
    }
}

Example response:

{
    "data": {
        "viewer": {
            "accounts": [
                {
                    "pipelinesUserErrorsAdaptiveGroups": [
                        {
                            "count": 679,
                            "dimensions": {
                                "date": "2026-02-19",
                                "errorFamily": "deserialization",
                                "errorType": "missing_field"
                            }
                        },
                        {
                            "count": 392,
                            "dimensions": {
                                "date": "2026-02-19",
                                "errorFamily": "deserialization",
                                "errorType": "type_mismatch"
                            }
                        },
                        {
                            "count": 363,
                            "dimensions": {
                                "date": "2026-02-19",
                                "errorFamily": "deserialization",
                                "errorType": "parse_failure"
                            }
                        },
                        {
                            "count": 44,
                            "dimensions": {
                                "date": "2026-02-19",
                                "errorFamily": "deserialization",
                                "errorType": "null_value"
                            }
                        }
                    ]
                }
            ]
        }
    },
    "errors": null
}

You can filter by a specific error type by adding errorType to the filter:

pipelinesUserErrorsAdaptiveGroups(
    limit: 100
    filter: {
        pipelineId: $pipelineId
        datetime_geq: $datetimeStart
        datetime_leq: $datetimeEnd
        errorType: "type_mismatch"
    }
    orderBy: [count_DESC]
)

To query errors across all pipelines on an account, omit the pipelineId filter and include pipelineId in the dimensions:

pipelinesUserErrorsAdaptiveGroups(
    limit: 100
    filter: {
        datetime_geq: $datetimeStart
        datetime_leq: $datetimeEnd
    }
    orderBy: [count_DESC]
) {
    count
    dimensions {
        pipelineId
        errorFamily
        errorType
    }
}

Note

In addition to pipelinesUserErrorsAdaptiveGroups, you can also query the pipelinesUserErrorsAdaptive dataset, which provides detailed error descriptions within the last 24 hours. Be aware that querying this dataset may return a large volume of data if your pipeline processes many events.

Was this helpful?

YesNo

On this page

Metrics and analytics (2) - 图1Docs

{"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/basin-pipelines/observability/metrics/#page","headline":"Metrics and analytics","description":"Query Basin Pipelines metrics for data ingested, processed, and delivered via the dashboard or GraphQL API.","url":"https://developers.cloudflare.com/basin-pipelines/observability/metrics/","inLanguage":"en","image":"https://developers.cloudflare.com/basin-pipelines/observability/metrics/og.png?v=770b8767b4e352df","dateModified":"2026-10-01","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}}

description: Account-level limits for Basin Pipelines streams, sinks, and pipelines. title: Limits image: https://developers.cloudflare.com/basin-pipelines/platform/limits/og.png?v=19b3ab56c5aa78b7

Skip to content

Documentation Index
Fetch the complete documentation index at: https://developers.cloudflare.com/basin-pipelines/llms.txt
Use this file to discover all available pages before exploring further.