第 17 组（U – U） - Understanding statistical significance for sign-up forms - 《Klaviyo 官方帮助中心完全手册：邮件与短信营销自动化》

Understanding statistical significance for sign-up forms

Estimated 3 minute read

Updated Jul 7, 2025, 12:00 AM EST

Table of contents

You will learn
How Klaviyo determines statistical significance for forms
Monitoring and ending an A/B test
Additional resources

You will learn

Learn when an A/B test’s results are considered to be statistically significant for Klaviyo sign-up forms. This guide will explain statistical significance for sign-up forms and when it’s decided.

Klaviyo considers an A/B test statistically significant when there is strong statistical evidence that 1 variation has a higher form submit rate than the others. You can configure an A/B test to automatically choose the winning form variation and end once the test results are considered to be statistically significant.

A/B tests allow you to measure how different elements of your forms perform to ensure they’re resonating well with your site visitors and driving strong engagement. It’s important to understand whether or not a test can be considered statistically significant so you can effectively analyze results and determine what improvements to make. If the test results are not statistically significant, then it’s more difficult to confirm that a particular variation will drive improved performance at scale.

How Klaviyo determines statistical significance for forms

Klaviyo considers win probability and form views along with additional checks when determining whether a sign-up form’s A/B test is statistically significant:

A win probability above 90% for the leading variation
More than 1,000 form views per variation

These thresholds alone are not enough. Klaviyo also requires at least 10 form submissions across all variations and checks that the estimated range of form submit rates for the leading variation has little overlap with those of the other variations.

Win probability of the leading variation
This is the probability that the form submit rate for the leading variation is truly higher than the form submit rate for all other variations. High win probability means there is strong evidence that the leading variation will win due to the form being better, rather than due to random chance.
Form views
A form needs to collect an ample number of views to reduce the effect of early-test anomalies. New A/B tests count unique visitors, so a visitor who views the same variation more than once counts as 1 view. For tests that compare display timing, Klaviyo counts visitors eligible to see each variation, including those who leave before the form appears.

Monitoring and ending an A/B test

On the A/B test settings page, under Winner selection, select Automatically end test when statistical significance is achieved, Set an end date, or both. If you select both, the test ends when either condition is met.

If the end date arrives before statistical significance is reached, Klaviyo chooses the variation with the highest form submit rate at that time. With either automatic ending option, Klaviyo sets the winning variation live. An automatically selected winner is not necessarily statistically significant.

Alternatively, clear both boxes under Winner selection to keep the test running until you end it manually. You decide when to end the test and which variation to use.

To monitor an in-progress test:

Go to Sign-up forms, open the form’s 3-dot menu, and select Analytics.
Select A/B test results, then choose the test from the dropdown.
In the Overview section, review Win probability and the data for each variation. Check the label beneath the probability: Statistically significant confirms that the test meets the significance checks. A high win probability alone does not confirm this.

To end a running test manually:

In the test’s Overview section, click Choose Winner.
Select the winning variation, then choose End test and set selected variation live or End test and set selected variation to draft. Choosing draft removes the form from your site.
Click Choose Winner to end the test and apply your selection.

For more details, head to our guide on reviewing A/B test results.

Additional resources

How to A/B test a sign-up form

Learn how to use A/B testing to evaluate the performance of various elements of your sign-up forms. By effectively analyzing the results, you can ensure that your forms resonate with site visitors and contribute to list growth. - Best practices for A/B testing reference

Learn best practices for using Klaviyo's A/B testing features to help optimize your communication with customers. - Understanding sign-up form analytics

Learn how to analyze your sign-up forms' performance so you can iterate and optimize based on knowledge of areas you excel in and areas where you can improve. This guide will help you gain a better understanding of your sign-up form analytics and each of the available reporting pages (i.e., Overview tab and A/B test results tab).

Was this article helpful?

YesNo

Use this form only for article feedback. Learn how to contact support.