Before you start
You need an existing Active Cancel Flow. There's no minimum volume requirement to start a test — the Start A/B test action is available on any eligible Active flow.
A/B testing is managed from a separate A/B Testing tab inside Cancel Flow, alongside your regular Flows tab.
Once a test is live, both the original and the variant flow are locked from editing until the test is paused or completed.
What is Cancel Flow A/B testing?
Cancel Flow A/B testing splits your cancellation traffic between two versions of a flow: your Original flow and a Variant you create by duplicating and editing it. Skio tracks save rate and other metrics for each version separately, so you can see which one performs better before rolling out a change to everyone.
Key terms
Original (A): Your existing, live cancel flow. This is the control in the test.
Variant (B): A duplicate of your original flow that you edit with the change you want to test.
Traffic split: The percentage of customers routed to each version. You can adjust this from 1% to 99% per test; it defaults to 50/50, and Skio enforces the split you set when assigning customers to a variant.
Winning metric: The metric Skio uses to declare a winner, such as churn reduction (save rate).
Statistical significance goal: The confidence level (for example, 95% confidence) Skio uses before recommending a winner.
Lift: The percentage difference in performance between the Variant and the Original.
Test statuses
A test moves through several statuses over its lifecycle. Here's what each one means:
Draft: You're still setting up the test. Nothing is live to customers yet.
In Progress: The test is live, and customers are being split between your Original and Variant.
Paused: The test is temporarily stopped. No new customers are assigned to a variant, but nothing you've already collected is lost.
Results ready (shown as Notified if you have email notifications turned on): Skio has reached a statistical result and stopped assigning traffic. It's time to review the results and decide what to do next — promote a variant, end without promoting, or extend the test.
Inconclusive (or Goal not met, for retention optimization tests): The test ran its full course without a clear result. This is a final status — no variant is picked as a winner.
Goal met: For retention optimization tests only. Your Variant's save rate held up within the threshold you allowed. Neither flow is marked a winner for this goal type.
Winner promoted (or Treatment promoted, for retention optimization tests): You chose to make the better-performing variant your new live flow.
No promotion: A variant performed significantly better, but you chose to keep your Original flow running instead.
Cancelled: You stopped the test before it reached a result.
Reaching Results ready happens automatically once Skio has enough data for a statistical result. Everything after that (such as promoting a winner, ending without promoting, or letting a test close as inconclusive) is a decision you make. Skio never promotes a variant on its own.
Key benefits
Data-driven decisions: Validate changes to your cancel flow with real subscriber behavior instead of intuition.
Lower risk: Test changes against a live control group so a change that hurts save rate doesn't roll out to every customer trying to cancel.
Revenue visibility: Each test tracks post-save revenue alongside save rate, so you can connect retention wins directly to recovered revenue.
Segment-level testing: Because tests run on top of your existing flow entry conditions, you can test a treatment change for one customer segment without affecting others.
Common use cases
Testing whether a different discount amount or offer order improves save rate for a specific segment.
Comparing a flow with an added pause option against your current flow to see if it reduces cancellations.
Validating a "best practice" change from Skio's guidance against your own store's actual subscriber behavior before rolling it out.