A/B Testing in Journeys

Marketing

Test different messages or paths to find what works best.

What is A/B testing?

A/B testing (or split testing) sends different versions to different customers:

• Variant A gets one version
• Variant B gets another version
• Compare results to find the winner

Creating a split test

1. Add a Split step in the Journey Builder
2. Name your variants (A, B, etc.)
3. Set the percentage split
4. Build different paths for each variant

Split percentages

Split	Recommendation
50/50	Standard A/B test
33/33/33	Testing 3 variants
10/10/80	Test small before scaling

What to test

Subject lines

Same content, different subject lines.

• "Don't miss out!" vs "Exclusive offer inside"

Content

Different messaging approaches.

• Formal vs casual tone
• Short vs detailed

Timing

Different wait periods.

• 1 day wait vs 3 day wait

Channels

Different communication methods.

• Email vs SMS

Only test one thing at a time. Testing multiple changes makes it hard to know what worked.

Measuring results

Key metrics

Metric	What it measures
Open rate	Subject line effectiveness
Click rate	Content engagement
Conversion	Goal achievement
Completion	Journey finish rate

Statistical significance

The system calculates whether results are statistically significant:

• Significant - Winner is clear
• Not yet - Need more data
• No winner - Too close to call

Wait for statistical significance before declaring a winner. Early results can be misleading.

Deploying winners

Once you have a winner:

1. Go to journey analytics
2. View split test results
3. Click "Deploy Winner"
4. All future customers get the winning variant

Best practices

Sample size

• Need enough customers for valid results
• Minimum ~100 per variant recommended
• Larger tests = more confidence

Test duration

• Run tests for at least 1-2 weeks
• Account for day-of-week variations
• Don't end early based on initial results

Documentation

• Record what you tested and why
• Note the winning approach
• Apply learnings to future journeys

Example split test

Testing: Welcome email subject line

Variants:

• A: "Welcome to the club!"
• B: "Your membership journey starts now"

Results after 2 weeks:

• A: 42% open rate
• B: 58% open rate
• Statistical significance: Yes

Action: Deployed Variant B

Next steps

• Analytics - Deep dive into metrics
• Best Practices - Optimization tips
• Creating Campaigns - One-time sends