PWA Split Testing
A split test divides traffic between PWA variants and shows what actually converts, not what looks better by eye. Sticky per visitor keeps stats clean, and unique-to-install-to-deposit metrics highlight the winner. Here is how to set up a correct split and when to stop it.
What you can test
You test showcases (icon, name, screenshots, reviews), prelanders, offer pages and even whole funnels per geo. The point is to find the pair that yields more installs and deposits on specific traffic. Change one element at a time, or you cannot tell what moved the result.
Weights and platforms
Traffic splits by weights between variants (50/50, 70/30, etc.) and can be cut by platform: Android and iOS behave differently, so mixing them in one split is often harmful. Separate per-platform splits give clean conclusions and do not blend incomparable audiences.
Sticky per visitor
The same user must always see one variant, or they land in both and blur the stats. Sticky binding per visitor locks a variant to a user for the session and later visits. Without it the split shows noise, not a real conversion difference.
Metrics: unique to install to deposit
Look at the whole funnel, not just installs. A variant may bring more installs but fewer deposits — then it loses on money. Unique-to-install-to-deposit metrics per variant show where the funnel actually earns and where it just gathers cheap but empty installs.
When to stop the test
Do not stop a split on the first dozens of clicks — that is still noise. Let it gather enough data, especially deposits, which happen with a delay. Fix the winner on a stable funnel difference, not one lucky day, and only then scale.
FAQ
- What is sticky in a split test?
- Binding one variant to a specific visitor so they do not land in both and blur the stats. Without sticky the split shows noise.
- Should I split by platform?
- Often yes: Android and iOS convert differently. Separate per-platform splits give clean conclusions and avoid mixing incomparable audiences.
- Which metric picks the winner?
- The whole unique-to-install-to-deposit funnel, not just installs. A variant with more installs may lose on deposits.
- How much traffic to send to a test?
- Enough for a stable difference, especially on delayed deposits. Stopping early on dozens of clicks gives a false result.
- How many elements to change at once?
- One. Changing several at once makes it impossible to tell what moved conversion.
Cloaking, anti-bot, push, split tests and your own domains — in one service.
Get started free