WriterzRoom does not split traffic. Exposures and conversions are entered or imported from the channel that served them; what the Experiment Lab owns is the decision, not the delivery.
The verdict
Each experiment needs exactly one control and at least one challenger. The evaluation runs a pooled two-proportion z-test and returns one of four verdicts:inconclusive and insufficient_data are kept separate on purpose: the first means keep collecting, the second means the sample cannot support the question yet.
Three guards
- Normal-approximation validity. Rather than an arbitrary impressions floor, the test requires at least 5 expected conversions and 5 expected non-conversions per arm under the pooled rate. Below that no p-value is reported, because one would not mean what it claims to.
- Multiple comparisons. Testing several challengers against one control inflates the false-positive rate, so the threshold is Bonferroni-corrected by the number of challengers. Adding a fifth headline makes winning harder, not easier.
- Direction. The p-value is two-sided, so a variant that performs significantly worse is never reported as a winner.
Promotion
A concluded experiment freezes its verdict, whatever it is. An inconclusive result is worth keeping: it is the record that stops the same hypothesis being re-run and read hopefully next quarter. Only an experiment that established a winner can be promoted into a pattern. Promotion is refused outright otherwise, with the verdict and its reason — a warning would be something to click through, and the result would be a coin flip becoming a house style rule nobody can trace. A promoted pattern carries its evidence forward — sample size and p-value — and can be scoped to verticals, channels, or audiences. Retiring a pattern deactivates it rather than deleting it, so work already written under it stays traceable.Audiences
Scope a promoted pattern to the audience it was proven on.
Campaigns
Group an experiment’s pieces under the initiative that ran it.