Creative Testing
Part of Creative testing programmes
Recording what a creative test cannot establish
Document a creative test's audience, delivery, measurement and uncertainty so its result is used within its limits.
A creative test record should state its limits as plainly as its result. Write down what changed, who could see the versions, what was measured, and what other explanations remain possible. This keeps a useful local finding from becoming an unsupported rule for every audience or channel.
Separate observation from interpretation
Use three parts in the result note:
- Observation:What did each version achieve on the measure chosen before launch, during the stated period?
- Interpretation:What does the comparison suggest about the changed creative?
- Limit:What does the design or data leave unresolved?
Consider a fictional service campaign in which a demonstration receives more qualified enquiries than a static explanation. If the video ran in different inventory, the result concerns the two delivered packages. It cannot isolate the demonstration from format or placement.
If lead quality was assessed only at submission, the test cannot establish which version produced more eventual sales.
Log the boundaries
Record these alongside the outcome:
- Creative:Which assets and combinations actually served?
- Population:Which audience and geography were eligible?
- Delivery:Were spend, impressions, placements or approval status uneven?
- Outcome:Was the measure a click, attributed conversion, qualified lead or completed sale? What was the conversion window?
- Timing:Were later conversions still expected when the report was read?
- Interference:Did the offer, site, tracking or other campaigns change during the run?
- Precision:Could the test resolve a difference large enough to matter for the decision?
When a platform report provides asset-level or experiment-level results, record the report, metric definitions and selected decision threshold. Avoid combining figures unless their definitions support it. If a result is still collecting data or remains inconclusive, record that status; an early directional indication is not conclusive.
Write the next defensible action
A limit is useful when it guides the next step. If format and message changed together, repeat with one held steady. If recent conversions are incomplete, schedule a later read using the same definition. If the test cannot resolve a commercially important difference, say it was inconclusive and decide whether a larger run is justified.
Do not call two versions equivalent merely because the test found no clear difference. This run may only have been unable to resolve one. Keep the record with the assets and decision so a future team can use the finding within its actual bounds.
