Creative Testing
Part of Offer and message alignment
Guide to testing service benefits against price incentives
“Better support” and “save now” can attract different customers. A useful test asks which truthful benefit produces a better customer outcome, not simply which version wins more clicks.
Use a randomised A/B split test to compare one service-benefit offer with one genuine price incentive. Randomly split the audience and change the offer while keeping the audience, channel, timing, format and destination as consistent as possible.
The service offer can state a specific setup or support commitment. The price offer should state a genuine saving and its full terms; make sure the business can deliver the service and substantiate both claims.
A price offer might use a percentage discount or a claimed saving. The Emma Sleep case involved strikethrough prices, discounts such as “50% OFF” and savings such as “Save as much as $3,531”; these were part of sale-price representations found to be misleading.
Choose one primary measure tied to the decision, such as qualified enquiries or completed purchases. Track guardrails such as refunds, complaints, support demand and eligibility questions; clicks alone do not show which offer produces a better customer outcome.
Compare the margin left by each offer after accounting for the cost of delivering the service benefit or providing the discount. A higher conversion rate may not be the better result if the offer reduces margin or attracts customers who expected something else.
Set the required sample size and test duration before launch. Run both versions over the same dates, avoid changing other campaign elements, and assess statistical significance after the planned test rather than stopping when one version moves ahead.
If the sample is small or seasonal demand could affect results, treat an apparent lead cautiously. If one version gets more clicks but also more eligibility questions, review the offer wording before scaling.
The ACCC says it can require businesses to back up claims about their products or services. Its price-displays guidance says it may investigate if price-display rules are broken, but does not provide legal advice.
On 24 April 2026, the Federal Court ordered Emma Sleep Pty Ltd and Emma Sleep Southeast Asia Inc to pay $15 million in penalties for false or misleading sale-price representations. Of 74 products shown with a higher strikethrough price, 58 had not previously been for sale at that price or without the discount; the remaining 16 had almost never been. The ACCC said the conduct breached the Australian Consumer Law.
Service Benefit vs. Price Incentive: Key Differences in Customer Impact
- Offer Type
- Service Benefit (e.g., free setup, extended support)
- Customer Outcome Measure
- Qualified enquiries or completed purchases
- Risk of Misleading Claims
- High if service delivery cannot be substantiated
- Risk of Misleading Claims
- High if strikethrough prices were never valid
- Margin Impact
- May reduce margin if service cost is high
- Margin Impact
- Direct reduction from discount or rebate
ACCC Enforcement Actions: Emma Sleep Case Highlights
Key Steps for Valid A/B Testing of Offers
- Randomly split audience using A/B testing
- Keep channel, timing, format and destination consistent
- Ensure both offers are truthful and substantiated
- Define one primary conversion metric (e.g., purchases)
- Track guardrailsrefunds, complaints, eligibility questions
- Calculate post-offer margin after costs
- Set sample size and test duration before launch
- Assess statistical significance only after planned test period
