FIZLY Launch Measurement

How much does FIZLY's launch video return, and why does every tool give a different number?

Estimate with intervalTruth for the same questionBreak-even in shelf sales at 30% contribution

What a measurement partner did in this case

Agreed the questions first. A one-page plan with the advertiser and agency before launch: which study answers which decision, read dates, and rules nobody can change once results are in (chapter 0).
Sized every study from the advertiser's own history. Brand lift holdout by answers needed, conversion lift holdout by detectable lift, geo regions by detectable return and launch sales at risk (chapters 1-3).
Ran four instruments in one flight without collisions. Brand lift first; conversion lift and a store-level geo test together; an independent tracker alongside, with a brief to its vendor.
Reconciled the numbers instead of picking one. Each answer tied to the decision it should drive, and the gaps between them explained (chapter 4).
Handed over and fed back. A readout in English and Korean, a playbook for the next launch, a program view for other advertisers, and a memo to the measurement product team (chapters 5-6).

Role-play on synthetic data, written the way the work would be done.

How to read this page

Chapters follow a measurement partner's work: agree a plan (0), run the studies (1-3), reconcile them (4), report and hand over (5), feed the product team (6).
Each tab shows the design, the readout and the truth. "Truth" exists only because the world is simulated: each person's and region's outcome with and without ads is known.
Methods are written in platform-neutral terms. The README maps them to named lift tools.

Before launch: what will each study answer, when, and what does it need?

Rules agreed before launch

Beyond FIZLY: which study can each advertiser in the market read?

The same design math, run for other budgets and sales mixes. A program offers each tier the studies it can read and names the one that answers its payback question.

Service levels and what one partner can carry

How to read this tab

The timeline is the launch half-year: the heavy flight (weeks 79-90) and when each study runs and is read.
"Minimum" values come from each study's design step, computed on the advertiser's own history before launch.

Did the launch video build awareness among the people it reached, and which creative did it better?

Lift by metric

Randomized lift, 95% CISaw vs never reached (naive)Truth

Design: what each holdout can detect

Answers come from people reached, so precision is bought with the holdout share, not the budget.

Awareness lift by frequency

Holdout matched on would-have-been frequencyWhole holdout (naive)Truth

Creative A (product first) minus B (lifestyle)

Only recall and awareness were named for the creative test before launch. Differences on the others are noise until shown otherwise.

How to read this tab

Each respondent answers one question, so every metric has its own sample.
"Weighted" lift re-weights answers by each person's chance of seeing a poll, so it describes everyone reached rather than heavy users.

How many online orders did video add, and how does that compare with what the platform credits?

Orders in the test group, weeks 83-88

Design: holdout size

When signals get thin: the same study at lower pixel match rates

Privacy changes cut the share of online orders a platform can match to a user. Same people, holdout and ads; only the match rate changes (15 runs each).

How to read this tab

The study counts orders the brand's pixel can match to a user (about 75%) and scales the gap between groups to all orders.
"Credited" orders use the platform's 7-day click / 1-day view rule.

With store sales counted, what did video add? And is awareness growing in the whole market?

Go-dark regions: actual vs predicted from control regions (stores + online, KRW M per week)

ActualPredicted without the test

Readouts against the truth

Design and checks

Vendor tracker: aided awareness of FIZLY Zero, adults 18-44

Tracker wave, 95% CITruth, whole populationTruth without any ads

How to read this tab

Video was off in 12 of 40 regions (29% of sales) for six weeks. Store sales come from a weekly sell-out panel projected to each region.
The tracker has no holdout: it tells the level of awareness, not what moved it.

FIZLY Zero: measuring a store-led drink launch

What a measurement partner did in this case

Rules agreed before launch

Service levels and what one partner can carry

Lift by metric

Design: what each holdout can detect

Awareness lift by frequency

Creative A (product first) minus B (lifestyle)

Orders in the test group, weeks 83-88

Design: holdout size

When signals get thin: the same study at lower pixel match rates

Go-dark regions: actual vs predicted from control regions (stores + online, KRW M per week)

Readouts against the truth

Design and checks

Vendor tracker: aided awareness of FIZLY Zero, adults 18-44

Awareness: platform brand lift vs vendor tracker

Brief for the research vendor's next waves

Each channel: platform report vs calibrated model

Decision table

Same pipeline, two worlds: does the readout follow the data?

Repeated runs: bias and interval coverage

What the checks caught (work log)