Search Experiment Design & Analysis

Test a structural change before it reaches the whole campaign.

A campaign draft mirrors the live campaign, allowing a structural or strategic change to be prepared without altering current delivery. The experiment then splits traffic and budget between the draft and original for a defined period. We isolate one change, agree the split and decision rule before launch, and interpret the result only after the full window closes. For a real structural or strategic hypothesis with a named decision owner and enough campaign traffic to split without starving either arm.

A record of which structural changes were tested, what the traffic split actually showed, and why each one got applied, extended, or dropped.

A strategist splitting traffic between an original campaign and a draft variant on a monitored experiment dashboard

Some of the 500+ brands we've worked with

See all references
  • Hyundai
  • Decathlon
  • Enerjisa
  • Axa Sigorta
  • Otsimo
  • Doremusic
  • Odamax

How the work runs

The draft keeps the live campaign intact

AI can monitor the experiment status and draft the readout. The record shows what happened at each step, while a person decides whether to launch, extend, or apply a change to the live campaign.

How we hold ourselves to it

  • Test the change as a draft before it ever touches the live campaign
  • Set the split, duration, and decision rule before launch
  • Hold the read until the window closes
  • One experiment per campaign, so the result stays attributable

  1. Define the intervention as a draft

    Mirror the live campaign into a draft and make only the one structural or strategic change under test, so nothing else moves at the same time.

    Draft campaign mirroring the original with exactly one structural or strategic change.

    AI assist
    Drafts the intervention definition from the stated hypothesis.
    Human gate
    Paid-search lead confirms the draft changes only the intended variable.
    Owners
    Paid-search lead + strategy lead

  2. Set the split, duration, and decision rule

    Agree the traffic and budget split, the run length, the guardrail metrics, and exactly what result triggers what action, before the experiment starts.

    Experiment brief fixing the split, duration, guardrail metrics and the result-to-action rule.

    AI assist
    Drafts the experiment brief template with the proposed split and duration.
    Human gate
    Client owner approves the decision rule before launch.
    Owners
    Client owner + paid-search lead

  3. Convert the draft to an experiment and launch

    Start the split run so the draft competes against the original campaign for the agreed window.

    Live split run competing against the original campaign for the agreed window.

    AI assist
    Confirms the experiment launched with the approved split and duration.
    Human gate
    Paid-search lead approves the go-live.
    Owners
    Paid-search lead + media specialist

  4. Monitor status and isolate confounds

    Watch the experiment's status through the run and flag any external change, such as a seasonal shift, policy change, or landing-page edit, that could contaminate the read.

    Status log with every flagged external change that could contaminate the read.

    AI assist
    Tracks experiment status and flags any external change during the run.
    Human gate
    Strategy lead reviews flagged confounds weekly.
    Owners
    Strategy lead + paid-search lead

  5. Read the result against the decision rule

    Wait for the full planned duration, then compare the primary metric and guardrails against the rule agreed before launch.

    Readout comparing the primary metric and guardrails against the pre-agreed rule at full duration.

    AI assist
    Drafts the readout once the planned duration completes.
    Human gate
    Paid-search lead signs the readout before any action is taken.
    Owners
    Paid-search lead + strategy lead

  6. Apply, extend, or discard, and record why

    Act on the pre-set decision rule, whether that means applying the draft to the original campaign, extending the run, or discarding it, and log the reasoning either way.

    Decision record naming the action taken and the reasoning, including any deviation from the rule.

    AI assist
    Drafts the decision record from the signed readout.
    Human gate
    Client owner approves applying any change to the live campaign.
    Owners
    Client owner + paid-search lead

What lands with your team

What got tested, and why it ended the way it did

You get the brief, the log, and the readout for every experiment, so a decision can be reconstructed later.

  • Experiment brief

    The intervention, the split, the duration, and the decision rule, all agreed before the experiment ever launches.

    Accepted when

    The intervention, split, duration and decision rule are agreed and recorded before the experiment launches.

    Cadence: Before every experiment

  • Draft and experiment status log

    Status changes through the run, plus any flagged confound that could affect how the result gets read.

    Accepted when

    Every status change and flagged confound during the run is recorded, so the read is not defended from memory.

    Cadence: Each experiment's duration

  • Guardrail-metric note

    The metrics that could stop or flag an experiment early, and the threshold each one is watched against.

    Accepted when

    Each guardrail names the threshold it is watched against and what happens when that threshold is crossed.

    Cadence: At brief, through run

  • Readout and decision record

    What the split traffic showed, what got decided, and the reasoning behind applying, extending, or discarding the draft.

    Accepted when

    The readout cites the pre-set rule, and the apply, extend or discard call states its reasoning either way.

    Cadence: Close of every experiment

Before the work starts

When this work is the right next step

Changing a live campaign without a test removes the fallback. Google's own experiment mechanism runs a draft alongside the original campaign for a set duration, splitting traffic and budget so you can compare before committing. Skip that step and a structural change lands with no fallback if it underperforms. Even teams that do run an experiment often read it days early, or start a second one on the same campaign when Google only allows one experiment to run there at a time.

A good fit when

  • Your team has a structural or strategic campaign change in mind, but the draft has not tested that single intervention against the original campaign.
  • The experiment can produce a readout, yet nobody has agreed who approves the decision rule or makes the call to apply, extend, or discard the draft.
  • The campaign has enough traffic for the original and draft to run side by side, so neither arm is starved before the planned window closes.
  • A structural change gets applied directly to the live campaign, with no draft to fall back on if it underperforms.
  • An experiment gets read and acted on days before its planned duration ends.
  • A second structural question gets tested on the same campaign while the first experiment is still running.
  • Nobody checked which reports go dark for the length of the draft, so a mid-test change to the account gets missed.

Better handled as other work when

  • The change is low-risk enough to apply directly, so running a draft through the full experiment window would delay a decision without answering a real question.
  • Your change belongs to Video, App, or Shopping, which Google's draft-and-experiment mechanism does not cover and this method scopes separately.
  • The team plans to read the test early, so the pre-set decision rule cannot govern the final call after the full window closes.

People who own a single channel

Paid Search, Paid Social, CRO, and Programmatic each run under a named owner at Zeo. The consultants below are matched to the channel this page is about, so you can see who you'd actually work with.

Tools we use

Tools behind this work

  • Optmyzr

    watches a running experiment so it gets judged at its planned end date, not before

Test the change before you commit to it

Design a search experiment with a decision rule set before the result exists

Bring us the structural or strategic change you're weighing. We'll build it into a draft-and-split experiment with the guardrails and decision rule agreed up front.
Design a search experiment

The questions we hear most

A draft mirrors your campaign so you can prepare changes without affecting live performance. An experiment is what you get when you convert that draft to run against the original with a real traffic and budget split for a set duration.