Most reported conversion wins are noise. A test runs for a week, shows an eleven per cent lift on two hundred visitors, gets declared a winner and is never seen again in the revenue figures. Without deciding the sample size in advance, stopping when the result looks good is guaranteed to produce a stream of false positives. The honest version of this discipline involves research first, fewer and larger tests, and telling clients when their traffic cannot support testing at all.
What the work involves
- Research first — session recordings, funnel analysis, support tickets and actually talking to customers
- Hypotheses that state the mechanism, not just the change
- Sample size and duration calculated before the test starts, and honoured
- Sequential or Bayesian methods where they suit the traffic better than fixed-horizon testing
- Losing and flat results documented, since they are most of what you learn
- Where traffic is too low for testing, qualitative research and best-practice fixes instead
Why we are credible at this
We wrote the attribution pipeline our own tests are measured with, including the reconciliation pass that exists because the live path is not trusted to be complete. If the measurement is wrong, every test result on top of it is decoration.
What moves the price
We quote after discovery and the number is fixed before anything starts. These are the things that move it, so you can see roughly where your project sits before you talk to us.
- Traffic volume, which determines whether testing is viable at all
- Whether analytics are trustworthy today, which frequently must be fixed first
- How much engineering each test requires to implement
- Whether research is included or you already have it
Questions we get asked
Other Web Marketing work
Tell us what you are trying to do
A short conversation is usually enough to tell whether we are the right people. If we are not, we will say so and point you somewhere better.