A/B Testing Method

A disciplined way to learn what improves conversion: one hypothesis, one primary metric, a sample size set in advance and a decision rule written before launch.

ID: MK-T0902
Tactic: Capture & Conversion
Sub-techniques: 3
Type: How-to
Level: Intermediate
Time: 2–6 weeks per test

Sub-techniques

ID	Name
MK-T0902.001	Hypothesis Writing
MK-T0902.002	Sample Size Planning
MK-T0902.003	Decision Rules

Procedure

Write the hypothesis.
Choose one primary metric and guardrail metrics such as lead quality.
Estimate the required sample size with a calculator.
Run for full weeks and avoid stopping early on a promising peek.
Decide by the pre-written rule and log the result in the test library.

Common mistakes

Mistake	Mitigation
Testing low-traffic pages	Test bigger changes or use qualitative research
Peeking and stopping	Fix the duration in advance

Sub-technique details

MK-T0902.001

Hypothesis Writing

Because [evidence], changing [element] to [variant] will improve [metric] for [audience].

MK-T0902.002

Sample Size Planning

Calculating required traffic from the baseline rate and minimum detectable effect.

MK-T0902.003

Decision Rules

Pre-written criteria for shipping, rejecting or extending a test.

Was this page helpful?