Knowledge · Conversion Engineering
What Is Conversion Engineering (and Why It Beats CRO Theater)
Most “CRO” you’ve been sold is theater. Color-changing buttons called “tests” on pages with 47 visitors. Heatmap PDFs that confirm what the agency wanted to recommend anyway. Quarterly retainers that produce 3 slides and a 1.2% lift that wasn’t statistically significant. Conversion engineering is the opposite: instrumentation first, math second, opinions last.
№ 01The CRO theater pattern (and why agencies sell it)
The standard CRO retainer pitch: $4K-$8K/month, monthly heatmap report, “A/B test” cycles, executive summary deck. What’s missing from that pitch is math. No mention of how many sessions are needed for 95% confidence on a 10% lift detection. No mention of test duration. No mention of guardrail metrics. Because if the agency mentioned it, you’d realize most months they don’t have the traffic to test anything.
The retainer exists because agencies need recurring revenue, not because your site needs monthly tests. A mid-market B2B site doing 8K sessions/month physically cannot complete a properly-powered A/B test in 30 days. The “wins” reported are noise, picked at convenient stopping points. This is the dirty secret of the CRO industry.
№ 02What conversion engineering actually is
Conversion engineering is the discipline of treating a website like an instrumented system. The work has four parts:
- Instrumentation. GA4 events on every meaningful interaction, GTM-managed, with a dataLayer specification. Microconversion paths defined before macroconversions.
- Measurement. Statistical literacy. Sample size calculators. Sequential testing when traffic is thin. Pre/post comparison with proper baselining when testing isn’t feasible.
- Hypothesis design. Specific, falsifiable hypotheses tied to friction points the data already surfaced. Not “let’s try a new button color.”
- Implementation. Fast. One sprint, not a quarter. Ship the change, instrument the result, move on.
The deliverable isn’t a deck. It’s a shipped change with a tracked lift, or a documented null result that closes the hypothesis.
№ 03The math beats opinions thesis, applied
Every conversion decision is one of three things: (1) backed by data that meets a significance threshold, (2) backed by data that doesn’t meet a significance threshold but is directional, or (3) an opinion dressed up as analysis. Conversion engineers say which one out loud.
A real example: a Tampa B2B SaaS site we audited had a 1.8% homepage conversion rate. The CRO agency’s recommendation was “test a new headline.” Our recommendation was “your form has 11 fields and the dropoff is at field 4 — cut to 5 fields, then look at the headline.” The form change shipped in 4 hours and lifted conversion to 3.1%. The headline never got tested. The data already told us where to spend the engineering hour.
№ 04When you don’t have the traffic for A/B testing
Most mid-market sites don’t. The math: to detect a 20% relative lift on a 3% baseline conversion rate with 95% confidence and 80% power, you need roughly 4,800 sessions per variant. For a 10% lift, 19,000 per variant. Below 30K sessions/month total, classical A/B testing on a homepage will take 3-6 months per test — long enough that seasonality contaminates the result.
The alternatives that actually work at this scale: pre/post comparison with multi-week baselines, sequential testing (Bayesian methods that allow early stopping), holdout cohorts on email campaigns, and directional analysis on microconversions (which need 10x less traffic to move). We’ll tell you when each applies.
№ 05What conversion engineering does NOT mean
It doesn’t mean monthly retainers. We sell a $500 Conversion Audit (5-day delivery, refundable against build) and a $3,500 CRO Sprint (14 days, implements the top 5 audit recommendations). After that, instrumentation runs itself and the next sprint happens when the data says it should — not when the calendar says it should.
It doesn’t mean “test everything.” It means test the things where math will pay back the engineering hours. For a 5K-session/month site, that’s usually 1-2 changes per quarter, not 8. The discipline is knowing when not to test.
⚠What to avoid
- Calling a 2-week color-test on 200 visitors a “winning A/B test.” The math says you can’t detect anything under a 60% lift at that volume. The “win” is noise.
- Reporting conversion lift without reporting confidence intervals. A 14% lift with a 95% CI of [-6%, +34%] is a coin flip. Reporting just the point estimate is dishonest math.
- Stopping a test early because it’s “winning.” Frequentist tests have predefined sample sizes. Peeking and stopping inflates false positives 3-5x. If you want to stop early, use sequential/Bayesian methods on purpose — not by accident.
Related questions
Go deeper
Three Ways to Start · No Sales Pitch
Want this analyzed on your site?
$500 audit. 5-day delivery. Refundable on engagement.