Technote

Performance optimization Practical

Series Speed and conversion Part 3 of 8

When field and lab data disagree, which do you believe?

Lab data is one reproducible run under controlled conditions; field data is the distribution of what real customers experienced. When they disagree, they are answering different questions.

The tool says 90 and customers say it is slow. Or the reverse: the real-user report looks poor, yet the page opens instantly when you try it. The usual mistake here is treating one of them as broken. Both are behaving correctly; they were built to answer different questions.

The question each one answers

Lab explains why; field tells you who experienced what

Lab data is an experiment with the conditions pinned down. That is what makes before-and-after comparison possible and what lets it point at the items consuming time. The trade-off is that those conditions are one assumption you chose.

Field data is collected from the browsers of actual visitors — a distribution across varied devices, connections, locations and cache states. It answers “what did our customers actually experience”. The trade-off is that it never explains why, it takes time to accumulate, and with little traffic there is simply nothing there.

Common reasons they disagree

When you find a gap, first check whether you are measuring the same thing. Most discrepancies are explained right here.

What explains a discrepancy — work down the list

The first line dominates. You test the home page while customers arrive from an ad onto a landing page. The two screens carry completely different weight, so the two numbers differ.

The last line is lag, not breakage. Field metrics accumulate over a window, so a fix deployed today appears only as that window rolls forward. Do not declare a change a failure because field numbers have not moved a week later.

The working rule

In practice: let field data decide what to fix first — the screens customers genuinely experience as slow set the priority — and let lab data explain why it is slow. After the change, confirm immediately with an identical lab measurement, then check weeks later whether the field data follows.

Report both numbers together. Give only one and somebody will inevitably reply that another tool says something different, and the only person who can answer that is the one who wrote down the difference in advance. Further performance writing lives in the performance archive, and the way before-and-after measurement is pinned into a procedure is on our process page.

Next part

Pages that are slow now were rarely slow at the start. The next part follows the route by which a landing page gains weight, one campaign at a time.

More on this topic

All technotes

Performance optimization Practical

Who pays for the milliseconds a tag adds?

A tracking script does not stop at downloading. While it is parsed and executed the main thread is busy, and a busy page ignores fingers. That cost is…

Marketers 5 min read

Performance optimization Practical

The object cache: what Redis actually removes

Repeated identical lookups disappear. One slow query does not. Miss that distinction and you will wire up Redis and then ask why nothing got faster.

Developers 7 min read

Performance optimization Practical

This is how a landing page gets heavy

Nobody decides to make a landing page heavy. It accumulates: one addition per campaign, and nothing ever removed when the campaign ends.

Marketers 6 min read

₩270,000 · Join the program