Technote

Performance optimization Practical

Series Speed and conversion Part 3 of 8

When field and lab data disagree, which do you believe?

Lab data is one reproducible run under controlled conditions; field data is the distribution of what real customers experienced. When they disagree, they are answering different questions.

The tool says 90 and customers say it is slow. Or the reverse: the real-user report looks poor, yet the page opens instantly when you try it. The usual mistake here is treating one of them as broken. Both are behaving correctly; they were built to answer different questions.

The question each one answers

Lab explains why; field tells you who experienced what

Lab data is an experiment with the conditions pinned down. That is what makes before-and-after comparison possible and what lets it point at the items consuming time. The trade-off is that those conditions are one assumption you chose.

Field data is collected from the browsers of actual visitors — a distribution across varied devices, connections, locations and cache states. It answers “what did our customers actually experience”. The trade-off is that it never explains why, it takes time to accumulate, and with little traffic there is simply nothing there.

Common reasons they disagree

When you find a gap, first check whether you are measuring the same thing. Most discrepancies are explained right here.

What explains a discrepancy — work down the list

The first line dominates. You test the home page while customers arrive from an ad onto a landing page. The two screens carry completely different weight, so the two numbers differ.

The last line is lag, not breakage. Field metrics accumulate over a window, so a fix deployed today appears only as that window rolls forward. Do not declare a change a failure because field numbers have not moved a week later.

The working rule

In practice: let field data decide what to fix first — the screens customers genuinely experience as slow set the priority — and let lab data explain why it is slow. After the change, confirm immediately with an identical lab measurement, then check weeks later whether the field data follows.

Report both numbers together. Give only one and somebody will inevitably reply that another tool says something different, and the only person who can answer that is the one who wrote down the difference in advance. Further performance writing lives in the performance archive, and the way before-and-after measurement is pinned into a procedure is on our process page.

Next part

Pages that are slow now were rarely slow at the start. The next part follows the route by which a landing page gains weight, one campaign at a time.

More on this topic

All technotes

Performance optimization Advanced

A front-end budget you can actually hold

Pages do not get heavy in one go. They get heavy through small, individually reasonable additions. A ceiling you actually check is what stops the accumulation.

Developers 8 min read

Performance optimization Advanced

A conversion audit: finding where people leave

Polishing a page nobody reaches is the most common waste. Walk the whole path end to end and the leak shows itself first.

Marketers 7 min read

Performance optimization Practical

The cost of pop-ups — search and conversion at once

An interstitial that covers the content on mobile is treated unfavourably in search, and the same interstitial costs conversions too. The levers are timing, trigger and dismissal.

Marketers 8 min read

₩270,000 · Join the program