Technote

Servers & infrastructure Practical

Series When traffic starts to grow Part 4 of 8

Knowing your site is down before your customers do

If a customer tells you the site is down, you are already late. Deciding what to watch and how to be alerted takes about half an hour.

The worst moment in operations is not the outage itself. It is learning about it from a customer. The message “your site won’t open” means that for however many hours came before it, everyone else saw the same thing and simply left.

Technically this is not a hard problem: have something open your site regularly from outside and tell you when it cannot. What matters is what you have it open and where the alert lands.

Watch from outside, always

Monitoring installed on the server only works while the server does. When it stops, the thing that would have alerted you stops with it. So the watcher has to live outside the site — an external service opening your pages the way a visitor would.

For the same reason, alerts must travel by a path unrelated to the site. If notifications are sent by the site’s own mail, and mail is bound to the same server, the outage takes the alert with it.

What to watch

Watching only the home page is not enough. If the home page is sitting in a cache, WordPress can be dead while the home page still looks perfect. Splitting the checks by kind removes that illusion.

Four things to watch — the bottom two are calendar problems, not failures

For the dynamic page, pick any address a cache will not serve — a search results page, or the login screen. If it loads, WordPress and the database are both alive.

Certificate and domain expiry are the most avoidable outages there are. Nothing broke; a renewal was missed. Knowing one day earlier would have made it a non-event, and a calendar entry is most of the fix.

Keep alerts worth believing

Monitoring introduces a new problem: false alarms. A momentary network hiccup fires an alert, that happens a few times, and you start ignoring alerts — at which point you are back to having none.

The point of alerting is not volume — it is that every alert turns out to be real

Do not skip recovery notifications. With outage alerts and nothing else, you keep checking a site that came back twenty minutes ago.

Decide what happens after the alert

Write down what to look at when one arrives: whether other sites load too (is it your connection?), whether your host has posted an incident, and what changed recently. Three lines are enough to make the panicked moment much faster.

Articles on server operations and incident response live in the servers & infrastructure archive, and if you would rather have the watching handled for you, that is what InfraGuard continuous monitoring does.

Next part

Next we turn to something that grows quietly. It never announces itself like an outage, but it gets heavier every month — what happens as the database grows.

More on this topic

All technotes

Servers & infrastructure Practical

Sending email yourself, or handing it over

Run your own mail server and the invoice is near zero — you pay in deliverability instead. Which is cheaper depends on what one enquiry is worth to…

Founders 7 min read

Servers & infrastructure Practical

Displaying business details: what belongs on the site

If you sell online in Korea, displaying business registration details is a legal requirement — and it is also the first thing a visitor about to pay looks…

Founders 7 min read

₩270,000 · Join the program