Case study

Two outages where every local check passed

RecoverNode began because monitoring once reported that a site was down without saying why (that story is on why RecoverNode was built). This page covers two later incidents with the opposite problem: the checks said everything was fine, and people still could not reach the site.

Incident 1: healthy server, HTTP 200, users not reaching the site

This incident happened before RecoverNode v2.8, which was released on September 2, 2026. Every check running inside our own infrastructure passed: the application was healthy, the reverse proxy returned HTTP 200, and DNS resolved correctly from inside the network.

None of that proved that unrelated people on the Internet were reaching our website. A status code of 200 proves that something answered, not that your site answered. Correct DNS inside your own network proves nothing about what the rest of the Internet sees.

What we changed in v2.8

Domain Watch now checks the public path one layer at a time:

1. Registry delegation: are the domain's nameservers the ones you expect?
2. Authoritative nameservers: queried directly, do they agree with each other?
3. Independent public resolvers: do they return your expected address?
4. HTTPS content identity: does the page contain a stable marker that only your site carries? This is what catches a provider parking page or a different site behind an HTTP 200.

Because each layer is reported separately, an alert says which layer broke instead of only saying that the site is down.

Incident 2: up for the Internet, down for the local network

This incident happened before RecoverNode v2.9, which was released on September 15, 2026. The site was healthy for the public: the reverse proxy, the application and public DNS were all fine. Users on the local network still could not open it.

The network's router did not support hairpin NAT, so local clients had to receive the site's private address from the local DNS resolver. The internal reference resolver returned the correct private address, but the resolver actually handed to clients still held the public address in its cache. From inside a network without hairpin NAT, that address does not work.

A healthy internal reference resolver does not prove that clients receive its answer. You have to test the resolver the clients actually use.

What we changed in v2.9

• DNS Path Integrity monitoring queries both the internal reference resolver and the client-facing resolver, over UDP and TCP.
• Answers are compared with the expected internal and public addresses.
• The remaining TTL of each answer is recorded, so an alert can say how long a wrong cached answer may keep affecting clients.
• RecoverNode diagnoses this condition but does not flush gateway, Pi-hole, operating-system or browser caches on its own. That stays a human decision.

What both incidents have in common

In both cases the server was healthy, so restarting or rebooting it would have fixed nothing. That is why RecoverNode separates observation from recovery authority: it identifies which layer failed, restarts a service only when local evidence shows that service is the problem, and alerts instead of acting when the evidence points at DNS, the network or a provider.

Check your own setup

• Does your uptime monitor verify page content, or only the status code? A parking page returns 200 too.
• Do you know your domain's expected nameservers, and would you be alerted if they changed?
• If you use split DNS or Pi-hole, do you test the resolver your clients actually use?

For the server itself, the free Linux Resilience Audit checks the problems that quietly take sites down, and the DNS monitoring and troubleshooting guide covers the DNS side in depth.