Technical guide

Linux server health monitoring: what to check and why

A Linux server is not simply healthy or unhealthy. DNS, networking, TLS, Nginx, applications, systemd, resource pressure, the operating system, and hardware can fail independently. Useful monitoring keeps those signals separate so the response matches the actual problem.

Start with layers instead of one uptime check

A public website check answers an important question: can the service be reached from the observer? It does not explain why the service is reachable or unreachable.

A complete health model evaluates multiple layers independently and then correlates the results.

Monitor DNS separately

A healthy server can appear offline if its hostname resolves to the wrong address or the selected resolver is unreachable.

DNS monitoring can evaluate expected addresses, direct resolver responses, and where appropriate differences between UDP and TCP results.

Monitor TLS separately from HTTP

An HTTPS endpoint may fail because of an expired certificate, hostname mismatch, incomplete chain, or TLS negotiation problem even when Nginx and the application are running normally.

Certificate expiration, validation, and connection behavior should therefore be recorded as their own health signals.

Monitor HTTP behavior

HTTP monitoring can confirm status codes, redirects, response time, and expected content.

A successful TCP connection alone does not prove the application is returning useful responses.

Monitor listening ports

TCP checks help confirm whether services are reachable on expected ports. They are especially useful for application services that sit behind a reverse proxy.

A port check should complement, rather than replace, a functional application-health test.

Monitor Nginx independently from applications

Nginx can be healthy while the upstream application is unavailable. The application can also be healthy while Nginx has an invalid configuration or stopped process.

Separate monitoring prevents every 502 or 504 response from being misclassified as the same failure.

Monitor systemd state

systemd exposes useful process and service state including active, failed, stopped, and restarting conditions.

That state should be combined with functional checks because a process can remain active while the service it provides is unhealthy.

Monitor resource pressure

CPU saturation, memory exhaustion, swap pressure, storage depletion, inode exhaustion, and excessive load can degrade a server without immediately stopping its services.

Resource monitoring is useful evidence, but thresholds should be interpreted in context rather than treated as automatic reboot triggers.

Monitor storage availability

Applications can fail when a filesystem fills even though the host and service processes remain running.

Monitoring should consider both available space and conditions such as inode exhaustion where relevant.

Monitor the network path separately from the host

Routing, gateways, switches, NAT, firewalls, ISP connectivity, and the observer's own network can fail independently of the server.

Remote loss of visibility should therefore be evidence of a problem, not automatic proof that the Linux host requires a reboot.

Use local checks to authorize local recovery

The protected host can inspect its own services, configuration, listening ports, and local requests without relying entirely on the remote observer.

This allows recovery decisions to remain close to the component that can actually prove it is unhealthy.

Monitor watchdog health separately

A hardware watchdog addresses a different class of failure: a complete operating-system or userspace freeze where ordinary service monitoring can no longer function.

It should remain distinct from application restarts and network monitoring.

Include hardware and power evidence where supported

On supported platforms, hardware-specific health information can add important context.

For example, Raspberry Pi systems can expose evidence of current or historical undervoltage and throttling. Power instability should be reported rather than blindly triggering additional reboot activity.

Preserve evidence during incidents

A useful incident record can include:

• DNS answers and selected resolver
• TLS validation and certificate status
• HTTP status and latency
• TCP reachability
• Nginx and application service state
• systemd results
• CPU, memory, storage, and load evidence
• network and route information
• watchdog state
• hardware or power evidence when supported
• recovery actions already attempted

Evidence captured while the failure is occurring is more valuable than a clean snapshot collected after the incident disappears.

Correlate failures without hiding individual symptoms

If several sites fail through the same resolver or network path, the monitoring system should be able to identify the shared infrastructure problem.

Correlation should reduce duplicate alert noise while preserving the symptoms needed for diagnosis.

Use recovery budgets and cooldowns

Monitoring becomes dangerous when every failed check authorizes a restart or reboot.

Consecutive-failure thresholds, cooldowns, exponential backoff, persistent counters, action budgets, and lockout place limits around automated recovery.

Know when not to act

Ambiguous evidence should not produce increasingly disruptive actions. If local services are healthy while remote checks disagree, the safest response may be to preserve the running state and alert a human.

Refusing an unjustified reboot is part of successful monitoring.

Related Linux monitoring guides

For the broader recovery model, read the Linux server automatic recovery guide.

For reverse-proxy monitoring, read the Nginx monitoring and automatic recovery guide.

For intermittent name-resolution problems, read the DNS monitoring and split-DNS troubleshooting guide.

For process and application recovery, read the systemd service monitoring and automatic recovery guide.

For HTTPS certificate health, read the SSL certificate monitoring and TLS expiration alerting guide.

RecoverNode also documents its testing and safety methodology, including controlled failures, correlation, recovery budgets, watchdog testing, and operational verification.