Monitoring and Troubleshooting

Introduction

When a server feels slow or a deploy fails, you need a repeatable way to see CPU, memory, disk, logs, and ports. This chapter covers live resource checks, log locations, and a troubleshooting map for errors you already met in earlier chapters—so production issues become boring checklists.

Prerequisites

Quick Health Snapshot

bash
# Uptime and load averages (1, 5, 15 minutes)
uptime
 
# Memory summary
free -h
 
# Disk usage
df -h
 
# Who is using CPU/memory right now
top -bn1 | head -20

Install htop for interactive view:

bash
sudo apt install -y htop
htop

Load average on a 4-core box: sustained above 4.0 suggests CPU saturation—investigate with top / htop.

CPU and Memory Deeper Look

bash
# One-line CPU stats every 2 seconds, 5 times
vmstat 2 5
 
# Per-process memory sort
ps aux --sort=-%mem | head
 
# Java heap is inside JVM — use app metrics or jcmd when needed
ps aux | grep java

iostat (package sysstat) helps disk bottlenecks—high %util on a disk with slow response points to storage limits.

systemd and journalctl

bash
# Boot messages and recent errors
sudo journalctl -p err -b --no-pager | tail -50
 
# Follow all logs
sudo journalctl -f
 
# Specific service since last hour
sudo journalctl -u nginx --since "1 hour ago"

Kernel OOM killer messages appear in journal when RAM is exhausted—apps die mysteriously without app-level stack traces.

Classic Log Files

FileOften contains
/var/log/syslogGeneral system messages (Ubuntu)
/var/log/auth.logSSH login success/failure
/var/log/nginx/access.logHTTP requests
/var/log/nginx/error.logNginx errors
/var/log/mysql/error.logDatabase issues

Tail practice:

bash
sudo tail -f /var/log/auth.log
sudo tail -f /var/log/nginx/error.log

Application logs may live under /opt/myapp/logs/ or only in journalctl -u myapp—document your team’s paths in runbooks.

Troubleshooting Playbook

command not found

bash
echo $PATH
which java

Fix: install package, fix ~/.bashrc, or use full path /usr/bin/java.

Permission denied

bash
ls -l /path/to/file
id

Fix: chown/chmod, run as correct user, or sudo for system paths—not for routine app data.

Address already in use

bash
sudo ss -tlnp | grep :8080

Fix: stop duplicate service, change app port, or fix stale process after crash.

Disk full / no space left on device

bash
df -h
sudo du -sh /var/* 2>/dev/null | sort -hr | head
sudo journalctl --vacuum-time=7d

Fix: rotate logs, remove old releases, expand volume on cloud console.

Cannot SSH

From provider console (VNC/serial):

  • Is sshd running? sudo systemctl status ssh
  • Firewall allows 22? sudo ufw status
  • Keys correct? ~/.ssh/authorized_keys permissions 600, directory 700
  • Wrong IP or security group rule

HTTP works locally but not from internet

  • App listens 127.0.0.1 only vs 0.0.0.0
  • Nginx proxy_pass misconfigured
  • Cloud firewall vs UFW
  • DNS not pointing to server

Database connection refused

  • MySQL running: systemctl status mysql or docker ps
  • Port open on 127.0.0.1:3306 only (good) vs firewall blocking wrong host
  • JDBC URL host/port typo—see JDBC track

Scripted Checks

Combine tools from shell scripting:

Run after deploy or from cron for a poor man’s monitoring.

When to Escalate Beyond CLI

SignalAction
Repeated OOMMore RAM, fix leak, JVM -Xmx tuning
Disk growth dailyLog rotation, separate volume
SSL expiry soonCertbot renew, calendar reminder
Need history graphsHosted monitoring (Prometheus, cloud vendor metrics)

This course stays CLI-first; SaaS APM is optional at scale.

FAQ

Load average 8 on 2 CPU machine?

Overloaded—reduce traffic, optimize code, or scale vertically/horizontally.

Logs empty in /var/log/myapp?

App may log only to stdout—check journalctl -u myapp. Configure logging.file.path in Spring Boot if files are required.

dmesg: read permission denied?

Use sudo dmesg | tail for hardware and OOM kernel lines.

Restart fixes everything?

Intermittent resource leak or race—capture logs before restart next time.

What comes next?

Security basics.