Linux Incident Response: Log Preservation and Analysis on a VPS

Capture auth.log, syslog and the systemd journal from a Linux VPS during a security incident, hash the copies so they cannot be disputed, and read them to rebuild an attacker timeline.

  • Advanced
  • Linux VPS
  • 4 min read

By RunRacks Team · Published

Linux incident response log preservation starts with copying /var/log/auth.log, syslog and the systemd journal off the server before anything else on it changes. This guide shows you how to capture those files, hash them so you can prove they were not edited, and read them to rebuild a timeline of what an intruder did.

Do this before you reboot, before you reinstall, and before you run any cleanup script. Logs are the only record of the intrusion that the attacker cannot rewrite once they are in your hands.

Before you start

You need:

  • SSH access with a sudo-capable account (see create a sudo user).
  • A second machine to hold the evidence — your laptop or another VPS you control. Do not keep the only copy on the compromised server.
  • Enough free space for the logs. Check with df -h /var/log.
  • The server's timezone and clock state, so timestamps line up later.

Record the basics first

Capture the context that makes the logs readable later.

Terminal
export CASE=/root/ir-$(date -u +%Y%m%dT%H%M%SZ)
sudo mkdir -p "$CASE"
{
  date -u
  timedatectl
  hostname
  uname -a
  uptime
  who -a
  last -Faiw | head -40
} | sudo tee "$CASE/00-context.txt"

The CASE variable keeps every later command short. Open a second SSH session and leave it open — if the attacker kills your first session, you still have a shell.

Linux incident response log preservation

Copy, do not move. cp -a keeps timestamps and permissions, which matters if you later need to show file metadata.

Terminal
sudo cp -a /var/log/auth.log*   "$CASE"/ 2>/dev/null
sudo cp -a /var/log/syslog*     "$CASE"/ 2>/dev/null
sudo cp -a /var/log/secure*     "$CASE"/ 2>/dev/null
sudo cp -a /var/log/messages*   "$CASE"/ 2>/dev/null
sudo cp -a /var/log/wtmp* /var/log/btmp* /var/log/lastlog "$CASE"/ 2>/dev/null
sudo cp -a /etc/passwd /etc/shadow /etc/group /etc/sudoers "$CASE"/ 2>/dev/null

Debian and Ubuntu write authentication events to auth.log and general events to syslog. AlmaLinux and Rocky Linux use secure and messages instead. Copying both sets costs nothing and covers a distribution you misremember.

Then pull the journal, which often holds events rsyslog never wrote to disk:

Terminal
sudo journalctl --no-pager > "$CASE/journal-all.txt"
sudo journalctl -b -1 --no-pager > "$CASE/journal-previous-boot.txt" 2>/dev/null
sudo journalctl -u ssh -u sshd --no-pager > "$CASE/journal-ssh.txt"

-b -1 reads the previous boot. It only works if persistent journal storage is enabled — if the file is empty, this host keeps its journal in /run/log/journal only.

Hash the evidence

A hash proves the copy has not changed since you took it. Generate one manifest for the whole case directory.

Terminal
cd "$CASE"
sudo find . -type f ! -name SHA256SUMS -exec sha256sum {} + | sudo tee SHA256SUMS
sudo sha256sum SHA256SUMS | sudo tee SHA256SUMS.sha256

Store SHA256SUMS and SHA256SUMS.sha256 with the logs. Anyone you hand the evidence to can run sha256sum -c SHA256SUMS and confirm nothing was touched.

Move the copy off the server

Push the case directory to a host the attacker does not control.

Terminal
sudo rsync -a --progress "$CASE/" [email protected]:/evidence/$(hostname)/

If you have no second host, pull it from your workstation instead:

Terminal
scp -r [email protected]:"$CASE" ./evidence/

Verify the hashes on the copy before you rely on it:

Terminal
sha256sum -c SHA256SUMS

Run that inside the copied case directory. If every line reports OK, the evidence is intact and you can analyse it safely.

Was this guide helpful?

Still stuck?

Ask RunRacks AI for a quick answer, or open a ticket and our team will take a look at your server.

Security

Check the Windows Firewall rule for RDP port 12230

Find and verify the inbound TCP 12230 rule that lets Remote Desktop reach your RunRacks server, recreate it if it was deleted and scope it without locking yourself out.

  • Intermediate
  • Windows VPS · Admin RDP
  • 5 min read

Security

iptables basics: list, allow, drop and save rules

Learn how iptables chains and rule order work, then build a small ruleset that allows SSH and web traffic, drops everything else and can be edited by line number.

  • Intermediate
  • Linux VPS
  • 6 min read