DevOps Peach

Security (DevSecOps) · Hardening a Linux server · Lesson 5 of 5

Watching logins and audit logs

Know who logged in, who failed and who changed key files; read the journal and auth.log, use last/lastb, set watches with auditd and ship logs to a central system.

Practice interactively →

Where login logs live

Source Contents
/var/log/auth.log (Ubuntu), /var/log/secure (RHEL) SSH, sudo, su, PAM
journalctl -u ssh The SSH service log in the systemd journal
last Successful login history (from /var/log/wtmp)
sudo lastb Failed logins (from /var/log/btmp)
who, w Who is logged in right now
sudo grep "Failed password" /var/log/auth.log | tail
sudo journalctl -u ssh --since today
sudo journalctl _COMM=sudo --since "1 hour ago"

Many Failed password lines from one IP mean brute force; Accepted publickey for root is a sign root login should be disabled.

auditd: who changed what

Application logs do not tell you who edited /etc/passwd or ran chmod. auditd (the Linux Audit system) records kernel-level events according to rules you set:

sudo auditctl -w /etc/passwd -p wa -k identity       # watch writes and attribute changes
sudo auditctl -w /etc/sudoers -p wa -k sudoers
sudo ausearch -k identity -i                          # find events by key
sudo aureport --login --summary                       # login report

-p wa means writes (w) and attribute changes (a); -k attaches a key for searching. auditctl rules vanish at reboot; permanent rules go in /etc/audit/rules.d/*.rules. Events carry the auid, the original login UID, so you still know the real person after they used sudo.

flowchart LR
  U["alice sudo vi /etc/passwd"] --> K["Kernel audit"]
  K --> L["/var/log/audit/audit.log<br/>auid=alice key=identity"]
  L --> C["SIEM / central logs"]

Getting logs off the machine

An attacker with root can delete local logs. Ship logs to a central system (rsyslog to a log server, the Azure Monitor agent, CloudWatch, Loki or a SIEM such as Microsoft Sentinel or Wazuh) and alert on root logins, unusual sudo use or many failures.

Traps: reading logs only after an incident; keeping logs for so few days that there is nothing left to investigate; forgetting that auth.log needs root or the adm group to read.

Practice interactively →