Linux: Keep Disk Usage of System Journal (systemd-journald) in check

systemd-journald sometimes uses up a ridiculous amount of disk space under /var/log/journal. Here are the steps to limit systemd-journald to use 16MB of disk space under /var/log/journal.

1) Edit the configuration file  /etc/systemd/journald.conf and set SystemMaxUse=16M uncommenting it if necessary.

2) Stop the service and its trigger sockets:

# systemctl stop systemd-journald-dev-log.socket systemd-journald.socket systemd-journald
# systemctl status systemd-journald-dev-log.socket systemd-journald.socket systemd-journald

3) If /var/log/journal does not  exist, create it:

# ls -l /var/log/journal/*

If not found:

# mkdir -p /var/log/journal
# systemd-tmpfiles --create --prefix /var/log/journal

4) Clear out existing transient and persistent journals.

# rm -rvf /var/log/journal/*
# rm -rvf /run/log/journal/*

5) Start the service and its trigger sockets:

# systemctl start systemd-journald-dev-log.socket systemd-journald.socket systemd-journald
# systemctl status systemd-journald-dev-log.socket systemd-journald.socket systemd-journald

6) Flush any transient journal entries into persistent storage:

#killall -USR1 systemd-journald # forces in-memory journal to be written to /var/log/journal/*

7) Now you can check the disk space used by journals periodically using:

# du -h /var/log/journal

Note: A quick cleanup of journal storage can be done using the following command which clears up all old entries so that disk usage falls below the indicated parameter:

# journalctl --vacuum-size=2M --rotate