journalctl Reference: Finding What You Need in System Logs

Almost every Linux troubleshooting session ends at journalctl, and almost everyone uses it the same three ways: tail everything, filter by unit, filter by time. The tool does considerably more than that. It can filter by numeric priority in ranges, search message text without piping to grep, select a specific boot, emit JSON for machine consumption, and tell you how much disk the journal is using. This is the reference for the rest.

The one concept that unlocks the rest is that the journal is a field-structured log, not a text file. Every entry has named fields, and journalctl‘s filtering language is built on them. Once you filter with _SYSTEMD_UNIT= instead of grep-ing for a name, and understand that multiple terms are ANDed while multiple values for one field are ORed, the behaviour of the tool stops being surprising. Add -b to talk about the previous boot, which is where post-crash questions actually live.

Key takeaways:

• The journal is structured data. Filter on fields; do not grep the formatted output.
• Space-separated terms are ANDed. Repeated values of the same field are ORed.
• -b selects a boot. -b -1 is the previous one, and --list-boots shows what you still have.
• -p err..alert is a priority range. A bare -p err means “exactly err”.
• -k is shorthand for kernel messages, and -g searches message text natively.
• --vacuum-size= and --vacuum-time= reclaim journal space, but journald also enforces its own limits continuously.
• If logs are missing, run journalctl --verify. A truncated journal drops entries silently.

The shape of the command

journalctl [OPTIONS...] [MATCHES...]

Matches come last, as FIELD=VALUE pairs. Options come first. That ordering is worth internalising because it means the intuitive thing — appending a condition to lengthen a command — always works.

Time selection

Time is where most people under-use the tool, because the formats accepted are more flexible than the documentation suggests:

journalctl --since today
journalctl --since yesterday
journalctl --since "1 hour ago"
journalctl --since "-30min"
journalctl --since "2026-10-01 14:30:00"
journalctl --since "2026-10-01" --until "2026-10-01 18:00"
journalctl --since "08:00" --until "09:00"

# Last 200 lines rather than everything
journalctl -n 200
journalctl -n 50 -f          # last 50, then follow

The last two are the ones that change how you work. -n combined with -f gives you recent history plus live tail in one command, which is almost always what you actually want.

Boots

Each boot has an identifier. Selecting a boot is how you investigate something that happened before a restart — and that is most of what post-incident logging is for:

journalctl --list-boots          # what is still available
journalctl -b                    # current boot
journalctl -b -1                 # previous boot
journalctl -b -2 -p err..alert   # two boots ago, errors only
journalctl -k -b -1              # kernel messages from the previous boot

If a machine has been running for months without restarting, --list-boots may show a single entry. If a host reboots unexpectedly, the previous boot’s final entries are the single most valuable diagnostic you have — which is why Storage=persistent is worth setting on anything you care about.

Priority, and the range trap

Syslog priorities, from most to least severe: emerg, alert, crit, err, warning, notice, info, debug. Each has a numeric value.

journalctl -p err             # ONLY err
journalctl -p err..alert      # err and more severe
journalctl -p warning..info   # warning through info
journalctl -p 3..4            # numeric form: err(3) and warning(4)

The single most common journalctl mistake is -p err when you meant -p err..alert. The bare form is an equality match and returns almost nothing, which looks like “the service is fine”. If a filter seems to be hiding your problem, check for a missing range operator first.

Field matching

This is the part that makes the tool different from reading a text log. Matches are field-based, and the combination logic is specific:

# Fields you will actually use
journalctl _PID=1234
journalctl _UID=1000
journalctl _COMM=nginx
journalctl _HOSTNAME=web-01
journalctl SYSLOG_IDENTIFIER=sshd

# Terms separated by spaces are ANDed
journalctl _SYSTEMD_UNIT=sshd.service _UID=0

# Values for the SAME field are ORed
journalctl _SYSTEMD_UNIT=sshd.service _SYSTEMD_UNIT=cron.service

# Explicit disjunction with +
journalctl _SYSTEMD_UNIT=a.service + _SYSTEMD_UNIT=b.service

# Grep the message text natively, with no pipe
journalctl -g "Out of memory"
journalctl _SYSTEMD_UNIT=kernel.service -g "segfault|oom" -b -1

Two rules to hold onto: space is AND, and repeating a field is OR. Everything else follows from those. The + syntax exists for when you need OR across different fields, which the space-separated form cannot express.

Note that -g searches the message body properly, escaping your pattern. That is strictly better than journalctl | grep, which searches the formatted line including timestamps and prefixes and will match things you did not intend.

Output formats

# Feed a pipeline, strip the decoration
journalctl -u app.service -o cat --since "1 hour ago" | grep -c ERROR

# Export for analysis or archiving
journalctl -u app.service -o json > app-journal.json

# See every field on one entry
journalctl _PID=1234 -n 1 -o verbose

Disk usage and retention

journalctl --disk-usage

journalctl --vacuum-size=1G       # shrink to 1 GB
journalctl --vacuum-time=7d       # keep 7 days
journalctl --vacuum-files=1000    # keep 1000 files

--vacuum only reclaims what you already have. The journal’s steady-state size is set by configuration in journald.conf, so vacuuming without changing the configuration buys you a week:

# /etc/systemd/journald.conf
[Journal]
Storage=persistent
SystemMaxUse=2G
SystemKeepFree=1G
# SystemMaxFileSec=1month

systemctl restart systemd-journald

Storage= is the setting people miss. With auto, the journal lives on disk only when /var/log/journal exists. If it does not, logs go to a volatile tmpfs under /run and disappear on reboot. On a machine that does not reboot often this looks fine right up until the day you need the previous boot.

Two more worth knowing: Seal=yes enables forward-sealed and FSS cryptographic journals, which makes tampering detectable; and if you also run rsyslog or syslog-ng, your logs may exist in two places with different retention, and /var/log/syslog or /var/log/messages may be the one your log shipper is actually reading.

When logs are missing

If entries you expect are not there, work through these in order:

• Check for truncation. A full or crashed journal loses the oldest entries silently. journalctl --verify reports corruption.
• Check persistence. journalctl --list-boots showing one entry on a machine that has rebooted means the journal is volatile.
• Check the boot selection. Without -b you see the current boot. The entry may be in an earlier one.
• Check the range syntax. -p err rather than -p err..alert is the classic cause of an empty result.
• Check the unit name. A unit may be logging under a different name than you expect. journalctl -o verbose on one entry shows the real fields.
• Check the field name. _SYSTEMD_UNIT and SYSLOG_IDENTIFIER are different fields with different values, and a service can populate one and not the other.

Recipes

# Everything a unit logged, clean, since this morning
journalctl -u myapp.service -o cat --since today

# Live errors from any unit
journalctl -f -p err..alert

# Everything one user did
journalctl _UID=1000 --since today

# Did this process die, and why, last boot
journalctl _PID=4821 -b -1 -o short-precise

# Count errors per hour for the last day
journalctl --since "24 hours ago" -p err..alert -o short-iso 
  | cut -c1-13 | uniq -c

# Kernel panics and OOM kills
journalctl -k -g "Out of memory|panic" -b -1

# Follow a unit's file-descriptor errors live
journalctl -u nginx.service -f -g "too many open files"

The counting recipe is the one that turns logs into information. “There were errors” is not actionable; “forty errors at 03:00 and none since” is.

Getting your logs somewhere else

The journal is well suited to being a source for aggregation, precisely because it is structured:

# Stream JSON to a collector
journalctl -f -o json | your-collector

# Pull a specific unit's recent history
journalctl -u myapp.service -o json --since "1 hour ago" > out.json

# Follow with the most recent entries pre-loaded
journalctl -u myapp.service -n 100 -f

If you are building detection on top of this, building an AI-powered Linux security monitor shows how audit records and the journal get combined into a triage pipeline, and finding large AWS CloudWatch log groups covers the cost side of the same problem on a managed platform.

References

• journalctl(1) — the authoritative option reference
• systemd.journal-fields(7) — every field you can match on
• journald.conf(5) — storage and retention configuration
• journalctl(1) on Debian — the same tool, at the older section 8

Related Tools

Try these companion tools on DevStackIO Tools:

• timestamp converter — convert log timestamps to readable dates for journalctl analysis.