Back to lessons

Hosting Operations

Find the First Failure Line for One Unit

A failed service has repeated restart noise, and you need the earliest current-boot line that names an error or failed startup step.

Command

journalctl -u app-worker -b --no-pager -o short-iso | grep -m1 -E 'ERROR|Failed|status='

What changed

Nothing changes. journalctl prints the current-boot unit log and grep stops at the first matching failure line.

Danger

safe

When to use it

Use when restart loops bury the original clue under repeated systemd retry messages.

When not to use it

Do not use this as a complete log review; it intentionally stops at the first matching line and can miss later, more specific evidence.

Undo or recovery

No undo needed because the command is read-only.

Expected output

The earliest current-boot app error, failed step, or status line for the unit.

demo script

Disposable terminal steps

  1. journalctl -u app-worker -b --no-pager -o short-iso
  2. journalctl -u app-worker -b --no-pager -o short-iso | grep -m1 -E 'ERROR|Failed|status='

simulated output

What it looks like

disposable vessel
::fixture-ready::
$ journalctl -u app-worker -b --no-pager -o short-iso
2026-06-25T14:20:58-05:00 vps systemd[1]: Started app-worker.service - Background job worker.
2026-06-25T14:20:58-05:00 vps worker[2081]: loading /etc/app/worker.env
2026-06-25T14:20:58-05:00 vps worker[2081]: ERROR redis connection refused at 127.0.0.1:6379
2026-06-25T14:20:59-05:00 vps systemd[1]: app-worker.service: Failed with result 'exit-code'.
2026-06-25T14:21:04-05:00 vps systemd[1]: app-worker.service: Scheduled restart job, restart counter is at 4.
2026-06-25T14:22:17-05:00 vps systemd[1]: Started app-worker.service - Background job worker.
2026-06-25T14:22:17-05:00 vps systemd[2144]: app-worker.service: Failed to determine user credentials: No such process
2026-06-25T14:22:17-05:00 vps systemd[2144]: app-worker.service: Failed at step USER spawning /srv/app/bin/worker: No such process
2026-06-25T14:22:17-05:00 vps systemd[1]: app-worker.service: Main process exited, code=exited, status=217/USER
2026-06-25T14:22:17-05:00 vps systemd[1]: app-worker.service: Failed with result 'exit-code'.
::exit-code::0
$ journalctl -u app-worker -b --no-pager -o short-iso | grep -m1 -E 'ERROR|Failed|status='
2026-06-25T14:20:58-05:00 vps worker[2081]: ERROR redis connection refused at 127.0.0.1:6379
::exit-code::0

YouTube Short

Find the first failure line.

Restart loops repeat the same failure. Search the current boot journal and stop at the first error-like line.

LinkedIn hook

The first failure line is often more useful than the last restart message.

Question: When a service restarts repeatedly, do you inspect the first failure or the latest one first?

experiments

A/B tests to run

Metric: save_rate

A: The first failure line beats retry noise.

B: Stop at the first useful clue.