$ Linux One Liners
Back to lessons

Hosting Operations

Find Tests That Passed After Rerun

The pipeline passed after retry, but you need to identify tests that failed earlier in the same job.

Command

grep -RInE 'rerun|retry|flaky|passed on retry|failed attempt' artifacts logs

What changed

Nothing changes. Retry and flaky-test markers are printed.

Danger

safe

When to use it

Use after a successful retry or when CI instability is suspected.

When not to use it

Do not label a test flaky from one run alone; compare with history when possible.

Undo or recovery

No undo needed because this command is read-only.

Expected output

Lines mentioning reruns, retries, flaky markers, or failed attempts.

demo script

Disposable terminal steps

  1. cd /lab/ci-artifacts && grep -RInE 'rerun|retry|flaky|passed on retry|failed attempt' artifacts logs
  2. cd /lab/ci-artifacts && grep -RInE 'rerun|retry|flaky|passed on retry|failed attempt' artifacts logs | cut -d: -f1 | sort -u
  3. cd /lab/ci-artifacts && sed -n '1,120p' artifacts/test/pytest.log
Terminal transcript Demo JSON

simulated output

What it looks like

disposable vessel 
::fixture-ready::
$ cd /lab/ci-artifacts && grep -RInE 'rerun|retry|flaky|passed on retry|failed attempt' artifacts logs
artifacts/test/pytest.log:6:tests/test_billing.py::test_invoice passed on retry after failed attempt 1
artifacts/test/pytest.log:7:1 failed, 2 passed, 1 rerun
::exit-code::0
$ cd /lab/ci-artifacts && grep -RInE 'rerun|retry|flaky|passed on retry|failed attempt' artifacts logs | cut -d: -f1 | sort -u
artifacts/test/pytest.log
::exit-code::0
$ cd /lab/ci-artifacts && sed -n '1,120p' artifacts/test/pytest.log
============================= test session starts =============================
tests/test_api.py::test_health PASSED
tests/test_api.py::test_create_user FAILED
Traceback (most recent call last):
  AssertionError: expected 201 got 500
tests/test_billing.py::test_invoice passed on retry after failed attempt 1
1 failed, 2 passed, 1 rerun
::exit-code::0

YouTube Short

Green retry is still a signal.

If CI passed after retry, search the logs for rerun markers. Flaky tests leave tracks.

LinkedIn hook

A green retry can still hide a flaky test.

Question: Do you investigate tests that only pass after retry?

experiments

A/B tests to run

Metric: short_save_rate

A: A green retry can still hide a flaky test.

B: Find tests that only passed after retry.