Forem: Emjey

Your pytest retries are lying to you. The hidden cost of --reruns, and the plugin I wrote so I could actually see what my tests were doing.

Emjey — Wed, 22 Apr 2026 11:55:01 +0000

Picture this. A test fails in CI. It's been flaky all week — fails on push, passes when you rerun. So you add --reruns 2 to the pytest command. Now the suite passes. Green build. Ship it.

A week later, the same test fails in production in a way that only happens under load. You go back to look at the build that passed, and the report says... "passed." One line. No context. No hint that the test ever failed before, let alone what it failed with.

This is what pytest looks like to most of us: a final verdict. It's not wrong, exactly — the test did pass, eventually. But "eventually" is hiding the interesting information. Why did it fail the first two times? What error? Was it a race condition? A flaky fixture? A genuine bug that only manifests one in three runs?

pytest doesn't tell you, and by default pytest-rerunfailures doesn't preserve that context in a form you can easily inspect. Add -n auto via xdist and it gets worse — your reports become a collage of retry artifacts spread across worker JSONs, and figuring out which attempt ran first on which worker is its own forensic exercise.

When a test goes fail → fail → pass, I want to see all three attempts. I want to see each error message. I want to see the order. I want to be able to go, "oh, the first two failures were ConnectionError but the third was clean — that's a network flake, I'll mark it as such" — instead of assuming a pass is a pass is a pass.

So I wrote pytest-html-plus.

What it does (just this, for now)

pytest-html-plus hooks into pytest and pytest-rerunfailures and preserves the full retry history in its HTML report. When a test has multiple attempts, you see all of them — passed, failed, errored — with their individual logs, errors, and tracebacks, in the order they ran.

Crucially, it also merges xdist worker JSONs back into a single cohesive story, so even if a test ran across two workers, you see the full chronological attempt history in one place.

Everything else the plugin does — combined XML export, automatic screenshots on failure, markers, email — is secondary. The retry visibility is the reason I built it.

Install and see it
The setup is three commands:

pip install pytest-html-plus pytest-rerunfailures
pytest --reruns 2
open report_output/report.html

That's it. No config file. No conftest changes. No hooks to register. It works with or without xdist. If a test goes through retries, you'll see the full story in the HTML report automatically.

If you want to try it without touching your own suite, there's a live demo report here. The flaky tests on that page show exactly the retry history I'm describing.

Why this matters more than it sounds like

Knowing which attempt failed" sounds like a nice-to-have until you're actually triaging flakes in a 2,000-test suite. Three concrete things it unlocks:

Distinguishing flaky from genuinely broken.
If a test goes fail → pass, that's a flake. If it goes fail → fail → pass there's probably a real bug that just doesn't reproduce deterministically. The attempt count alone is a diagnostic signal, and you lose it in a standard report.

And when a test goes fail → fail → fail, the first failure is often the most diagnostic one. Later attempts can fail for downstream reasons, polluted state left behind by the first failure, for example: so the symptom you see on attempt three may have nothing to do with the original bug. A classic tell is a test that's quietly depending on state from another test that ran before it; you only see it clearly in the first attempt.

Finding correlated flakes. When three tests all fail their first attempt with ConnectionError but pass on retry, you don't have three flaky tests — you have one network issue. The cross-test retry log makes that pattern visible.

Honest CI reports. There's a real difference between "this build passed on the first try" and "this build passed after eight retries across twelve tests." Both show passed in a default pytest run. Both should not be treated the same by a reviewer.

What I didn't build (on purpose)

If you're looking for a full test-management platform with dashboards, trends, or historical tracking across runs — this isn't that. pytest-html-plus is a single-run reporter. It writes one self-contained HTML file per run. That's by design: it's portable, it works without a backend, you can archive it as a CI artifact, you can email it(by just enabling --email flag).

If you want Allure, use Allure — it's a different product solving a different problem. If you want a server with a database tracking flakes across months, that's also a different product. This plugin is for the specific moment when you're triaging a single build and want to see what actually happened.

Try it, tell me it's broken

The project is on GitHub at reporterplus/pytest-html-plus. I maintain it alone. If you install it and something doesn't work on your suite, please open an issue — the most valuable feedback is from people whose pytest setup is weirder than mine. Exotic fixtures, unusual xdist configurations, custom reruns logic — those are the corners where bugs hide, and I can't find them alone.

And if the retry history helps you catch one bug you would have otherwise shipped, that's enough. That's what I wrote it for.

pytest-html-plus — Your Default Pytest Reporter

Emjey — Sat, 24 Jan 2026 08:37:47 +0000

Why another Pytest HTML reporter?

If you’ve been running Pytest for a while, chances are you’ve used pytest-html. It works. It’s stable. It does the job.

But over time, many teams run into the same quiet problems:

Reports feel static
Debugging still requires jumping between logs, terminals, and CI artifacts
Customization often means patching or workarounds
The report exists… but it makes you think than looking at quick debugging.

pytest-html-plus was built to solve those gaps — not by replacing testing or test fundamentals, but by rewriting how simpler debugging has to be or what a default test report should feel like in 2025.

What “default reporter” actually means

A default reporter should:
Work out of the box
Require zero configuration and codechanges to be useful
Guide you when data is missing
Stay readable even as test suites grow
Fit naturally into CI pipelines

pytest-html-plus is designed with exactly that mindset.

You install it, run Pytest, and you already get something meaningful. No Java needed, No extra decorators needed, no conftest, no merge plugins(even if you use xdist or rerun-failures)

No ceremony.

What makes pytest-html-plus different

1. Reports that explain themselves

Instead of silently showing empty values, the report tells you why something is missing and how to fix it.

Example:

Missing environment?
→ The report nudges you to use --rp-env

Metadata not populated?
→ It tells you what flag or config to add

This removes tribal knowledge and makes reports usable even for someone new to the project.

2. Designed for real debugging with auto screenshots

The report is structured to help you answer real questions:

What failed? pytest-html-plus displays all the failures right there
Where did it fail? pytest-html-plus displays all the errors, traces, logs in seperate block that you can simply click copy to send to your team or debug.
Under which environment and configuration? pytest-html-plus lets you use your own filters when you pass pytest.mark.{filtername}
What context do I need right now? pytest-html-plus shows the external JIRA tickets or test cases if you had tagged it using markers like pytest.mark.link in your codebase

You spend less time re-running tests just to reproduce context.

3. CI-first, not CI-afterthought

pytest-html-plus plays nicely with:

GitHub Actions
Archived artifacts
JSON and optional xml outputs that can be reused by other tools or test management tools.

The report is not just an HTML file — it’s part of a pipeline and its unified without making you use a dozen of other plugins or keep writing conftest.

4. Opinionated, but not heavy

The goal is clarity, not feature overload.

Instead of adding 20 toggles and panels, the focus is on:

Clean layout
Warm, readable colors
Information density without noise

If a feature makes the report harder to scan, it doesn’t belong in pytest-html-plus.

Why should it be my default reporter?

If pytest-html is the reliable base,
pytest-html-plus is the version shaped by years of daily test runs, CI failures at 2 a.m., and “why didn’t the report tell me this?” moments.

For many teams, it naturally becomes the default reporter.

Teams running Pytest daily
Engineers debugging from CI and in the local runs
Projects where reports are read by humans, not just machines
Anyone who wants reports to help, not just exist

Closing thought

Good tooling fades into the background. Great tooling quietly makes you faster.

pytest-html-plus aims to be the reporter you stop thinking about — because it already gave you what you needed.

A Small VS Code Extension to Shorten the Pytest Fix Loop

Emjey — Thu, 01 Jan 2026 09:36:15 +0000

When working with pytest locally, the loop after a test failure usually looks like this:

run tests
scan terminal output
find the failed test
locate the file
scroll to the failure
repeat if there are multiple failures

It works, but it’s not particularly calm — especially when failures are spread across files or when you’re re-running tests frequently.

As part of maintaining pytest-html-plus, we’ve been thinking about how to make the local fix loop a little smoother — without introducing new concepts or replacing existing tools.

We’ve taken a similar approach before with the GitHub Actions integration: instead of asking users to learn a new workflow, the action simply plugs into existing CI YAML and publishes the same report artifact automatically.

Along the same lines, we’ve been experimenting with a small VS Code extension that:

reads the generated report
surfaces failed tests in a sidebar
lets you jump directly to the failure location in code

One deliberate design choice is that the extension does not try to run or discover tests.

Instead, it consumes a stable artifact — final_report.json — produced by pytest-html-plus.

That means it doesn’t matter:

where pytest was run (terminal, CI, container, xdist, docker etc.)
how the tests were executed
whether they ran inside or outside the editor
As long as a report exists, the extension can surface failures and navigate to their source.

This keeps the extension:

runner-agnostic
predictable
decoupled from test execution itself

What we added

We’ve released a small VS Code extension that focuses on one thing:

helping you stay in the editor while fixing failures, and only switch to the HTML report when you actually need it.

The extension reads the existing generated report produced by pytest-html-plus and presents:

a quick summary (passed / failed / skipped)
failed tests grouped by file
inline error context
one-click navigation to the failure line

There’s no test execution, no report rendering, and no screenshots in the editor.

What this extension is not

This is important to be clear about.

The VS Code extension does not:

replace the HTML report
render screenshots or CI artifacts
analyze failures
re-run tests
change how pytest works
Try to be just another extension to run pytest and show results

The HTML report remains the source of truth for:

detailed inspection
screenshots
metadata
CI usage

The extension is intentionally scoped to local development, when the goal is simply to move from failure → fix as quickly as possible while already inside the editor.

Why we kept it small

There’s a temptation to turn editor extensions into dashboards.

We resisted that.

Instead, the extension is designed to:

complement the terminal, not replace it
respect existing workflows
stay out of the way when you don’t need it
You open it when you want orientation.
You close it when you don’t.

Configuration, on your terms

The extension supports a few lightweight ways to configure the report path:

browse for the report file
enter the path manually
auto-detect reports in the workspace
Nothing is auto-configured without asking, and nothing is mandatory.

Where this fits in the ecosystem

With this addition, the pytest-html-plus ecosystem now looks like this:

Core library → generate a unified report (local + CI)
GitHub Action → publish reports automatically in CI
VS Code extension → act on failures locally, inside the editor

Each piece does one thing, at one moment, for one context.

If your workflow already feels smooth with the terminal alone, you may not need this.

But if you often:

deal with multiple failures
jump between test files
re-run tests while iterating locally

this extension might reduce a bit of friction in that loop.

As with the rest of pytest-html-plus, the goal isn’t to change how you work — just to remove a few unnecessary steps.

Links

VS Code extension: Pytest HTML Plus

Core library: pytest-html-plus

Github actions: pytest-html-plus