Files with Coverage Reduction % is kinda confusing for uninitiated/unalert readers

[jsoref]

PowerDNS/pdns#16531 (comment)

I think I understand the % in this table as "percentage of changed/added lines that are covered":

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
modules/lmdbbackend/lmdbbackend.cc	0	1	0.0%
pdns/backends/gsql/gsqlbackend.cc	1	2	50.0%
pdns/dnsdistdist/dnsdist-configuration-yaml.cc	0	1	0.0%
...	...	...	...
pdns/recursordist/pdns_recursor.cc	5	6	83.33%
pdns/recursordist/rec-lua-conf.cc	0	1	0.0%
pdns/recursordist/rec-main.cc	0	1	0.0%

But this table is different, as a naive human, "why are the percentages wildly different when the only displayed number is about the same for each row?"

Files with Coverage Reduction	New Missed Lines	%
pdns/backends/gsql/gsqlbackend.hh	1	97.77%
pdns/dnsdistdist/dnsdist-tcp.cc	1	77.7%
pdns/recursordist/syncres.cc	1	81.1%
pdns/recursordist/aggressive_nsec.cc	2	66.69%
modules/godbcbackend/sodbc.cc	3	71.43%
pdns/recursordist/test-syncres_cc2.cc	3	89.18%
pdns/recursordist/test-syncres_cc1.cc	6	90.19%
pdns/tcpiohandler.cc	9	60.29%

This table only has one percent and it clearly hints that the other thing is a comparison to a baseline:

Totals
Change from base Build 19494496269:	0.02%
Covered Lines:	128023
Relevant Lines:	164483

That said, it probably should say +0.02% to hint that it could go down by 0.02% in the form of -0.02% assuming that's what it's saying (if it's saying it went down by 0.02%, then, obviously it should say -0.02%, but, I'm charitably assuming it's going up).

Note that for this PR (which should only be changing spelling), I'd expect the change to be exactly 0. I understand that this usually means that the test harness didn't run the same tests for both the baseline and the tested runs which should explain any variance. I don't mind terribly, but it is amusing.

---

My guess is that there's an implied line count for each file and an implied covered line count and somehow these numbers relate. But I really can't deal w/ this and don't like hidden numbers.

Disclaimer: I'm really upset at certain bean counters who have "withheld" (please read lied) about the numbers they were giving to me recently.

[afinetooth]

Hi @jsoref — thanks for taking the time to share such detailed feedback. First-impression notes like these help us see where things are confusing for new users.

Let me break this into parts:

---

1. PR Comment Design

You’re right that some of the terminology and numbers presented in our PR Comments feels unclear, or at least inconsistent with Build Pages. A redesign is already in our backlog as part of a broader refresh, but your examples make a strong case for doing an interim cleanup sooner.

In fact, trying to articulate these details helped me put my finger on some things that don't work for me either.

---

2. “Diff Tables” in PR Comments vs. Build Pages

First, the examples you’re showing, we refer to as "Diff Tables".

That said, the examples you're comparing come from two different builds:

• Your PR comment here:
  Spelling PowerDNS/pdns#16531 (comment)
  → comes from Coveralls Build 19536992955: https://coveralls.io/builds/76582846
• The diff tables you pasted
  → come from Coveralls Build 19514239601: https://coveralls.io/builds/76564005

So to avoid mixing signals, I’ll use the second build (where your pasted tables come from) to explain the diff types and details.

---

3. What These Tables Are For

Both tables highlight specific files and line ranges that lost or lack coverage for this commit.

They’re meant to be the most “actionable” part of the report — the fastest way to see:

• where newly added code isn’t tested, or
• where previously tested code became uncovered.

The two diff types:

1. “NEW MISSED LINES”

New lines added in the commit that don’t have test coverage.

2. “UNCOVERED EXISTING LINES”

Lines that used to be covered in previous commits but no longer are.

---

4. Differences between PR Comments & Build Pages:

The Diff Tables in PR Comments and the Diff Tables in Build Pages show the same underlying information, but they present it differently and use different denominators when calculating percentages. I think this is the root of the confusion.

1. Order and Labels

The diff tables appear in the same order in both PR Comments and Build Pages, but they use different labels:

Build Page Label	PR Comment Label
NEW MISSED LINES	Changes Missing Coverage
UNCOVERED EXISTING LINES	Files with Coverage Reduction

This is confusing—the labels should be consistent, or at least better clarified.

2. Line Counts and Coverage Percentages

In PR Comment View:

"Changes Missing Coverage" show "PR-level metrics." Just what's changed in the PR, so:

1. In “Changes Missing Coverage”
   The % =
   Covered Lines / Changed (Added/Modified) Lines

Whereas "Files with Coverage Reduction" show "File-level metrics," the same metrics as in Build Pages.

2. In “Files with Coverage Reduction”
   The % =
   Uncovered Lines / Total Lines

Here the denominator is how many lines are in the file, not just how many lines changed in this PR (which in this case is just the lines that lost coverage).

In Build Page View:

By contrast, NEW MISSED LINES in Build Pages show "File-level metrics"—the complete diff context for each file. So:

• In NEW MISSED LINES
  The % =
  Lines Missing Coverage / Total Lines

And this is the same context for UNCOVERED EXISTING LINES—just like in the PR Comment—only here, both diff tables includes one extra element:

• Total Coverage Change (Δ)

The design precedes me, so I'm not sure why, but I assume we add this element because: (A) we have more space in the Build Page; and (B) it's easier to compare the delta (Δ) with the file's previous build by using the "Previous Build" link and doing a side-by-side comparison with the file's previous version.

Example Difference:

An example that I find clearly illustrates the difference is this one:

• In the PR Comment:

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
...	...	...	...
pdns/backends/gsql/gsqlbackend.cc	1	2	50.0%
...	...	...	...

Translation: 50% of the changes in this file are uncovered.

• In the Build Page:

NEW MISSED LINES

Lines	Coverage	∆	File
...	...	...	...
1	81.45	0.0%	pdns/backends/gsql/gsqlbackend.cc
...	...	...	...

Translation: 1 line lost coverage, but is a vanishingly small change (~0.0%) for this 2506 line file.

3. Why the Percentages Look “Wildly Different”

Once the different denominators are understood, the variance makes sense:

• In PR Comments, "Changes Missing Coverage" answer:
  “Of the lines you touched in this PR, how many are uncovered?”

• In PR Comments, "Files with Coverage Reduction", and both Diff Tables in Build Pages answer:
  “Of the entire file, how many lines lost coverage, and how did that change coverage for the file as a whole?”

So, across "Changes Missing Coverage" and the other diff tables, one missed line can be:

• 50% (PR-level diff scope), and
• 0% of a 81.5% covered file (file-level scope),
  both accurately, because they’re answering different questions.

---

5. Closing Thoughts & Next Steps

Thanks again for calling this out — you're absolutely right that the current wording and percentages are easy to misinterpret without knowing the different scopes. We’ll capture this thread in our redesign notes so the terminology, denominators, and table labels can be made clearer and more consistent across PR Comments and Build Pages.

If you have additional examples or suggestions as you continue exploring, please keep them coming. Your feedback here has already been very helpful.

[afinetooth]

@jsoref I was incorrect about one table in my comparisons, so I updated my answer: UNCOVERED EXISTING LINES / "Files with Coverage Reduction" use the same denominators (ie. "File-level metrics") in both PR Comments and Build Pages. There's just the extra "delta (Δ)" column in both diff tables on Build Pages.