chunkers: do not re-read a file map that does not reach EOF

[ThomasWaldmann]

FileReader re-reads a file map that does not extend to EOF, which appends duplicate data to the file it is reading — or never terminates.

The bug

FileReader uses blockify_gen is None for two different things: "not started yet" and "exhausted". _fill_buffer() clears the attribute on StopIteration, and read() / readinto() take a cleared generator as their cue to build a fresh FileFMAPReader.blockify() — which replays the fmap from its start.

dread() reads sequentially and ignores its offset argument, so the consequences depend on the fmap:

• an fmap that ends before EOF is replayed from the current file position, so the caller gets the mapped ranges plus whatever follows them, up to EOF;
• an fmap that does not start at 0 dseek()s back to its start on every replay and loops forever.

FileFMAPReader.blockify() itself is fine — it yields exactly the mapped ranges and stops.

This stayed invisible because an fmap that does reach EOF is replayed into an immediate 0-byte read and stops there, and until now every caller produced such a map: sparsemap() covers the whole file, and so does create --map. create --map --reuse-from from #10137 is the first caller that deliberately reads only parts of a file — it skips the ranges whose chunks it reuses from the reference archive — and it hits this immediately. It is not specific to a chunker; the fixed chunker is affected the same way.

The fix

Track exhaustion in its own blockify_done flag and never restart the generator.

That alone is not enough, though: blockify() also returned on got < wanted, i.e. it treated any short read as EOF. The restart happened to paper over that, so removing the restart without fixing it truncates the input instead. buzhash_self_test.py::test_small_reads — whose fake file object returns one byte per read() call — passes on current master only because of the restart, and is the tripwire for getting this half-right. So blockify() now returns only on got == 0.

Tests

Two regression tests in reader_test.py:

• test_filereader_fmap_not_covering_eof — prefix-only, not-starting-at-0, and multi-range partial fmaps. Fails on master (assert 60 <= 30). The in-loop length assert stops a regression from hanging the suite instead of failing it.
• test_filereader_short_reads_are_not_eof — pins the short-read contract. This one passes on master too (the restart hides it); it fails on a half-applied fix, which is the point.

Verification

Built from this branch on Linux (Debian 13, Python 3.13):

• src/borg/testsuite/chunkers/ + create_cmd_test.py + extract_cmd_test.py: 328 passed, 346 skipped, 0 failed.
• Reading a 1 MiB file through partial fmaps returns exactly the mapped byte count in every case, including the previously non-terminating ones.
• Against #10137, where this surfaced: its intermittently failing test_create_map_reuse_from_cdc went from 5 failures / 100 runs to 0 / 200, and a create --read-special --map ... --reuse-from of a real LVM thin snapshot — which previously spun for ~12 minutes of CPU on a single file — now completes and borg extract --stdout ... | cmp - /dev/vgtest/snap2 matches byte for byte over 4 GiB.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.55%. Comparing base (00fd667) to head (7bc4647).
⚠️ Report is 5 commits behind head on master.
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #10166      +/-   ##
==========================================
- Coverage   87.00%   86.55%   -0.45%     
==========================================
  Files         101      101              
  Lines       17993    17996       +3     
  Branches     2738     2738              
==========================================
- Hits        15654    15577      -77     
- Misses       1635     1717      +82     
+ Partials      704      702       -2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.