Skip to content

Resolve companion ranges across filename case differences - #19

Draft
imnasnainaec wants to merge 6 commits into
mainfrom
ranges-companion-filename-case
Draft

Resolve companion ranges across filename case differences#19
imnasnainaec wants to merge 6 commits into
mainfrom
ranges-companion-filename-case

Conversation

@imnasnainaec

@imnasnainaec imnasnainaec commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

A LIFT folder written on Windows can spell its pair inconsistently — Dict.LIFT beside Dict.lift-ranges, or the reverse — and load fine there, because the filesystem folds case. On Linux the sibling candidate is built from the .lift's own name (_model.py:536), so it missed, the companion was skipped silently, and every range it defined went absent.

Not zip-specific: sil_lift.load("/data/Foo.LIFT") on Linux hits it with no archive involved, since parse_document never inspects the extension.

Approach

A candidate that matches no file exactly now falls back to one whose name folds onto it.

  • Reached only after an exact miss, so a case-folding filesystem never enters the fallback and behaves exactly as before. The cost lands only where the lookup was already failing.
  • One directory read per folder, cached across the candidate list, so a header with several range/@href entries doesn't re-scan.
  • Both directions, since an exporter sloppy about one is sloppy about the other.
  • Case and normalization formcasefold() over NFC, not lower(). FLEx writes the .lift in NFC and its companion in NFD within one export, and only APFS folds the two forms on its own; NTFS does not, so a mismatched pair fails to resolve on Windows as well.
  • Only the final path component, so an href under a directory spelled in another case still does not resolve. The hrefs this serves are bare basenames or same-folder relatives.
  • dangling-ranges-href decides existence the same way, so validation and loading agree on what is there.

Two things the fallback would otherwise have widened:

  • A candidate that folds onto the .lift itself — a header href spelling Dict.lift beside a Dict.LIFT — is skipped. RangesFile.load rejects a <lift> root, so it would fail the whole load. Reachable on Windows before this PR, by an href naming the lexicon at all.
  • A candidate that folds onto a companion already tracked is skipped. Path.resolve() leaves case alone on macOS, so one file reached under two spellings was loaded and tracked twice — and written twice by save().

Also in the area: the sibling candidate is built with with_name rather than with_suffix, which raised ValueError: Invalid suffix '-ranges' for a document loaded under a name with no extension. The two agree on every name that has one.

The tie-break

Where several names fold together, the first in code point order wins (Dict.LIFT-ranges ahead of Dict.lift-ranges). Arbitrary, but fixed — and fixed is the property that matters: directory order varies between filesystems and between runs, and a companion that resolved differently on consecutive reads would be worse than one that never resolved. The sort key is the name rather than the Path, since PurePath ordering is itself case-folded on Windows and would leave the choice to directory order there. Documented in _existing_file and asserted by a test.

Open to a different rule if you'd rather have one — e.g. preferring an exact case match on the stem — but I'd want it to stay deterministic.

Tests

Eight in test_ranges_folder.py: uppercase .lift, uppercase companion, an NFC/NFD filename mismatch, the tie-break, a negative case pinning that the fallback doesn't reach past a folder for a name that isn't in it, an extensionless .lift, an href folding onto the lexicon itself, and a case-variant companion that must not be reported dangling-ranges-href.

Only the tie-break needs both spellings to coexist, so it skips off Linux. The normalization test exercises the fallback on Windows and Linux alike (APFS resolves it without one), and the two self-reference tests are platform-independent.

python scripts/check.py green: 545 passed, 1 skipped (the tie-break, on Windows), 97% coverage.

Draft: opening for a read on the tie-break rule before marking ready.

🤖 Generated with Claude Code


Devin review: https://app.devin.ai/review/sillsdev/python-sil-lift/pull/19


This change is Reviewable

@imnasnainaec imnasnainaec self-assigned this Aug 6, 2026
@imnasnainaec
imnasnainaec force-pushed the ranges-companion-filename-case branch 2 times, most recently from dededeb to cd9cb67 Compare August 14, 2026 18:40
imnasnainaec and others added 3 commits August 18, 2026 16:22
A LIFT folder written on Windows can spell its pair inconsistently —
Dict.LIFT beside Dict.lift-ranges, or the reverse — and load fine there,
because the filesystem folds case. On Linux the sibling candidate is built
from the .lift's own suffix, so it missed, the companion was skipped
without a word, and every range it defined went absent.

Candidates that match no file exactly now fall back to one whose name
differs only in case. The fallback is reached only after an exact miss, so
a case-folding filesystem never enters it and behaves as before; a
case-sensitive one gets one directory read per folder, cached across the
candidate list.

Where several names fold together the lexicographically first wins. The
choice is arbitrary but fixed, which matters more than which file it picks:
directory order varies between filesystems and runs, and a companion that
loads differently on consecutive reads would be worse than one that never
loads.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
0.1.0 has not shipped, so there is no released behavior for an Unreleased
entry to be fixing — the tolerance is simply part of what companion
discovery does in the first release. Fold it into that bullet.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The directory listing the fallback builds called its files "entries", the
word this module uses for a LIFT <entry> everywhere else — the same
collision that keeps byte regions from being called spans. Name them
files.

Spell the surrounding prose the way the rest of the package does: a
fallback that runs rather than fires, a name that matched no file rather
than missed, a helper named for the filesystem it probes rather than
abbreviating it, and fixture names deliberately not taken from the
corpus file. Unpack the two densest clauses so each reads in one pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@imnasnainaec
imnasnainaec force-pushed the ranges-companion-filename-case branch from cd9cb67 to a565d67 Compare August 18, 2026 20:23
imnasnainaec and others added 3 commits August 18, 2026 17:23
Companion lookup folds names with casefold() over NFC rather than
lower(), so a Turkish-cased or NFD-spelled name resolves the way it does
on the filesystem that wrote it — FLEx mixes normalization forms within
one export. Ties break in code point order on every platform; sorting
Path objects left the choice to directory order on Windows, where
PurePath ordering is itself case-folded. An unstattable exact spelling
now falls through to the folded lookup instead of giving up.

A candidate that folds onto the .lift itself is skipped: RangesFile.load
rejects a <lift> root, so a header href naming the lexicon in another
case took the whole load down. One that folds onto a companion already
tracked is skipped too — Path.resolve() leaves case alone on macOS, so a
single file reached under two spellings was loaded and tracked twice,
and written twice by save().

The sibling candidate is built with with_name, which agrees with
with_suffix on every name that has an extension and does not raise on a
name without one. Nothing upstream requires the .lift extension:
parse_document never inspects it.

dangling-ranges-href decides existence with that same lookup, so a
companion spelled in another case is no longer reported missing on a
case-sensitive filesystem.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The 0.1.0 entry said companion names fold on case alone; they fold on
Unicode normalization form as well. The folder guide listed the
candidates tried but never mentioned the folding at all.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The three helpers each stated a rule, defended it, then disclaimed it.
What is left is the reasoning the code cannot show: casefold over lower,
NFC, why only the final component folds, why code point order, and what
the fold pre-check protects the inode comparison from.

The folder guide drops the folding sentence outright — it describes
behavior no reader acts on, in a paragraph otherwise about which
candidate wins. The 0.1.0 entry keeps the fact and loses the
justification, which now lives only in the docstrings.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant