Resolve companion ranges across filename case differences - #19
Draft
imnasnainaec wants to merge 6 commits into
Draft
Resolve companion ranges across filename case differences#19imnasnainaec wants to merge 6 commits into
imnasnainaec wants to merge 6 commits into
Conversation
imnasnainaec
force-pushed
the
ranges-companion-filename-case
branch
2 times, most recently
from
August 14, 2026 18:40
dededeb to
cd9cb67
Compare
A LIFT folder written on Windows can spell its pair inconsistently — Dict.LIFT beside Dict.lift-ranges, or the reverse — and load fine there, because the filesystem folds case. On Linux the sibling candidate is built from the .lift's own suffix, so it missed, the companion was skipped without a word, and every range it defined went absent. Candidates that match no file exactly now fall back to one whose name differs only in case. The fallback is reached only after an exact miss, so a case-folding filesystem never enters it and behaves as before; a case-sensitive one gets one directory read per folder, cached across the candidate list. Where several names fold together the lexicographically first wins. The choice is arbitrary but fixed, which matters more than which file it picks: directory order varies between filesystems and runs, and a companion that loads differently on consecutive reads would be worse than one that never loads. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
0.1.0 has not shipped, so there is no released behavior for an Unreleased entry to be fixing — the tolerance is simply part of what companion discovery does in the first release. Fold it into that bullet. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The directory listing the fallback builds called its files "entries", the word this module uses for a LIFT <entry> everywhere else — the same collision that keeps byte regions from being called spans. Name them files. Spell the surrounding prose the way the rest of the package does: a fallback that runs rather than fires, a name that matched no file rather than missed, a helper named for the filesystem it probes rather than abbreviating it, and fixture names deliberately not taken from the corpus file. Unpack the two densest clauses so each reads in one pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
imnasnainaec
force-pushed
the
ranges-companion-filename-case
branch
from
August 18, 2026 20:23
cd9cb67 to
a565d67
Compare
Companion lookup folds names with casefold() over NFC rather than lower(), so a Turkish-cased or NFD-spelled name resolves the way it does on the filesystem that wrote it — FLEx mixes normalization forms within one export. Ties break in code point order on every platform; sorting Path objects left the choice to directory order on Windows, where PurePath ordering is itself case-folded. An unstattable exact spelling now falls through to the folded lookup instead of giving up. A candidate that folds onto the .lift itself is skipped: RangesFile.load rejects a <lift> root, so a header href naming the lexicon in another case took the whole load down. One that folds onto a companion already tracked is skipped too — Path.resolve() leaves case alone on macOS, so a single file reached under two spellings was loaded and tracked twice, and written twice by save(). The sibling candidate is built with with_name, which agrees with with_suffix on every name that has an extension and does not raise on a name without one. Nothing upstream requires the .lift extension: parse_document never inspects it. dangling-ranges-href decides existence with that same lookup, so a companion spelled in another case is no longer reported missing on a case-sensitive filesystem. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The 0.1.0 entry said companion names fold on case alone; they fold on Unicode normalization form as well. The folder guide listed the candidates tried but never mentioned the folding at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The three helpers each stated a rule, defended it, then disclaimed it. What is left is the reasoning the code cannot show: casefold over lower, NFC, why only the final component folds, why code point order, and what the fold pre-check protects the inode comparison from. The folder guide drops the folding sentence outright — it describes behavior no reader acts on, in a paragraph otherwise about which candidate wins. The 0.1.0 entry keeps the fact and loses the justification, which now lives only in the docstrings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A LIFT folder written on Windows can spell its pair inconsistently —
Dict.LIFTbesideDict.lift-ranges, or the reverse — and load fine there, because the filesystem folds case. On Linux the sibling candidate is built from the.lift's own name (_model.py:536), so it missed, the companion was skipped silently, and every range it defined went absent.Not zip-specific:
sil_lift.load("/data/Foo.LIFT")on Linux hits it with no archive involved, sinceparse_documentnever inspects the extension.Approach
A candidate that matches no file exactly now falls back to one whose name folds onto it.
range/@hrefentries doesn't re-scan.casefold()over NFC, notlower(). FLEx writes the.liftin NFC and its companion in NFD within one export, and only APFS folds the two forms on its own; NTFS does not, so a mismatched pair fails to resolve on Windows as well.dangling-ranges-hrefdecides existence the same way, so validation and loading agree on what is there.Two things the fallback would otherwise have widened:
.liftitself — a header href spellingDict.liftbeside aDict.LIFT— is skipped.RangesFile.loadrejects a<lift>root, so it would fail the whole load. Reachable on Windows before this PR, by an href naming the lexicon at all.Path.resolve()leaves case alone on macOS, so one file reached under two spellings was loaded and tracked twice — and written twice bysave().Also in the area: the sibling candidate is built with
with_namerather thanwith_suffix, which raisedValueError: Invalid suffix '-ranges'for a document loaded under a name with no extension. The two agree on every name that has one.The tie-break
Where several names fold together, the first in code point order wins (
Dict.LIFT-rangesahead ofDict.lift-ranges). Arbitrary, but fixed — and fixed is the property that matters: directory order varies between filesystems and between runs, and a companion that resolved differently on consecutive reads would be worse than one that never resolved. The sort key is the name rather than thePath, sincePurePathordering is itself case-folded on Windows and would leave the choice to directory order there. Documented in_existing_fileand asserted by a test.Open to a different rule if you'd rather have one — e.g. preferring an exact case match on the stem — but I'd want it to stay deterministic.
Tests
Eight in
test_ranges_folder.py: uppercase.lift, uppercase companion, an NFC/NFD filename mismatch, the tie-break, a negative case pinning that the fallback doesn't reach past a folder for a name that isn't in it, an extensionless.lift, an href folding onto the lexicon itself, and a case-variant companion that must not be reporteddangling-ranges-href.Only the tie-break needs both spellings to coexist, so it skips off Linux. The normalization test exercises the fallback on Windows and Linux alike (APFS resolves it without one), and the two self-reference tests are platform-independent.
python scripts/check.pygreen: 545 passed, 1 skipped (the tie-break, on Windows), 97% coverage.Draft: opening for a read on the tie-break rule before marking ready.
🤖 Generated with Claude Code
Devin review: https://app.devin.ai/review/sillsdev/python-sil-lift/pull/19
This change is