fix(cli): name the real --tp-size flag in the 2D-parallelism error - #1129
Merged
Conversation
The 2D (PP x TP) validator in `validate_pipeline_parallel_args` rejected a `--tp-size > 1` run without a pipeline topology by telling the user to pass `--tensor-parallel-size > 1`. No binary has ever accepted that spelling, so following the instruction produced `error: unexpected argument '--tensor-parallel-size' found`. The rest of the sentence named `--pp-size` and `--pp-layers`, both real, which made the one wrong name more misleading than a vague message: the two names a reader could check both held up. The same wrong spelling was also in `tests/pp_tp_2d_real_models.rs`, and that is why nothing caught this. `pp_tp_2d_validator_accepts_combination` is not `#[ignore]`d, so it runs, but it passed vacuously: clap rejected `--tensor-parallel-size` before the process ever reached the validator, and the test's only assertion was that a specific old rejection string is absent from the output, which trivially held. The test now passes `--tp-size` and asserts up front that the arguments were not rejected at parse time, so it cannot pass again without reaching the validator it claims to exercise. Also repoints the comment above the validator. It cited `docs/en/distributed/pipeline-parallelism.md` and `docs/en/distributed/tensor-parallelism.md`; `docs/en/` has never existed in this repository, and those were the only two `docs/en/` references in `src/`. The real operator guide is `docs/distributed.md`, which documents the PP and TP knobs separately but does not yet write up the 2D composition, so the comment says that rather than implying coverage it does not have. Swept every flag named anywhere in `src/commands/generate.rs` against the built binary's `mlxcel generate --help` (64 accepted long flags). `--tensor-parallel-size` was the only one the binary does not accept; `--pp-micro-batch-size`, `--pp-size`, `--pp-layers`, `--estimate-memory`, `--no-memory-check`, `--max-tokens`, `--recommend-quant` and `--surgery` all resolve. Verified: `cargo test --profile test-fast --features cuda --test pp_tp_2d_real_models` passes (1 passed, 1 ignored); the binary rejects `--tensor-parallel-size` and accepts `--pp-size 2 --tp-size 2`; `cargo fmt --all -- --check` clean. Closes #1112
Three corrections from review, two of which invalidated claims the first commit made. The integration test still could not reach the validator, and the first commit made it worse. `run_generate` resolves `-m` before calling `validate_pipeline_parallel_args`, because the validators read the resolved model directory, so a subprocess invocation cannot reach the validator without a real model on disk. The test's `-m` value is a valid bare repo segment, so once the flag name was fixed and clap stopped rejecting the argv at parse time, the resolver expanded it against `$MLXCEL_DEFAULT_ORG` and went to HuggingFace: running the exact argv fails with `authentication failed (HTTP 401)` after an outbound request, in a test that is not `#[ignore]`d. The test is now scoped to what a subprocess can actually check, that the parser accepts `--pp-size 2 --tp-size 2`, by appending `--help` so clap exits as soon as parsing succeeds. It asserts positively on exit status rather than on the absence of a string, so it cannot pass by dying early. Verified both ways against the built binary: with `--tp-size` it exits 0, with `--tensor-parallel-size` it exits 2. Runtime dropped from 1.22s to 0.09s because the network call is gone. The validator keeps its direct unit coverage in `validate_pipeline_parallel_args_accepts_2d_pp_tp`. The `docs/en/` paths were not broken references. `mkdocs.yml` sets `docs_dir: docs/en` and its nav names `distributed/tensor-parallelism.md` and `distributed/pipeline-parallelism.md`, which resolve to exactly the two paths the comment cited. They are pages of the operator manual whose sources are maintained in a separate documentation repository; `docs/README.md` calls that split "deliberate, not drift" and the `docs-guard` Makefile target from #1122 is built on the same fact. Deleting them is the exact failure #1122's report listed as a high-impact risk. The comment now keeps both manual pages, explains why their sources are not in this tree, and names `docs/distributed.md` as the in-checkout summary. This knowingly does not satisfy the issue's "no `docs/en/` reference remains in `src/`" criterion, which rested on a false premise. The CHANGELOG entry is removed rather than reworded. The `ensure!` condition is the exact negation of the early return above it, so the guard can never fail and the message can never be emitted; the repository's own `validate_pipeline_parallel_args_rejects_2d_without_pp_enabled` asserts `is_ok()` for that input. No user has seen this message, so there is no user-visible change to record. The unreachable guard is left alone, since changing it would violate the issue's "validation logic is unchanged" criterion, and is filed as a follow-up. Reports rewritten to match. Refs #1112
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The 2D (PP x TP) guard in
validate_pipeline_parallel_argsnamed--tensor-parallel-size, a spelling no mlxcel binary accepts. It now names--tp-size. Review turned up two things that matter more than the original defect, and both changed what this PR does.The guard is unreachable. Its
ensure!condition is the exact negation of the early return eight lines above it, so it can never fail and the message can never be emitted. The repository's ownvalidate_pipeline_parallel_args_rejects_2d_without_pp_enabledassertsis_ok()forpp_size = 1, tp_size = 2and its comment says the validator returns early. No user has ever seen this message, so this is a latent text fix with no user-visible change, and it ships with no CHANGELOG entry. The dead condition is left alone (the issue's fourth criterion is that the validation logic is unchanged) and filed as a follow-up.The
docs/en/paths are not broken references. The issue's premise was wrong and I initially acted on it. Details below.What changed
src/commands/generate.rs: theensure!message names--tp-size. Message text only; the condition, thetotal_ranksguard, and every other arm of the function are byte-identical.src/commands/generate.rs: the validator comment keeps both operator-manual pages, explains that their sources live in the separate documentation repository by design, and addsdocs/distributed.mdas the in-checkout summary.tests/pp_tp_2d_real_models.rs: both tests used--tensor-parallel-sizeon the command line. The non-ignored one is rescoped, made hermetic, and now asserts positively. Details below.The test was vacuous, and the obvious fix made it worse
pp_tp_2d_validator_accepts_combinationis not#[ignore]d, so it ran in CI, but it passed vacuously: clap rejected--tensor-parallel-sizeat argument parsing, and the test's only assertion was that a particular old rejection string is absent from the output, which is trivially true of a process that died before producing any.My first attempt fixed the flag name and added a guard asserting the args were not rejected at parse time. That was still wrong, because a subprocess cannot reach this validator at all.
run_generateresolves-mfirst, since the validators read the resolved model directory:And the test's
-m nonexistent-model-path-for-validator-only-checkis a valid bare repo segment, so once clap stopped rejecting the argv, the resolver expanded it against$MLXCEL_DEFAULT_ORGand went to the network. Running the exact argv:Exit 1, before either validator, after an outbound HuggingFace request on every CI run. An offline runner would eat a connect timeout and still pass.
The shipped version appends
--help, so clap exits as soon as parsing succeeds, and asserts positively on exit status. It is hermetic and cannot pass by dying early. Verified both directions against the built binary:--pp-size 2 --tp-size 2 --helpexits 0.--pp-size 2 --tensor-parallel-size 2 --helpexits 2 witherror: unexpected argument '--tensor-parallel-size' found.Runtime went from 1.22s to 0.09s because the network call is gone. Nothing is lost by narrowing the scope: the validator has direct unit coverage in
validate_pipeline_parallel_args_accepts_2d_pp_tp. This also restores what the test's own comment always claimed it did; the code had drifted from the comment.The
docs/en/references were real, and one acceptance criterion is knowingly not metThe issue asserted
docs/en/"has never been part of this repository" and made "nodocs/en/reference remains insrc/" a criterion. True of this git tree, but the conclusion does not follow:Under
docs_dir: docs/enthose resolve to exactly the two paths the comment cited.docs/README.mdstates thatdocs/en,docs/koanddocs/sharedare maintained in a separate documentation repository and that the root mkdocs configs name paths into it, verbatim: "That is deliberate, not drift." Thedocs-guardMakefile target from #1122 is built on the same fact, and that PR's own report lists "someone fixes the dangling navs by pointing them atdocs/*.md, breaking the tree that owns them" as a high-impact risk. Deleting the references would have been that exact mistake.So the comment keeps both manual pages and explains the split, and
docs/distributed.mdis named alongside as the in-checkout summary (it covers the PP and TP knobs in separate sections and has no 2D section, so it does not replace them). The third acceptance criterion is deliberately not satisfied, and the second only in the sense that no broken reference remains.The sweep
Every
--flagtoken insrc/commands/generate.rswas diffed against the 64 long flags the built binary advertises, rather than read by eye:That was the only one.
--pp-micro-batch-size,--pp-size,--pp-layers,--estimate-memory,--no-memory-check,--max-tokens,--recommend-quantand--surgeryall resolve.Test plan
MLX_CUDA_ARCHITECTURES=121 cargo test --profile test-fast --features cuda --test pp_tp_2d_real_models: 1 passed, 1 ignored, 0 failed.grep -rn -- "--tensor-parallel-size" src/returns nothing.cargo fmt --all -- --checkclean.cargo clippy --profile test-fast --features cuda --lib --tests -- -D warningsclean.Acceptance criteria
Nodeliberately not met: the paths name real manual pages, see above. No broken reference remains.docs/en/reference remains insrc/grep -rn -- "--tensor-parallel-size" src/returns nothing.Follow-ups
ensure!, andvalidate_pipeline_parallel_args_rejects_2d_without_pp_enabled, whose name says "rejects" while it assertsis_ok().tests/cli_help_consistency.rsso the sweep above is enforced rather than performed once.Closes #1112