Skip to content

fix(cli): name the real --tp-size flag in the 2D-parallelism error - #1129

Merged
inureyes merged 3 commits into
mainfrom
fix/issue-1112-cli-flag-name-in-2d-error
Aug 13, 2026
Merged

fix(cli): name the real --tp-size flag in the 2D-parallelism error#1129
inureyes merged 3 commits into
mainfrom
fix/issue-1112-cli-flag-name-in-2d-error

Conversation

@inureyes

@inureyes inureyes commented Aug 13, 2026

Copy link
Copy Markdown
Member

Summary

The 2D (PP x TP) guard in validate_pipeline_parallel_args named --tensor-parallel-size, a spelling no mlxcel binary accepts. It now names --tp-size. Review turned up two things that matter more than the original defect, and both changed what this PR does.

The guard is unreachable. Its ensure! condition is the exact negation of the early return eight lines above it, so it can never fail and the message can never be emitted. The repository's own validate_pipeline_parallel_args_rejects_2d_without_pp_enabled asserts is_ok() for pp_size = 1, tp_size = 2 and its comment says the validator returns early. No user has ever seen this message, so this is a latent text fix with no user-visible change, and it ships with no CHANGELOG entry. The dead condition is left alone (the issue's fourth criterion is that the validation logic is unchanged) and filed as a follow-up.

The docs/en/ paths are not broken references. The issue's premise was wrong and I initially acted on it. Details below.

What changed

  • src/commands/generate.rs: the ensure! message names --tp-size. Message text only; the condition, the total_ranks guard, and every other arm of the function are byte-identical.
  • src/commands/generate.rs: the validator comment keeps both operator-manual pages, explains that their sources live in the separate documentation repository by design, and adds docs/distributed.md as the in-checkout summary.
  • tests/pp_tp_2d_real_models.rs: both tests used --tensor-parallel-size on the command line. The non-ignored one is rescoped, made hermetic, and now asserts positively. Details below.

The test was vacuous, and the obvious fix made it worse

pp_tp_2d_validator_accepts_combination is not #[ignore]d, so it ran in CI, but it passed vacuously: clap rejected --tensor-parallel-size at argument parsing, and the test's only assertion was that a particular old rejection string is absent from the output, which is trivially true of a process that died before producing any.

My first attempt fixed the flag name and added a guard asserting the args were not rejected at parse time. That was still wrong, because a subprocess cannot reach this validator at all. run_generate resolves -m first, since the validators read the resolved model directory:

args.model.model =
    resolve_model_source_with_override(&args.model.model, args.model.models_dir.as_deref())?;

validate_tensor_parallel_args(&args)?;
validate_pipeline_parallel_args(&args)?;

And the test's -m nonexistent-model-path-for-validator-only-check is a valid bare repo segment, so once clap stopped rejecting the argv, the resolver expanded it against $MLXCEL_DEFAULT_ORG and went to the network. Running the exact argv:

[mlxcel] 'nonexistent-model-path-for-validator-only-check' -> mlx-community/nonexistent-...
[mlxcel] model '...' not found locally; downloading into the mlxcel store...
Error: failed to download model '...': authentication failed (HTTP 401).

Exit 1, before either validator, after an outbound HuggingFace request on every CI run. An offline runner would eat a connect timeout and still pass.

The shipped version appends --help, so clap exits as soon as parsing succeeds, and asserts positively on exit status. It is hermetic and cannot pass by dying early. Verified both directions against the built binary:

  • --pp-size 2 --tp-size 2 --help exits 0.
  • --pp-size 2 --tensor-parallel-size 2 --help exits 2 with error: unexpected argument '--tensor-parallel-size' found.

Runtime went from 1.22s to 0.09s because the network call is gone. Nothing is lost by narrowing the scope: the validator has direct unit coverage in validate_pipeline_parallel_args_accepts_2d_pp_tp. This also restores what the test's own comment always claimed it did; the code had drifted from the comment.

The docs/en/ references were real, and one acceptance criterion is knowingly not met

The issue asserted docs/en/ "has never been part of this repository" and made "no docs/en/ reference remains in src/" a criterion. True of this git tree, but the conclusion does not follow:

mkdocs.yml:8:docs_dir: docs/en
mkdocs.yml:160:      - Tensor Parallelism: distributed/tensor-parallelism.md
mkdocs.yml:161:      - Pipeline Parallelism: distributed/pipeline-parallelism.md

Under docs_dir: docs/en those resolve to exactly the two paths the comment cited. docs/README.md states that docs/en, docs/ko and docs/shared are maintained in a separate documentation repository and that the root mkdocs configs name paths into it, verbatim: "That is deliberate, not drift." The docs-guard Makefile target from #1122 is built on the same fact, and that PR's own report lists "someone fixes the dangling navs by pointing them at docs/*.md, breaking the tree that owns them" as a high-impact risk. Deleting the references would have been that exact mistake.

So the comment keeps both manual pages and explains the split, and docs/distributed.md is named alongside as the in-checkout summary (it covers the PP and TP knobs in separate sections and has no 2D section, so it does not replace them). The third acceptance criterion is deliberately not satisfied, and the second only in the sense that no broken reference remains.

The sweep

Every --flag token in src/commands/generate.rs was diffed against the 64 long flags the built binary advertises, rather than read by eye:

$ grep -o -- '--[a-z][a-z0-9-]*' src/commands/generate.rs | sort -u \
    | comm -23 - <(mlxcel generate --help | grep -o -- '--[a-z][a-z0-9-]*' | sort -u)
--tensor-parallel-size

That was the only one. --pp-micro-batch-size, --pp-size, --pp-layers, --estimate-memory, --no-memory-check, --max-tokens, --recommend-quant and --surgery all resolve.

Test plan

  • MLX_CUDA_ARCHITECTURES=121 cargo test --profile test-fast --features cuda --test pp_tp_2d_real_models: 1 passed, 1 ignored, 0 failed.
  • Hermetic: no network request; the argv exits inside clap.
  • Non-vacuous, verified in both directions against the built binary (exit 0 vs exit 2, above).
  • grep -rn -- "--tensor-parallel-size" src/ returns nothing.
  • cargo fmt --all -- --check clean.
  • cargo clippy --profile test-fast --features cuda --lib --tests -- -D warnings clean.

Acceptance criteria

  • Every flag named in the message is one the binary accepts.
  • No docs/en/ reference remains in src/ deliberately not met: the paths name real manual pages, see above. No broken reference remains.
  • grep -rn -- "--tensor-parallel-size" src/ returns nothing.
  • Message text only; the validation logic is unchanged.

Follow-ups

  • The unreachable ensure!, and validate_pipeline_parallel_args_rejects_2d_without_pp_enabled, whose name says "rejects" while it asserts is_ok().
  • Machine-check the flag names in tests/cli_help_consistency.rs so the sweep above is enforced rather than performed once.

Closes #1112

The 2D (PP x TP) validator in `validate_pipeline_parallel_args` rejected a `--tp-size > 1` run without a pipeline topology by telling the user to pass `--tensor-parallel-size > 1`. No binary has ever accepted that spelling, so following the instruction produced `error: unexpected argument '--tensor-parallel-size' found`. The rest of the sentence named `--pp-size` and `--pp-layers`, both real, which made the one wrong name more misleading than a vague message: the two names a reader could check both held up.

The same wrong spelling was also in `tests/pp_tp_2d_real_models.rs`, and that is why nothing caught this. `pp_tp_2d_validator_accepts_combination` is not `#[ignore]`d, so it runs, but it passed vacuously: clap rejected `--tensor-parallel-size` before the process ever reached the validator, and the test's only assertion was that a specific old rejection string is absent from the output, which trivially held. The test now passes `--tp-size` and asserts up front that the arguments were not rejected at parse time, so it cannot pass again without reaching the validator it claims to exercise.

Also repoints the comment above the validator. It cited `docs/en/distributed/pipeline-parallelism.md` and `docs/en/distributed/tensor-parallelism.md`; `docs/en/` has never existed in this repository, and those were the only two `docs/en/` references in `src/`. The real operator guide is `docs/distributed.md`, which documents the PP and TP knobs separately but does not yet write up the 2D composition, so the comment says that rather than implying coverage it does not have.

Swept every flag named anywhere in `src/commands/generate.rs` against the built binary's `mlxcel generate --help` (64 accepted long flags). `--tensor-parallel-size` was the only one the binary does not accept; `--pp-micro-batch-size`, `--pp-size`, `--pp-layers`, `--estimate-memory`, `--no-memory-check`, `--max-tokens`, `--recommend-quant` and `--surgery` all resolve.

Verified: `cargo test --profile test-fast --features cuda --test pp_tp_2d_real_models` passes (1 passed, 1 ignored); the binary rejects `--tensor-parallel-size` and accepts `--pp-size 2 --tp-size 2`; `cargo fmt --all -- --check` clean.

Closes #1112
@inureyes inureyes added type:bug Bug fixes, error corrections, or issue resolutions priority:low Low priority status:review Under review labels Aug 13, 2026
Three corrections from review, two of which invalidated claims the first commit made.

The integration test still could not reach the validator, and the first commit made it worse. `run_generate` resolves `-m` before calling `validate_pipeline_parallel_args`, because the validators read the resolved model directory, so a subprocess invocation cannot reach the validator without a real model on disk. The test's `-m` value is a valid bare repo segment, so once the flag name was fixed and clap stopped rejecting the argv at parse time, the resolver expanded it against `$MLXCEL_DEFAULT_ORG` and went to HuggingFace: running the exact argv fails with `authentication failed (HTTP 401)` after an outbound request, in a test that is not `#[ignore]`d. The test is now scoped to what a subprocess can actually check, that the parser accepts `--pp-size 2 --tp-size 2`, by appending `--help` so clap exits as soon as parsing succeeds. It asserts positively on exit status rather than on the absence of a string, so it cannot pass by dying early. Verified both ways against the built binary: with `--tp-size` it exits 0, with `--tensor-parallel-size` it exits 2. Runtime dropped from 1.22s to 0.09s because the network call is gone. The validator keeps its direct unit coverage in `validate_pipeline_parallel_args_accepts_2d_pp_tp`.

The `docs/en/` paths were not broken references. `mkdocs.yml` sets `docs_dir: docs/en` and its nav names `distributed/tensor-parallelism.md` and `distributed/pipeline-parallelism.md`, which resolve to exactly the two paths the comment cited. They are pages of the operator manual whose sources are maintained in a separate documentation repository; `docs/README.md` calls that split "deliberate, not drift" and the `docs-guard` Makefile target from #1122 is built on the same fact. Deleting them is the exact failure #1122's report listed as a high-impact risk. The comment now keeps both manual pages, explains why their sources are not in this tree, and names `docs/distributed.md` as the in-checkout summary. This knowingly does not satisfy the issue's "no `docs/en/` reference remains in `src/`" criterion, which rested on a false premise.

The CHANGELOG entry is removed rather than reworded. The `ensure!` condition is the exact negation of the early return above it, so the guard can never fail and the message can never be emitted; the repository's own `validate_pipeline_parallel_args_rejects_2d_without_pp_enabled` asserts `is_ok()` for that input. No user has seen this message, so there is no user-visible change to record. The unreachable guard is left alone, since changing it would violate the issue's "validation logic is unchanged" criterion, and is filed as a follow-up.

Reports rewritten to match.

Refs #1112
@inureyes inureyes added status:done Completed and removed status:review Under review labels Aug 13, 2026
@inureyes
inureyes merged commit 33ffc7f into main Aug 13, 2026
8 checks passed
@inureyes
inureyes deleted the fix/issue-1112-cli-flag-name-in-2d-error branch August 13, 2026 18:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

priority:low Low priority status:done Completed type:bug Bug fixes, error corrections, or issue resolutions

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(cli): 2D-parallelism error tells the user to pass --tensor-parallel-size, which does not exist

1 participant