fix(cli): IDEs with plugins using BSP can now use aspect cli instead of vanilla bazel - #1370
fix(cli): IDEs with plugins using BSP can now use aspect cli instead of vanilla bazel#1370mcook42 wants to merge 7 commits into
Conversation
The runner only suppressed its default target pattern for aspect's own
`--target-pattern-file` arg. Bazel's own spelling arriving as
`--bazel-flag=--target_pattern_file=<path>` was invisible to it, so
`ctx.args.targets` fell back to its default and Bazel rejected the
invocation outright:
ERROR: Command-line target pattern and --target_pattern_file
cannot both be specified
That is exactly the shape the IntelliJ Bazel plugin produces: the
`tools/bazel` wrapper rewrites every Bazel-native flag to
`--bazel-flag=`, so an IDE sync never reaches Bazel.
Suppress the default when the caller forwarded the flag and gave no
explicit patterns. Only the *default* is suppressed — explicit patterns
still reach Bazel alongside the flag so Bazel emits the error above
itself, rather than this runner growing a second spelling of it.
The new `forwarded_flag_value` helper reads `ctx.args.bazel_flags` rather
than the parsed rc because target patterns are resolved before
`parse_rc` runs: the resolved pattern file has to be in the `base_flags`
the rc is built from. `--target_pattern_file` is per-invocation and not
rc material, so nothing is lost.
`Build::spawn` appends the CLI's own `--build_event_binary_file` after every user flag. Bazel's option is single-valued, so last-wins meant a caller who asked for a BEP file silently got nothing — the file was created by whoever made the temp path and never written to. The IntelliJ Bazel plugin drives sync on exactly that flag, so even with the target-pattern collision fixed it would parse an empty file. Collect a file sink for the caller's path instead of stripping or reordering flags. The CLI's path still wins on the command line; the caller's file is re-created from Bazel's own byte stream, since file sinks share the BES reader's raw-bytes path. Placed in `collect_bes_sinks` so all eight bazel-spawning tasks are covered — every one of them hit the same clobbering. Two consequences: `--build_event_binary_file_upload_mode` no longer governs that file (the caller's existing `sink.wait()` completes it, before the task concludes), and `--build_event_json_file` / `--build_event_text_file` are untouched — different flags, so Bazel still writes those itself.
`BAZEL_VERBS` was generated from `bazel help`'s "Available commands" list, which hides `config`. A verb missing from that table is treated as a custom aspect task, so `bazel config` became `aspect config` → `error: unrecognized subcommand 'config'`. The IntelliJ Bazel plugin calls `bazel config --dump_all --output=json` during sync and parses stdout as JSON, so this failed the sync outright. Regenerate from `bazel help completion`'s `BAZEL_COMMAND_LIST`, which includes hidden commands — verified against Bazel 9.0.1, `config` is the sole difference. Record the regeneration recipe and *why* it must not be `bazel help` in the comment, and in tools/bazel.md. The gap also silently degraded the pre-verb disambiguation walk, since KNOWN_VERBS_STR is built from the same table.
The unit tests cover the two resolvers, but nothing exercised the pair end to end: that the composed bazel command line is one Bazel accepts, and that the caller's BEP file actually lands on disk. Extend the existing `test-flags-task` step in both pipelines — it already writes the pattern file — with one `aspect build` in the shape the IntelliJ Bazel plugin produces (Bazel's own flag spellings behind `--bazel-flag`), asserting a non-empty BEP at the caller's path.
Both are generated: .idea/ by the IDE, .bazelbsp/ by the JetBrains Bazel plugin (its injected aspect .bzl files), and both showed as untracked after an IntelliJ sync.
✨ Aspect Workflows Tasks📅 Wed Aug 5 14:37:45 UTC 2026 ❌ 1 failed task
|
| """ | ||
| eq_prefix = name + "=" | ||
| value = "" | ||
| for i, flag in enumerate(bazel_flags): |
There was a problem hiding this comment.
this sounds really unprincipled/workaroubd. can we understand what's really wrong and fix it?
I'd rather let bazel decide what's last to win rather than us deciding it.
There was a problem hiding this comment.
It was a workaround, and I should have caught this before posting, so thanks for holding me accountable.
The runner already avoided sending a target pattern when Aspect's own --target-pattern-file flag was set. The bug is that it only checked that one spelling. When the pattern file arrives as a forwarded --bazel-flag=--target_pattern_file=<path>, as an IntelliJ sync sends, the runner never sees it and falls back to its default pattern of .... Bazel then gets both that pattern and the pattern file and refuses to run, since it has no last-wins rule for that pair.
The fix is to check the flags Bazel will actually run with, rather than a specific arg. We now ask the parsed command with rc.flag_value("--target_pattern_file", ...), which detects the flag regardless of how it arrived, and skips sending a pattern if it is set. If the user typed patterns, we always send them and let Bazel report the conflict, so our own error in that case is removed.
Getting to that point took a reordering. The parsed command did not yet exist when patterns were being decided, because we were adding Aspect's own pattern file flag to the same list of flags that gets parsed. Moving that flag out of that list let target resolution happen after the parse, which is what makes rc.flag_value available to ask.
This reverts commit c993267.
|
This seems like 3 separate issues that can have independent fixes? |
You're right that these are three separate issues, and I could split them into three separate PRs. However, until all three of these issues are fixed, the IntelliJ/JetBrains Bazel IDE plugin won't work. The three fixes in this PR need to land before the plugin works with aspect CLI, so in my mind it was best to land them all in a single review. I'm happy to close this PR and open three independent PRs for smaller reviews if that's preferred. |
`run_bazel_task` only suppressed its target-pattern default for aspect's own
`--target-pattern-file` arg. Bazel's spelling reaching aspect as
`--bazel-flag=--target_pattern_file=<path>` — what `tools/bazel` produces for
every Bazel-native flag, and so what an IntelliJ BSP sync sends — lands in
`ctx.args.bazel_flags` instead, invisible there. `ctx.args.targets` then fell
back to its declared default `["..."]`, which reaches Bazel as residue
alongside the forwarded flag:
ERROR: Command-line target pattern and --target_pattern_file
cannot both be specified
That pair has no last-wins rule in Bazel — it is a hard conflict. So the only
decision this runner owns is whether to invent a pattern the user never typed;
every precedence question belongs to Bazel's own option parser.
A `RunCommand` is the *effective* option set, not just the rc file — CLI flags
live in it as the `<command line>` source — so
`rc.flag_value("--target_pattern_file", ...)` answers for the forwarded
spelling with the same last-wins / `=`-form / two-token matching as
`crates/bazelrc`. Reaching it only needed the ordering fixed: aspect's own
`--target-pattern-file` moves from `base_flags` (an *input* to `parse_rc`) to
the per-invocation `flags`, freeing patterns to resolve after the rc parse.
That also keeps the flag out of the rc's `always` bucket, so it is no longer
expanded for every command.
Explicit patterns are always forwarded, so Bazel emits the error above itself
and the runner's second spelling of it is dropped. The existence check on
`--target-pattern-file` stays, since aspect resolves that path.
Supersedes the reverted `forwarded_flag_value` scan (c993267 / 2af86d1),
which reimplemented Bazel's option parsing in AXL.
d50e9df to
f7876f0
Compare
The JetBrains Bazel plugin (
bazelbsp:3.2.0) drives IDE sync by shelling out tobazelonPATH. In a repo with this project'stools/bazelhook, that hook routesbuild/testtoaspectand rewrites every Bazel-native flag to--bazel-flag=…. Three independent defects in that path caused an IntelliJ sync to fail outright, so the only way to use the plugin was to setASPECT_WRAPPER_SKIP=1and give up aspect's BES streaming, remote config, and task reporting entirely. See the Build Server Protocol (BSP) used by the plugin, if curious.Each fix is a separate commit.
1. A caller-forwarded
--target_pattern_filecollided with the default target patternrun_bazel_tasksuppressed its default target pattern only for aspect's own--target-pattern-filearg. Bazel's spelling arrives as--bazel-flag=--target_pattern_file=<path>, which was invisible to the runner, soctx.args.targetsfell back to its default. Bazel rejected the invocation before doing any work:Both sync passes died here. That pair has no last-wins rule in Bazel — it is a hard conflict — so the only decision this runner owns is whether to invent a pattern the user never typed. Every precedence question belongs to Bazel's own option parser.
A new
target_patternshelper inbazel_runner.axlmakes exactly that one decision, and asksrc.flag_value("--target_pattern_file", command = …)whether a pattern file is in effect. ARunCommandis the effective option set rather than just the rc file — CLI flags live in it as the synthetic<command line>source — so that answers for the forwarded spelling using the same last-wins /=-form / two-token matching ascrates/bazelrc, and picks up a pattern file from.bazelrcor a--configexpansion for free.Reaching it only required the ordering to be fixed. Aspect's own
--target-pattern-filewas appended tobase_flags, which is an input toparse_rc, forcing patterns to resolve before anyrcexisted. It now goes through the per-invocationflagsinstead, so resolution moves after the rc parse. Two consequences, both wanted:alwaysbucket, so it is no longer expanded for every command — including any sidecarquery/inforeusing the active run command (queryhas no--target_pattern_file; it uses--query_file).--target-pattern-filenow comes after a forwarded--bazel-flag=--target_pattern_file, so Bazel's last-wins picks the aspect arg when both are given.Only the default is suppressed. An explicit pattern still reaches Bazel alongside the flag, so Bazel itself emits the error above — and the runner's own
fail("--target-pattern-file cannot be combined with command-line target patterns")is dropped, leaving one spelling of that error. The existence check on--target-pattern-filestays, since aspect resolves that path itself.2. A caller's
--build_event_binary_filewas silently never writtenBuild::spawnappends the CLI's own--build_event_binary_fileafter every user flag. Bazel's option is single-valued, so last-wins meant a caller who asked for a BEP file got a path that was created but never written to. The plugin parses sync results from that file, so even with (1) fixed, it would have read an empty file.Fixed by collecting a file sink for the caller's path in
collect_bes_sinks, rather than stripping or reordering flags. The CLI's path still wins on the command line; the caller's file is recreated from Bazel's own byte stream, since file sinks share the BES reader's raw-bytes path. Reordering was not an option in any case — the CLI's own "file" is a named pipe it reads the stream from, not a file on disk.Placing it in
collect_bes_sinkscentralizes every bazel-spawning task; all of them hit the same clobbering. Deliberately not routed through_drop_bazel_streamed: a local file dump is not a second upload to an endpoint Bazel already streams to, and that helper's contract already keeps items whoseuri_ofis empty.Two consequences, both documented in the docstring:
--build_event_binary_file_upload_modeno longer governs that file. The caller's existingsink.wait()completes it before the task concludes — which is what the plugin'swait_for_upload_completewas asking for anyway.--build_event_json_file/--build_event_text_fileare untouched; different flags, so Bazel still writes those itself.3.
bazel configwas misrouted toaspectBAZEL_VERBSintools/bazelwas generated frombazel help's "Available commands" list, which hidesconfig. A verb missing from that table is treated as a custom aspect task and routed toaspect. The plugin callsbazel config --dump_all --output=jsonduring sync and parses stdout as JSON, so this failed the sync on its own.The authoritative list is
bazel help completion'sBAZEL_COMMAND_LIST, which includes hidden commands — verified against Bazel 9.0.1,configis the sole difference. The regeneration recipe and why it must not bebazel helpare now recorded in both the script comment andtools/bazel.md:The gap also silently degraded the pre-verb disambiguation walk, since
KNOWN_VERBS_STRis built from the same table.Also ignores
.idea/and.bazelbsp/, both of which are generated by the plugin.Changes are visible to end-users: yes
tools/bazel.mddocumentsBAZEL_VERBSand now carries the regeneration source and rationaleThe Aspect CLI now serves as the
bazelbinary for IDE and BSP tooling, such as the IntelliJ Bazel plugin. A caller-forwarded--target_pattern_fileno longer collides with the CLI's default target pattern; a caller's--build_event_binary_fileis now written instead of being silently clobbered by the CLI's own; andbazel configreaches Bazel through thetools/bazelwrapper rather than erroring as an unknown aspect subcommand. An IntelliJ sync no longer needsASPECT_WRAPPER_SKIP=1, so IDE-driven builds keep BES streaming, remote config, and task reporting.Test plan
target_patternsinbazel_runner_test.axl(default forwarded when nothing supplies patterns; suppressed by a forwarded flag; suppressed by aspect's own arg; explicit patterns always forwarded; the lookup is command-scoped). The=-form / two-token / last-wins matching is deliberately not re-asserted against a fakedrc— that behavior isflag_value_list's and is covered incrates/bazelrc/src/lib.rs(flag_value_eq_form_last_wins,flag_value_two_token_form). Plusbazel_bep_fileresolution and thecollect_bes_sinksfile-sink append inbazel/build_events_test.axlaspect tests axl(910 cases) and the fullaspect testsuite (26/26) both passtest-flags-taskstep in both.buildkite/pipeline.yamland.github/workflows/ci-workflows.yamlnow runs a singleaspect buildin the plugin's exact shape and asserts a non-empty BEP at the caller's path. The unit tests cover the two resolvers in isolation; this asserts that the composed command line is accepted by Bazel and that the file lands on disk.The failure mode was captured on
mainfirst, so each result below is a before/after rather than an assertion that nothing broke:main)cannot both be specified, exit 2Found 1 targetfrom the fileWARNING: BES was not properly closed--target_pattern_file <path>cannot both be specified.bazelrcline--target-pattern-file+ a forwarded onebazel config --dump_all --output=jsonunrecognized subcommand 'config', exit 2BAZEL_VERBSvsBAZEL_COMMAND_LISTconfig-- //... -//exclude/...End-to-end in IntelliJ IDEA 2026.2 with the Bazel plugin
Both the plugin's build and Sync Project actions complete against this branch. The build was confirmed from aspect's own BEP for the invocation (
/tmp/<uuid>.bep.binpb), which carries the plugin's full fingerprint:bazelbsp:3.2.0--target_pattern_file=…/IntelliJIdea2026.2/tmp/targets-8273310712140375819--build_event_binary_file=/tmp/bazel-bep-output9212484283891080652.tmpalongside--build_event_binary_file_upload_mode=wait_for_upload_complete//examples/deliverable:py_deliverable,:py_deliverable2SUCCESS, with nocannot both be specifiedand nounrecognized subcommandTwo things that will waste a reviewer's time if unstated:
target/ci/, as above..aspect/version.axlprefers local sources over the pinned release, butbazel-binis a symlink to the last-built configuration, and the IDE's own build applies a Starlark transition that repoints it atk8-fastbuild-ST-<hash>/bin— where noaspect-cliexists. The launcher then falls through to the pinned release, and the IDE reports an unrelatedpath "bazel/build_metadata.axl" does not exist in module aspecterror, which looks like a failure of this change. This is the same hazard that.aspect/bootstrap.shcopies totarget/ci/to avoid.mcp__idea__build_projectneedsrebuild=true. Withrebuild=falseit returnsisSuccess: truewithout invoking Bazel at all, and the wrapper trace goes to the Bazel plugin's build console, notidea.log— so neither the status nor the IDE log is usable evidence. Assert on the BEP artifact instead.Unrelated failure worth knowing about
bazel test //...fails withCI exclusion regression: //exclude was not excluded. That is the deliberate tripwire inexclude/BUILD.bazelfiring correctly: an explicit//...overrides the CLI's default pattern, so//excludegets built. It is cause (1) in the tripwire's own message and is unrelated to this PR — CI runsaspect testwith no patterns. The one-command discriminator:Not measured here, reported for follow-up:
aspectspawns Bazel with--ignore_all_rc_files --max_idle_secs=1200while wrapper-passthrough verbs (config,query,info,mod) reach Bazel with no startup flags. Bazel keys its server on startup options, so the mismatch could force a server restart per sync, costing the IDE its analysis cache.