Skip to content

Release/9.1 into maintenance-10.x - #11793

Merged
sensei-hacker merged 8 commits into
maintenance-10.xfrom
release/9.1
Aug 17, 2026
Merged

Release/9.1 into maintenance-10.x#11793
sensei-hacker merged 8 commits into
maintenance-10.xfrom
release/9.1

Conversation

@sensei-hacker

Copy link
Copy Markdown
Member

No description provided.

Stub only -- captures the buffer-decoupling and shrink-audit technique
from the MSP tunnel reply buffer fix before it's lost, ahead of the
tracked document-ram-flash-optimization-practices project writing the
full guide and linking it from Development.md.
Surfaces RAM/flash regressions and creep on every PR instead of only via a
hard CI failure (motivated by PR #11718's terrain-cache RAM overflow, which
was only caught because CI happened to fail on that specific board).
Extracts flash/RAM usage from each build's .elf right after compiling
(no second build of the base branch), persists a baseline per branch as a
release asset in the pr-test-builds companion repo, and diffs 4
representative targets (one per MCU family) against it on PR builds.
Several steps spliced \${{ }} expressions directly into run: script text
instead of passing them through env: — including one value (base_ref) that
round-trips through an artifact from a less-trusted PR build before being
used to construct a release tag. Route all of them through env: and
validate base_ref against safe git-ref characters before use.
…iene

- Clarify that the 4 representative targets are size tiers, not one-per-
  silicon-family (two are STM32F7, none are AT32); flag the AT32 gap in
  code comments/README rather than silently changing the manager-approved
  target list.
- Build size-report.json via jq instead of manual string concatenation, so
  target names are JSON-escaped rather than assumed safe.
- Carry baseFlash/baseRam through diffSizeReports instead of reconstructing
  them by subtracting the delta back out in renderComment.
- Drop a test that only re-asserted a literal copy of a source constant
  (no behavior exercised); derive the threshold-boundary test from
  NOISE_THRESHOLD_BYTES instead of hardcoding 32/31.
- Note the accepted tradeoff that this (non-gating) script runs on the
  PR's own checkout, so its numbers are informational only.
Delete-then-recreate left a window where the release didn't exist at all,
which a concurrent PR's baseline-fetch could hit and misreport as "no
baseline available yet" for a baseline that actually existed. Create the
release once, then only ever replace the asset in place on later pushes.
Also add a short retry on the fetch side, since an individual asset
replace still involves a brief delete-then-upload under the hood.
…f-comment-9.1

Add CI PR comment for RAM/flash usage delta vs. base branch
@qodo-code-review

Copy link
Copy Markdown
Contributor

ⓘ Qodo reviews are paused because the subscription is no longer active. Ask your workspace admin to reactivate the subscription to resume reviews. Manage billing

@qodo-free-for-open-source-projects

Copy link
Copy Markdown

PR Summary by Qodo

Add CI RAM/flash size-delta PR comments and seed optimization guide

✨ Enhancement ⚙️ Configuration changes 📝 Documentation 🧪 Tests 🕐 40+ Minutes

Grey Divider

AI Description

• Extract per-target flash/RAM usage from built ELF files and publish as CI artifacts.
• Persist per-branch size baselines and post PR comments showing size deltas vs base.
• Seed a RAM/flash optimization guide and add unit tests for comment rendering logic.
Diagram

graph TD
  A["ci.yml (Build firmware)"] --> B["extract-size-report.sh"] --> C[/"size-report-* artifacts"/] --> D["merge-size-reports.sh"] --> E[/"size-report artifact"/]
  E --> F["ci-size-report.yml"] --> G[("PR comment")]
  F --> H[("Baseline release asset")]
  F --> I["ram-and-flash-optimization.md"]
  subgraph Legend
    direction LR
    _wf["Workflow/job"] ~~~ _art[/"Artifact"/] ~~~ _store[("Release asset / comment")]
  end
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. Rebuild the base branch for every PR to compute a baseline
  • ➕ Always compares against an exact build of the base commit being targeted
  • ➕ Avoids maintaining cross-run baseline storage
  • ➖ Doubles CI build cost/time for PRs
  • ➖ Adds more moving parts (two checkouts/builds) and increases flake risk
2. Store baselines as long-lived artifacts in the same repo (instead of pr-test-builds releases)
  • ➕ Keeps all data co-located within the main repo’s Actions storage model
  • ➕ Potentially simpler credential model (no cross-repo write)
  • ➖ Artifacts are not a good fit for durable per-branch state and retention policies
  • ➖ Harder to address branch-scoped lookup and lifecycle cleanly across runs
3. Publish size deltas as a GitHub Check/Step Summary instead of a PR comment
  • ➕ Keeps signal in the Checks UI and avoids comment churn
  • ➕ Can be easier to scan alongside other CI outputs
  • ➖ Less visible during review discussion compared to a pinned/updated comment
  • ➖ Harder to persist a stable, always-updated artifact in the PR conversation

Recommendation: The chosen approach (extract sizes during the existing build, persist a per-branch baseline externally, then diff and comment via workflow_run) is a strong cost/visibility tradeoff: no second build, and reviewers see regressions early. Keeping the privileged commenting workflow on the default-branch checkout and validating artifact-derived base_ref appropriately addresses the main security risk for fork PRs.

Files changed (8) +850 / -0

Enhancement (1) +108 / -0
size-diff-comment.jsImplement pure size-diff logic and PR comment markdown renderer +108/-0

Implement pure size-diff logic and PR comment markdown renderer

• Adds a dependency-free Node module that diffs a PR size report against a baseline for four representative targets, applies a noise threshold, and renders a stable markdown comment body with an identification marker and optional doc link.

.github/scripts/size-diff-comment.js

Tests (1) +288 / -0
size-diff-comment.test.jsAdd Node unit tests for size-diff formatting and rendering +288/-0

Add Node unit tests for size-diff formatting and rendering

• Introduces comprehensive tests using node:test and node:assert to validate delta formatting, thresholding, missing-baseline behavior, doc-link inclusion, and consistent trailing newline behavior.

.github/scripts/size-diff-comment.test.js

Documentation (2) +101 / -0
README.mdDocument the new ci-size-report workflow behavior and data flow +29/-0

Document the new ci-size-report workflow behavior and data flow

• Extends the workflows README with an overview of the RAM/Flash delta comment feature, representative targets, baseline persistence strategy, and the scripts involved.

.github/workflows/README.md

ram-and-flash-optimization.mdSeed draft RAM/flash optimization techniques from PR #11718 +72/-0

Seed draft RAM/flash optimization techniques from PR #11718

• Adds a stub guide capturing concrete techniques: avoid reusing worst-case buffer sizing across subsystems, audit all writers when shrinking shared buffers, and survey real message sizes before choosing buffer sizes. Includes related links and notes that the guide is not yet complete or linked from Development.md.

docs/development/ram-and-flash-optimization.md

Other (4) +353 / -0
extract-size-report.shAdd ELF size extraction script producing per-target JSON +51/-0

Add ELF size extraction script producing per-target JSON

• Introduces a bash script that discovers built .elf files, runs arm-none-eabi-size, and computes flash (.text+.data) and RAM (.data+.bss) usage per target. Emits a JSON report via jq with proper escaping and warns gracefully when no ELF outputs exist.

.github/scripts/extract-size-report.sh

merge-size-reports.shAdd jq-based merger for per-shard size reports +19/-0

Add jq-based merger for per-shard size reports

• Adds a small helper script to merge multiple shard JSON reports into a single aggregated size-report.json for downstream consumption.

.github/scripts/merge-size-reports.sh

ci-size-report.ymlAdd workflow_run job to publish baselines and comment PR size deltas +230/-0

Add workflow_run job to publish baselines and comment PR size deltas

• Adds a new workflow that runs after successful firmware builds to (a) publish per-branch baselines as release assets in iNavFlight/pr-test-builds and (b) on PR runs, fetch the base branch baseline and post/update a PR comment. Hardens against script injection by routing interpolated values via env and validating base_ref from artifacts before use.

.github/workflows/ci-size-report.yml

ci.ymlEmit size-report artifacts from builds and publish metadata for workflow_run +53/-0

Emit size-report artifacts from builds and publish metadata for workflow_run

• Updates the build jobs to generate and upload per-shard size-report.json artifacts after compilation, then merges them in upload-artifacts to a single size-report artifact. Also uploads PR-number/base-ref (PR) and branch-name (push) artifacts consumed by ci-size-report.yml.

.github/workflows/ci.yml

@qodo-free-for-open-source-projects

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (2) 📘 Rule violations (0) 📜 Skill insights (0)

Grey Divider


Remediation recommended

1. Script execute-bit dependency 🐞 Bug ☼ Reliability
Description
ci.yml invokes the new .github/scripts/*.sh files directly; if they are not committed with +x, the
workflow will fail with a permission error. This repo’s existing workflow pattern executes scripts
via bash, so making these steps explicitly bash ... avoids relying on file mode.
Code

.github/workflows/ci.yml[R124-125]

+      - name: Extract size report
+        run: .github/scripts/extract-size-report.sh build build/size-report.json
Evidence
The workflow currently runs the new scripts directly, but an existing workflow in the same repo runs
scripts via bash, indicating the safer established convention. Direct execution will fail if the
scripts are not executable in the checkout.

.github/workflows/ci.yml[122-137]
.github/workflows/ci.yml[255-267]
.github/workflows/pg-version-check.yml[30-41]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`ci.yml` runs the new scripts as executables (e.g. `.github/scripts/extract-size-report.sh ...`). If the files are not committed with executable permissions, CI will fail at runtime with `Permission denied`.

## Issue Context
Elsewhere in this repo, workflows run scripts as `bash .github/scripts/<script>.sh`, which avoids depending on the file mode.

## Fix Focus Areas
- .github/workflows/ci.yml[124-126]
- .github/workflows/ci.yml[164-166]
- .github/workflows/ci.yml[260-262]

## Suggested fix
Change the `run:` commands to explicitly invoke bash, e.g.:
- `bash .github/scripts/extract-size-report.sh build build/size-report.json`
- `bash .github/scripts/merge-size-reports.sh ...`

(Alternatively, ensure the scripts are committed with mode `100755`, but using `bash` is the most robust and consistent with existing workflows.)

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


2. Broken JSON escaping 🐞 Bug ≡ Correctness
Description
extract-size-report.sh interpolates the ELF basename into a JSON literal passed to jq --argjson,
so target names containing quotes/backslashes/newlines will produce invalid JSON and make jq fail
(and fail the build due to set -e). The comment claims jq will JSON-escape the target, but the
escaping already needs to happen before jq parses the literal.
Code

.github/scripts/extract-size-report.sh[R40-43]

+    ram=$((data + bss))
+
+    JQ_ARGS+=(--argjson "entry_${#JQ_ARGS[@]}" "{\"target\":\"${target}\",\"flash\":${flash},\"ram\":${ram}}")
+done
Evidence
The script builds a JSON literal containing an unescaped ${target} and feeds it to --argjson,
which requires already-valid JSON; jq cannot escape a value that has already been interpolated into
JSON source text.

.github/scripts/extract-size-report.sh[33-49]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`extract-size-report.sh` constructs a JSON string in bash that includes `${target}` and passes it as `--argjson`. This is not JSON-safe for arbitrary target names (e.g., embedded `"`), and contradicts the comment claiming jq will escape it.

## Issue Context
The target name comes from `basename "$elf" .elf` and is not escaped before being embedded into a JSON literal.

## Fix Focus Areas
- .github/scripts/extract-size-report.sh[33-49]

## Suggested fix
Avoid embedding `${target}` into JSON text. Instead, pass it with `--arg` and build the object inside jq. One straightforward pattern:

1) For each elf, produce a small object safely:
```bash
obj=$(jq -n --arg target "$target" --argjson flash "$flash" --argjson ram "$ram" \
 '{($target): {flash: $flash, ram: $ram}}')
parts+=("$obj")
```
2) At the end, merge:
```bash
printf '%s\n' "${parts[@]}" | jq -s 'add' > "$OUTPUT_JSON"
```

This ensures the target string is JSON-escaped by jq rather than manually interpolated into JSON source.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Tip of the day
💡 Did you know, you can route each action level your way: inline, summary, both, or drop

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

Comment thread .github/workflows/ci.yml
Comment on lines +124 to +125
- name: Extract size report
run: .github/scripts/extract-size-report.sh build build/size-report.json

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remediation recommended

1. Script execute-bit dependency 🐞 Bug ☼ Reliability

ci.yml invokes the new .github/scripts/*.sh files directly; if they are not committed with +x, the
workflow will fail with a permission error. This repo’s existing workflow pattern executes scripts
via bash, so making these steps explicitly bash ... avoids relying on file mode.
Agent Prompt
## Issue description
`ci.yml` runs the new scripts as executables (e.g. `.github/scripts/extract-size-report.sh ...`). If the files are not committed with executable permissions, CI will fail at runtime with `Permission denied`.

## Issue Context
Elsewhere in this repo, workflows run scripts as `bash .github/scripts/<script>.sh`, which avoids depending on the file mode.

## Fix Focus Areas
- .github/workflows/ci.yml[124-126]
- .github/workflows/ci.yml[164-166]
- .github/workflows/ci.yml[260-262]

## Suggested fix
Change the `run:` commands to explicitly invoke bash, e.g.:
- `bash .github/scripts/extract-size-report.sh build build/size-report.json`
- `bash .github/scripts/merge-size-reports.sh ...`

(Alternatively, ensure the scripts are committed with mode `100755`, but using `bash` is the most robust and consistent with existing workflows.)

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Comment thread .github/scripts/extract-size-report.sh
cmake/arm-none-eabi-checks.cmake downloads its own toolchain into tools/
and adds it to PATH, but only inside that cmake process via
set(ENV{PATH} ...) — that doesn't persist to a later CI step's shell.
extract-size-report.sh defaulted to bare `arm-none-eabi-size` on PATH,
which happened to work in local dev sandboxes with a system copy
installed but fails on actual GitHub Actions runners, which have only
the project's downloaded toolchain. Fall back to the same tools/
location CMake uses when nothing is found on PATH.
…f-comment-9.1

Fix arm-none-eabi-size not found on real CI runners
@sensei-hacker
sensei-hacker merged commit 8994327 into maintenance-10.x Aug 17, 2026
23 checks passed
@github-actions

Copy link
Copy Markdown

RAM / Flash usage vs. base branch — commit d69afcd

No size baseline is available yet for this PR's base branch (first run after this feature shipped, or a new branch). This comment will show deltas once a baseline exists.

Target Flash Δ RAM Δ
MATEKF405 660387 B (no baseline) 143640 B (no baseline)
MATEKF722 463007 B (no baseline) 126220 B (no baseline)
MATEKF765 685695 B (no baseline) 151140 B (no baseline)
MATEKH743 716207 B (no baseline) 153380 B (no baseline)

See RAM/flash optimization guide for techniques to reduce usage.

@github-actions

Copy link
Copy Markdown

Test firmware build ready — commit d69afcd

Download firmware for PR #11793

244 targets built. Find your board's .hex file by name on that page (e.g. MATEKF405SE.hex). Files are individually downloadable — no GitHub login required.

Development build for testing only. Use Full Chip Erase when flashing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant