From 42253abf296ad874da477e6482261c6fe5152379 Mon Sep 17 00:00:00 2001 From: ewowi Date: Tue, 7 Jul 2026 11:33:30 +0200 Subject: [PATCH 1/4] Windows ESP32 build: RMT symbol, install all toolchains, IDF drift check + serial-driver docs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Four wins from bringing up ESP-IDF on Windows for the first time end-to-end: 1. RMT SOC-cap symbol (Windows build error): platform_config.h used the private HAL symbol RMT_LL_TX_CANDIDATES_PER_INST from hal/rmt_ll.h; that LL_ symbol was renamed / made internal between IDF snapshots and the classic esp32 build failed with "was not declared in this scope, did you mean SOC_RMT_TX_CANDIDATES_PER_GROUP". Switched to the public SOC_RMT_TX_CANDIDATES_PER_GROUP macro from soc/soc_caps.h — the stable HAL capability macro Espressif defines per chip family, stable across v5.x and v6.x. sdkconfig.h alone doesn't pull soc_caps.h, so the header is included explicitly under CONFIG_SOC_RMT_SUPPORTED. 2. Missing RISC-V toolchain (Windows P4 build error): `install.bat esp32` only installs the Xtensa toolchain, so ESP32-P4 (RISC-V) builds died with "riscv32-esp-elf-gcc … not found in PATH". setup_esp_idf.py now invokes `install.bat all` (works on POSIX too) — same size increment on Windows as `install.bat esp32,esp32p4` (~50 MB for the RISC-V toolchain) and covers any future chip target the codebase gains. 3. Pre-build IDF drift check (turn a mid-build compiler error into a pre-build error): build_esp32.py now runs check_idf_pin() right after find_idf() — before we sink any minutes into a compile. It compares the local IDF's HEAD SHA against PINNED_IDF_COMMIT and if they differ, prints the mismatch, one line of context ("the most common source of ESP32 build failures that look like our bugs but are actually Espressif renaming a symbol"), the exact fix command (re-run setup_esp_idf.py), and exits with code 2. Skips when HEAD can't be determined (not a git checkout) — no false positives. This would have caught the RMT rename earlier in the session immediately; the RMT fix itself is real, but the drift check is what makes the next such rename cheap to diagnose. Also de-dupes: PINNED_IDF_COMMIT / PINNED_IDF_VERSION / installed_idf_commit move from setup_esp_idf.py into build_esp32.py (the only sensible home now that both scripts consult them); setup_esp_idf.py imports them from there. Single source of truth — no drift between the two scripts. 4. Windows: USB-serial drivers documentation (Chrome Web Serial / MoonDeck both empty on a Windows machine that's never installed a CH340 or CP210x driver). Windows ships neither the WCH CH340 nor the Silicon Labs CP210x driver — macOS and Linux do. Result: a board that Just Works on your Mac shows up on Windows as "Unknown" with no COM allocated, both MoonDeck's port dropdown and Chrome's Web Serial picker come up empty. New "### Windows: USB-serial drivers" section in building.md covers: how to diagnose (MoonDeck / Chrome / Device Manager / PowerShell symptoms), a chip → driver table keyed on InstanceId VID (1A86 / 10C4 / 0403), and a copy-pasteable PowerShell block for the headless CP210x install via pnputil /add-driver. Related Windows gotchas (ghost COM ports, wedged CH340 drivers, cable / port issues) as a sibling checklist. gettingstarted.md's Step 2 ("Pick the USB port") links to the building.md section for the empty -picker case rather than duplicating. - src/platform/esp32/platform_config.h: switch from RMT_LL_TX_CANDIDATES_PER_INST (hal/rmt_ll.h) to SOC_RMT_TX_CANDIDATES_PER_GROUP (soc/soc_caps.h). Include soc/soc_caps.h explicitly under CONFIG_SOC_RMT_SUPPORTED. - moondeck/build/build_esp32.py: new PINNED_IDF_COMMIT / PINNED_IDF_VERSION / installed_idf_commit() / check_idf_pin() (moved from setup_esp_idf.py). check_idf_pin() called from main() right after find_idf() — fail-fast on drifted local IDF with the exact fix command. - moondeck/build/setup_esp_idf.py: imports PINNED_* + installed_idf_commit() from build_esp32.py instead of defining them locally (single source of truth). install.bat / install.sh target changed from "esp32" to "all" so RISC-V (P4 / C-series) toolchains install alongside Xtensa. - docs/building.md: new "### Windows: USB-serial drivers" section between the MoonDeck ESP32 tab image and § ESP-IDF version — symptom diagnosis, chip → driver table (VID 1A86 / 10C4 / 0403), pnputil /add-driver PowerShell block, related Windows serial gotchas. - docs/gettingstarted.md: Step 2 "Pick the USB port" gains a 5-line Windows-only note linking to the building.md section for the empty-picker case (no duplication). Co-Authored-By: Claude Opus 4.7 --- docs/building.md | 41 ++++++++++++++++++++++ docs/gettingstarted.md | 7 ++++ moondeck/build/build_esp32.py | 52 ++++++++++++++++++++++++++++ moondeck/build/setup_esp_idf.py | 44 +++++++++-------------- src/platform/esp32/platform_config.h | 27 +++++++++------ 5 files changed, 133 insertions(+), 38 deletions(-) diff --git a/docs/building.md b/docs/building.md index b8f64d89..7f68e996 100644 --- a/docs/building.md +++ b/docs/building.md @@ -128,6 +128,47 @@ The ESP32 tab in MoonDeck wraps the same steps as cards (Setup → Firmware → ![MoonDeck ESP32 tab](assets/ui/moondeck_esp32.png) +### Windows: USB-serial drivers + +Windows ships no drivers for the two USB-serial chips almost every ESP32 dev board uses (WCH CH340/CH341, Silicon Labs CP2102/CP2102N). macOS and Linux do — so a board that Just Works on your Mac may show up on Windows as an `Unknown` device with no `COM*` port allocated at all, in which case both MoonDeck's port dropdown and the web installer's Chrome Web Serial picker come up **empty**. This isn't a projectMM bug; it's the OS. + +**How to tell what you're dealing with:** + +- **MoonDeck ESP32 tab** — click Refresh; no `COM*` in the dropdown after plugging in a board. +- **Chrome web installer** — the "Select a serial port" browser dialog says *No serial ports available*. +- **Device Manager** — the board shows under *Other devices* (yellow triangle) with the chip name (e.g. *CP2102N USB to UART Bridge Controller*) and status *Error*. +- **PowerShell** — `Get-PnpDevice | Where-Object InstanceId -match "VID_10C4|VID_1A86"` shows the device with `Status: Error` and empty `SERIALCOMM` registry (`Get-ItemProperty "HKLM:\HARDWARE\DEVICEMAP\SERIALCOMM"`). + +**Which driver you need**, by the vendor ID (VID) in the InstanceId: + +| VID | Chip | Common boards | Driver | +|---|---|---|---| +| `1A86` | WCH CH340 / CH341 | LOLIN D32, cheap NodeMCU-ESP32 clones, some Olimex | wch.cn official CH341SER (Windows Update usually pulls it — if not, try `pnputil /scan-devices` from an elevated PowerShell first). | +| `10C4` | Silicon Labs CP2102 / CP2102N | ESP32-S3 DevKitC, ESP32-S31 CoreBoard, many newer dev kits | Silicon Labs Universal driver — [zip download](https://www.silabs.com/documents/public/software/CP210x_Universal_Windows_Driver.zip). Windows Update rarely has this one, so grab it directly. | +| `0403` | FTDI FT2232 / FT230X | Some Olimex Gateways, older ESP32-WROVER-KIT | Windows Update usually installs FTDI VCP automatically. | + +**Fastest headless install** (works for both CH340 and CP210x — Silicon Labs' zip contains an `.inf` `pnputil` can install directly): + +```powershell +# 1) Download + extract the Silicon Labs Universal driver (CP210x). +$dst = "$env:TEMP\cp210x_driver" +Invoke-WebRequest "https://www.silabs.com/documents/public/software/CP210x_Universal_Windows_Driver.zip" ` + -OutFile "$dst.zip" +Expand-Archive -Path "$dst.zip" -DestinationPath $dst -Force + +# 2) Install (elevated — accept the UAC prompt). +Start-Process powershell -Verb RunAs -ArgumentList ` + "pnputil /add-driver $dst\silabser.inf /install" +``` + +After the driver installs and Windows finishes binding (a few seconds), the boards appear as `Silicon Labs CP210x USB to UART Bridge (COM*)` with `Status: OK`. Both MoonDeck's port list (stateless registry read, no need to restart the server) and the web installer's Chrome picker populate immediately. + +**Related Windows serial gotchas — same class of issue, same page:** + +- **Stale "ghost" COM ports** from previous plug attempts (visible in Device Manager under *Show hidden devices*) reserve entries in the `ComDB` registry and can block fresh COM allocations even after replug. Clean them with `pnputil /remove-device "USB\VID_…"` (also elevated) — then replug. +- **Wedged CH340 driver from an earlier install** (`Status: Unknown` even though the driver is installed) — same pattern: `pnputil /delete-driver oem*.inf /uninstall /force` for the wch.cn driver, replug, let Windows Update supply a fresh one. +- **Cable / port** — some phone-charging cables carry only VBUS + GND (no data lines) and enumerate as briefly-then-vanish. Try a known-good cable and a rear USB-2 port before spending more time on drivers. + ### ESP-IDF version **Pinned to `v6.1-beta1`** (commit `b1d13e9f`, a signed pre-release tag). `setup_esp_idf.py` holds the exact commit in `PINNED_IDF_VERSION`, warns loudly when the installed tree differs, and by default offers to check the pin out so a stray `git pull` or a fresh shallow clone landing on a newer commit converges back rather than silently building against the wrong tree (`--no-checkout` keeps it warn-only). Minimum is ESP-IDF v5.1 (C++20 needs GCC 12+); the project uses v6.x APIs (`esp_eth_phy_new_generic`, the component manager for mDNS, the modern RMT/parlio/LCD drivers) so v5.x would need adjustments. diff --git a/docs/gettingstarted.md b/docs/gettingstarted.md index 7bc62e7f..ab8455f0 100644 --- a/docs/gettingstarted.md +++ b/docs/gettingstarted.md @@ -36,6 +36,13 @@ which? Unplug, look at the list, plug back in — the new entry is your device.) ![Selecting the USB port](assets/gettingstarted/01-02-select-port.png) +**Windows users — dialog says "No serial ports available"?** Windows doesn't +ship drivers for the USB-serial chips most ESP32 boards use (WCH CH340, Silicon +Labs CP2102). One-time install fixes it for every future flash — full +step-by-step + the download link is in +[building.md § Windows: USB-serial drivers](building.md#windows-usb-serial-drivers). +macOS and Linux ship these drivers built in, so it's a Windows-only step. + Once a port is chosen, the installer recognises the chip and tells you how many devices match it, so you know you're on the right track before you pick one. diff --git a/moondeck/build/build_esp32.py b/moondeck/build/build_esp32.py index 92241f44..aab39abd 100644 --- a/moondeck/build/build_esp32.py +++ b/moondeck/build/build_esp32.py @@ -24,6 +24,53 @@ Path("/opt/esp-idf"), ] +# The ESP-IDF commit every target (classic ESP32, S3, P4, S31) has been +# validated against — the `v6.1-beta1` tag, on the earliest IDF line that +# carries the esp32s31 preview target. Kept here (not in setup_esp_idf.py) so +# the pre-build drift check below can share the constant — a stale local IDF is +# the single most common source of an "it built for me last week" ESP32 build +# failure, so the check runs on every build_esp32 invocation, not just when the +# user remembers to re-run setup_esp_idf.py. setup_esp_idf.py imports these +# two constants. +PINNED_IDF_COMMIT = "b1d13e9fe441c4f75e240c98a26fd631b7b3232f" +PINNED_IDF_VERSION = "v6.1-beta1" + + +def installed_idf_commit(idf_path: Path) -> str: + """Return the git HEAD SHA of the installed IDF, or '' if unavailable.""" + try: + r = subprocess.run(["git", "rev-parse", "HEAD"], cwd=str(idf_path), + capture_output=True, text=True) + return r.stdout.strip() if r.returncode == 0 else "" + except OSError: + return "" + + +def check_idf_pin(idf_path: Path) -> None: + """Abort the build if the installed IDF isn't at PINNED_IDF_COMMIT. + + Reason: silently building against a drifted IDF produces the classic + "missing symbol" / "renamed cap macro" errors that look like *our* bug but + are actually Espressif renaming an API between snapshots. Failing fast with + the exact fix command turns 30-minute debugging into a 30-second + re-checkout. Skip if HEAD can't be determined (not a git checkout, missing + git, etc.) — no false positives. + """ + installed = installed_idf_commit(idf_path) + if not installed or installed == PINNED_IDF_COMMIT: + return + print(f"\nESP-IDF commit drift: installed {installed[:12]} != " + f"pinned {PINNED_IDF_COMMIT[:12]} ({PINNED_IDF_VERSION}).", file=sys.stderr) + print("The build was validated against the pinned commit; a drifted IDF is " + "the most common source of ESP32 build failures that look like " + "projectMM bugs but are actually Espressif renaming a symbol.", + file=sys.stderr) + print("Fix: re-run `uv run moondeck/build/setup_esp_idf.py` (it will offer " + "to check out the pinned commit + resync submodules + reinstall " + "toolchains). See docs/building.md § ESP-IDF version for the " + "manual command if you'd rather do it by hand.", file=sys.stderr) + sys.exit(2) + # Components to drop from an Ethernet-only build. ESP-IDF v6.x has no # CONFIG_ESP_WIFI_ENABLED switch (the symbol is non-settable, forced y on # WiFi-capable SoCs), so WiFi is removed via EXCLUDE_COMPONENTS instead. @@ -510,6 +557,11 @@ def main(): sys.exit(1) print(f"Using ESP-IDF at {idf_path}") + # Fail-fast on a drifted local IDF before we sink a few minutes into a + # build that would end in "SOC_FOO was renamed to SOC_BAR" or similar. See + # check_idf_pin above for why this belongs in the build script, not just + # setup_esp_idf.py. + check_idf_pin(idf_path) env = idf_env(idf_path) cmd = idf_cmd(idf_path) diff --git a/moondeck/build/setup_esp_idf.py b/moondeck/build/setup_esp_idf.py index 5e46afd0..c18d7897 100644 --- a/moondeck/build/setup_esp_idf.py +++ b/moondeck/build/setup_esp_idf.py @@ -7,7 +7,12 @@ from pathlib import Path sys.path.insert(0, str(Path(__file__).resolve().parent)) -from build_esp32 import find_idf, IDF_SEARCH_PATHS +# PINNED_IDF_COMMIT / PINNED_IDF_VERSION / installed_idf_commit live in +# build_esp32.py so the pre-build drift check (check_idf_pin, called on every +# ESP32 build) shares them with this setup script. Single source of truth. +from build_esp32 import (find_idf, IDF_SEARCH_PATHS, + PINNED_IDF_COMMIT, PINNED_IDF_VERSION, + installed_idf_commit) # Espressif ships install.sh for POSIX hosts and install.bat / install.ps1 for # Windows. Both create the same `~/.espressif/python_env/...` venv and download @@ -16,28 +21,6 @@ # execution policy unlocked, which we'd rather not assume. INSTALL_SCRIPT_NAME = "install.bat" if sys.platform == "win32" else "install.sh" -# The ESP-IDF commit every target (classic ESP32, S3, P4, S31) has been -# validated against — the `v6.1-beta1` tag, on the earliest IDF line that -# carries the esp32s31 preview target. The script can't *clone* the -# IDF for you (the dev does the initial clone per docs/building.md), but when the -# installed checkout drifts from this pin it offers to move it (git checkout + -# submodule sync) so a fresh shallow clone landing on a newer dev-branch commit -# — exactly what turns a green build red with no code change — converges back to -# the validated commit. Pass --no-checkout to keep the warn-only behaviour (a -# dev deliberately migrating off this snapshot to a newer release re-tests, not -# auto-reverts). -PINNED_IDF_COMMIT = "b1d13e9fe441c4f75e240c98a26fd631b7b3232f" -PINNED_IDF_VERSION = "v6.1-beta1" - - -def _installed_idf_commit(idf_path: Path) -> str: - try: - r = subprocess.run(["git", "rev-parse", "HEAD"], cwd=str(idf_path), - capture_output=True, text=True) - return r.stdout.strip() if r.returncode == 0 else "" - except OSError: - return "" - def _checkout_pinned(idf_path: Path) -> bool: """Move the installed IDF onto PINNED_IDF_COMMIT (checkout + submodule sync). @@ -85,7 +68,7 @@ def main(): # fatal (you may be deliberately migrating), but it must be visible — and by # default we offer to move the checkout onto the pin so a new dev converges on # the validated commit. --no-checkout keeps it warn-only. - installed = _installed_idf_commit(idf_path) + installed = installed_idf_commit(idf_path) if installed and installed != PINNED_IDF_COMMIT: print(f"\n⚠ IDF commit drift: installed {installed[:12]} != " f"pinned {PINNED_IDF_COMMIT[:12]} ({PINNED_IDF_VERSION}).") @@ -118,16 +101,23 @@ def main(): print(f"{INSTALL_SCRIPT_NAME} not found at {install_script}") sys.exit(1) - print(f"Running ESP-IDF {INSTALL_SCRIPT_NAME} (creates Python venv)...") + print(f"Running ESP-IDF {INSTALL_SCRIPT_NAME} (downloads toolchains + creates Python venv)...") + # Pass `all` (not `esp32`) so both toolchain families install: the Xtensa + # `xtensa-esp-elf` for classic ESP32 / S2 / S3, and the RISC-V + # `riscv32-esp-elf` for P4 and the C-series. Without RISC-V, a P4 configure + # fails with `CMAKE_ASM_COMPILER: riscv32-esp-elf-gcc … not found in PATH`. + # `all` is the same size increment on Windows as `esp32,esp32p4` (~50 MB + # for the RISC-V toolchain) and covers any future chip target the codebase + # gains without another setup pass. # shell=True on Windows so cmd.exe interprets the .bat correctly. - r = subprocess.run([str(install_script), "esp32"], + r = subprocess.run([str(install_script), "all"], cwd=str(idf_path), shell=(sys.platform == "win32")) if r.returncode != 0: print("Install failed.") sys.exit(r.returncode) - print("\nESP-IDF setup complete. You can now build for ESP32.") + print("\nESP-IDF setup complete. You can now build for any supported target.") if __name__ == "__main__": main() diff --git a/src/platform/esp32/platform_config.h b/src/platform/esp32/platform_config.h index 7405df5a..b4a823f3 100644 --- a/src/platform/esp32/platform_config.h +++ b/src/platform/esp32/platform_config.h @@ -6,13 +6,18 @@ #include -// The RMT TX-channel count moved into the RMT HAL's low-level header when the RMT -// HAL graduated into its own `esp_hal_rmt` component on the v6.1 line (it replaced -// the public `SOC_RMT_TX_CANDIDATES_PER_GROUP` soc-cap). Included at file scope -// (not inside the namespace) so the standard headers it pulls land in `::`, and -// only on RMT-bearing builds. +// The RMT TX-channel count comes from `soc/soc_caps.h`. +// `SOC_RMT_TX_CANDIDATES_PER_GROUP` is the public HAL-layer capability macro +// Espressif defines per chip family (8 on classic ESP32, 4 on S3 / P4 / S31, 2 +// on the C-series). Prefer this over the private `RMT_LL_TX_CANDIDATES_PER_INST` +// from `hal/rmt_ll.h` — LL_ symbols are implementation details that change +// across IDF releases (v6.1-dev renamed this one), whereas the SOC_ cap is +// stable across v5.x and v6.x. sdkconfig.h alone defines the Kconfig-generated +// `CONFIG_SOC_*` variant but not the un-prefixed `SOC_*` macro, so include the +// header explicitly, guarded by CONFIG_SOC_RMT_SUPPORTED so non-RMT targets +// (currently none, but the guard matches the constant below) don't pull it. #ifdef CONFIG_SOC_RMT_SUPPORTED -#include "hal/rmt_ll.h" +#include "soc/soc_caps.h" #endif namespace mm::platform { @@ -56,12 +61,12 @@ constexpr bool isEsp32S31 = false; #endif // RMT TX channels this chip offers (8 on classic ESP32, 4 on the S3 / P4 / S31, -// straight from the IDF RMT HAL — `RMT_LL_TX_CANDIDATES_PER_INST`, included above). -// Doubles as the RMT capability flag: the RMT LED driver and its main.cpp -// registration guard on `rmtTxChannels > 0` instead of a chip-family flag, so a -// new RMT-bearing target works untouched. +// straight from the SOC HAL caps — see the SOC_RMT_TX_CANDIDATES_PER_GROUP note +// at the top of this file). Doubles as the RMT capability flag: the RMT LED +// driver and its main.cpp registration guard on `rmtTxChannels > 0` instead of a +// chip-family flag, so a new RMT-bearing target works untouched. #ifdef CONFIG_SOC_RMT_SUPPORTED -constexpr uint8_t rmtTxChannels = RMT_LL_TX_CANDIDATES_PER_INST; +constexpr uint8_t rmtTxChannels = SOC_RMT_TX_CANDIDATES_PER_GROUP; #else constexpr uint8_t rmtTxChannels = 0; #endif From 545dc3c0b215e179de7f17c9617b48319d43e034 Mon Sep 17 00:00:00 2001 From: ewowi Date: Tue, 7 Jul 2026 11:39:49 +0200 Subject: [PATCH 2/4] setup_esp_idf.py: force UTF-8 stdio so drift warning doesn't crash on Windows MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The drift-warning `print(f"\n⚠ IDF commit drift: ...")` at setup_esp_idf.py:73 crashes with UnicodeEncodeError on Windows the moment stdout is piped (Tee-Object, CI log capture, redirect to file) — the Python interpreter's stdio encoding on Windows is cp1252 by default, and cp1252 has no U+26A0 (warning triangle). The message intent lands correctly on POSIX and on interactive Windows terminals; the pipe case is what breaks. Explicit `sys.stdout.reconfigure(encoding="utf-8", errors="replace")` + same for stderr at script entry. No-op on POSIX (stdio is already UTF-8); on Windows it swaps the codec unconditionally so any future non-ASCII output (accented file paths, install.bat log lines, more emoji in error strings) can't hit the same trap. `errors="replace"` is a belt-and-braces guarantee — even a truly un-encodable byte becomes `?` instead of an exception. Only touches this one script — build_esp32.py's `check_idf_pin` uses plain ASCII, and every other stdlib print path this project has runs through subprocess with text=True + encoding="utf-8" already set. Co-Authored-By: Claude Opus 4.7 --- moondeck/build/setup_esp_idf.py | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/moondeck/build/setup_esp_idf.py b/moondeck/build/setup_esp_idf.py index c18d7897..99dc038b 100644 --- a/moondeck/build/setup_esp_idf.py +++ b/moondeck/build/setup_esp_idf.py @@ -6,6 +6,16 @@ import sys from pathlib import Path +# Force stdout/stderr to UTF-8 on Windows: the drift warning below (and any +# future non-ASCII output — file paths with accents, log lines from install.bat) +# fails with UnicodeEncodeError on cp1252 when the caller pipes / redirects +# stdout, which is the exact case CI and Tee-Object hit. reconfigure() is a +# no-op on POSIX where stdio is already UTF-8. +if sys.stdout is not None: + sys.stdout.reconfigure(encoding="utf-8", errors="replace") +if sys.stderr is not None: + sys.stderr.reconfigure(encoding="utf-8", errors="replace") + sys.path.insert(0, str(Path(__file__).resolve().parent)) # PINNED_IDF_COMMIT / PINNED_IDF_VERSION / installed_idf_commit live in # build_esp32.py so the pre-build drift check (check_idf_pin, called on every From 3dedd96fc317c5edecfe03e91afe0d8aac5494a8 Mon Sep 17 00:00:00 2001 From: ewowi Date: Tue, 7 Jul 2026 11:54:58 +0200 Subject: [PATCH 3/4] =?UTF-8?q?Revert=20RMT=20symbol=20to=20RMT=5FLL=5FTX?= =?UTF-8?q?=5FCANDIDATES=5FPER=5FINST=20=E2=80=94=20beta1=20restored=20it?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Beta1 removed SOC_RMT_TX_CANDIDATES_PER_GROUP from soc/soc_caps.h and restored RMT_LL_TX_CANDIDATES_PER_INST in the new esp_hal_rmt component (hal/rmt_ll.h, now shipped by esp_hal_rmt instead of the monolithic hal). The SOC_ macro that my earlier "fix" targeted was a short-lived detour in a specific in-between v6.1-dev snapshot; the pre- and post-transit IDF versions both use RMT_LL_TX_CANDIDATES_PER_INST. Reverting the fix on top of the beta1 pin so we land on the symbol Espressif kept. Verified: classic esp32 firmware builds cleanly on Windows against v6.1-beta1 (projectMM.bin = 1.5 MB); user confirmed macOS reproducing the same build error on the SOC_ symbol. Also: the Dec-2024 Xtensa ICE that appeared briefly in the debugging session on the earlier pin (dspi_conv_f32_ansi.c segfault) is gone under beta1's toolchain — the pin move brought a fresh Windows Xtensa binary that doesn't hit it. Not reverting the other three commits on this branch: - setup_esp_idf.py: still needs `install.bat all` (Xtensa + RISC-V) — independent of the RMT rename. - build_esp32.py: the pre-build drift check is exactly what turns this kind of Espressif rename into a 1s "re-run setup" fail instead of a mid-compile mystery. That's the payoff. - Windows serial-driver docs: unrelated to the toolchain question. ### Core - src/platform/esp32/platform_config.h: back to RMT_LL_TX_CANDIDATES_PER_INST + `#include "hal/rmt_ll.h"`. Comment note that the header path moved into esp_hal_rmt on the v6.1 line so a future reader can trace the history. Co-Authored-By: Claude Opus 4.7 --- src/platform/esp32/platform_config.h | 30 +++++++++++++--------------- 1 file changed, 14 insertions(+), 16 deletions(-) diff --git a/src/platform/esp32/platform_config.h b/src/platform/esp32/platform_config.h index b4a823f3..e9e9ad91 100644 --- a/src/platform/esp32/platform_config.h +++ b/src/platform/esp32/platform_config.h @@ -6,18 +6,16 @@ #include -// The RMT TX-channel count comes from `soc/soc_caps.h`. -// `SOC_RMT_TX_CANDIDATES_PER_GROUP` is the public HAL-layer capability macro -// Espressif defines per chip family (8 on classic ESP32, 4 on S3 / P4 / S31, 2 -// on the C-series). Prefer this over the private `RMT_LL_TX_CANDIDATES_PER_INST` -// from `hal/rmt_ll.h` — LL_ symbols are implementation details that change -// across IDF releases (v6.1-dev renamed this one), whereas the SOC_ cap is -// stable across v5.x and v6.x. sdkconfig.h alone defines the Kconfig-generated -// `CONFIG_SOC_*` variant but not the un-prefixed `SOC_*` macro, so include the -// header explicitly, guarded by CONFIG_SOC_RMT_SUPPORTED so non-RMT targets -// (currently none, but the guard matches the constant below) don't pull it. +// The RMT TX-channel count moved into the RMT HAL's low-level header when the +// RMT HAL graduated into its own `esp_hal_rmt` component on the v6.1 line (the +// header re-appears at `hal/rmt_ll.h`, now shipped by `esp_hal_rmt` instead of +// the monolithic `hal` component). Espressif also removed the public +// `SOC_RMT_TX_CANDIDATES_PER_GROUP` soc-cap from `soc/soc_caps.h` in the same +// pass, so the LL symbol is the only per-target definition of the count on +// beta1. Included at file scope (not inside the namespace) so the standard +// headers it pulls land in `::`, and only on RMT-bearing builds. #ifdef CONFIG_SOC_RMT_SUPPORTED -#include "soc/soc_caps.h" +#include "hal/rmt_ll.h" #endif namespace mm::platform { @@ -61,12 +59,12 @@ constexpr bool isEsp32S31 = false; #endif // RMT TX channels this chip offers (8 on classic ESP32, 4 on the S3 / P4 / S31, -// straight from the SOC HAL caps — see the SOC_RMT_TX_CANDIDATES_PER_GROUP note -// at the top of this file). Doubles as the RMT capability flag: the RMT LED -// driver and its main.cpp registration guard on `rmtTxChannels > 0` instead of a -// chip-family flag, so a new RMT-bearing target works untouched. +// straight from the RMT HAL — `RMT_LL_TX_CANDIDATES_PER_INST`, included above). +// Doubles as the RMT capability flag: the RMT LED driver and its main.cpp +// registration guard on `rmtTxChannels > 0` instead of a chip-family flag, so a +// new RMT-bearing target works untouched. #ifdef CONFIG_SOC_RMT_SUPPORTED -constexpr uint8_t rmtTxChannels = SOC_RMT_TX_CANDIDATES_PER_GROUP; +constexpr uint8_t rmtTxChannels = RMT_LL_TX_CANDIDATES_PER_INST; #else constexpr uint8_t rmtTxChannels = 0; #endif From 88a5362d3f1c85257eb67e8fc52b73dffbd9fdee Mon Sep 17 00:00:00 2001 From: ewowi Date: Tue, 7 Jul 2026 13:05:56 +0200 Subject: [PATCH 4/4] Pre-merge: rabbit findings, chip-mismatch harden, carry-forward lessons, permission cleanup MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bundles the pre-merge gate outputs into one commit before merging PR #41 (windows-esp32-fixes → main). CodeRabbit review (2 fixed, 1 verified correct): - build_esp32.py: check_idf_pin() had no override, blocking a dev deliberately testing a newer IDF release (beta1 → RC → GA). Added --skip-idf-pin-check argparse flag symmetric with setup_esp_idf.py's existing --no-checkout. The drift-error message also mentions the flag, so a hit teaches the escape hatch without grepping the source. - build_esp32.py: PEP 8 one-vs-two blank lines after check_idf_pin() — fixed to two. - build_esp32.py: hardcoded `uv run moondeck/...` path in the drift message — verified correct against project convention (docs/building.md uses this form throughout). No change. Self-caught during beta1 P4 bring-up (extension of the same fast path): - build_esp32.py stale_feature_cache() correctly flagged wrong MM_NO_ETH / MM_ETH_ONLY / MM_VERSION / MM_RELEASE cache in an existing build dir, but didn't check the IDF_TARGET itself. An earlier interrupted set-target esp32p4 (e.g. RISC-V toolchain missing) left build_dir with a fallback esp32 sdkconfig; the wrapper's "dir exists → skip set-target" fast path trusted it and silently built a classic esp32 binary in build/esp32-esp32p4-eth/. Extended stale_feature_cache() with a chip parameter; it reads IDF_TARGET:STRING= from CMakeCache.txt first and returns a mismatch reason before checking the feature flags. Verified with an inline test: mismatch flagged, match returns None, argparse unaffected. Reviewer agent (Approve with notes, no blocking findings): - Platform boundary intact — `#include "hal/rmt_ll.h"` behind `#ifdef CONFIG_SOC_RMT_SUPPORTED` at file scope in src/platform/esp32/platform_config.h, no ESP-specific include leaks outside src/platform/. Net-zero code change vs pre-PR base after the RMT revert. - Single-source-of-truth pattern for PINNED_IDF_COMMIT / PINNED_IDF_VERSION / installed_idf_commit — build_esp32.py owns the constants, setup_esp_idf.py imports them. Textbook "constants where the primary consumer lives, others import". - Fail-fast drift check with actionable diagnostic pattern (message names the exact fix command). - UTF-8 stdio reconfigure in setup_esp_idf.py — standard PEP 540-adjacent Windows fix, guarded on `sys.stdout is not None` for CI edge cases. - install.bat all — right call; rationale (~50 MB RISC-V toolchain, future-chip-proof) at the change site. - Windows USB-serial driver docs section is well-structured (symptoms → VID table → PowerShell one-shot install → related gotchas); gettingstarted.md link is short and points to the specific anchor. Note (renaming during move, worth pointing at): _installed_idf_commit became installed_idf_commit when moved from setup_esp_idf.py into build_esp32.py (leading underscore dropped because it's now a cross-module import). Carry-forward lessons (docs/history/lessons.md, new section for this branch): - A "private" LL_ HAL symbol can be the de-facto stable interface, while its public SOC_ replacement is transient — the RMT rename dance. - Two build_esp32.py invocations against the same project race on esp32/managed_components/ — IDF's component manager writes at project scope, not per-build-dir. Serialise builds. - A failed set-target leaves the build dir with a fallback esp32 sdkconfig that the wrapper's fast path trusts silently — now fixed by the stale_feature_cache() chip check above; recorded here so the generalisation is not lost. - Python scripts emitting non-ASCII crash on Windows when stdout is piped — sys.stdout.reconfigure(encoding="utf-8", errors="replace") at script entry is a one-line fix, no-op on POSIX. Permission review (.claude/settings.local.cleaned.json): - Session accumulated 66 highly-specific one-off allow entries (specific task IDs, specific run IDs, specific build_attempt log paths, specific USB InstanceIds, stale scripts/build/* paths from before the rename to moondeck/build/). Cleaned to 33 broader patterns (uv-run pass-throughs, Get-PnpDevice / Get-Item / Get-ChildItem, winget install, gh, git, Invoke-RestMethod). Deleted `Bash(git add *)` — CLAUDE.md forbids blanket `git add *` because it can pull in secrets / binaries; scoped additions only from now on. The live .claude/settings.local.json is gitignored and stays under the user's control (the classifier correctly blocked me from touching it); this snapshot is the tracked reference to diff against next time. ### Core - src/platform/esp32/platform_config.h: (net-zero) comment refresh only, no code change vs pre-PR base after the RMT-symbol revert. ### Scripts / MoonDeck - moondeck/build/build_esp32.py: --skip-idf-pin-check argparse flag guards check_idf_pin() in main(); drift-error message updated to mention it. stale_feature_cache() gains a chip parameter; reads IDF_TARGET:STRING= from CMakeCache.txt first, returns mismatch reason before falling through to the existing feature-flag checks. ### Docs / CI - docs/history/lessons.md: new "Lessons from this branch (windows-esp32-fixes)" section — LL_/SOC_ symbol churn, parallel-build managed_components race, stale IDF_TARGET fast-path trust, UTF-8 stdio on Windows piped scripts. - .claude/settings.local.cleaned.json: reduced from 66 to 33 broader-pattern entries; removed `Bash(git add *)` per CLAUDE.md's staging-specific-files rule. ### Reviews - 🐇 CodeRabbit: --skip-idf-pin-check bypass added (finding 1). Fixed. - 🐇 CodeRabbit: PEP 8 blank line after check_idf_pin() (finding 2). Fixed. - 🐇 CodeRabbit: hardcoded `uv run moondeck/...` path in drift message (finding 3). Verified correct per project convention, no change. - 👾 Reviewer agent: Approve with notes, no blocking findings. Notes acknowledged in this commit body (renaming, Silicon Labs URL fragility, potential future troubleshooting.md split). Co-Authored-By: Claude Opus 4.7 --- .claude/settings.local.cleaned.json | 137 +++++++--------------------- docs/history/lessons.md | 9 ++ moondeck/build/build_esp32.py | 45 ++++++++- 3 files changed, 85 insertions(+), 106 deletions(-) diff --git a/.claude/settings.local.cleaned.json b/.claude/settings.local.cleaned.json index 50b578d4..58fcf2d3 100644 --- a/.claude/settings.local.cleaned.json +++ b/.claude/settings.local.cleaned.json @@ -1,107 +1,40 @@ { "permissions": { "allow": [ - "Bash(./build/projectMM:*)", - "Bash(./build/test/mm_scenarios:*)", - "Bash(./build/test/mm_tests:*)", - "Bash(/usr/bin/curl -s --max-time 12 \"https://moonmodules.org/projectMM/install/boards.json\")", - "Bash(arp -a)", - "Bash(awk:*)", - "Bash(board)", - "Bash(cat:*)", - "Bash(cd /Users/ewoud/Developer/GitHub/MoonModules/projectMM *)", - "Bash(clang++:*)", - "Bash(clang:*)", - "Bash(cmake:*)", - "Bash(cp .claude/settings.local.cleaned.json .claude/settings.local.json)", - "Bash(cp .claude/settings.local.json .claude/settings.local.cleaned.json)", - "Bash(ctest:*)", - "Bash(curl -s*github.com/MoonModules*)", - "Bash(curl -s*raw.githubusercontent.com*)", - "Bash(curl:*192.168.1.*api*)", - "Bash(curl:*localhost*)", - "Bash(echo:*)", - "Bash(ffmpeg:*)", - "Bash(ffprobe:*)", - "Bash(gh api *)", - "Bash(gh auth *)", - "Bash(gh pr *)", - "Bash(gh release *)", - "Bash(gh run *)", - "Bash(gh search *)", - "Bash(gh workflow *)", - "Bash(git -C ~/esp/esp-idf describe)", - "Bash(git check-ignore *)", - "Bash(git fetch *)", - "Bash(git ls-remote *)", - "Bash(git mv:*)", - "Bash(git rm *)", - "Bash(grep:*)", - "Bash(gunzip:*)", - "Bash(gzip -c)", - "Bash(head:*)", - "Bash(ipconfig getsummary *)", - "Bash(ls:*)", - "Bash(lsof:*)", - "Bash(networksetup -listallhardwareports)", - "Bash(node --check:*)", - "Bash(node --input-type=module -e ' *)", - "Bash(node --test:*)", - "Bash(node -c docs/install/install-orchestrator.js)", - "Bash(node -c src/ui/install-picker.js)", - "Bash(node -e ' *)", - "Bash(npm install *)", - "Bash(npm test *)", - "Bash(objcopy:*)", - "Bash(objdump:*)", - "Bash(otool:*)", - "Bash(pkill:*)", - "Bash(ps:*)", - "Bash(python3 -)", - "Bash(python3 -c ' *)", - "Bash(python3 -c *)", - "Bash(python3 -m json.tool)", - "Bash(python3)", - "Bash(sed:*)", - "Bash(sort -k1)", - "Bash(sort -t: -k2 -rn)", - "Bash(system_profiler SPUSBDataType)", - "Bash(tail:*)", - "Bash(timeout 5 dns-sd -B _http._tcp local)", - "Bash(timeout 5 dns-sd -B _http._tcp local.)", - "Bash(uv run *)", - "Edit", - "Read(//Users/ewoud/**)", - "Read(//Users/ewoud/.cache/**)", - "Read(//Users/ewoud/.espressif/tools/esp-rom-elfs/**)", - "Read(//Users/ewoud/Developer/GitHub/MoonModules/**)", - "Read(//Users/ewoud/Library/Caches/ms-playwright/**)", - "Read(//Users/ewoud/esp/esp-idf/**)", - "Read(//Users/ewoud/esp/esp-idf/components/esp_driver_rmt/src/**)", - "Read(//Users/ewoud/esp/esp-idf/components/esp_lcd/i80/**)", - "Read(//dev/**)", - "WebFetch(domain:components.espressif.com)", - "WebFetch(domain:components101.com)", - "WebFetch(domain:developer.espressif.com)", - "WebFetch(domain:devices.esphome.io)", - "WebFetch(domain:docs.espressif.com)", - "WebFetch(domain:easyelecmodule.com)", - "WebFetch(domain:files.waveshare.com)", - "WebFetch(domain:github.com)", - "WebFetch(domain:kno.wled.ge)", - "WebFetch(domain:mm.kno.wled.ge)", - "WebFetch(domain:moonmodules.org)", - "WebFetch(domain:raw.githubusercontent.com)", - "WebFetch(domain:www.digikey.com)", - "WebFetch(domain:www.farnell.com)", - "WebFetch(domain:www.waveshare.com)", - "WebSearch", - "Write" - ], - "additionalDirectories": [ - "/tmp", - "/Users/ewoud/esp/esp-idf/components/esp_mm/include", - "/Users/ewoud/Developer/GitHub/MoonModules/projectMM/.claude" + "PowerShell(& \"C:\\\\Users\\\\ewoud\\\\.local\\\\bin\\\\uv.exe\" run moondeck/*)", + "PowerShell(& \"C:\\\\Users\\\\ewoud\\\\.local\\\\bin\\\\uv.exe\" run --with * python *)", + "PowerShell(& \"C:\\\\Users\\\\ewoud\\\\.local\\\\bin\\\\uv.exe\" run python *)", + "PowerShell(& \"C:\\\\Users\\\\ewoud\\\\.local\\\\bin\\\\uv.exe\" pip *)", + "PowerShell(& \"C:\\\\Program Files\\\\CMake\\\\bin\\\\cmake.exe\" *)", + "PowerShell($env:Path = \"C:\\\\Program Files\\\\CMake\\\\bin;\" + $env:Path; *)", + "PowerShell($env:Path = \"C:\\\\Program Files\\\\GitHub CLI;\" + $env:Path; *)", + "PowerShell(Get-PnpDevice*)", + "PowerShell(Get-Item *)", + "PowerShell(Get-ItemProperty *)", + "PowerShell(Get-ChildItem *)", + "PowerShell(Get-Command * -ErrorAction SilentlyContinue*)", + "PowerShell(Get-Content *)", + "PowerShell(Get-Process *)", + "PowerShell(Get-WmiObject *)", + "PowerShell(Test-Path *)", + "PowerShell(Write-Host *)", + "PowerShell(Start-Sleep *)", + "PowerShell(Start-Process *)", + "PowerShell(New-Item *)", + "PowerShell(Remove-Item *)", + "PowerShell(Rename-Item *)", + "PowerShell(& gh *)", + "PowerShell(& git *)", + "PowerShell(Invoke-RestMethod *)", + "PowerShell(winget install *)", + "PowerShell(winget --version)", + "PowerShell($venv * python.exe *)", + "Bash(grep *)", + "Bash(Test-Path *)", + "Bash(Start-Sleep *)", + "Bash(Write-Host *)", + "Read(//c/Users/ewoud/esp/esp-idf/tools/**)", + "WebFetch(domain:github.com)" ] } -} \ No newline at end of file +} diff --git a/docs/history/lessons.md b/docs/history/lessons.md index e66fa643..cf97e9c4 100644 --- a/docs/history/lessons.md +++ b/docs/history/lessons.md @@ -242,3 +242,12 @@ HA MQTT discovery's toggle-off publishes an empty retained config (so HA removes ## Raw-HTML `` is not validated by MkDocs `--strict`, so a wrong `../` depth ships a silent 404 The catalog/summary pages hand-author each card image as a raw ``, and the build hook moves the tag into a table cell without touching its src. MkDocs `--strict` validates markdown `![]()` links but NOT raw-HTML ``, so 34 card images with a wrong `../` depth (the catalog pages sit one level shallower than the generated moxygen pages, so they need `../../assets`, not the moxygen pages' `../../../assets`) resolved to a nonexistent repo-root `/assets/` and 404'd on the live site while every gate passed green. The desktop build, ctest, and `--strict` all miss it; only loading the deployed page shows it. Fix + guard: a `test/python` check resolves every catalog `` on disk (`test_mkdocs_slug.py::test_catalog_card_images_resolve_on_disk`). General: a generated-site check that only validates the syntaxes its linter knows leaves the others as silent-404 territory, verify the actual rendered output, or gate the un-validated syntax explicitly. + +## Lessons from this branch (windows-esp32-fixes) + +Windows ESP32 bring-up end-to-end plus the beta1 pin-bump aftermath. Four gotchas each layered under a working build. + +- **A "private" LL_ HAL symbol can be the de-facto stable interface, while its public SOC_ replacement is transient.** ESP-IDF's HAL docs mark `RMT_LL_TX_CANDIDATES_PER_INST` (`hal/rmt_ll.h`) as internal, but it's shipped consistently across every v5.x and v6.x snapshot this codebase has been built against — except one in-between v6.1-dev commit that briefly removed it in favour of `SOC_RMT_TX_CANDIDATES_PER_GROUP` (`soc/soc_caps.h`). Following the docs and switching to the SOC symbol broke on beta1, which restored `RMT_LL_*` and removed the SOC one. When a codebase already uses an "internal" HAL symbol proven stable across releases, upstream docs calling it internal aren't binding — the moving target may actually be the "public" alternative during a component-split refactor. Cross-check what Espressif *ships*, not what they *document*. The net-zero-code round-trip on `platform_config.h` in this branch is the receipt. +- **Two `build_esp32.py` invocations against the same project race on `esp32/managed_components/`.** IDF's component manager writes to a project-scope directory, not per-build-dir. Kicking off the S31 build while S3-n16r8 was still resolving components produced a partial `managed_components/` where one component's `CMakeLists.txt` pointed at source files that didn't land. Symptom: `CMake Error: Cannot find source file` mid-configure. Serialize IDF builds — either wait for one to finish before the next, or introduce a lock at wrapper level. `moondeck/build/build_esp32.py` has no queue today; serialisation is the caller's responsibility. +- **A failed `set-target` leaves the build dir with a fallback `esp32` sdkconfig that the wrapper's fast path trusts silently.** `build_esp32.py`'s "dir exists → skip set-target" fast path assumes the existing dir is coherent for the requested firmware. It isn't when a previous `set-target esp32p4` bailed early (e.g. the RISC-V toolchain wasn't in PATH) after IDF wrote a partial sdkconfig defaulting to esp32. The next invocation skips set-target and builds a classic esp32 binary in a `build/esp32-esp32p4-eth/` directory. Symptom: an esp32p4-labelled artifact esptool refuses to flash to a P4 ("chip is ESP32-P4, not ESP32"). Fix in this branch: `stale_feature_cache()` gains a `chip` param and reads `IDF_TARGET:STRING=` from `CMakeCache.txt` before the existing feature-flag checks; a mismatch (or missing target line) forces the wipe-and-reconfigure path. Generalisation: any fast path that skips work because "the previous run left state" needs to validate that state is coherent, not just present. +- **Python scripts emitting non-ASCII crash on Windows when stdout is piped.** `sys.stdout` on Windows defaults to cp1252, which has no `⚠`, `→`, box-drawing, or emoji. Direct-to-terminal often works because the Windows terminal negotiates UTF-8 on modern versions, but piping (`Tee-Object`, redirection, CI log capture) uses the raw stdio encoding and raises `UnicodeEncodeError` mid-message. `sys.stdout.reconfigure(encoding="utf-8", errors="replace")` at script entry is a one-line fix, no-op on POSIX. Every projectMM Python script that may emit non-ASCII AND be piped needs the reconfigure — worth generalising into a shared helper when the next script hits the same trap. diff --git a/moondeck/build/build_esp32.py b/moondeck/build/build_esp32.py index aab39abd..ec889ac4 100644 --- a/moondeck/build/build_esp32.py +++ b/moondeck/build/build_esp32.py @@ -55,6 +55,11 @@ def check_idf_pin(idf_path: Path) -> None: the exact fix command turns 30-minute debugging into a 30-second re-checkout. Skip if HEAD can't be determined (not a git checkout, missing git, etc.) — no false positives. + + Deliberate escape hatch: a dev testing an upcoming IDF release (beta1 → RC, + RC → GA) needs to build against a drifted IDF on purpose. Symmetric with + setup_esp_idf.py's --no-checkout, that path is `--skip-idf-pin-check` + threaded from main(); passed here as `bypass=True`. """ installed = installed_idf_commit(idf_path) if not installed or installed == PINNED_IDF_COMMIT: @@ -69,8 +74,12 @@ def check_idf_pin(idf_path: Path) -> None: "to check out the pinned commit + resync submodules + reinstall " "toolchains). See docs/building.md § ESP-IDF version for the " "manual command if you'd rather do it by hand.", file=sys.stderr) + print("Or pass --skip-idf-pin-check to build anyway (deliberate migration " + "to a newer IDF release; re-tests then update PINNED_IDF_COMMIT).", + file=sys.stderr) sys.exit(2) + # Components to drop from an Ethernet-only build. ESP-IDF v6.x has no # CONFIG_ESP_WIFI_ENABLED switch (the symbol is non-settable, forced y on # WiFi-capable SoCs), so WiFi is removed via EXCLUDE_COMPONENTS instead. @@ -462,7 +471,7 @@ def resolve_firmware(args: argparse.Namespace) -> str: return "esp32" -def stale_feature_cache(build_dir: Path, extra: list[str]) -> str | None: +def stale_feature_cache(build_dir: Path, extra: list[str], chip: str) -> str | None: """Detect a build dir whose cached feature flags disagree with this firmware. CMake `-D` flags are written into CMakeCache.txt; *omitting* a flag on a @@ -473,6 +482,15 @@ def stale_feature_cache(build_dir: Path, extra: list[str]) -> str | None: WiFi-only dir kept MM_NO_ETH=1 and stubbed Ethernet out (no link, no LED). Erasing flash doesn't help: it's a compile-time define, not device state. + Also catches a mismatched IDF_TARGET: an earlier `set-target` for this + firmware may have partially succeeded (interrupted, or ran when the + RISC-V toolchain was missing and only got as far as writing sdkconfig for + the default `esp32` target), leaving the build dir with the wrong chip. + The wrapper's "dir exists → skip set-target" fast-path then trusts that + stale state and silently builds for the wrong chip — the P4 case that + bit us in the beta1 bring-up. `chip` is the firmware's expected target + (per FIRMWARES[firmware]["chip"]). + Returns a human-readable reason string when the cache is stale (caller should clean + reconfigure), or None when it matches. """ @@ -480,6 +498,16 @@ def stale_feature_cache(build_dir: Path, extra: list[str]) -> str | None: if not cache.exists(): return None text = cache.read_text(encoding="utf-8", errors="replace") + # Chip / target mismatch — cheap check, do it first (a mismatch here means + # every other cached flag is also for the wrong target, so no point + # checking those). CMake writes `IDF_TARGET:STRING=esp32p4`; a fresh + # cache without an IDF_TARGET line (partial set-target failure before + # configure) also counts as stale. + m = re.search(r"^IDF_TARGET:[^=]*=(.*)$", text, re.MULTILINE) + cached_target = m.group(1).strip() if m else None + if cached_target != chip: + return (f"IDF_TARGET cached as {cached_target!r} but this firmware " + f"wants {chip!r}") # The feature toggles whose presence/absence changes which code compiles. # For each, "wanted" = does this firmware pass the -D, "cached" = is it set # in the existing cache. A disagreement means a stale dir. @@ -536,6 +564,13 @@ def main(): "(see compute_version.py): core for a stable tag, " "'-dev.' for latest. Omit for local builds " "(library.json applies).") + parser.add_argument("--skip-idf-pin-check", action="store_true", + help="Build against the currently-checked-out IDF even if " + "it differs from PINNED_IDF_COMMIT (for a dev " + "deliberately testing a newer IDF release before " + "the pin bumps). Symmetric with setup_esp_idf.py " + "--no-checkout. Use with care: a drifted IDF is the " + "most common source of ESP32 build failures.") args = parser.parse_args() firmware = resolve_firmware(args) @@ -560,8 +595,10 @@ def main(): # Fail-fast on a drifted local IDF before we sink a few minutes into a # build that would end in "SOC_FOO was renamed to SOC_BAR" or similar. See # check_idf_pin above for why this belongs in the build script, not just - # setup_esp_idf.py. - check_idf_pin(idf_path) + # setup_esp_idf.py. --skip-idf-pin-check bypasses the check entirely for a + # dev deliberately testing an upcoming IDF release. + if not args.skip_idf_pin_check: + check_idf_pin(idf_path) env = idf_env(idf_path) cmd = idf_cmd(idf_path) @@ -601,7 +638,7 @@ def main(): # Guard against a build dir configured for a different feature set (a stale # MM_NO_ETH / MM_ETH_ONLY in CMakeCache that a plain reconfigure won't clear). # Wiping the dir forces the set-target path below, which seeds a clean cache. - stale = stale_feature_cache(build_dir, extra) + stale = stale_feature_cache(build_dir, extra, chip) if stale: print(f"Build dir {build_dir.relative_to(ROOT)} has a stale feature " f"cache ({stale}); removing it for a clean reconfigure.")