feat: add memory read/write functionality#7
Merged
Conversation
Implement GDB RSP Bridge + GDB Session to replace pyelftools for ELF/DWARF parsing, achieving ~100-300x speedup for symbol and type queries on large ELF files (>100MB). New modules: - core/gdb_bridge.py: Minimal GDB RSP protocol server bridging memory read/write to fl serial commands - core/gdb_session.py: GDB subprocess manager (MI3 protocol) with high-level APIs: lookup_symbol, search_symbols, get_struct_layout - core/gdb_manager.py: Start/stop orchestration with async support Key features: - Auto-detect GDB binary (arm-none-eabi-gdb -> gdb-multiarch fallback) - Const detection from declaration text and ptype qualifier - Address resolution via 'info address' for debug-format symbols - Skip full symbol preload when GDB available (avoids timeout on 200k+ symbols), use per-query search instead - Pyelftools fallback when GDB is not available Route integration: - symbols.py: GDB-first paths for lookup, search, struct layout - connection.py: start_gdb_async on elf_path change, stop on disconnect - state.py: Added gdb_bridge and gdb_session fields Tests: 1518 passed, 7 skipped, coverage 85.2%
- Remove all pyelftools imports and symbol functions from elf_utils.py (get_symbols, search_symbols, lookup_symbol, read_symbol_value, get_struct_layout, DWARF helpers, _classify_symbol) - Add read_symbol_value() to GDBSession using 'x/<N>bx' command - Convert symbols.py routes to GDB-only paths (no fallback) - Stub fpb_inject.get_symbols() for backward compat - Remove pyelftools from requirements.txt - Clean helpers.py: remove symbol preload block - Update all tests to use GDB session mocks - Fix unused imports (time, os, MagicMock)
Replace stubbed get_symbols() calls in inject() and inject_multi() with _resolve_symbol_addr() that queries GDB session directly. This fixes 'Target function not found in ELF' and symbol address=0 bugs introduced by the pyelftools removal refactor.
FASTSHIFT
force-pushed
the
feat_symbol_monitor
branch
from
a3f0d88 to
b71e44b
Compare
Add [GDB] prefixed logs to all GDB session methods: - _execute_cli: log slow commands (>1s as warning, others as debug) - _lookup_symbol_impl: log start/done with address and timing - _search_symbols_impl: log query, result count, and timing - _resolve_addresses: log unresolved count and resolution stats - _get_struct_layout_impl: log member count and timing Add [symbols] prefixed logs to route handlers: - _lookup_symbol: log GDB lookup timing - api_search_symbols: log search delegation and results Signed-off-by: VIFEX <vifextech@foxmail.com>
FASTSHIFT
force-pushed
the
feat_symbol_monitor
branch
from
7eb2b6b to
e90edbc
Compare
- Add minimum query length (3 chars) to avoid broad GDB searches - Cap address resolution to 'limit' results instead of all matches - Add search generation counter for stale search cancellation - Guard gdb_session.get_struct_layout with is_gdb_available check - Fix test_symbol_not_found to mock GDB availability - Reset gdb_session in test tearDown to prevent cross-test leaks
- read_symbol_value: use x/Nwx (word reads) instead of x/Nbx (byte reads) to reduce GDB output volume by 4x, fixing 10s timeout on 616-byte reads - ptype /o: increase timeout to 30s for large structs like lv_global - Add dynamic timeout based on read size
- read_symbol_value: use x/Nwx (word reads) instead of x/Nbx (byte reads) reducing GDB output volume by 4x, fixing timeout on large symbols - ptype /o: increase timeout to 30s for large structs - Guard get_struct_layout with is_gdb_available check - Add dynamic timeout based on symbol size
…tion - Replace hand-written GDB/MI parser with pygdbmi IoManager - Auto-start GDB session on WebServer startup when ELF is configured - Fix address parsing for 'is at 0x...' format (no-debug symbols) - Strip array suffix from symbol names before GDB queries - Add linker-name fallback for scoped C++ symbols (sizeof/ptype) - Log raw GDB output on parse failures for easier debugging - Add pygdbmi to requirements.txt
FASTSHIFT
force-pushed
the
feat_symbol_monitor
branch
from
0591696 to
44a9335
Compare
…ut cache - Replace all hardcoded English text with t() i18n calls - Add translations for zh-CN, zh-TW, en locale files - Replace colored emoji icons with monochrome codicon icons - Change buttons from vscode-button to vscode-btn (match inject style) - Add periodic auto-read with configurable interval - Cache struct_layout results to avoid repeated GDB queries - Clear struct layout cache on symbol reload - Widen auto-read interval input to prevent text clipping
- Restore nm-based get_symbols() in elf_utils.py and fpb_inject.py - Rewrite _ensure_symbols_loaded() to use nm via fpb.get_symbols() - Rewrite api_search_symbols() to use nm-loaded dict for fast search - Add _symbol_detail_cache for GDB detail results (separate from nm) - Update _lookup_symbol() with nm int-addr fallback when GDB unavailable - Handle state.gdb_session being None even when is_gdb_available=True - Update tests to use nm-loaded state.symbols instead of GDB mocks nm loads all symbols (~1s for 200k+) for instant search. GDB is only used for per-symbol detail queries (ptype/sizeof/struct).
Restores nm-based symbol listing via CLI. Supports --filter for pattern matching and --limit for result count control.
…const
nm output column 2 provides symbol type codes (T=function, D/B=variable,
R=const). Now get_symbols() returns {name: {addr, sym_type}} instead of
{name: int}, enabling the frontend to know whether to start GDB for
variable detail queries.
- Add _NM_TYPE_MAP for nm type code -> category mapping
- Update get_symbols() return format to include sym_type
- Update all consumers: symbols.py routes, fpb_cli.py, tests
- Search/list APIs now include type field in response
- Restore missing @bp.route decorators lost during editCode operations
- Add test_symbols.c fixture covering structs, unions, arrays, const, BSS, static, weak and padded types - Add test_symbols_cpp.cpp fixture covering namespaces, classes, vtable, guard variables, templates, operator overloads and mangled names - Add build_test_elf.sh to compile both fixtures with arm-none-eabi-gcc/g++ - Add test_elf_symbols.py with 82 tests (9 classes) for nm + GDB pipeline - Fix gdb_session: recognize C++ 'class' keyword in struct layout parser - Fix gdb_session: detect functions by .text section for mangled names - Fix elf_utils: add nm type 'A'/'a' mapping to 'other' - Fix gdb_session: handle const/volatile qualifiers in struct detection - Add symbol_test_report.md with pipeline analysis and known issues
- Rename all docs to consistent kebab-case naming convention - Unify document titles to English Title Case - Merge 4 redundant log-file docs into single log-file-recording.md
Backend: - Add dynamic timeout based on data size for serial reads - Support partial writes via offset parameter in /api/symbols/write - Add generic memory read/write endpoints (/api/memory/read, /api/memory/write) - Add SSE streaming memory read with progress (/api/memory/read/stream) - Add _parse_addr helper for flexible address parsing Frontend: - Add float/double decode in _decodeFieldValue - Replace single auto-read timer with Map for multi-symbol support - Enforce 500ms minimum auto-read interval - Add writeSymbolField for field-level partial writes - Add readMemoryAddress for arbitrary address reading - Add i18n key symbols.invalid_params to all locales Tests: - Add 28 backend tests (offset write, memory read/write, stream, helpers) - Add 25 frontend tests (auto-read Map, writeSymbolField, readMemoryAddress, float/double decode, toggleAutoRead multi-symbol) - Fix SSE content_type assertion for Flask charset suffix - All CI checks pass: Python 86.6%/85%, JS 80.7%/80% Docs: - Add symbol-viewer-improvement-plan.md with 3-phase roadmap
- WatchEvaluator core: type resolution, address lookup, array slices - REST API: evaluate, deref, CRUD watch list endpoints - Frontend: watch.js module with panel rendering and interaction - Sidebar: watch panel with input, refresh/clear toolbar - Activity bar: add watch shortcut button (codicon-eye) - i18n: watch.* keys for en, zh-CN, zh-TW locales - CSS: watch panel styles in workbench.css - Tests: Python unit tests for evaluator + routes (86.7% coverage) - Tests: JS tests for watch module (80.24% coverage) - Tests: regression test ensuring all sidebar sections have activity bar buttons
- Backend: detect pointer types via GDB whatis, return is_pointer/pointer_target - Backend: skip struct_layout for pointer variables (avoid showing target struct as pointer data) - Backend: add deref flag to /api/symbols/read to read pointed-to data - Frontend: show pointer value prominently with 0x address display - Frontend: add Dereference checkbox to read and display target struct data - Add get_sizeof public method to GDBSession - Add pointer-related CSS styles and i18n keys (en/zh-CN/zh-TW)
…ange - Cache symbol tab data for language re-render - Listen for i18n:translated event to re-render all open symbol viewer tabs - Save/restore scrollTop on readSymbolFromDevice to preserve scroll position - Add 17 tests for pointer rendering, _decodeLittleEndianHex, _renderStructTable
Replace flat struct table with IDE-style collapsible tree view: - Nested structs/arrays render as expandable nodes (click to expand) - Use GDB 'print' for native value decoding (handles typedefs like lv_coord_t) - Pointer members display as 0x hex addresses - Typedef fallback: unknown types with size 1/2/4/8 decoded as integers - Deref checkbox state persisted across auto-read cycles - Add _splitGdbFields, _renderStructTree, _renderTreeNode, _isExpandableValue - Remove unused i18n keys (symbols.field/type/offset/value) - Fix GDB session test mocks for whatis pointer detection
parse_struct_values was receiving sym_name as type_name, causing 'No struct type named disp_def' errors. Now: 1. Try 'print sym_name' directly (works for normal variables) 2. Fallback: whatis to get real type, then cast to address 3. Deref scenario uses 'print *sym_name' to dereference pointer
- Fix values all 0: use whatis + address cast instead of print sym_name to read live device memory instead of ELF initial values - Fix function pointer display: track parenthesis depth in both _parse_gdb_struct_body (Python) and _splitGdbFields (JS) so commas inside function pointer signatures don't split fields - Fix expandable nodes can't toggle: replace inline onclick with event delegation using data-expandable attribute and counter-based IDs - Add Python tests: _parse_gdb_struct_body with function pointers, _parse_struct_values_impl whatis/cast logic (107 passed) - Add JS tests: _splitGdbFields with function pointers, array index display, event delegation tree toggle (1396 passed, coverage 80%)
Replace fragile text parsing of GDB 'print' output with a custom 'json-print' GDB command implemented via the GDB Python API. The new approach: - Registers a 'json-print EXPR [MAX_DEPTH]' command in GDB via gdb_json_print.py, sourced on GDB startup - Uses gdb.parse_and_eval() + recursive gdb.Value traversal to produce structured JSON output - Correctly handles all C types: scalars, pointers, function pointers, enums, bitfields, nested structs, arrays, typedefs - Falls back to legacy text parsing when GDB Python is unavailable Frontend updated to consume structured JSON: - _isExpandableValue: detects objects (structs) and arrays - _formatGdbValue: renders ptr/func_ptr/enum/_kind types - _renderExpandableChildren: handles JSON objects and arrays natively - _renderLegacyExpandableChildren: kept for backward compat - _splitGdbFields: kept as legacy helper Tests: 1417 JS passed (80.25% coverage), 110 Python passed
Move hex-to-value decoding from frontend JS to backend Python so the /symbols/read (device read) route returns fully decoded gdb_values. Problem: device reads via serial returned correct hex_data, but gdb_values was obtained by re-reading device memory through GDB RSP, which returned stale ELF initial values (zeros for .bss variables). Solution: - Add _decode_field_value() and _decode_struct_values() in symbols.py to decode hex bytes using struct_layout type info (pointers, integers, floats, char arrays, typedef fallback) - /symbols/read now calls _decode_struct_values(struct_layout, hex_data) instead of _get_gdb_values() which re-read via GDB RSP - /symbols/value (ELF initial load) still uses GDB json-print for rich type info (enum names, function pointer signatures) - Frontend keeps hex decode as fallback safety net, primary values come from backend gdb_values Tests: 238 Python passed (28 new decode tests), 1416 JS passed, lint clean
- nested struct: query GDB for sub-struct layout and decode recursively, producing nested dicts for frontend tree-view expansion - progress bar: add indeterminate CSS animation during device read, shown via .sym-read-progress.active class toggle - deref log: add log.success() for dereferenced pointer reads - GDB fallback: /symbols/value returns success with nm-only info (addr, size, type) when GDB unavailable, instead of error - test speed: share single GDB process across integration tests, reduce pygdbmi poll interval from 0.3s to 0.05s (18s -> 4.5s) - test fix: update test_no_gdb to match degraded-success behavior, add test_no_gdb_symbol_in_nm_cache for graceful degradation path
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
增强SYMBOL功能交互，支持读写设备内存。