Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 47 additions & 0 deletions .dev/decisions.md
Original file line number Diff line number Diff line change
Expand Up @@ -363,4 +363,51 @@ Backward compatible — NULL config uses default allocator.
**Precedents**: SQLite (`SQLITE_CONFIG_MALLOC`), Lua (`lua_newstate(alloc_fn, ud)`),
jemalloc, mimalloc — all accept custom allocators from the host.

---

## D129: Windows First-Class Support — Platform Abstraction

**Date**: 2026-03-15
**Status**: Complete (PR #8, commit 48f68a7)
**Decision**: Add Windows x86_64 as a first-class target via platform abstraction
layer, without compromising Mac/Linux code quality.

**Problem**: zwasm used POSIX APIs directly (mmap, mprotect, signals, fd_t).
Windows requires VirtualAlloc, VEH, HANDLE-based I/O.

**Design**:

1. **`platform.zig`** — Unified OS abstraction for page-level memory:
- `reservePages`/`commitPages`/`protectPages`/`freePages` (mmap ↔ VirtualAlloc)
- `flushInstructionCache` (sys_icache_invalidate / __clear_cache / FlushInstructionCache)
- `appCacheDir`/`tempDirPath` (cross-platform paths)

2. **`guard.zig`** — OOB trap via VEH on Windows:
- POSIX: SIGSEGV signal handler modifies ucontext PC
- Windows: VEH handler modifies CONTEXT.Rip/Pc on EXCEPTION_ACCESS_VIOLATION
- Same recovery logic (JIT code range check → redirect to OOB exit stub)

3. **`wasi.zig`** — HostHandle abstraction:
- `posix.fd_t` → `HostHandle { raw: Handle, kind: .file|.dir }`
- POSIX file ops (read/write/lseek) → `std.fs.File` methods
- `path_open`: Windows uses `Dir.openDir`/`createFile`; POSIX keeps `openat`
- `FdEntry.append` field for Windows O_APPEND emulation

4. **`x86.zig`** — Win64 ABI support:
- SysV: RDI/RSI/RDX args, RDI/RSI caller-saved
- Win64: RCX/RDX/R8 args, RDI/RSI callee-saved, 32-byte shadow space
- Compile-time dispatch via `abiRegsArg()`/`abiVmArg()`/`abiInstArg()`

5. **Test infrastructure** — bash → Python migration:
- All test runners rewritten in Python for cross-platform support
- bash wrappers retained for Mac/Linux backward compatibility
- `select.select()` → `queue.Queue` + threading (Windows stdio)

**Scope**: x86_64 Windows only. ARM64 Windows deferred (no test hardware).

**Trade-offs**:
- `writeFilestat`: nlink always 1 on portable path (std.fs.File.Stat lacks nlink)
- `path_filestat_get`: POSIX retains fstatat for SYMLINK_NOFOLLOW; Windows always follows
- Binary size/memory checks skipped on Windows CI (no strip/time -v equivalents)

Related: D126 (C API), D127 (conditional compilation), CW D110, cw-new D13.
42 changes: 19 additions & 23 deletions .dev/memo.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,38 +4,34 @@ Session handover document. Read at session start.

## Current State

- Stages 0-46 + Phase 1, 3, 5 complete. **v1.3.0 released** (tagged 7570170).
- Spec: 62,263/62,263 Mac+Ubuntu (100.0%, 0 skip). E2E: 792/792 (100.0%, 0 leak).
- Stages 0-46 + Phase 1, 3, 5, 8, 11, 15 complete. **v1.5.0** (tagged 48342ab).
- Spec: 62,263/62,263 Mac+Ubuntu+Windows (100.0%, 0 skip). E2E: 792/792. Real-world: 50/50.
- Wasm 3.0: all 9 proposals. WASI: 46/46 (100%). WAT parser complete.
- JIT: Register IR + ARM64/x86_64. Size: 1.20MB stripped. RSS: 4.48MB.
- Module cache: `zwasm run --cache`, `zwasm compile` (D124).
- **C API**: `libzwasm.so`/`.dylib`/`.a` — 25 exported `zwasm_*` functions (D126).
- JIT: Register IR + ARM64/x86_64 (macOS, Linux, Windows x86_64). Size: 1.22MB stripped.
- **Windows x86_64**: First-class support (D129, PR #8). platform.zig abstraction,
VEH guard pages, Win64 ABI, HostHandle WASI, Python test runners, CI 3-OS.
- **C API**: `libzwasm` (.dll/.lib, .dylib/.a, .so/.a) — 25 exported functions (D126).
- **Conditional compilation**: `-Djit=false`, `-Dcomponent=false`, `-Dwat=false` (D127).
Minimal build: ~940KB stripped (24% reduction).
- **Phase 8 merged to main** (d770bfe). Real-world compat: 50/50 (Mac+Ubuntu).
- **Phase 11 merged to main** (49f99e5). C API allocator injection (D128).
- **main = stable**: v1.5.0 tagged (48342ab). ClojureWasm updated to v1.5.0.
- **main = stable**: v1.5.0. ClojureWasm updated to v1.5.0.

## Current Task

**Fix JIT fuel bypass + PR #6 timeout merge**
**PR #8 Windows support review + merge** (branch: `fix/pr8-review-fixes`)

- [x] Code review: 26 files, platform.zig / guard.zig / wasi.zig / x86.zig / CI
- [x] Fix run_compat.py rust_file_io regression (guest path alias)
- [x] Fix path_filestat_get SYMLINK_NOFOLLOW restoration
- [x] Fix README Stage 33 duplicate, run_spec.py .exe detection
- [x] Code quality: VEH constant, HostHandle.close(), fd placeholder docs
- [x] Doc updates: embedding.md, security.md, roadmap.md, decisions.md (D129)
- [ ] Merge Gate (Mac + Ubuntu)
- [ ] Benchmark recording
- [ ] Push to PR branch → merge via GitHub

### Pending: JIT fuel bypass + PR #6 timeout
Checklist: `@./.dev/checklist-jit-fuel-timeout.md`
PR review: `@./private/pr6-timeout-review.md`

### Phase A: Fix JIT fuel bypass (branch: `fix/jit-fuel-bypass`)
- [x] A1. Add `jitSuppressed()` — suppress JIT when `fuel != null` (6 locations in vm.zig)
- [x] A2. Test: infinite loop with fuel=1M terminates (`30_infinite_loop.wasm`)
- [ ] A3. Commit Gate: `zig build test` pass, spec/e2e/realworld/bench (running)
- [ ] A4. Merge Gate (Mac + Ubuntu)

### Phase B: Merge timeout support (PR #6 + additions)
- [ ] B1. Apply PR #6 changes (TimeoutExceeded, deadline, consumeInstructionBudget)
- [ ] B2. Add `--timeout <ms>` CLI option
- [ ] B3. Tests + verify with JIT enabled
- [ ] B4. Commit + Merge Gate
- [ ] B5. Comment on PR #6, credit DeanoC

## Handover Notes

### JIT fuel/timeout suppression — current fix vs proper solution
Expand Down
20 changes: 11 additions & 9 deletions .dev/roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,16 +148,17 @@ Largest technical challenge.

**Gate**: SIMD bench faster than scalar. zwasm v2.0.0 candidate.

### Phase 15: Windows Port (3 days)
### Phase 15: Windows Port (3 days) — DONE (PR #8, D129)

- D## decision record (SEH, VirtualAlloc, CI strategy)
- Memory management OS abstraction (mmap → VirtualAlloc)
- Signal handler port (SIGSEGV → SEH)
- JIT W^X port (VirtualProtect + FlushInstructionCache)
- WASI filesystem Windows branch
- CI Windows job + release binaries
- [x] D129 decision record (VEH, VirtualAlloc, CI strategy)
- [x] Memory management OS abstraction (mmap → VirtualAlloc) — `platform.zig`
- [x] Signal handler port (SIGSEGV → VEH) — `guard.zig`
- [x] JIT W^X port (VirtualProtect + FlushInstructionCache) — `jit.zig`
- [x] WASI filesystem Windows branch — `wasi.zig` HostHandle abstraction
- [x] CI Windows job + release binaries — `ci.yml`, `release.yml`
- [x] x86_64 JIT Win64 ABI (RCX/RDX/R8, shadow space) — `x86.zig`

**Gate**: Windows x86_64 all tests pass. 3-OS support complete.
**Gate**: Windows x86_64 all tests pass. 3-OS CI complete.

### Phase 18: Book i18n + Lazy Compilation + CLI Extensions (3 days)

Expand Down Expand Up @@ -195,7 +196,8 @@ JIT deferred to first call. Trampoline → direct jump patch.
| **v1.3.0** | 1, 3 | Guard pages, cache, CI automation, ARCHITECTURE.md |
| **v1.4.0** | 5, 8 | C API, conditional compilation, 50+ real-world, WAT parity |
| **v1.5.0** | 11 | Allocator injection, C API config, embedding docs |
| **v2.0.0** | 13, 15 | SIMD JIT, Windows, 3-OS support |
| **v1.6.0** | 15 | Windows x86_64, 3-OS CI, platform abstraction |
| **v2.0.0** | 13 | SIMD JIT (NEON/SSE) |

## Benchmark History

Expand Down
125 changes: 86 additions & 39 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,13 @@ on:

jobs:
test:
defaults:
run:
shell: bash
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest]
os: [ubuntu-latest, macos-latest, windows-latest]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
Expand All @@ -21,13 +24,19 @@ jobs:
with:
version: 0.15.2

- name: Install Python
uses: actions/setup-python@v5
with:
python-version: '3.x'

- name: Cache Zig build artifacts
uses: actions/cache@v4
with:
path: |
.zig-cache
zig-cache
~/.cache/zig
~/AppData/Local/zig
key: zig-${{ runner.os }}-${{ hashFiles('build.zig', 'build.zig.zon', 'src/**/*.zig') }}
restore-keys: |
zig-${{ runner.os }}-
Expand All @@ -38,54 +47,59 @@ jobs:
- name: Run unit tests
run: zig build test

- name: Run C API tests
run: zig build c-test

- name: Setup Rust
run: |
source .github/tool-versions
rustup install "$RUST_VERSION" --no-self-update
rustup default "$RUST_VERSION"
rustup target add wasm32-wasip1
rustc --version

- name: Install wasm-tools
run: |
source .github/tool-versions
ARCH=$(uname -m)
[ "$ARCH" = "arm64" ] && ARCH="aarch64"
OS=${{ runner.os == 'Linux' && '"linux"' || '"macos"' }}
ARCHIVE="wasm-tools-${WASM_TOOLS_VERSION}-${ARCH}-${OS}"
curl -L -o /tmp/wasm-tools.tar.gz \
"https://github.com/bytecodealliance/wasm-tools/releases/download/v${WASM_TOOLS_VERSION}/${ARCHIVE}.tar.gz"
tar xzf /tmp/wasm-tools.tar.gz -C /tmp
echo "/tmp/${ARCHIVE}" >> "$GITHUB_PATH"
cargo install wasm-tools --locked --version "$WASM_TOOLS_VERSION"

- name: Download WebAssembly spec testsuite
run: git clone --depth 1 https://github.com/WebAssembly/spec.git /tmp/wasm-spec
run: git clone --depth 1 https://github.com/WebAssembly/spec.git "${{ runner.temp }}/wasm-spec"

- name: Convert spec tests
run: bash test/spec/convert.sh /tmp/wasm-spec/test/core
run: python test/spec/convert.py "${{ runner.temp }}/wasm-spec/test/core"

- name: Run spec tests (strict)
run: python3 test/spec/run_spec.py --build --summary --strict
run: python test/spec/run_spec.py --build --summary --strict

- name: Download wasmtime misc_testsuite
run: |
git clone --depth 1 --filter=blob:none --sparse \
https://github.com/bytecodealliance/wasmtime.git /tmp/wasmtime
cd /tmp/wasmtime
https://github.com/bytecodealliance/wasmtime.git "${{ runner.temp }}/wasmtime"
cd "${{ runner.temp }}/wasmtime"
git sparse-checkout set tests/misc_testsuite

- name: Run E2E tests
env:
WASMTIME_MISC_DIR: /tmp/wasmtime/tests/misc_testsuite
run: bash test/e2e/run_e2e.sh --convert --summary
WASMTIME_MISC_DIR: ${{ runner.temp }}/wasmtime/tests/misc_testsuite
run: python test/e2e/run_e2e.py --convert --summary

- name: Build ReleaseSafe
run: zig build -Doptimize=ReleaseSafe

- name: Binary size check
if: runner.os != 'Windows'
run: |
BINARY=zig-out/bin/zwasm
# Strip debug info for size measurement (Linux ELF includes DWARF
# inline while macOS uses separate .dSYM). We check distributable size.
cp "$BINARY" /tmp/zwasm_size_check
strip /tmp/zwasm_size_check 2>/dev/null || true
SIZE_BYTES=$(wc -c < /tmp/zwasm_size_check | tr -d ' ')
SIZE_MB=$(python3 -c "print(f'{$SIZE_BYTES / 1048576:.2f}')")
SIZE_MB=$(python -c "print(f'{$SIZE_BYTES / 1048576:.2f}')")
LIMIT_BYTES=1572864
RAW_BYTES=$(wc -c < "$BINARY" | tr -d ' ')
RAW_MB=$(python3 -c "print(f'{$RAW_BYTES / 1048576:.2f}')")
RAW_MB=$(python -c "print(f'{$RAW_BYTES / 1048576:.2f}')")
echo "Binary size (raw): ${RAW_MB} MB ($RAW_BYTES bytes)"
echo "Binary size (stripped): ${SIZE_MB} MB ($SIZE_BYTES bytes)"
if [ "$SIZE_BYTES" -gt "$LIMIT_BYTES" ]; then
Expand All @@ -95,6 +109,7 @@ jobs:
echo "PASS: Within 1.5 MB limit"

- name: Memory usage check
if: runner.os != 'Windows'
run: |
BINARY=zig-out/bin/zwasm
LIMIT_KB=4608 # 4.5 MB
Expand All @@ -106,7 +121,7 @@ jobs:
MEM_OUTPUT=$(/usr/bin/time -v "$BINARY" --invoke sieve bench/wasm/sieve.wasm 1000000 2>&1 >/dev/null)
MEM_KB=$(echo "$MEM_OUTPUT" | grep "Maximum resident set size" | awk '{print $NF}')
fi
MEM_MB=$(python3 -c "print(f'{int(${MEM_KB}) / 1024:.2f}')")
MEM_MB=$(python -c "print(f'{int(${MEM_KB}) / 1024:.2f}')")
echo "Peak memory: ${MEM_MB} MB (${MEM_KB} KB)"
if [ "$MEM_KB" -gt "$LIMIT_KB" ]; then
echo "FAIL: Peak memory exceeds 4.5 MB limit"
Expand All @@ -117,49 +132,81 @@ jobs:
- name: Install wasmtime
run: |
source .github/tool-versions
if [ "$(uname)" = "Darwin" ]; then
ARCH="aarch64"; OS="macos"
if [ "${{ runner.os }}" = "macOS" ]; then
ARCH="aarch64"; OS="macos"; EXT="tar.xz"; BIN_NAME="wasmtime"
elif [ "${{ runner.os }}" = "Windows" ]; then
ARCH="x86_64"; OS="windows"; EXT="zip"; BIN_NAME="wasmtime.exe"
else
ARCH="x86_64"; OS="linux"
ARCH="x86_64"; OS="linux"; EXT="tar.xz"; BIN_NAME="wasmtime"
fi
ARCHIVE="wasmtime-v${WASMTIME_VERSION}-${ARCH}-${OS}.tar.xz"
curl -L -o /tmp/wasmtime.tar.xz \
ARCHIVE="wasmtime-v${WASMTIME_VERSION}-${ARCH}-${OS}.${EXT}"
curl -L -o "${{ runner.temp }}/wasmtime.${EXT}" \
"https://github.com/bytecodealliance/wasmtime/releases/download/v${WASMTIME_VERSION}/${ARCHIVE}"
mkdir -p "$HOME/.wasmtime/bin"
tar xJf /tmp/wasmtime.tar.xz -C /tmp
cp "/tmp/wasmtime-v${WASMTIME_VERSION}-${ARCH}-${OS}/wasmtime" "$HOME/.wasmtime/bin/"
if [ "$EXT" = "zip" ]; then
unzip -q "${{ runner.temp }}/wasmtime.${EXT}" -d "${{ runner.temp }}/wasmtime-extract"
cp "${{ runner.temp }}/wasmtime-extract/wasmtime-v${WASMTIME_VERSION}-${ARCH}-${OS}/${BIN_NAME}" "$HOME/.wasmtime/bin/"
else
tar xJf "${{ runner.temp }}/wasmtime.${EXT}" -C "${{ runner.temp }}"
cp "${{ runner.temp }}/wasmtime-v${WASMTIME_VERSION}-${ARCH}-${OS}/${BIN_NAME}" "$HOME/.wasmtime/bin/"
fi

- name: Install WASI SDK
if: runner.os != 'Windows'
run: |
source .github/tool-versions
if [ "$(uname)" = "Darwin" ]; then
if [ "${{ runner.os }}" = "macOS" ]; then
ARCH="arm64"; OS="macos"
else
ARCH="x86_64"; OS="linux"
fi
ARCHIVE="wasi-sdk-${WASI_SDK_VERSION}.0-${ARCH}-${OS}.tar.gz"
curl -L -o /tmp/wasi-sdk.tar.gz \
curl -L -o "${{ runner.temp }}/wasi-sdk.tar.gz" \
"https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-${WASI_SDK_VERSION}/${ARCHIVE}"
sudo mkdir -p /opt/wasi-sdk
sudo tar xzf /tmp/wasi-sdk.tar.gz -C /opt/wasi-sdk --strip-components=1
mkdir -p "${{ runner.temp }}/wasi-sdk"
tar xzf "${{ runner.temp }}/wasi-sdk.tar.gz" -C "${{ runner.temp }}/wasi-sdk" --strip-components=1

- name: Setup Rust wasm32-wasip1 target
- name: Install WASI SDK (Windows)
if: runner.os == 'Windows'
shell: python
run: |
source .github/tool-versions
rustup install "$RUST_VERSION" --no-self-update
rustup default "$RUST_VERSION"
rustup target add wasm32-wasip1
rustc --version
import pathlib
import tarfile
import urllib.request

tool_versions = pathlib.Path(".github/tool-versions").read_text(encoding="utf-8").splitlines()
wasi_sdk_version = next(
line.split("=", 1)[1].strip().strip('"')
for line in tool_versions
if line.startswith("WASI_SDK_VERSION=")
)
archive = pathlib.Path(r"${{ runner.temp }}\wasi-sdk.tar.gz")
dest = pathlib.Path(r"${{ runner.temp }}\wasi-sdk")
dest.mkdir(parents=True, exist_ok=True)

url = (
f"https://github.com/WebAssembly/wasi-sdk/releases/download/"
f"wasi-sdk-{wasi_sdk_version}/wasi-sdk-{wasi_sdk_version}.0-x86_64-windows.tar.gz"
)
urllib.request.urlretrieve(url, archive)

with tarfile.open(archive, "r:gz") as tf:
for member in tf.getmembers():
parts = pathlib.PurePosixPath(member.name).parts
if len(parts) <= 1:
continue
member.name = str(pathlib.PurePosixPath(*parts[1:]))
tf.extract(member, dest)

- name: Build real-world wasm programs
env:
WASI_SDK_PATH: /opt/wasi-sdk
run: bash test/realworld/build_all.sh
WASI_SDK_PATH: ${{ runner.temp }}/wasi-sdk
run: python test/realworld/build_all.py

- name: Run real-world compat tests
run: |
export PATH="$HOME/.wasmtime/bin:$PATH"
bash test/realworld/run_compat.sh
python test/realworld/run_compat.py

size-matrix:
runs-on: ubuntu-latest
Expand Down
Loading
Loading