Introduce fastly-compute-py build tool

.github/workflows/python-ci.yml

@@ -1,15 +1,14 @@
 name: Python CI
 
 on:
-  push:
-    branches:
-      - main
+  push:  # Runs on pushes to all branches
   pull_request:
     branches:
       - main
+  workflow_dispatch:  # Allows manual triggering from GitHub Actions UI
 
 env:
-  VICEROY_TAG: v0.16.2
+  VICEROY_TAG: v0.16.3
 
 jobs:
   build:
@@ -29,34 +28,76 @@ jobs:
     - name: Set up Rust toolchain
       uses: actions-rust-lang/setup-rust-toolchain@v1
       with:
-        toolchain: '1.86.0'
+        toolchain: 1.92.0
         target: wasm32-unknown-unknown
+        components: rustfmt, clippy
+        # Disable the built-in cargo cache - we'll use a more sophisticated setup below
+        cache: false
+    - name: Set up nightly Rust toolchain with rust-src
+      run: |
+        rustup toolchain install nightly --component rust-src
+        rustup target add wasm32-wasip1 --toolchain nightly
     - name: Set up Python
       uses: actions/setup-python@v5
       with:
         python-version: '3.12'
     - name: Install uv
       run: pip install uv
-    - name: Install dependencies
-      run: uv sync
-    - name: Cache viceroy, wasm-tools, etc.
-      id: cache-cargo
+
+    # Cache cargo binaries (viceroy, wasm-tools, etc.)
+    - name: Cache cargo binaries
+      id: cache-cargo-bins
       uses: actions/cache@v4
       with:
-        key: "viceroy-$VICEROY_TAG"
+        key: cargo-bins-${{ runner.os }}-${{ env.VICEROY_TAG }}
         path: |
-            ~/.cargo/bin/
+            ~/.cargo/bin/viceroy*
+            ~/.cargo/bin/wasm-tools*
+            ~/.cargo/bin/wac*
     - name: Install wasm-tools and wac
-      if: steps.cache-cargo.outputs.cache-hit != 'true'
+      if: steps.cache-cargo-bins.outputs.cache-hit != 'true'
       run: cargo install wasm-tools wac-cli
     - name: Install viceroy
-      if: steps.cache-cargo.outputs.cache-hit != 'true'
+      if: steps.cache-cargo-bins.outputs.cache-hit != 'true'
       run: cargo install --git https://github.com/fastly/Viceroy.git --tag "$VICEROY_TAG" viceroy
-    - name: Install dependencies
-      run: uv sync --extra dev --extra test
+
+    # Cache Rust dependencies and build artifacts
+    # CRITICAL: ~/.cargo/git/checkouts/ contains componentize-py source + built CPython (5+ min build)
+    - name: Cache Rust dependencies
+      uses: actions/cache@v4
+      with:
+        path: |
+          ~/.cargo/registry/index/
+          ~/.cargo/registry/cache/
+          ~/.cargo/git/db/
+          ~/.cargo/git/checkouts/
+          target/
+        key: cargo-deps-${{ runner.os }}-${{ hashFiles('**/Cargo.lock') }}-v3
+        restore-keys: |
+          cargo-deps-${{ runner.os }}-${{ hashFiles('**/Cargo.lock') }}-
+          cargo-deps-${{ runner.os }}-
+
+    # Setup sccache for Rust compilation caching
+    - name: Run sccache-cache
+      uses: mozilla-actions/sccache-action@v0.0.8
+    - name: Configure sccache
+      run: |
+        echo "SCCACHE_GHA_ENABLED=true" >> $GITHUB_ENV
+        echo "RUSTC_WRAPPER=sccache" >> $GITHUB_ENV
+
+    # Build the Rust native extension (this is the slow part - builds componentize-py + CPython)
+    - name: Build native extension
+      run: uv run maturin develop --release
+
+    # Install Python dependencies
+    - name: Install Python dependencies
+      run: uv sync --extra dev --extra test --extra examples
+
+    # Finally, do our final checks
     - name: Check formatting
       run: make format-check
     - name: Run linting
       run: make lint
     - name: Run tests
-      run: make test
+      # Use DEV_MODE=0 to use the installed fastly-compute-py binary instead of rebuilding with cargo
+      run: DEV_MODE=0 make test

.gitignore

@@ -7,3 +7,9 @@ __pycache__
 
 # Build artifacts
 /build/
+bin/
+
+# Rust
+target/
+*.so
+target/

.gitmodules

@@ -1,3 +1,3 @@
-[submodule "vendor/wasiless"]
-	path = vendor/wasiless
+[submodule "crates/wasiless"]
+	path = crates/wasiless
 	url = git@github.com:fastly/wasiless.git

CONTRIBUTING.md

@@ -0,0 +1,143 @@
+# Contributing to Fastly Compute Python SDK
+
+Thank you for your interest in contributing! This guide will help you get set up for development.
+
+## Prerequisites
+
+### Required Tools
+
+The build process requires several tools to be installed:
+
+1. **Python 3.12+**
+   ```bash
+   python --version  # Should be 3.12 or higher
+   ```
+
+2. **uv** - Python package manager
+3. **Rust toolchain** (stable)
+4. **wasm32-unknown-unknown target** (required by build.rs)
+   ```bash
+   rustup target add wasm32-unknown-unknown
+   ```
+
+5. **wasm-tools** (required by build.rs for WIT merging and componentization)
+6. **Viceroy** - Fastly's local testing server
+
+## Getting Started
+
+1. **Clone the repository**
+   ```bash
+   git clone <repo-url>
+   cd compute-sdk-python/build-tool-impl
+   ```
+
+2. **Initialize submodules** (if applicable)
+   ```bash
+   git submodule update --init --recursive
+   ```
+
+3. **Install Python dependencies**
+   ```bash
+   uv sync --extra dev --extra test
+   ```
+
+4. **Verify setup**
+   ```bash
+   make help  # Should show available commands
+   ```
+
+## Development Workflow
+
+### Building Examples
+
+The default development workflow uses `cargo run` which automatically picks up Rust changes:
+
+```bash
+# Build an example
+make build/bottle-app.composed.wasm
+
+# Build all examples
+make
+
+# Serve an example for testing
+make serve EXAMPLE=bottle-app
+```
+
+### Making Changes to the Build Tool
+
+The build tool is in `crates/fastly-compute-py/`. When you make changes:
+
+```bash
+# The build system automatically rebuilds via `cargo run`
+make build/bottle-app.composed.wasm
+
+# Or test the installed entry point
+make DEV_MODE=0 build/bottle-app.composed.wasm
+```
+
+### Code Quality
+
+```bash
+# Format code (Python + Rust)
+make format
+
+# Check formatting
+make format-check
+
+# Run linters (Python + Rust)
+make lint
+
+# Auto-fix linting issues
+make lint-fix
+```
+
+### Testing
+
+```bash
+# Run all tests
+make test
+
+# Update snapshot tests
+make test-update-snapshots
+```
+
+## Project Structure
+
+```
+.
+├── crates/
+│   ├── fastly-compute-py/   # Rust build tool
+│   │   ├── build.rs         # Build script (requires wasm-tools)
+│   │   └── src/
+│   └── wasiless/            # WASM component for WASI removal
+├── examples/                 # Example applications
+│   ├── bottle-app/
+│   ├── flask-app/
+│   └── ...
+├── fastly_compute/          # Python SDK
+├── wit/                     # WIT (WebAssembly Interface Type) definitions
+└── tests/                   # Integration tests
+```
+
+## Build Process
+
+Understanding the build process helps when debugging issues:
+
+1. **build.rs runs** (during Rust compilation):
+   - Calls `wasm-tools component wit` to merge WIT files
+   - Builds `wasiless` crate for wasm32-unknown-unknown
+   - Calls `wasm-tools component new` to componentize wasiless
+
+2. **fastly-compute-py runs**:
+   - Resolves Python dependencies from virtualenv
+   - Calls `componentize-py` to build Python WASM component
+   - Composes with wasiless using WAC
+
+## Continuous Integration
+
+Our CI builds wheels for multiple platforms:
+- Linux: x86_64, aarch64
+- macOS: x86_64 (Intel), aarch64 (Apple Silicon)
+- Windows: x86_64
+
+The CI workflow (`.github/workflows/build-wheels.yml`) ensures all required tools are installed automatically.