Read-only VFS layer: user-defined directory listings and synthetic file contents

[congwang-mk]

Problem

Sandlock can synthesize file content per syscall today. python/examples/s3_handlers.py maps a /s3 namespace onto an object store using three handlers (openat, newfstatat, statx), and open("/s3/x").read() plus os.stat("/s3/x") both work.

What it cannot do ergonomically is synthesize a namespace. ls /s3 is empty because nothing serves getdents64, and that caveat is documented in the example itself (line 42).

Writing directory synthesis by hand is possible with the existing Handler API but hostile. A user must:

1. Resolve which directory a getdents64 dirfd refers to by readlinking /proc/<pid>/fd/<n> from the supervisor.
2. Pack linux_dirent64 records with correctly 8-byte-aligned d_reclen.
3. Maintain a per-(pid, fd) cursor, because getdents64 is a stream that must advance across calls and eventually return 0. Getting this wrong makes ls loop forever.
4. Pack struct stat by hand. The existing example does this with a hardcoded x86_64 layout (its own docstring admits this at line 50), which is wrong on our riscv64 test hosts.
5. Keep five or more handlers agreeing on one model of the namespace.

The core has the same problem from the other side. Roughly twelve bespoke synthetic files (/proc/cpuinfo, /proc/meminfo, /proc/uptime, /proc/loadavg, /proc/net/*, /proc/mounts, /etc/hosts, /etc/hostname, CA bundles) live as if path == "..." arms in procfs.rs:446-510, each separately wired into the dispatch table, and none of them appear in a directory listing.

Proposal

A read-only VFS layer in sandlock-core, exposed through the C ABI to Python.

pub trait Vfs: Send + Sync + 'static {
    /// Resolve a path relative to the mount point. `None` => ENOENT.
    /// The mount root itself arrives as an empty path.
    fn lookup(&self, path: &Path) -> Option<Arc<dyn VfsNode>>;
}

pub trait VfsNode: Send + Sync + 'static {
    fn attr(&self) -> VfsAttr;
    fn read(&self) -> io::Result<Vec<u8>>;          // default: EISDIR
    fn readdir(&self) -> io::Result<Vec<VfsEntry>>; // default: ENOTDIR
    fn readlink(&self) -> io::Result<PathBuf>;      // default: EINVAL
}

Registration follows the existing policy_fn precedent, stored as a #[serde(skip)] field on the policy struct next to it:

Sandbox::builder()
    .fs_readable("/usr")
    .vfs_mount("/s3", Arc::new(S3Vfs::new(backend)))
    .build()?

Python:

class Object(VfsNode):
    def __init__(self, body): self.body = body
    def attr(self): return VfsAttr(kind=VfsKind.FILE, size=len(self.body))
    def read(self): return self.body

class S3Vfs(Vfs):
    def lookup(self, path):
        return Object(...) if ... else None   # None => ENOENT

sb.vfs_mount("/s3", S3Vfs(backend))

Plus a shorthand for the dominant static case, which should not need a trait impl at all:

sb.vfs_files("/dev", {"foo": b"contents\n"})

Scope

Read-only. Writes to a VFS path return EROFS. No write, create, unlink, rename, or truncate in the node interface, and therefore no per-fd write state.

Out of scope: ranged reads, chdir into a mount, fstat(fd) reflecting attr() rather than the injected memfd, character-device impersonation, Go bindings.

Design decisions

Sync trait, always dispatched off-loop. Every node call runs via spawn_blocking, so a network-backed VFS cannot stall the supervisor's notification loop, which is the property s3_handlers.py currently buys with async def handle. An async trait would push Pin<Box<dyn Future>> across the C ABI, which is the nastiest part of any Python binding. Cost is one worker hop per call, negligible next to a seccomp notification round trip.

Paths relative to the mount point. lookup("hello.txt"), not lookup("/s3/hello.txt"), so an implementation is reusable at any mount point and nobody hand-strips a prefix the way Namespace.key_for does today.

Per-directory anchor dirs. getdents64 needs a real directory fd, and the handler must recognize which VFS directory that fd refers to. The supervisor keeps a private 0500 tree mirroring the guest namespace under the existing per-sandbox runtime dir (control.rs:45-62, tmpfs-backed, lifecycle already managed):

guest  /s3/docs
anchor /dev/shm/sandlock-<uid>/<name>/vfs/s3/docs

Because the mirror is verbatim under a fixed root, the reverse mapping is string arithmetic and needs no lookup table. Identity is path-based, so it survives dup and fork for free, and if the layer ever fails to fire the anchor is empty, so the guest sees nothing rather than host content.

Dispatch placement, forced rather than chosen.

  /etc/hosts open           (dispatch.rs:450)
  CA inject                 (dispatch.rs:485)
  /proc open                (dispatch.rs:516)
> VFS open + stat + access + readlink        <- new
  chroot handlers           (dispatch.rs:546)
  COW handlers              (dispatch.rs:550)
> VFS getdents64                             <- new
  /proc getdents PID filter (dispatch.rs:559)
  deterministic dirs        (dispatch.rs:636)

Opens must precede chroot and COW for the reason the comment at dispatch.rs:444 gives for /etc/hosts: the chroot handler intercepts every open inside the chroot and would serve the real file first.

getdents64 must precede deterministic_dirs, because handle_sorted_getdents (procfs.rs:765) readlinks the fd, reads the real directory, and returns a ReturnValue unconditionally. Registered after it, a VFS listing would never run and the guest would see the anchor's own empty contents. This breaks silently, so it gets a dedicated regression test.

Builtins register first, so a user mount at /proc cannot displace sensitive-path blocking or PID filtering.

Security

The strongest property is structural: VfsNode returns bytes, never file descriptors. A Vfs implementation is incapable of handing the guest a host fd, unlike a raw Handler, which can inject_fd_send anything the supervisor can open.

The layer answers before the kernel runs the syscall, so Landlock is never consulted and a mount needs no fs_readable grant. Since all content originates with the operator and no host fd can cross the boundary, this grants the guest nothing it did not already have.

readlink("/proc/self/fd/N") on a VFS directory fd would otherwise expose the anchor's host path, so the anchor mapping gets added to the to_virtual transform in procfs.rs:711, which exists to hide chroot host paths for exactly this reason.

Plan

Phase 1

• Vfs / VfsNode / VfsAttr / VfsEntry in sandlock-core, plus a MemVfs convenience impl.
• vfs_mounts on the policy struct, vfs_mount() builder, added to the UnsupportedForConfine list at sandbox.rs:190 (a checkpoint cannot serialize a user trait object).
• Dispatch integration at the two positions above, covering openat/openat2/open, newfstatat/statx/stat/lstat, faccessat2/faccessat/access, readlinkat/readlink, getdents64/getdents, with legacy spellings via arch::sys_*().
• vfs_notif_syscalls() in seccomp_plan.rs gated on a non-empty mount list, mirroring procfs_hosts_notif_syscalls(). Without it the cBPF program never raises the notification and the layer silently never fires.
• Anchor tree lifecycle and the (pid, child_fd, anchor_path) dirent cursor, mirroring procfs_dir_cache (procfs.rs:786-800).
• C ABI (sandlock_vfs_new, sandlock_sandbox_vfs_mount, sandlock_vfs_reply_*) and the Python wrapper.
• Rewrite python/examples/s3_handlers.py onto the VFS as the acceptance test. It should drop from roughly 360 lines to roughly 40, lose its x86_64-only _pack_stat, and gain working ls /s3.

Phase 2

Port exactly one builtin, /etc/hostname, onto an internal ProcVfs, diff-tested against current output. It has no security logic, so if the trait turns out to be the wrong shape we find out on a file nobody's confinement depends on. /proc's sensitive-path blocking and PID filtering are deliberately not migrated.

Known limits

Each open of a VFS file allocates a fresh memfd, so a guest looping open("/s3/huge") amplifies memory by the file size per held fd. The fix (cache one sealed memfd per node and re-open it through /proc/self/fd/N so each open gets its own offset over shared pages) is deferred, since it trades freshness away from dynamic backends and is not needed to prove the design.

fstat(fd) on an opened VFS file reports the injected memfd rather than attr(). This matches existing /etc/hosts and CA-injection behavior, but puts character-device impersonation out of reach in phase 1.

Symlink nodes are reported by stat, lstat, readlink, and listings, but the layer does not resolve them during open, which returns ELOOP. Following would mean reimplementing kernel path resolution for a case no current use needs.

[congwang-mk]

@dzerik Does this make sense to you?

[MuhtasimTanmoy]

Looking into it.

[pro-utkarshM]

@congwang-mk i’ve been digging into this and would like to help with the implementation.

i’m currently looking at the VFS core + directory handling side, especially getdents{,64}, anchor directories, and preserving directory-stream semantics across dup/fork. one thing i’m testing is whether we can use the anchor directory’s open-file-description offset as the synthetic readdir cursor instead of maintaining a separate (pid, fd) cursor.