Place code snapshots outside the repo when save_dir is in-repo

[eugenevinitsky]

What

isolate_code() in scripts/submit_cluster.py now checks whether the resolved save_dir lives inside the resolved project root. If it does, the code snapshot is placed in a sibling directory next to the repo — <repo>_code_snapshots/<save_dir relative path>/code — instead of save_dir/code inside the source tree. Out-of-repo save dirs keep the existing save_dir/code location. The code_vN versioning, symlink farm, pufferlib/ copy, and the return value threading into the job's project_root are all unchanged.

Why

With an in-repo save dir (the common --save_dir experiments), every submission planted a full pufferlib/ copy plus a symlink farm pointing at every top-level repo entry inside the source tree. setuptools egg_info/manifest processing walks the entire tree with os.walk(followlinks=True) (the MANIFEST.in global-include forces the walk; excludes only apply afterwards), so pip install -e . traversed every snapshot's pufferlib copy and chased the symlink farms. Snapshots under two different in-repo save roots symlink each other and form genuine cycles, making editable installs effectively endless — and worse with every submission. Keeping snapshots outside the tree removes them from setuptools file discovery entirely.

Notes

• Verification: py_compile passes; isolate_code was unit-exercised via importlib against fake repo trees — in-repo save_dir lands the snapshot in the <repo>_code_snapshots sibling (with correct copy/symlink structure and working code_vN versioning), out-of-repo save_dir keeps save_dir/code. Grep confirms the single call site (line ~459) needs no change. A real cluster submission was not run.
• Risks: In-repo runs' snapshots move to a new location, so tooling that assumed save_dir/code for such runs must look in the sibling dir (the submit-time >>> Code snapshot: log prints the actual path). The sibling dir requires write access next to the repo — true for the /scratch/$USER/code/ cluster layout.
• Merge-order dependencies: none; single-file change on top of origin/3.0.

🤖 Generated with Claude Code

🤖 Generated with Claude Code