feat(integrations): Harbor agent-phase fidelity

docs/v6/advanced/harbor-convert.mdx

@@ -4,80 +4,48 @@ description: "Load Harbor tasks into the HUD runtime or export HUD tasks to Harb
 icon: "ship"
 ---
 
-Everything that authors tasks - HUD's own `env.py`, platform rows, **Harbor**
-task dirs - is a *frontend* that loads into the same primitives (`Environment`,
-`Task`, `Taskset`). Integrations are **loaders, not converters**: no codegen
-roundtrip to run foreign tasks. Each one implements
-[`hud.environment.Integration`](https://github.com/hud-evals/hud-python/blob/main/hud/environment/integration.py)
-- `load(ref) -> Taskset` and `environment(ref) -> Environment` - and ships as
-the repository's `integrations/<format>`.
-
-Harbor's agent works *inside* its container, so its environment constructor is
-only meaningful in there. `adapt()` packages it: one HUD-speaking image per
-distinct build context, whose CMD serves `harbor:environment`.
-The same rows then run on any container placement.
+`adapt()` turns Harbor task directories into the same `Environment`, `Task`, and
+`Taskset` primitives used by authored HUD environments. It builds the Harbor
+environment first, then wraps that image with the checked-in
+`integrations/harbor/Dockerfile`, `env.py`, and `install.sh`. The generated
+directory contains task data, not generated source.
 
 ## Prerequisites
 
 - A Harbor task directory - each task has `task.toml` + `instruction.md`, and
   usually an `environment/` (with a `Dockerfile`) and `tests/`.
 
-## Load Harbor tasks
+## Run Harbor tasks
 
-`load(path)` parses a Harbor task dir (or a dataset of them) into a `Taskset`
-directly - one row per task dir (`id` = the dir name), sharing one declarative
-`Environment` per distinct `environment/` build context:
+Build the images and receive a runnable taskset:
 
 ```python
 from integrations import harbor
+from hud.eval import DockerRuntime
 
-assert harbor.detect("./terminal-bench")
-taskset = harbor.load("./terminal-bench")
-
-for task in taskset:
-    print(task.env, task.id, task.columns["difficulty"])
-```
-
-Each row carries what the task declared: `[metadata]` as `columns` (difficulty,
-category, tags - the platform's filter/leaderboard facets) and
-`[environment]` cpu/memory/gpu as `runtime_config`. Time budgets stay off the
-row, since they bound the *rollout* rather than the substrate - read one with
-`harbor.agent_timeout(task_dir)` and pass it as `rollout_timeout`.
-
-## Run Harbor tasks
-
-Build the images once, then place the rows anywhere:
-
-```python
-images = await harbor.adapt("./terminal-bench")          # {env name: image ref}
-taskset = harbor.load("./terminal-bench", images=images) # rows carry the image
-job = await taskset.run(agent, runtime=harbor.docker_runtime())
+taskset = await harbor.adapt("./terminal-bench")
+job = await taskset.run(agent, runtime=DockerRuntime())
 ```
 
-`harbor.docker_runtime()` is a `DockerRuntime` that permits nested
-namespaces: an adapted image sandboxes *inside itself* to keep the baked
-tests and the verdict away from the agent, which plain images neither need
-nor should be given.
+`DockerRuntime` supplies the inner workspace sandbox needed by adapted images;
+the setting is automatic for all HUD Docker environments.
 
-The mapping is a value you hold - nothing is written into the dataset - so
-there is no cache to go stale. `adapt(path, push="registry.io/acme")` pushes
-instead; `hud deploy` of the generated contexts under `.hud-adapt/` works the
-same way, since both run the image's own CMD.
+`adapt(path, push="registry.io/acme")` pushes the wrapped images and returns
+rows bound to those refs. The readable build inputs are left under
+`.hud-adapt/`; `adapt()` owns the two-image build because the wrapper's base is
+the Harbor environment it just built.
 
-The image also carries what the task declared about its environment:
-`[environment.env]`, `workdir` and `user` become `ENV`, `WORKDIR` and `USER`
-directives, so Docker applies them to every process - the agent's shells, the
-verifier, the serving process alike.
+Each row carries `[metadata]` as `columns` and CPU, memory, and GPU requirements
+as `runtime_config`. The image's `tasks.json` carries workdir, environment,
+network, and phase-user declarations into the authored HUD environment.
 
 Tasks whose declared behaviour cannot be reproduced faithfully are refused
 rather than silently downgraded. Adaptation replaces the container's own boot
-process with the serving command, so anything depending on that boot process is
-refused: a Dockerfile `ENTRYPOINT`, `healthcheck` (nothing would start the
-services it awaits), and `mcp_servers` (nothing would start the servers they
-point at) - alongside compose environments, `network_mode = "allowlist"`, a
-verifier with its own environment, prebuilt `docker_image` environments,
-non-linux `os`, TPUs, multi-step `[[steps]]` tasks, and an agent and verifier
-that must run as *different* users (one image, one `USER`).
+process with the HUD server, so `healthcheck` and `mcp_servers` declarations are
+refused when they depend on that process. Compose environments, separate
+verifier environments, non-Linux tasks, TPUs, skills directories, and
+multi-step tasks are also refused. Prebuilt images, allowlists, and distinct
+agent/verifier users are supported.
 
 ## Export HUD tasks to Harbor
 

docs/v6/more/faq.mdx

@@ -100,7 +100,7 @@ Evals are a complete use on their own - write tasks, run them across models, rea
 </Accordion>
 
 <Accordion title="Can I bring an existing benchmark or tasks?">
-Yes. The Harbor integration loads Harbor-format tasks straight into a `Taskset` (`integrations.harbor.load`), no conversion round-trip needed. And a whole benchmark can become one generative task definition. See [Harbor interop](/v6/advanced/harbor-convert).
+Yes. `integrations.harbor.adapt()` builds Harbor-format tasks as a runnable HUD `Taskset`. And a whole benchmark can become one generative task definition. See [Harbor interop](/v6/advanced/harbor-convert).
 </Accordion>
 
 <Accordion title="Does HUD support robotics / VLA policies?">

docs/v6/reference/cli.mdx

@@ -132,8 +132,8 @@ hud sync tasks my-taskset      # publish tasks as a named taskset
 hud sync env                   # sync environment metadata
 ```
 
-External benchmark formats (currently Harbor) load directly into the runtime
-as `Taskset`s - no conversion step. See [Harbor interop](/v6/advanced/harbor-convert).
+External benchmark formats (currently Harbor) can be adapted into runnable
+`Taskset`s. See [Harbor interop](/v6/advanced/harbor-convert).
 
 ## Inspect
 

docs/v6/reference/runtime.mdx

@@ -115,21 +115,20 @@ SubprocessRuntime(path, *, env=None, ready_timeout=120.0)
 ### `DockerRuntime`
 
 ```python
-DockerRuntime(image=None, *, port=8765, run_args=(), runtime_config=None, nested_sandbox=False)
+DockerRuntime(image=None, *, port=8765, run_args=(), runtime_config=None)
 ```
 
 - **`image`** - image name to run; shorthand for `runtime_config.image`.
 - **`port`** - port the image's CMD serves inside the container (the scaffolded `Dockerfile.hud` serves `8765`).
 - **`run_args`** - extra `docker run` flags, e.g. `["--gpus", "all"]` or `["-e", "KEY=VAL"]`.
 - **`runtime_config`** - a `RuntimeConfig` (image, resources) for finer control.
-- **`nested_sandbox`** - the image sandboxes *inside* the container (a `Workspace` using bubblewrap). Off by default.
-
-An image that sandboxes internally needs nested user/mount/proc namespaces,
-which Docker's default seccomp profile and masked `/proc` block. Passing
-`nested_sandbox=True` starts the container with `--security-opt
-seccomp=unconfined --security-opt systempaths=unconfined`; without it such an
-environment refuses to serve rather than running unisolated. Images that do
-not sandbox internally keep Docker's full containment, so this is opt-in.
+
+`DockerRuntime` always starts with HUD's compact seccomp profile and the system
+path configuration required by `Workspace`'s bubblewrap sessions. The profile
+is intentionally default-allow for Docker compatibility, allowing the
+namespace syscalls needed for the inner workspace sandbox while denying a
+small set of unrelated kernel interfaces. It is not Docker's default
+deny-by-default profile.
 
 ### `ModalRuntime`
 

hud/agents/openai_compatible/tools/filesystem.py

@@ -66,9 +66,6 @@ async def execute(self, arguments: dict[str, Any]) -> MCPToolResult:
         path = arguments.get("filePath")
         if not isinstance(path, str) or not path:
             raise ValueError("filePath is required")
-        # Map once so the directory predicate and the file read agree on
-        # the same workspace-anchored path.
-        path = self.client.map_path(path)
         offset = _read_offset(arguments.get("offset"))
         limit = _positive_int(arguments.get("limit"), default=DEFAULT_READ_LIMIT, name="limit")
         if not (await self.bash(f"test -d {shlex.quote(path)}")).isError:
@@ -196,9 +193,6 @@ async def execute(self, arguments: dict[str, Any]) -> MCPToolResult:
         path = arguments.get("filePath")
         if not isinstance(path, str) or not path:
             raise ValueError("filePath is required")
-        # Map once so existence checks, mkdir, and the write all target
-        # the same workspace-anchored path.
-        path = self.client.map_path(path)
         old = arguments.get("oldString")
         new = arguments.get("newString")
         if not isinstance(old, str):
@@ -258,7 +252,6 @@ async def execute(self, arguments: dict[str, Any]) -> MCPToolResult:
         path = arguments.get("filePath")
         if not isinstance(path, str) or not path:
             raise ValueError("filePath is required")
-        path = self.client.map_path(path)
         content = arguments.get("content")
         if not isinstance(content, str):
             raise ValueError("content is required")
@@ -294,7 +287,7 @@ async def execute(self, arguments: dict[str, Any]) -> MCPToolResult:
         pattern = arguments.get("pattern")
         if not isinstance(pattern, str):
             raise ValueError("pattern is required")
-        path = self.client.map_path(str(arguments.get("path") or "."))
+        path = str(arguments.get("path") or ".")
         cmd = f"grep -rn {shlex.quote(pattern)} {shlex.quote(path)}"
         include = arguments.get("include")
         if isinstance(include, str) and include:
@@ -318,7 +311,7 @@ async def execute(self, arguments: dict[str, Any]) -> MCPToolResult:
         pattern = arguments.get("pattern")
         if not isinstance(pattern, str):
             raise ValueError("pattern is required")
-        path = self.client.map_path(str(arguments.get("path") or "."))
+        path = str(arguments.get("path") or ".")
         return await self.bash(f"find {shlex.quote(path)} -name {shlex.quote(pattern)}")
 
 

hud/agents/tests/test_provider_native_tools.py

@@ -319,54 +319,27 @@ async def test_openai_compatible_write_stores_file_via_ssh_exec() -> None:
     assert ssh.files["/REPORT.md"] == b"done"
 
 
-async def test_absolute_paths_anchor_to_the_capability_cwd() -> None:
-    """The old SFTP chroot resolved ``/REPORT.md`` against the workspace root;
-    exec-channel file helpers must keep that contract via the capability cwd."""
-    ssh = _FakeSSH(cwd="/workspace", files={"/workspace/f.txt": b"inside"})
+async def test_paths_reach_the_session_verbatim() -> None:
+    """A path means what it says in the session's own namespace: file helpers
+    and shell commands must never disagree about what a path names, so
+    nothing is anchored or rewritten on the way through."""
+    ssh = _FakeSSH(cwd="/app", files={"/app/f.txt": b"inside"})
     tool = WriteTool(spec=WriteTool.default_spec("qwen"), client=cast("SSHClient", ssh))
 
-    await tool.execute({"filePath": "/REPORT.md", "content": "done"})
+    await tool.execute({"filePath": "/tmp/probe.txt", "content": "done"})
 
-    assert ssh.files["/workspace/REPORT.md"] == b"done"
-    # Paths already inside the workspace are untouched.
-    assert await cast("SSHClient", ssh).read_text("/workspace/f.txt") == "inside"
-
-
-def test_map_path_clamps_traversal_like_a_chroot() -> None:
-    ssh = cast("SSHClient", _FakeSSH(cwd="/workspace"))
-    assert ssh.map_path("/workspace/../etc/passwd") == "/workspace/etc/passwd"
-    assert ssh.map_path("/../etc/passwd") == "/workspace/etc/passwd"
-    assert ssh.map_path("../../etc/passwd") == "/workspace/etc/passwd"
-    assert ssh.map_path("a/../b.txt") == "/workspace/b.txt"
-    assert ssh.map_path("/") == "/workspace"
-    assert ssh.map_path(".") == "/workspace"
-
-
-def test_map_path_handles_windows_native_paths() -> None:
-    """The workspace publishes cwd via as_posix(); callers pass native
-    backslash paths, and NTFS compares case-insensitively."""
-    cap = Capability(
-        name="shell",
-        protocol="ssh/2",
-        url="ssh://localhost:22",
-        params={"shell": "powershell", "cwd": "C:/work"},
-    )
-    ssh = SSHClient(cap, cast("Any", None))
-    assert ssh.map_path("C:\\work\\file.txt") == "C:/work/file.txt"
-    assert ssh.map_path("C:\\Work\\sub\\f.txt") == "C:/work/sub/f.txt"
-    assert ssh.map_path("D:\\other\\f.txt") == "C:/work/other/f.txt"
-    assert ssh.map_path("\\temp\\f.txt") == "C:/work/temp/f.txt"
-    assert ssh.map_path("sub\\f.txt") == "C:/work/sub/f.txt"
-    assert ssh.map_path("C:\\work\\..\\secrets.txt") == "C:/work/secrets.txt"
+    assert ssh.files["/tmp/probe.txt"] == b"done"
+    assert "/app/tmp/probe.txt" not in ssh.files
+    assert await cast("SSHClient", ssh).read_text("/app/f.txt") == "inside"
 
 
 async def test_read_maps_the_directory_predicate_and_listing_together() -> None:
-    """`test -d`, listing, and reads must agree on the anchored path, or
-    absolute workspace dirs are misclassified as files."""
+    """`test -d`, listing, and reads must agree on the same path, or
+    workspace dirs are misclassified as files."""
     ssh = _FakeSSH(cwd="/workspace", files={"/workspace/pkg/mod.py": b"x = 1\n"})
     tool = ReadTool(spec=ReadTool.default_spec("qwen"), client=cast("SSHClient", ssh))
 
-    result = await tool.execute({"filePath": "/pkg"})
+    result = await tool.execute({"filePath": "/workspace/pkg"})
 
     text = result_text(result)
     assert "<type>directory</type>" in text
@@ -538,26 +511,6 @@ async def test_gemini_edit_creates_file_when_old_string_empty() -> None:
     assert ssh.files["/n.txt"] == b"fresh"
 
 
-def test_map_path_leaves_a_symlinked_spelling_of_the_workspace_alone() -> None:
-    """A workspace made at /tmp/w is served as /private/tmp/w on macOS; re-anchoring
-    the caller's spelling instead of stripping it nests the path under itself."""
-    cap = Capability(
-        name="shell",
-        protocol="ssh/2",
-        url="ssh://localhost:22",
-        params={"cwd": "/private/tmp/w", "cwd_aliases": ["/tmp/w"]},
-    )
-    ssh = SSHClient(cap, cast("Any", None))
-
-    assert ssh.map_path("/tmp/w/calc.py") == "/private/tmp/w/calc.py"
-    assert ssh.map_path("/private/tmp/w/calc.py") == "/private/tmp/w/calc.py"
-    assert ssh.map_path("/tmp/w") == "/private/tmp/w"
-    # Workspace-relative addressing still anchors, and an unrelated absolute
-    # path is still clamped into the workspace like a chroot.
-    assert ssh.map_path("/REPORT.md") == "/private/tmp/w/REPORT.md"
-    assert ssh.map_path("/tmp/elsewhere/f.txt") == "/private/tmp/w/tmp/elsewhere/f.txt"
-
-
 async def test_reading_a_missing_file_is_a_tool_error_not_a_raised_traceback() -> None:
     """Reading before creating is the first thing an editor tool does; that failure
     must come back as a tool result carrying the shell's message."""

hud/capabilities/base.py

@@ -7,12 +7,9 @@
 import sys
 from abc import ABC, abstractmethod
 from dataclasses import dataclass, field
-from typing import TYPE_CHECKING, Any, ClassVar, Self
+from typing import Any, ClassVar, Self
 from urllib.parse import urlsplit
 
-if TYPE_CHECKING:
-    from collections.abc import Sequence
-
 #: Matches the scheme prefix of a URL (RFC 3986).
 SCHEME_RE: re.Pattern[str] = re.compile(r"^([a-zA-Z][a-zA-Z0-9+\-.]*):")
 
@@ -82,7 +79,6 @@ def ssh(
         client_key_path: str | os.PathLike[str] | None = None,
         shell: str | None = None,
         cwd: str | None = None,
-        cwd_aliases: Sequence[str] | None = None,
     ) -> Capability:
         """``ssh/2`` — SSH daemon with publickey auth.
 
@@ -93,10 +89,8 @@ def ssh(
         type (``bash``, ``powershell``, ``cmd``). Defaults to auto-detect
         from ``sys.platform`` at construction time. Agents read this to
         format commands correctly. ``cwd`` is the absolute path sessions
-        start in (the served workspace); clients anchor file paths to it.
-        ``cwd_aliases`` are other names the same directory answers to (paths
-        that reach it through symlinks), which clients treat as already
-        anchored rather than as workspace-relative addresses.
+        start in. Paths are the session namespace's own — clients pass them
+        verbatim, and nothing is anchored or rewritten.
         """
         normalized = normalize_url(url, default_scheme="ssh", default_port=22)
         if shell is None:
@@ -108,8 +102,6 @@ def ssh(
             params["client_key_path"] = os.fspath(client_key_path)
         if cwd is not None:
             params["cwd"] = cwd
-        if cwd_aliases:
-            params["cwd_aliases"] = list(cwd_aliases)
         return cls(name=name, protocol="ssh/2", url=normalized, params=params)
 
     @classmethod