From b7fddb1f7cf2b899ba4ca63bc85f52124be68944 Mon Sep 17 00:00:00 2001 From: reacher-z Date: Mon, 27 Jul 2026 17:24:02 +0800 Subject: [PATCH 1/2] docs: reference ClawBench as an external benchmark --- README.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/README.md b/README.md index 40611b4d..24b6e570 100644 --- a/README.md +++ b/README.md @@ -65,6 +65,12 @@ AgentLab Features: | [OSWorld](https://os-world.github.io/) | [setup](https://github.com/ServiceNow/AgentLab/blob/main/src/agentlab/benchmarks/osworld.md) | 369 | None | - | - | self hosted | soon | | [TimeWarp](https://timewarp-web.github.io/) | [setup](https://github.com/sparklabutah/timewarp) | 1386 | None | 30 | yes | self hosted | soon | +## 📎 External live-web benchmarks + +The following independent benchmark is useful for evaluating web and computer-use agents, but is not currently integrated with or supported by AgentLab: + +* **[ClawBench](https://github.com/reacher-z/ClawBench)** — [paper](https://arxiv.org/abs/2604.08523) and [project page](https://claw-bench.com/). It evaluates agents on 283 tasks across 163 live platforms using isolated browser sessions, request-level outcome checks, and recorded action, screenshot, network, and message traces. See the project's own [runner and setup instructions](https://github.com/reacher-z/ClawBench#readme) for execution details. Task definitions are available on [Hugging Face](https://huggingface.co/datasets/NAIL-Group/ClawBench) and mirrored at [TIGER-Lab/ClawBench](https://huggingface.co/datasets/TIGER-Lab/ClawBench). + ## 🛠️ Setup AgentLab From f73ce417626822192049ec9938bec1327159999f Mon Sep 17 00:00:00 2001 From: Yuxuan Zhang <68239571+reacher-z@users.noreply.github.com> Date: Tue, 28 Jul 2026 16:59:25 +0800 Subject: [PATCH 2/2] docs: use canonical ClawBench repository links --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 24b6e570..38f1dc0b 100644 --- a/README.md +++ b/README.md @@ -69,7 +69,7 @@ AgentLab Features: The following independent benchmark is useful for evaluating web and computer-use agents, but is not currently integrated with or supported by AgentLab: -* **[ClawBench](https://github.com/reacher-z/ClawBench)** — [paper](https://arxiv.org/abs/2604.08523) and [project page](https://claw-bench.com/). It evaluates agents on 283 tasks across 163 live platforms using isolated browser sessions, request-level outcome checks, and recorded action, screenshot, network, and message traces. See the project's own [runner and setup instructions](https://github.com/reacher-z/ClawBench#readme) for execution details. Task definitions are available on [Hugging Face](https://huggingface.co/datasets/NAIL-Group/ClawBench) and mirrored at [TIGER-Lab/ClawBench](https://huggingface.co/datasets/TIGER-Lab/ClawBench). +* **[ClawBench](https://github.com/TIGER-AI-Lab/ClawBench)** — [paper](https://arxiv.org/abs/2604.08523) and [project page](https://claw-bench.com/). It evaluates agents on 283 tasks across 163 live platforms using isolated browser sessions, request-level outcome checks, and recorded action, screenshot, network, and message traces. See the project's own [runner and setup instructions](https://github.com/TIGER-AI-Lab/ClawBench#readme) for execution details. Task definitions are available on [Hugging Face](https://huggingface.co/datasets/NAIL-Group/ClawBench) and mirrored at [TIGER-Lab/ClawBench](https://huggingface.co/datasets/TIGER-Lab/ClawBench). ## 🛠️ Setup AgentLab @@ -343,4 +343,4 @@ Here is an example of how they can be used: We use the AgentLab framework to run and manage our experiments \cite{workarena2024,chezelles2025browsergym}. ``` ## Traces -Traces from “The BrowserGym Ecosystem for Web Agent Research” paper are available in [Huggingface](https://huggingface.co/datasets/agentlabtraces/agentlabtraces/tree/main). +Traces from “The BrowserGym Ecosystem for Web Agent Research” paper are available in [Huggingface](https://huggingface.co/datasets/agentlabtraces/agentlabtraces/tree/main). \ No newline at end of file