Add CVDP benchmark resource server with apptainer instead of docker

[arti4nvj]

Part of Customer Eval Bench, this is adding CVDP (non-agentic, non-commercial) support to Gym. This is a single pass evaluation using vLLM as a backend. The current code matches the existing public CVDP infra as of 3/5.

resources_server/cvdp --> All the helper scripts and files needed to run the benchmark
resources_server/cvdp/cvdp_lib --> contain files and code straight from CVDP Public Github that are needed for the final report generation
resources_server/cvdp/scripts --> contain all the helper scripts to convert the dataset to gym, create the final report
responses_api_agents/cvdp_agent --> copy of Simple Agent except added support for retries

Instead of Docker, this runs with Apptainer

[jmabry]

Code review

Found 2 issues:

1. Usage double-counting in responses_api_agents/cvdp_agent/app.py — usage = model_response.usage on the first iteration stores a reference, then the if usage: block immediately adds model_response.usage.input_tokens to itself, doubling the first call's token counts. PR feat: Fix duplicated usage counting and errors on empty usage in subsequent model calls #939 (commit c7bb3191) fixed this exact pattern in simple_agent by adding model_response.usage = None immediately after capture and guarding with if usage and model_response.usage:. The cvdp_agent was copied from the pre-fix version.

Gym/responses_api_agents/cvdp_agent/app.py

Lines 105 to 113 in 5a0ffa9

	if not usage:
	usage = model_response.usage
	if usage:
	usage.input_tokens += model_response.usage.input_tokens
	usage.output_tokens += model_response.usage.output_tokens
	usage.total_tokens += model_response.usage.total_tokens

2. responses_api_agents/cvdp_agent/client.py is an unmodified copy-paste from example_single_tool_call / simple_agent — it hardcodes server_name="example_single_tool_call_simple_agent", calls a get_weather tool, and uses "going out in sf tn" as example input. It has module-level executable code that runs on import and would connect to the wrong server. This file should be removed or updated for CVDP.

Gym/responses_api_agents/cvdp_agent/client.py

Lines 22 to 27 in 5a0ffa9

	server_client = ServerClient.load_from_global_config()
	task = server_client.post(
	server_name="example_single_tool_call_simple_agent",
	url_path="/v1/responses",
	json=NeMoGymResponseCreateParamsNonStreaming(
	input=[

🤖 Generated with Claude Code

_{- If this code review was useful, please react with 👍. Otherwise, react with 👎.}

[jmabry]

LG%C

[roclark]

LGTM, thanks @arti4nvj!

[cmunley1]

would be better to import rather than duplicate if you can

[cmunley1]

could you add a sentence or two here at least pointing to the resources server readme

[cmunley1]

and clarify data availability

[cmunley1]

just pointing to resources server docs is fine i guess