Skip to content

Commit 7d7dc2d

Browse files
shrutiyam-glitchshrutiyam-glitch
committed
Gateway tests
1 parent 7ed7d38 commit 7d7dc2d

13 files changed

Lines changed: 454 additions & 5 deletions

File tree

‎Makefile‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,10 +24,18 @@ deploy-kind:
2424
-p '{"spec": {"template": {"spec": {"containers": [{"name": "agent-sandbox-controller", "args": ["--extensions=true"]}]}}}}'; \
2525
fi
2626

27+
.PHONY: deploy-cloud-provider-kind
28+
deploy-cloud-provider-kind:
29+
./dev/tools/deploy-cloud-provider
30+
2731
.PHONY: delete-kind
2832
delete-kind:
2933
kind delete cluster --name ${KIND_CLUSTER}
3034

35+
.PHONY: kill-cloud-provider-kind
36+
kill-cloud-provider-kind:
37+
killall cloud-provider-kind
38+
3139
.PHONY: test-unit
3240
test-unit:
3341
./dev/tools/test-unit

‎clients/python/agentic-sandbox-client/gateway-kind/README.md‎

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
# Gateway API Support for KinD Clusters
2+
3+
This directory contains resources to enable Kubernetes Gateway API support within KinD (Kubernetes in Docker) clusters. Since KinD doesn't have a native load balancer, we use `cloud-provider-kind` to provision an IP address for the Gateway resource. This enables the agentic-sandbox-client to connect to sandboxes in the cluster via a Kubernetes Gateway.
4+
5+
### Step-by-step Guide for Local Kind Cluster
6+
7+
This guide explains how to manually set up the Sandbox Router and Gateway API on a local KinD cluster.
8+
9+
1. **Setup Agentic Sandbox and Python SDK Client:** Follow the instructions in the [main client README](../README.md), which includes prerequisites, installing the SDK, and deploying the Sandbox Router.
10+
11+
2. **Install and Run cloud-provider-kind:**
12+
This component provides a load balancer implementation for KinD.
13+
14+
Install the latest version and run `cloud-provider-kind` in the background, enabling the Gateway API controller::
15+
```bash
16+
make deploy-cloud-provider-kind
17+
```
18+
19+
*See [cloud-provider-kind documentation](https://github.com/kubernetes-sigs/cloud-provider-kind) for more details.*
20+
21+
3. **Deploy KinD Gateway Resources:**
22+
Apply the Gateway, and HTTPRoute from this directory:
23+
```bash
24+
kubectl apply -f gateway-kind.yaml
25+
```
26+
27+
4. **Test the Setup:**
28+
a. **Check Gateway Status:** After a short time, verify that the Gateway has been assigned an IP address by `cloud-provider-kind`:
29+
```bash
30+
kubectl get gateway
31+
```
32+
You should see output similar to this, with an ADDRESS populated:
33+
```bash
34+
NAME CLASS ADDRESS PROGRAMMED AGE
35+
kind-gateway cloud-provider-kind 192.168.8.3 True 2m
36+
```
37+
If the ADDRESS is empty, wait a bit longer or check the `cloud-provider-kind` logs.
38+
39+
b. **Run Test Client:** Use the `agentic-sandbox-client` in Gateway mode to test the end-to-end flow:
40+
```bash
41+
python ../test_client.py --gateway_name="kind-gateway"
42+
```
43+
44+
5. **Clean up:** To stop the cloud-provider-kind process, run:
45+
```bash
46+
make kill-cloud-provider-kind
47+
```
48+
49+
### Automated Setup & Test
50+
51+
For a fully automated setup and test run, you can use the `run-test-kind.sh` script provided in this directory:
52+
53+
```bash
54+
./run-test-kind.sh
55+
```

‎clients/python/agentic-sandbox-client/gateway-kind/gateway-kind.yaml‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
apiVersion: gateway.networking.k8s.io/v1
2+
kind: Gateway
3+
metadata:
4+
name: kind-gateway
5+
spec:
6+
gatewayClassName: cloud-provider-kind
7+
listeners:
8+
- name: http
9+
protocol: HTTP
10+
port: 80
11+
allowedRoutes:
12+
kinds:
13+
- kind: HTTPRoute
14+
namespaces:
15+
from: All
16+
---
17+
# The HTTPRoute to connect the Gateway to the router service.
18+
apiVersion: gateway.networking.k8s.io/v1beta1
19+
kind: HTTPRoute
20+
metadata:
21+
name: sandbox-router-route
22+
spec:
23+
parentRefs:
24+
- name: kind-gateway # This must match the name of the Gateway
25+
rules:
26+
- matches:
27+
- path:
28+
type: PathPrefix
29+
value: /
30+
backendRefs:
31+
- name: sandbox-router-svc # Route all traffic to the router service
32+
port: 8080

‎clients/python/agentic-sandbox-client/gateway-kind/python-sandbox-template.yaml‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
apiVersion: extensions.agents.x-k8s.io/v1alpha1
2+
kind: SandboxTemplate
3+
metadata:
4+
# The test_client.py expects the template to have this name
5+
name: python-sandbox-template
6+
namespace: default
7+
spec:
8+
podTemplate:
9+
spec:
10+
containers:
11+
- name: python-runtime
12+
# The test_client.py expects the image to be agent-sandbox/examples/python-runtime-sandbox
13+
image: IMAGE_PLACEHOLDER
14+
ports:
15+
- containerPort: 8888
16+
readinessProbe:
17+
httpGet:
18+
path: "/"
19+
port: 8888
20+
initialDelaySeconds: 0
21+
periodSeconds: 1
22+
livenessProbe:
23+
httpGet:
24+
path: "/"
25+
port: 8888
26+
initialDelaySeconds: 2
27+
periodSeconds: 10
28+
resources:
29+
requests:
30+
cpu: "250m"
31+
memory: "512Mi"
32+
ephemeral-storage: "512Mi"
33+
restartPolicy: "OnFailure"

‎clients/python/agentic-sandbox-client/gateway-kind/run-test-kind.sh‎

Lines changed: 98 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,98 @@
1+
#!/bin/bash
2+
# Copyright 2025 The Kubernetes Authors.
3+
#
4+
# Licensed under the Apache License, Version 2.0 (the "License");
5+
# you may not use this file except in compliance with the License.
6+
# You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
16+
17+
set -e
18+
19+
export KIND_CLUSTER_NAME="agent-sandbox"
20+
21+
# Get the image tag using a temporary Python script
22+
REPO_ROOT=$(git rev-parse --show-toplevel)
23+
24+
TEMP_PY_SCRIPT=$(mktemp /tmp/get_tag_XXXXXX.py)
25+
cat <<EOF > "$TEMP_PY_SCRIPT"
26+
import sys
27+
sys.path.append('$REPO_ROOT/dev/tools/shared')
28+
from utils import get_image_tag
29+
print(get_image_tag())
30+
EOF
31+
32+
IMAGE_TAG=$(python3 "$TEMP_PY_SCRIPT")
33+
34+
rm "$TEMP_PY_SCRIPT"
35+
echo "Using image tag: $IMAGE_TAG"
36+
37+
38+
export SANDBOX_ROUTER_IMG="kind.local/sandbox-router:${IMAGE_TAG}"
39+
export SANDBOX_PYTHON_RUNTIME_IMG="kind.local/python-runtime-sandbox:${IMAGE_TAG}"
40+
41+
# following develop guide to make and deploy agent-sandbox to kind cluster
42+
cd $REPO_ROOT
43+
make build
44+
make deploy-kind EXTENSIONS=true
45+
make deploy-cloud-provider-kind
46+
47+
cd clients/python/agentic-sandbox-client/gateway-kind
48+
echo "Applying CRD for template - Sandbox claim will be applied by the sandbox client in python code"
49+
sed -i "s|IMAGE_PLACEHOLDER|${SANDBOX_PYTHON_RUNTIME_IMG}|g" python-sandbox-template.yaml
50+
kubectl apply -f python-sandbox-template.yaml
51+
52+
53+
cd ../sandbox-router
54+
echo "Applying CRD for router template"
55+
sed -i "s|IMAGE_PLACEHOLDER|${SANDBOX_ROUTER_IMG}|g" sandbox_router.yaml
56+
kubectl apply -f sandbox_router.yaml
57+
kubectl rollout status deployment/sandbox-router-deployment --timeout=60s
58+
59+
60+
cd ../gateway-kind
61+
echo "Setting up Cloud Provider Kind Gateway in the kind cluster..."
62+
echo "Applying Gateway configuration..."
63+
kubectl apply -f gateway-kind.yaml
64+
kubectl wait --for=condition=Programmed=True gateway/kind-gateway --timeout=60s
65+
66+
cd ../
67+
68+
69+
# Cleanup function
70+
cleanup() {
71+
echo "Cleaning up virtual environment..."
72+
deactivate
73+
rm -rf .venv
74+
75+
cd $REPO_ROOT
76+
echo "Cleaning up cloud provider kind..."
77+
make kill-cloud-provider-kind
78+
79+
echo "Deleting kind cluster..."
80+
make delete-kind || true
81+
82+
echo "Cleanup completed."
83+
}
84+
85+
86+
echo "Starting virtual environment for Python client and install agentic sandbox client..."
87+
python3 -m venv .venv
88+
source .venv/bin/activate
89+
pip install -e .
90+
91+
92+
echo "========= $0 - Running the Python client tester... ========="
93+
python3 ./test_client.py --gateway-name kind-gateway
94+
echo "========= $0 - Finished running the Python client with gateway and router tester. ========="
95+
96+
trap cleanup EXIT
97+
98+
echo "Test finished."

‎clients/python/agentic-sandbox-client/sandbox-router/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -54,7 +54,7 @@ previous step, and then apply the manifest.
5454

5555
```bash
5656
sed -i "s|IMAGE_PLACEHOLDER|${SANDBOX_ROUTER_IMG}|g" sandbox_router.yaml
57-
kubectl apply -f sandbox_router.yaml
57+
kubectl apply -f sandbox_router.yaml --namespace=default
5858
```
5959

6060
### Deploy the Gateway

‎dev/ci/presubmits/test-e2e‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,10 +34,14 @@ def main():
3434
result = subprocess.run([f"{repo_root}/dev/tools/push-images", "--kind-cluster-name", "e2e-test", "--image-prefix", "kind.local/"])
3535
if result.returncode != 0:
3636
return result.returncode
37-
result = subprocess.run([f"{repo_root}/dev/tools/deploy-to-kube", "--image-prefix", "kind.local/"])
37+
result = subprocess.run([f"{repo_root}/dev/tools/deploy-to-kube", "--image-prefix", "kind.local/", "--gateway"])
38+
if result.returncode != 0:
39+
return result.returncode
40+
result = subprocess.run([f"{repo_root}/dev/tools/deploy-cloud-provider"])
3841
if result.returncode != 0:
3942
return result.returncode
4043
result = subprocess.run([f"{repo_root}/dev/tools/test-e2e", "--image-prefix", "kind.local/"])
44+
subprocess.run(["killall", "cloud-provider-kind"])
4145

4246
# Always create junit file whether tests pass or fail
4347
artifact_dir = os.getenv("ARTIFACTS")

‎dev/tools/deploy-cloud-provider‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
#!/usr/bin/env python3
2+
# Copyright 2025 The Kubernetes Authors.
3+
#
4+
# Licensed under the Apache License, Version 2.0 (the "License");
5+
# you may not use this file except in compliance with the License.
6+
# You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
16+
import os
17+
import subprocess
18+
19+
from shared import utils
20+
21+
22+
def main():
23+
repo_root = utils.get_repo_root()
24+
25+
print("Setting up cloud-provider-kind for gateway ...")
26+
cmd = ["go", "install", "sigs.k8s.io/cloud-provider-kind@latest"]
27+
subprocess.run(cmd, cwd=repo_root, check=True)
28+
29+
# Get current path
30+
path = os.environ.get("PATH", "")
31+
32+
# Append the go/bin path
33+
go_bin = os.path.expanduser("~/go/bin")
34+
os.environ["PATH"] = f"{path}:{go_bin}"
35+
36+
print(
37+
"Starting cloud-provider-kind in the background and enabling the Gateway API controller ..."
38+
)
39+
process = subprocess.Popen(
40+
["cloud-provider-kind", "--gateway-channel", "standard"],
41+
stdout=subprocess.DEVNULL, # Silence output so it doesn't clutter terminal
42+
stderr=subprocess.DEVNULL, # Silence errors
43+
)
44+
45+
print(f"Process started with PID: {process.pid}")
46+
47+
48+
if __name__ == "__main__":
49+
main()

‎dev/tools/shared/utils.py‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,10 @@ def get_image_tag():
3232

3333
def get_image_prefix(args):
3434
"""Constructs the image prefix for a container image."""
35+
if not args.image_prefix and "IMAGE_PREFIX" in os.environ:
36+
return os.environ["IMAGE_PREFIX"]
3537
if args.image_prefix:
38+
os.environ["IMAGE_PREFIX"] = args.image_prefix
3639
return args.image_prefix
3740
raise Exception(f"--image-prefix arg or IMAGE_PREFIX environment variable must be set")
3841

0 commit comments

Comments
 (0)