edge-endpoint can be configured by python-sdk

[timmarkhuff]

Summary

Adds HTTP endpoints for reading and modifying the edge endpoint configuration at runtime, without requiring a Helm redeploy. A Python SDK companion PR provides the client-side methods.

New endpoints:

• GET /edge-config -- returns the active EdgeEndpointConfig
• PUT /edge-config -- replaces the active config (diffs against current state, adds/removes detector pods accordingly)
• GET /edge-detector-readiness -- reports which configured detectors have inference pods ready to serve

Key design decisions

Config as a shared file on PVC

Multiple uvicorn workers cannot share in-memory state. The active config is persisted to a YAML file on the existing PVC (/opt/groundlight/edge/config/active-edge-config.yaml). Workers read it via an mtime-based cache (EdgeConfigManager.active()), which calls os.path.getmtime on each access and only re-parses when the file has changed. This gives cross-worker consistency with negligible overhead.

Config seeding via init container

An apply-edge-config init container copies the Helm ConfigMap to the active config file on PVC. It uses a revision fingerprint (revision + deploy timestamp) stored on PVC to distinguish helm install/upgrade from restarts -- only copying when the fingerprint changes. This ensures Helm config is applied on deploy while SDK changes survive restarts.

At startup, main.py checks: EDGE_CONFIG env var (for Balena) > active config on PVC > Pydantic defaults.

Backward compatibility

Older Helm charts lack the init container. In that case, main.py detects the missing active config file and falls back to copying the Helm ConfigMap directly.

Detector reconciliation

PUT /edge-config triggers reconcile_config(), which:

1. Diffs the desired detectors against DB records (compute_detector_diff)
2. Marks removed detectors as pending_deletion in the DB
3. Creates new deployment records for added detectors
4. Saves the new config to disk

The inference-model-updater picks up these DB changes on its next loop iteration -- deleting pods for pending_deletion records, then creating pods for new ones. This could cause the config update process to take a bit longer than necessary (because we have to wait for the refresh loop to occur). I thought about adding logic to make the inference-model-updater poll for config updates while it waits, and terminate early if it sees one. In the end, I decided to not add that (yet). YAGNI.

Deprecation of default-edge-config.yaml

The static deploy/helm/groundlight-edge-endpoint/files/default-edge-config.yaml has been removed. Default config is now defined by Pydantic model defaults in the python-sdk. When no config file is provided to Helm, _helpers.tpl generates an empty YAML object, and the system uses Pydantic defaults.

Notable refactorings

• EdgeConfigManager (new, replaces edge_config_loader.py): Singleton-style class with class methods for config lifecycle -- save(), active(), detector_configs(), detector_config().

• EdgeInferenceManager simplification: Removed stored detector_inference_configs, inference_client_urls, oodd_inference_client_urls, min_times_between_escalations, and the 30-second scheduler poll. The manager now computes pod URLs on the fly from detector IDs and reads config from EdgeConfigManager.active(). Only last_escalation_times remains as runtime state.

• naming.py (new): Extracted get_edge_inference_service_name and get_edge_inference_model_name out of edge_inference.py to break a circular import between edge_inference.py and edge_config_manager.py.

• pending_deletion column: Added to the InferenceDeployment DB model. The column has a server default of False, so existing databases are compatible without migration.

• apply-edge-config init container (new): Busybox-based init container added to the edge-endpoint Deployment. Uses a revision fingerprint to conditionally copy Helm config to PVC. Script lives in files/apply-edge-config.sh, embedded in the edge-config ConfigMap.

Authentication

These new endpoints are unauthenticated, consistent with all other non-inference endpoints on the edge endpoint (status page, metrics, health checks). The edge endpoint assumes a trusted local network.

GET /edge-config and GET /edge-detector-readiness are comparable in sensitivity to the existing status page and metrics.json, which already expose detector IDs, model versions, and readiness.

PUT /edge-config is the only unauthenticated write endpoint that modifies system state. An attacker with network access could add or remove detectors. This is an accepted tradeoff under the trusted-network assumption. If edge endpoints are ever exposed to untrusted networks, this endpoint should be the first to get auth.

Release strategy

This PR must be deployed before the companion SDK PR (python-sdk 0.26.0). The SDK's gl.edge.* methods call endpoints that only exist after this PR is deployed. Releasing the SDK first would cause those methods to 404.

The edge endpoint's groundlight dependency does not need to change for this PR -- it uses SDK features already available in 0.25.x. However, once the SDK is released at 0.26.0, a follow-up bump to >=0.26.0, <0.27.0 is recommended.

Tests

• test/api/test_edge_config.py: GET returns 200, PUT validates body (422 on invalid, 200 on valid)
• test/core/test_edge_config_manager.py: compute_detector_diff (6 cases), apply_detector_changes (3 cases), EdgeConfigManager (save/load/active roundtrip, mtime caching, startup priority, detector config lookups)

Load Testing

This PR touches some code in the hot path of inference (in what I hope is a trivial way). To ensure that I am not introducing any performance regressions, I performed the following benchmark. The results appear nearly identical, each test achieving a "Max Steady RPS" of 30 with the countstep-yolox-tracking pipeline.

uv run python multiple_client_throughput_test.py COUNT --edge-pipeline-config count-step-yolox-tracking --time-between-ramp 10 --max-clients 8

With changes:

Before changes:

[timmarkhuff]

We no longer have default-edge-config.yaml (since we rely on Pydantic defaults), so we need to construct a config from scratch here.

[brandon-wada]

I'm reminded how rusty I am on the edge code. At least all the parts look like they make sense to me

[brandon-wada]

When is detector_id falsey?

[timmarkhuff]

It should not be. I have adjusted the Pydantic model to ensure that it is not, and removed all the defensive code that assumes it might be.

[brandon-wada]

Do we want to reuse the existing volume for this?

[timmarkhuff]

Yes, this reuses the existing edge-endpoint-pvc. We just added a new mountPath at /opt/groundlight/edge/config on the same PVC that already mounts /opt/groundlight/edge/sqlite and /opt/groundlight/edge/serving/model-repo. No new PVC.