test: track remaining OpenWebUI runtime loading and registration validation gap

[rlippmann]

Summary

This issue has been decomposed from the earlier broad runtime-integration tracker.

Recent work closed the LiteLLM Proxy runtime lifecycle portion of that gap:

• opt-in runtime smoke coverage now starts a real LiteLLM Proxy process
• verifies startup/teardown and request routing
• covers both the basic hook and the directive-drafter hook

Open WebUI verification guidance has also improved, but the remaining automated runtime-validation gap is still true OpenWebUI runtime loading/registration coverage.

Remaining gap tracked here

• True OpenWebUI runtime loading/registration validation rather than import-only or fixture-only coverage.

This remaining gap should cover:

• loading the pipe into a real Open WebUI runtime
• confirming registration/import behavior in the actual runtime environment
• verifying the basic pipe path at the runtime boundary
• verifying the directive-drafter pipe path at the runtime boundary
• confirming local-only state updates vs forwarded backend requests in the real runtime

Decomposed follow-up issues

The broader tracker has been split so the remaining non-OpenWebUI work can progress independently:

• Streaming/chunked runtime-path validation: test: add streaming or chunked runtime-path validation where integration behavior depends on it #37
• Cross-process or multi-worker runtime validation: test: add cross-process or multi-worker runtime validation for example integrations #38

The dedicated OpenWebUI follow-up that matches the remaining scope here is:

• OpenWebUI runtime loading/registration validation: test: add true Open WebUI runtime loading and registration validation #36

Why this remains open

The repository still relies on stubbed/in-process tests plus documented manual verification for the Open WebUI integrations. That is better than the previous state, but it is not yet equivalent to automated runtime loading/registration validation in a real Open WebUI runtime.

Notes

This issue remains open until the OpenWebUI runtime loading/registration gap is covered or its tracking is fully handed off to #36 and this issue is intentionally closed.

[rlippmann]

Progress update on runtime coverage gaps tracked here:

Completed:

• LiteLLM Proxy runtime lifecycle validation is now in place.
• The repo has an opt-in runtime smoke test that starts a real LiteLLM Proxy process, verifies startup/teardown, blocked-before-upstream behavior, forwarded contract injection, and covers both the basic hook and the directive-drafter hook.

Partially addressed:

• Open WebUI verification guidance is much clearer now.
• The Open WebUI README documents runtime verification steps and advanced forwarded-request checks, but this is still manual/documented verification rather than automated runtime loading/registration validation.

Still open:

• True Open WebUI runtime loading/registration validation.
• Streaming/chunked response-path validation where runtime behavior depends on it.
• Cross-process or multi-worker runtime validation.

Recommendation:

• Keep this issue open until Open WebUI runtime loading/registration validation is covered or split into a dedicated follow-up issue.
• Consider moving streaming/chunked and cross-process/multi-worker validation into separate issues so this tracker can narrow to the remaining primary runtime gaps.

[rlippmann]

Superseded by #36, which now owns the remaining OpenWebUI runtime loading and registration validation scope.