Custom prompt configuration feature and Pyright fixes#297
Merged
Thirunayan22 merged 37 commits intobuerokratt:wipfrom Feb 10, 2026
Merged
Custom prompt configuration feature and Pyright fixes#297Thirunayan22 merged 37 commits intobuerokratt:wipfrom
Thirunayan22 merged 37 commits intobuerokratt:wipfrom
Conversation
Pulling changes from Burokratt WIP to rootcodelabs/RAG-Module wip
Get update from RAG-201-Fix into encrypt-llm-keys
update cron manager vault script
Streaming response formatting
Encrypt llm keys
…G-Module into encrypt-llm-keys
Sync rootcodelabs/RAG-Module wip with buerokratt/RAG-Module wip
… encrypt-llm-keys
…G-Module into encrypt-llm-keys
Refactor docker-compose-ec2.ym l file with new vault agent containers
There was a problem hiding this comment.
Pull request overview
Adds support for centrally managed “custom prompt instructions” loaded from Ruuter/Resql and applied during response generation, alongside several Pyright/type-safety fixes across the codebase.
Changes:
- Introduces a
PromptConfigurationLoaderwith TTL caching + retry logic, and wires it intoLLMOrchestrationService+ResponseGeneratorAgent. - Adds an API + DSL flow to force-refresh the prompt cache after admin updates (
POST /prompt-config/refreshand Ruuter save hook). - Fixes/adjusts typing and runtime contracts (Pyright) in metrics, guardrails streaming/provider registration, and contextual retrieval.
Reviewed changes
Copilot reviewed 14 out of 14 changed files in this pull request and generated 12 comments.
Show a summary per file
| File | Description |
|---|---|
| src/utils/prompt_config_loader.py | New HTTP loader for custom prompt config with caching/retry and monitoring stats. |
| src/llm_orchestrator_config/llm_ochestrator_constants.py | Adds constants for prompt-config endpoint and cache TTL. |
| src/llm_orchestration_service.py | Initializes loader, warms cache, and injects custom prefix into response generator. |
| src/response_generator/response_generate.py | Adds custom_instructions_prefix and prepends it in streaming + non-streaming generation. |
| src/llm_orchestration_service_api.py | Adds /prompt-config/refresh endpoint to force refresh of the cache. |
| DSL/Ruuter.public/.../get-prompt.yml | Adds Ruuter public endpoint to fetch prompt config from Resql. |
| DSL/Ruuter.private/.../prompt-configuration/save.yml | Triggers refresh endpoint after saving prompt configuration. |
| constants.ini | Adds RAG_SEARCH_PROMPT_REFRESH used by the DSL save flow. |
| docs/CUSTOM_PROMPT_CONFIGURATION.md | Documents the new prompt configuration flow end-to-end. |
| src/optimization/metrics/generator_metrics.py | Pyright-friendly import/use of SemanticF1 and ensures float typing. |
| src/llm_orchestrator_config/exceptions.py | Uses Optional[str] for error_id args for typing correctness. |
| src/guardrails/nemo_rails_adapter.py | Adjusts provider registration to satisfy typing (but has a runtime risk). |
| src/guardrails/dspy_nemo_adapter.py | Updates streaming return types to GenerationChunk for LangChain compatibility. |
| src/contextual_retrieval/contextual_retriever.py | Improves exception type narrowing for asyncio.gather(..., return_exceptions=True). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+721
to
+723
| @app.post("/prompt-config/refresh") | ||
| def refresh_prompt_config(http_request: Request) -> Dict[str, Any]: | ||
| """ |
There was a problem hiding this comment.
New endpoint /prompt-config/refresh is not covered by the existing integration tests (there are tests for /health and the inference flows). Add an integration test that saves a prompt via the Ruuter save route and asserts refresh succeeds and the updated instructions are reflected (or at least that cached prompt length changes).
Thirunayan22
approved these changes
Feb 10, 2026
This was referenced Feb 11, 2026
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.