[Bug] Embedding requests for gemini_cli incorrectly routed to acompletion

[kevincojean]

Summary

The proxy fails with a 500 error when generating embeddings via gemini_cli (e.g., gemini-embedding-001). Requests are incorrectly routed to the chat completion endpoint instead of an embedding endpoint.

Technical Details

1. Routing Bug: In rotator_library/client.py, the _execute_with_retry method hardcodes provider_plugin.acompletion for custom providers, ignoring whether the original call was for embeddings.
2. Missing Implementation: GeminiCliProvider in gemini_cli_provider.py does not implement aembedding.
3. Endpoint Mismatch: Requests are sent to :streamGenerateContent, which returns 404/400 for embedding models, resulting in a 500 error for the client.

Steps to Reproduce

curl http://localhost:8000/v1/embeddings \
  -H "Authorization: Bearer <token>" \
  -d '{"input": "test", "model": "gemini_cli/gemini-embedding-001"}'
Suggested Fix
- Update client.py to check the api_call type before delegation.
- Implement aembedding in GeminiCliProvider using the Google :embedContent endpoint.

[mirrobot-agent]

@kevincojean Thank you for submitting this issue. I am now beginning my analysis and will report back shortly.

[mirrobot-agent]

Initial Analysis Report

Summary: Embedding requests for gemini_cli are incorrectly routed to the chat completion endpoint due to a hardcoded delegation in the rotation client and a missing implementation in the provider.
Issue Validation: Confirmed.
Reproducibility Assessment: Reproducible.
Root Cause Analysis:

1. Routing Bug: In src/rotator_library/client.py, the _execute_with_retry method (line 1443) hardcodes a call to provider_plugin.acompletion whenever a provider has has_custom_logic() == True. This ignores the api_call parameter (which would be litellm.aembedding for embedding requests).
2. Missing Implementation: GeminiCliProvider in src/rotator_library/providers/gemini_cli_provider.py does not override the aembedding method from ProviderInterface, which by default raises NotImplementedError.
3. Endpoint Mismatch: This results in embedding requests being treated as chat completions, leading to 400/404 errors from the Google API and a 500 error for the proxy client.

Suggested Labels: bug, provider:gemini_cli, routing
Proposed Next Steps:

1. Update client.py: Modify _execute_with_retry to check if api_call is litellm.aembedding and call provider_plugin.aembedding accordingly.
2. Implement aembedding in GeminiCliProvider: Add the aembedding method to src/rotator_library/providers/gemini_cli_provider.py, targeting the :embedContent endpoint as suggested.
3. Add Tests: Add a test case for embeddings using the gemini_cli provider to prevent regression.

Missing Information (if any): None.

This analysis was generated by an AI assistant.

[Mirrowel]

Embeddings are not even really supported. I made that ages ago and never worked on that since.

Also, gemini cli has an embedding model?

[kevincojean]

Yep it has 2; and they are offered with the Gemini CLI sub :)

[kevincojean]

Perhaps you can forward embedding requests to LiteLLM?

[Mirrowel]

Yeah i need to research embedding and all that. It's a lot of stuff to do. Even litellm never really did work fine with it.