Skip to content

Refactor Unified API for stable MoE execution with Multi-Modal Routin…#77

Open
harshdesaii wants to merge 2 commits intodevelopfrom
process_request_error_solved
Open

Refactor Unified API for stable MoE execution with Multi-Modal Routin…#77
harshdesaii wants to merge 2 commits intodevelopfrom
process_request_error_solved

Conversation

@harshdesaii
Copy link
Collaborator

Description

Refactored the Unified API to fix MoE initialization and routing issues by correcting factory setup, expert creation, and router integration. Added deterministic multi-modal routing with equal weight distribution across active experts, standardized response formats, and improved validation.

Fixes # (issue)

  • Fixed incorrect MoE factory usage that caused Unknown MoE model type errors.
  • Fixed router instantiation bug caused by using abstract ExpertRouter.
  • Updated modality router to distribute weights evenly across matched experts.
  • Added real experts in the API:

    text_expert
    Default LLM provider via factory

    image_expert
    OpenAI Vision if API key exists
    otherwise Ollama Vision fallback

    audio_expert
    OpenAI Whisper if API key exists

  • Fixed request packing to support multiple inputs per modality.
  • Standardized API response format.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants