Skip to content

v2.0.0 roadmap #292

@SachinVarghese

Description

@SachinVarghese

P0

  • [Loadgen] Multi-model or LoRA support and traffic splitting
  • [Datagen] Multi-modal support (vision language model)

P1

  • [Datagen] Support different input / output distribution for different stages (helpful for autoscaling)

P2

  • [Metrics] SLO support and conformance of specific latency SLOs
  • [Metrics] GPU utilization / other hardware metrics from Prometheus

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions