Skip to content

feat(melious): add Melious provider with 56 models - #4438

Closed
pvdspek wants to merge 9 commits into
anomalyco:devfrom
pvdspek:claude/melious-provider-integration-ad54cf
Closed

feat(melious): add Melious provider with 56 models#4438
pvdspek wants to merge 9 commits into
anomalyco:devfrom
pvdspek:claude/melious-provider-integration-ad54cf

Conversation

@pvdspek

@pvdspek pvdspek commented Aug 10, 2026

Copy link
Copy Markdown

Adds Melious (Melious AI GmbH), a European relay for open-weight models with an OpenAI-compatible endpoint at https://api.melious.ai/v1.

What's here

providers/melious/ provider.toml, logo.svg, and 56 model entries — 49 chat + 7 embedding
models/ 16 new lab metadata entries, needed as base_model targets
labs/ descriptions for 5 new labs; logos for 3 of them

Every provider entry uses base_model and is override-only. New labs: nousresearch, hcompany, baai, intfloat, sentence-transformers.

Sources

Pricing: which figure I used

Melious publishes two prices per model, and they disagree for 8 of 49 chat models. I used the hub balanced-route price, because a bare model ID resolves to balanced — that is what you are billed when you call model="gpt-oss-120b". The /pricing rate card agrees for the other 41.

The sharpest divergence is gpt-oss-120b: rate card €0.04/€0.20 vs balanced €0.22/€0.66. Happy to switch these 8 to the rate card if maintainers prefer the published table as the canonical figure.

All costs are USD/MTok, converted from EUR at 1.1535 USD/EUR — the ECB euro reference rate for 2026-08-07 (last publication before 2026-08-10; ECB publishes on working days only). Rate and date are recorded in a leading comment in every model file.

cost.cache_read is set where Melious charges a discounted cached-prompt rate. Note their docs say prompt caching is billed but not user-controllable — there is no cache-control knob in the API.

Scope

Only models listed on https://melious.ai/hub/models are included. apertus-70b appears on the /pricing rate card but has no hub page (404, and absent from all 64 hub listings), so it is left out. Image and audio models are excluded — they price per-image and per-minute, which the cost schema does not express. models/swiss-ai/apertus-70b.toml is untouched.

Reasoning options

Melious documents exactly one reasoning control: reasoning_effort ("Use reasoning_effort: high when you want them to think harder, low to keep it quick"). There is no reasoning on/off field and no reasoning budget field, so no entry uses toggle or budget_tokens. For hybrid models, none in the effort list carries "thinking disabled". Granularity per model follows the underlying model plus same-surface OpenAI-compatible relays already in this repo (greenpt, cortecs, scaleway). [] is used only where peer entries agree there is no caller-side control — never as a stand-in for uncertainty.

preset: reasoning|non_reasoning is a routing control, not a reasoning control, so it is not modelled as a reasoning_option.

Lab logos

baai and nousresearch come from lobe-icons (MIT); hcompany is the official mark from hcompany.ai. All normalised to currentColor, no fixed width/height, square viewBox.

intfloat and sentence-transformers ship no logo: intfloat is an individual's Hugging Face handle with no brand mark, and sentence-transformers has no SVG anywhere (only _static/logo.png on sbert.net). Both fall back to the default logo, matching the existing behaviour of 10 other logo-less labs. I did not want to trace bitmaps into fake vectors.

LAB_NAME_OVERRIDES in packages/web/src/render.tsx gains 4 entries — without them labName() title-cases the directory id and the pages render "Nousresearch", "Hcompany", "Baai", "Intfloat".

Test plan

  • bun validate — exits 0
  • bun test — 164 pass, 4 fail; all 4 fail identically on a clean dev checkout (2 DeepInfra sync assertions, 2 missing packages/sdk/src/snapshot.js build artifact). Unrelated to this change.
  • cd packages/web && bun run build — exits 0; /providers/melious renders, all 5 /labs/<id> pages render with correct display names, dist/logos/melious.svg and the 3 lab logos are emitted
  • Generated _api.json contains 56 Melious models; every base_model resolves; every resolved model has limit.context + limit.output, and no limit.output/limit.input exceeds its context
  • Every model with reasoning = true has reasoning_options; none set it with reasoning = false
  • Lab logos checked visually at 20/24/32/48/72 px

Reviewer notes

Two judgement calls worth a look:

  1. labs/nousresearch/logo.svg is 19.7 KB — 5× the largest existing lab logo. It is their real mark from the MIT icon set, but densely traced: legible at 48 px+, muddy at 24 px. Easy to drop for the fallback if that is preferred.
  2. structured_output = true on 18 entries where the lab metadata is silent and Melious reports support. These are additions rather than restatements of base values, but say the word if provider files should stay quieter.

🤖 Generated with Claude Code

Melious AI GmbH is a European relay for open-weight models, exposing an
OpenAI-compatible endpoint at https://api.melious.ai/v1.

- providers/melious: provider.toml, currentColor logo.svg, and 56 model
  entries (49 chat + 7 embedding), all override-only via base_model.
- models/: 16 new lab metadata entries required as base_model targets,
  under new labs nousresearch, hcompany, baai, intfloat and
  sentence-transformers.
- labs/: descriptions for those five labs, logos for baai, nousresearch
  and hcompany, plus LAB_NAME_OVERRIDES so they render as "Nous Research",
  "H Company", "BAAI" and "intfloat" rather than a title-cased directory id.

Only models listed on https://melious.ai/hub/models are included.

Costs are converted from Melious's EUR list price for the default
`balanced` route at 1.1535 USD/EUR (ECB euro reference rate, 2026-08-07).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Checking Mistral/Nemotron peers and whether the PR documents Melious reasoning wire format.

@github-actions

Copy link
Copy Markdown
Contributor

Action items

  • [high] [violation] providers/melious/models/deepseek-v4-pro.toml:929 - Check: Relay reasoning_options must follow lab/same-surface peer baseline (AGENTS.md Reasoning options; DeepSeek V4 = toggle + graded effort, not invented L/M/H). Why: Melious is a multi-model relay. First-party DeepSeek V4 Pro is toggle + effort high/max (thinking.type + reasoning_effort); Flash is toggle + low/high/max. These files use effort none/high/max with no toggle and no wire comment—wrong control shape and drops Flash’s low. Action: Align deepseek-v4-pro, deepseek-v4-flash, and deepseek-v4-flash-0731 with lab/peer options Melious actually forwards (typically toggle + the lab effort list), and add a leading wire-path comment for any toggle.
  • [high] [violation] providers/melious/models/deepseek-v3.1.toml:857 - Check: Do not invent GPT-style L/M/H when lab/peers use a narrower control. Why: Established relays (OpenRouter, Novita, Vercel) expose DeepSeek V3.1/V3.2 as toggle only. Melious authors none/low/medium/high with no Melious docs/test evidence for graded effort. Action: Change deepseek-v3.1 and deepseek-v3.2 to the peer baseline (toggle, with leading wire comment) unless you cite Melious-specific proof of graded reasoning_effort values that actually differ.
  • [high] [violation] providers/melious/models/qwen3.5-397b-a17b.toml:1671 - Check: Qwen hybrid reasoners on relays should copy lab/peer controls (toggle and/or budget_tokens), not a free-form effort enum. Why: Alibaba first-party is toggle + budget_tokens; OpenRouter peers are typically toggle. Melious invents mixed effort lists (none/high or none/low/medium/high) across Qwen reasoners with no host wire evidence. Action: Re-author reasoning options for qwen3.5-122b-a10b, qwen3.5-397b-a17b, qwen3.5-9b, qwen3.6-27b, qwen3.6-35b-a3b, and qwen3-32b from lab + same-surface peers (and only what Melious forwards); do not keep invented L/M/H.
  • [high] [violation] providers/melious/models/gemma-4-31b.toml:1002 - Check: Gemma 4 reasoning baseline is toggle, not effort L/M/H. Why: Google first-party and OpenRouter use { type = "toggle" }. Melious sets none/low/medium/high on both Gemma 4 entries. Action: Set gemma-4-31b and gemma-4-26b-a4b to toggle (with leading wire comment) or provide Melious evidence that graded effort is real on this host.
  • [high] [violation] providers/melious/models/kimi-k2.6.toml:1190 - Check: Kimi reasoning controls must match Moonshot lab + peers. Why: Lab K2.5/K2.6 are toggle only; K2.7-code is [] (always-on / no control); K3 is toggle + low/high/max. Melious invents none/high (K2.5), L/M/H (K2.6), L/M/H on K2.7-code (contradicts lab []), and folds K3 off into effort=none instead of toggle. Action: Rewrite kimi-k2.5, kimi-k2.6, kimi-k2.7-code, and kimi-k3 to lab/peer shapes Melious actually exposes; do not invent effort on K2.7-code.
  • [high] [violation] providers/melious/models/minimax-m3.toml:1329 - Check: MiniMax-M3 baseline is toggle, not L/M/H effort. Why: First-party MiniMax-M3 is { type = "toggle" }. Melious authors none/low/medium/high with no host evidence for graded levels. Action: Use toggle (leading wire comment) or document Melious-specific graded effort that is real on the wire.
  • [high] [violation] providers/melious/models/nemotron-3-super-120b-a12b.toml:1451 - Check: Nemotron lab baseline is toggle. Why: NVIDIA entries use { type = "toggle" }. Melious invents none/high (nano) and none/low/medium (super). Action: Align both Nemotron files with toggle (or proven Melious controls), not invented effort enums.
  • [high] [violation] providers/melious/models/glm-5.2.toml:1058 - Check: GLM effort/toggle must match Zhipu/Z.AI native mapping. Why: Lab/Z.AI GLM-5.2 is effective high/max (with documented low/medium→high and none/minimal as off). Melious uses none/low/medium/high and omits max. GLM-4.7/5/5.1 lab is toggle, not none/high effort. Action: Fix glm-5.2 to the lab effort set Melious forwards; set glm-4.7, glm-5, and glm-5.1 to toggle (or effort-with-none only if Melious’s API is documented as effort-only binary) with wire comments.
  • [medium] [possible mistake] providers/melious/models/bge-large-en-v1.5.toml:795 - Check: Provider limit overrides must be real host deltas. Why: Lab metadata sets context 512; Melious overrides to 8192 without a cited Melious/hub limit for this embedding ID (other BGE entries keep lab context). Action: Confirm Melious’s hub/API context for bge-large-en-v1.5 is 8192; if not, drop the override or set the true value.
  • [low] [possible mistake] .pr-review/pull-request.json:4 - Check: Data-changing PRs should map evidence to reasoning claims. Why: Pricing/docs citations are present, but there is no mapped evidence that Melious’s OpenAI-compatible API accepts the many invented effort enums above (vs lab toggle/budget surfaces). Action: In the PR body, state Melious’s reasoning request fields and which models were verified (docs or live calls), and update TOML options to match.

The API reference for this host documents exactly one reasoning control:
`reasoning_effort` accepting low|medium|high for reasoning models, ignored by
non-reasoning models. There is no thinking/toggle field and no reasoning budget
field, so the previous per-model effort enums invented values this API does not
accept (`none`, `max`) and narrowed others without evidence.

- All 34 reasoning models now use the documented low|medium|high enum.
- provider.toml records the wire field and the reference URL.
- Drop the context overrides on bge-large-en-v1.5 and qwen3-embedding-8b: the
  hub reports 8192 and 41000, above the models' architectural maxima of 512 and
  32768, so the lab limits are kept.

Ref: https://melious.ai/docs/reference/chat-completions

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@pvdspek

pvdspek commented Aug 18, 2026

Copy link
Copy Markdown
Author

Thanks — the "no mapped evidence for the effort enums" item (the [low] one) was the right thread to pull, and it turns out to invalidate most of the specific fixes proposed above it. Pushed in f05c753.

The missing evidence

I had been reasoning from the prose in /docs/concepts/models ("Use reasoning_effort: highlow to keep it quick"). There is a proper request schema at https://melious.ai/docs/reference/chat-completions, which states:

Reasoning

Parameter Type Default Description
reasoning_effort string model default "low", "medium", "high" for reasoning models. Ignored by non-reasoning models.

That is the complete reasoning surface. The page has zero occurrences of thinking. No thinking.type, no enable_thinking, no thinking_budget, no reasoning object. preset is listed under "Melious-specific" and takes reasoning/non_reasoning to bias routing — it is not a thinking switch.

What I fixed

You were right that I invented values. none and max are not in this host's enum, and I had narrowed several models to ["none","high"] or ["none","low","medium"] with nothing backing it. All 34 reasoning models now carry exactly:

reasoning_options = [{ type = "effort", values = ["low", "medium", "high"] }]

provider.toml now records the wire field, its accepted values, and the reference URL in the leading comment block, per the AGENTS.md rule that sync keeps only a leading header.

Where I don't think the suggested fix holds

Most action items ask for { type = "toggle" } (Gemma 4, GLM 4.7/5/5.1, DeepSeek V3.1/V3.2/V4, Kimi K2.5/K2.6, MiniMax-M3, Nemotron, Qwen hybrids) or budget_tokens (Qwen). Those are the lab-native shapes, and Melious does not forward them — it flattens everything to reasoning_effort. Authoring them here would describe a request this endpoint does not accept, and AGENTS.md rules that out from both directions:

  • "Never invent budget_tokens unless this host (or the lab API it clearly proxies) has a real reasoning budget field." Melious has none.
  • "Every toggle needs a leading top-of-file comment with the exact wire path." There is no wire path to cite, because there is no toggle field.

The relay guidance to "copy the underlying model's controls" is about granularity, not about pasting the lab's field names onto a host that doesn't expose them. On this surface the graded enum is the whole control, which is why it comes out uniform: the flattening is a real property of the host, not a gap in research.

Two knock-on points:

  • kimi-k2.7-code "contradicts lab []". Under this schema [] would be the wrong claim — [] means no caller control, and reasoning_effort is accepted for reasoning models here. The 6 entries I had at [] (R1-0528, MiniMax M2.5/M2.7, Qwen3-Next-Thinking, both Hermes) were understating the host and are now L/M/H too.
  • GLM-5.2 "omits max". Correct that Z.AI natively has max; it is not in Melious's enum, so it cannot be authored here.

If a maintainer would rather this host report [] where the upstream control is genuinely binary, I'll make that change — but toggle/budget_tokens would misdescribe the API.

bge-large-en-v1.5 context — good catch, dropped

The 8192 came from Melious's hub payload, but BGE Large EN v1.5 is hard-capped at 512 positions, so 8192 cannot be real. Override removed; it inherits the lab's 512.

Auditing the rest on the same principle — trust a host cap below the model's architectural maximum, distrust one above it — turned up one more: qwen3-embedding-8b, where the hub claims 41000 against a native 32768. Also dropped. Both files carry a leading comment recording the hub figure and why it isn't used, so a future sync doesn't reinstate it.

The remaining upward overrides check out as genuine: DeepSeek R1-0528/V3.1/V3.2 at 164000 matches DeepSeek's 163840 max_position_embeddings (the lab entries under-report), and qwen3-next-80b-a3b-thinking at 262144 is Qwen3-Next's native window.

bun validate passes on the current base.

pvdspek and others added 2 commits August 18, 2026 15:24
…r-integration-ad54cf

# Conflicts:
#	models/alibaba/qwen3-vl-235b-a22b-instruct.toml
dev's canonical models/alibaba/qwen3-vl-235b-a22b-instruct.toml (taken over
mine in the merge) states a 131072 context and text+image modalities, and
06f44f5 set meta/llama-3.1-8b-instruct context to 128000. Both now match what
Melious serves, so the provider entries no longer need to restate them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Inspecting base models for suspicious overrides and whether new lab entries collide with existing files.

@github-actions

Copy link
Copy Markdown
Contributor

Action items

  • [high] [violation] models/meta/llama-3.1-8b-instruct.toml:1 / models/alibaba/qwen3-vl-235b-a22b-instruct.toml:1 - Check: New lab metadata must not replace existing models/ entries. Why: Both paths already exist on dev with different facts (e.g. Llama 8B context 128_000 and name Llama-3.1-8B-Instruct; Qwen3-VL context 131_072 and text+image only). The PR still ships them as new files with divergent values, which would clobber shared lab metadata used by other providers. Action: Drop those two lab-file additions from the PR and keep pointing Melious at the existing base paths (as commit 3 already assumes for provider overrides).
  • [high] [possible mistake] providers/melious/models/deepseek-r1-0528.toml:6 - Check: Relay reasoning_options must match this host’s real controls and the lab/peer baseline, not a blanket GPT-style enum. Why: DeepSeek R1 is always-on with no graded effort on established peers (providers/openrouter/models/deepseek/deepseek-r1*.toml and first-party reasoner use []). Authoring low|medium|high claims caller control peers treat as absent. Action: Use reasoning_options = [] unless Melious docs/tests show reasoning_effort meaningfully changes R1-0528 behavior; cite that evidence in a leading comment if keeping effort.
  • [high] [possible mistake] providers/melious/models/minimax-m2.5.toml:6 / providers/melious/models/minimax-m2.7.toml:6 - Check: Do not invent effort levels when lab and same-surface peers expose no control. Why: First-party MiniMax and OpenRouter entries for M2.5/M2.7 use reasoning_options = []. Melious still sets low|medium|high. Action: Switch both to [], or provide Melious-specific evidence that reasoning_effort is honored for these IDs.
  • [high] [possible mistake] providers/melious/models/kimi-k2.7-code.toml:5 - Check: First-party baseline for this model is no caller control. Why: providers/moonshotai/models/kimi-k2.7-code.toml authors reasoning_options = [], but Melious adds low|medium|high. Action: Match the lab with [], or document Melious-only graded effort with a leading wire/evidence comment.
  • [high] [possible mistake] providers/melious/models/qwen3-next-80b-a3b-thinking.toml:5 - Check: Always-on / thinking-suffix models should not get invented effort enums on relays. Why: OpenRouter peers use [] for this ID; the lab path is budget-oriented, not OpenAI L/M/H. Melious still sets low|medium|high. Action: Prefer [] (always-on thinking ID), or prove Melious maps reasoning_effort for this model and document it.
  • [medium] [possible mistake] providers/melious/models/hermes-4-70b.toml:5 / providers/melious/models/hermes-4-405b.toml:5 - Check: Hermes 4 hybrid reasoning controls on relays. Why: Established OpenRouter peers use toggle (not L/M/H); the first commit correctly noted system-prompt steering and used []. The follow-up blanket L/M/H rewrite conflicts with that peer baseline. Action: Revert to [] (or peer toggle only if Melious exposes an on/off wire field with a leading toggle comment)—do not keep unproven low|medium|high.

Melious only forwards `reasoning_effort` (low|medium|high), so authoring that
enum for models with no upstream effort concept overstated caller control. Set
[] for the always-on reasoners and the prompt-steered hybrids, each with a
leading comment citing the baseline:

- deepseek-r1-0528, minimax-m2.5, minimax-m2.7, kimi-k2.7-code: the first-party
  lab host authors [] (deepseek-reasoner, MiniMax-M2.5/M2.7, kimi-k2.7-code).
- qwen3-next-80b-a3b-thinking: Alibaba exposes only a reasoning budget, which
  Melious has no field for; OpenRouter authors [].
- hermes-4-405b, hermes-4-70b: thinking is system-prompt steered; OpenRouter
  models it as a toggle, which Melious has no wire field for.

The other 27 keep low|medium|high: their underlying models have a real on/off or
graded thinking control, and effort is the only dial this host exposes.

Also revert the [[links]] addition to models/meta/llama-3.1-8b-instruct.toml so
this PR no longer modifies shared lab metadata that already exists on dev.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@pvdspek

pvdspek commented Aug 19, 2026

Copy link
Copy Markdown
Author

This round landed — all six reasoning items were right, and I've adopted them plus one more the sweep turned up. Pushed in 56a1aea.

Reasoning options

My previous blanket low|medium|high overreached. The API only accepts that enum; it does not follow that every reasoning model has a dial for it to drive. I verified every citation in this repo, and the first-party lab hosts are decisive:

model first-party entry its options
kimi-k2.7-code providers/moonshotai/models/kimi-k2.7-code.toml []
minimax-m2.5 providers/minimax/models/MiniMax-M2.5.toml []
minimax-m2.7 providers/minimax/models/MiniMax-M2.7.toml []
deepseek-r1-0528 providers/deepseek/models/deepseek-reasoner.toml []

If the lab that built the model offers no effort control on its own API, a relay cannot invent one — reasoning_effort would be accepted at the edge and dropped downstream. Set to []:

  • deepseek-r1-0528, minimax-m2.5, minimax-m2.7, kimi-k2.7-code — first-party [] as above.
  • qwen3-next-80b-a3b-thinkingproviders/alibaba/models/qwen3-next-80b-a3b-thinking.toml exposes budget_tokens only, no effort grades; Melious has no budget field. OpenRouter authors [].
  • hermes-4-405b, hermes-4-70b — system-prompt steered; OpenRouter models it as toggle, which this host has no wire field for. (Reinstates what commit 1 had, which you correctly noted I'd regressed.)

kimi-k2.7-code is the one I'd have missed from the earlier round — thanks for pinning it to the first-party file.

Each of the seven carries a leading comment naming the baseline it follows.

Why the other 27 keep low|medium|high

The line is whether a dial exists upstream at all, not whether its shape matches:

  • [] — no dial: always-on reasoners, or the only upstream control is one this host cannot express.
  • effort low|medium|high — a dial exists (on/off or graded), and effort is the only one Melious exposes. Covers DeepSeek V3.x/V4, GLM, Gemma 4, Kimi K2.5/K2.6/K3, MiniMax-M3, Nemotron, the Qwen3 hybrids, Mistral, and gpt-oss.

That keeps toggle/budget_tokens out of the files, since neither field exists on this API, while not claiming graded control where nothing is controllable. gpt-oss-120b/20b are the clearest case for keeping effort — graded reasoning effort is native to gpt-oss and every peer in this repo authors low|medium|high.

Lab metadata — one real, one already resolved

models/meta/llama-3.1-8b-instruct.toml: fixed. The remaining diff was a single additive [[links]] model-card block — it did not change any fact (context 128_000 and the name were already dev's, inherited through the merge). Still, a provider PR shouldn't touch shared lab metadata, so I reverted the file to dev's exact blob. models/ changes in this PR are now additions only, 14 new files, zero modifications.

models/alibaba/qwen3-vl-235b-a22b-instruct.toml: already resolved before this review. dev added it while the PR was open; the merge in 699125c hit an add/add conflict and I took dev's version wholesale. It is byte-identical — same blob f6d7739a on both sides, and it does not appear in git diff origin/dev...HEAD at all. The facts quoted in the finding (context 131_072, text+image) are the values this PR ships, because they are dev's. Taking that version is also what made two Melious overrides redundant, which is why they were dropped in d390471.

bun validate passes.

@github-actions

Copy link
Copy Markdown
Contributor

Checking which new lab files already exist on base and reviewing more potential issues.

@pvdspek
pvdspek marked this pull request as draft August 19, 2026 09:06
@pvdspek
pvdspek marked this pull request as ready for review August 19, 2026 09:06
@github-actions

Copy link
Copy Markdown
Contributor

Action items

  • [high] [violation] models/meta/llama-3.1-8b-instruct.toml:1 - Check: Provider PRs must not rewrite existing shared lab metadata with divergent facts. Why: This file already exists on base (name = "Llama-3.1-8B-Instruct", context = 128_000, no license). The PR still lands a different lab definition (name = "Llama 3.1 8B Instruct", context = 131_072, added license). That rewrites canonical metadata for every host of this model, not just Melious. Patch 3 already assumes the base 128k lab entry. Action: Drop this path from the PR (keep only providers/melious/models/llama-3.1-8b-instruct.toml pointing at the existing base_model).
  • [high] [violation] models/alibaba/qwen3-vl-235b-a22b-instruct.toml:1 - Check: Do not replace an existing lab entry with incompatible modalities/limits. Why: Base already has this lab model as text+image, context = 131_072. The PR still introduces a divergent lab body (adds video, context = 262_144). Patch 3 removed Melious-side limit/modality overrides specifically because the base lab already matches what Melious serves; shipping the divergent lab file undoes that and changes shared metadata for other providers. Action: Remove this lab file from the PR and keep base_model = "alibaba/qwen3-vl-235b-a22b-instruct" against the existing entry.
  • [medium] [possible mistake] providers/melious/models/devstral-2-123b-instruct-2512.toml:13 - Check: Provider limit overrides should reflect the host’s real served window, not mirror context into output. Why: Lab mistral/devstral-2512 is 262k/262k; this entry sets both context and output to 200_000. Equal context/output caps are often hub misreads. Action: Verify Melious’s hub/API max output for this ID and correct limit.output (and context if needed) to the documented host values.
  • [medium] [possible mistake] providers/melious/models/mistral-medium-3.5-128b.toml:12 - Check: Same host-limit accuracy for equal context/output overrides. Why: Lab is 262k/262k; Melious sets both to 256_000. That may be intentional truncation, but pairing identical context/output is suspicious without a cited max-completion figure. Action: Confirm Melious’s served context and max output separately; keep only real deltas from the lab limits.
  • [medium] [possible mistake] providers/melious/models/nemotron-3-nano-30b-a3b.toml:14 - Check: Host limit.output must not be copied from context without evidence. Why: Lab is 262k/262k; Melious sets both to 128_000. Peers vary, but equal caps again look like a single hub “context” field applied twice. Action: Verify Melious max completion tokens for this model and set limit.output from that value.
  • [low] [possible mistake] providers/melious/models/qwen3-coder-next.toml:5 - Check: Provider capability overrides need host evidence when flipping lab defaults. Why: Lab alibaba/qwen3-coder-next has structured_output = true; this sets false. Some peers do the same, but Melious-specific evidence is not cited in the file/PR body for this flip. Action: Confirm Melious does not support structured/JSON output for this ID (or drop the override if it does).

The hub payload's `parameters` block carries a `max_tokens` alongside
`context_window`; the earlier scrape only read the latter, so limit.output was a
mechanical clamp to the served context instead of the host's real cap.

- devstral-2-123b-instruct-2512: output 200_000 -> 8_192
- nemotron-3-nano-30b-a3b: output 128_000 -> 8_192
- kimi-k2.5, mistral-medium-3.5-128b: the hub publishes no max_tokens for these,
  so the cap at the served context stays, now with a comment saying so.
- qwen3-coder-next: cite capabilities.structured_output = false behind the flip.

Cross-checked the whole provider against a fresh scrape: all 18
structured_output overrides and all 24 context overrides match the hub exactly.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@pvdspek

pvdspek commented Aug 19, 2026

Copy link
Copy Markdown
Author

Applied the limit and capability items in 3197444. The two [high] items are already satisfied on the branch — details at the end.

limit.output — you were right, and this found a field I had missed

The hub payload's parameters block carries a max_tokens next to context_window:

parameters: { top_p: .95, max_tokens: 8192, temperature: .15, context_window: 200000 }

My original scrape only read context_window, so limit.output was a mechanical clamp to the served context (to avoid inheriting a lab output larger than the overridden context) rather than the host's real cap. Where the hub publishes max_tokens, that clamp was badly wrong:

model was now source
devstral-2-123b-instruct-2512 200 000 8 192 parameters.max_tokens
nemotron-3-nano-30b-a3b 128 000 8 192 parameters.max_tokens

Only 4 of 56 models publish the field. The other two are the E5 embedders, where it just echoes context and output = 1 is the repo convention (matching providers/google/models/gemini-embedding-001.toml), so those are unchanged.

kimi-k2.5 is a fourth instance of the same pattern that the review did not flag. It shipped output = 256_000 from the same clamp. It and mistral-medium-3.5-128b publish no max_tokens, so capping at the served context remains the only defensible value — both now carry a leading comment saying exactly that, so it no longer reads as a hub figure.

qwen3-coder-next structured_output

The flip is correct — the hub reports capabilities.structured_output = false for this ID against the lab's true — and the file now cites that flag in its leading comment.

While confirming it I re-scraped all 56 hub pages and cross-checked the whole provider:

  • 18/18 structured_output overrides match the hub exactly, no mismatches.
  • 24/24 context overrides match the hub's served deployment context exactly.

bun validate passes, and no resolved model has limit.output exceeding limit.context.

The two [high] lab-metadata items

These are already resolved on the branch; both files are byte-identical to base:

models/meta/llama-3.1-8b-instruct.toml         base=ff2584e0  head=ff2584e0
models/alibaba/qwen3-vl-235b-a22b-instruct.toml base=f6d7739a  head=f6d7739a

Neither path appears in git diff origin/dev...HEAD, and models/ in this PR is additions only — 14 new files, zero modifications. models/meta/llama-3.1-8b-instruct.toml was reverted to base's exact blob in 56a1aea, and models/alibaba/qwen3-vl-235b-a22b-instruct.toml was resolved to base's version when the merge in 699125c hit an add/add conflict.

The values quoted in the finding (context = 131_072 plus a license for Llama; video and context = 262_144 for Qwen-VL) come from the first commit in the series, which later commits reverted. The review appears to be reading the cumulative patch series rather than the net diff — consistent with the phrasing "Patch 3 already assumes the base 128k lab entry." So there is no path left to drop; the requested end state is what the branch already ships.

Happy to squash the branch if you'd prefer no intermediate commit to have touched shared lab metadata at all — just say so and I'll force-push a cleaned history.

@github-actions

Copy link
Copy Markdown
Contributor

Action items

  • [high] [violation] providers/melious/models/deepseek-v4-pro.toml:6 - Check: Relay reasoning_options must follow lab + same-surface peer baselines, not a universal GPT-style enum (AGENTS.md Reasoning options; audit skill anti-pattern “forcing L/M/H onto DeepSeek V4”). Why: Melious is a multi-model relay. Final entries author effort = low|medium|high for essentially every reasoner, including DeepSeek V4 (lab/peers: toggle + high/max), toggle-only hosts (Kimi K2.5/K2.6, GLM-4.7/5/5.1, MiniMax M3, Gemma 4 — lab/OpenRouter use toggle or []), Qwen paths that are toggle/budget (not graded effort), and Mistral Medium/Small (lab none/high). That invents distinct L/M/H levels, drops none/max where native, and overstates caller control when peers correctly use [] or narrower sets. Host docs that only name an OpenAI-style field do not license one enum for every upstream surface. Action: Re-author per model from lab + peer intersection ∩ what Melious actually forwards. At minimum: DeepSeek V4 → not L/M/H (use host-real high/max and off if present, else document honest subset); toggle-only / no graded dial → [] or real on/off mapping, not invented grades; Mistral → keep none if off exists; stop the blanket L/M/H copy-paste across the 27 files.
  • [high] [violation] models/alibaba/qwen3-vl-235b-a22b-instruct.toml:1 - Check: Do not overwrite existing shared lab metadata with third-party scrape values. Why: Base already has this lab entry (context = 131_072, input = ["text", "image"]). The PR still introduces a divergent lab file (context = 262_144, adds video, different name/description). That mutates provider-agnostic facts for every consumer, after the PR itself notes dev’s canonical entry should win. Same class of problem for models/meta/llama-3.1-8b-instruct.toml (base name/context differ from the PR’s redefinition). Action: Drop lab-file creates/edits for IDs that already exist on base; keep only providers/melious/ overrides. Add new models/ files only for missing lab IDs.
  • [medium] [violation] providers/melious/models/glm-5.1.toml:4 - Check: base_model provider files must be override-only — no restated identical fields. Why: structured_output = true is already true on models/zhipuai/glm-5.1.toml, so the provider restatement is redundant noise and fails the override-only rule. Action: Remove redundant structured_output = true (and audit other Melious entries for the same restatement where the lab already matches).
  • [low] [possible mistake] providers/melious/models/qwen3.5-122b-a10b.toml:14 - Check: Modality overrides must be real host deltas, not incomplete scrapes. Why: Several Qwen/Kimi entries narrow lab modalities (e.g. drop audio and/or video on qwen3.5-122b/397b, qwen3.6-27b/35b-a3b, qwen3.5-9b, kimi-k2.5) while gemma-4 entries add video. Without hub evidence that Melious truly drops those inputs, catalog consumers get wrong capability flags. Action: Confirm each modality delta against Melious hub/API for that model ID; drop overrides that only restate or incorrectly shrink the lab set.

@pvdspek

pvdspek commented Aug 19, 2026

Copy link
Copy Markdown
Author

Superseded by #5047 — closing this one.

Same final content, squashed onto current dev as a single commit. The automated reviewer reads the cumulative patch series rather than the net diff, so the two lab-metadata findings kept resurfacing against the first commit in this branch even though later commits reverted them and the files were byte-identical to base. Squashing removes that ambiguity: models/ in #5047 is additions only, and neither models/meta/llama-3.1-8b-instruct.toml nor models/alibaba/qwen3-vl-235b-a22b-instruct.toml appears in the diff at all.

All review feedback from this thread is carried over — the reasoning_effort wire format and the seven [] entries, limit.output from the hub's parameters.max_tokens, the dropped over-max context overrides on bge-large-en-v1.5 and qwen3-embedding-8b, and the cited structured_output flip on qwen3-coder-next. Thanks for the passes here.

@pvdspek pvdspek closed this Aug 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant