Skip to content

sambanova API key fails test #34

@lesshaste

Description

@lesshaste

I have a sambanova key which I can test successfully using curl. However the free-coding-models test fails. This is because it is using the first model in the list which is Qwen3-235B-A22B-Instruct-2507, but this model doesn't exist in the API.

To find out which models it has, I ran:

curl -sS 'https://api.sambanova.ai/v1/models' \
  -H "Authorization: Bearer $SAMBANOVA_API_KEY" |
jq -r '.data[].id' |
while IFS= read -r model; do
  code=$(curl -sS -o /dev/null -w '%{http_code}' \
    'https://api.sambanova.ai/v1/chat/completions' \
    -H 'Content-Type: application/json' \
    -H "Authorization: Bearer $SAMBANOVA_API_KEY" \
    --data "$(jq -nc --arg model "$model" '{model:$model,messages:[{role:"user",content:"Reply with OK"}]}')")

  [ "$code" = "200" ] && echo "$model: PASS" || echo "$model: FAIL (HTTP $code)"
done

This tests all the models https://api.sambanova.ai/v1/models claims to have. Many of them fail the test. The ones that currently pass are:

  1. DeepSeek-V3-0324
  2. DeepSeek-V3.1-Terminus
  3. DeepSeek-V3.2
  4. Llama-3.3-Swallow-70B-Instruct-v0.4
  5. Meta-Llama-3.1-8B-Instruct
  6. Qwen3-235B
  7. gpt-oss-120b

Metadata

Metadata

Assignees

No one assigned

    Labels

    help wantedExtra attention is needed

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions