Skip to content

Engine failure - Not enough memory for KV cache, try reducing --num page #2

Description

@freyandere

PC:
64gb RAM
4090 24gb Vram
14900k
With 50% safety margin, QWEN 3.6 27b nvfp4, wasn't able to load .

Log too long for Github, here's pastebin link - https://pastebin.com/xJUBXRi2

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions