ExLlamaV3: Fix regression in max_seq_len limit

turboderp · turboderp · commit 92cb48c38d6d · 2026-03-22T00:34:47.000+01:00
diff --git a/backends/exllamav3/model.py b/backends/exllamav3/model.py
@@ -264,7 +264,7 @@ async def create(cls, model_directory: pathlib.Path, hf_model: HFModel, **kwargs
             )
             cache_size = cache_size_default
 
-        if max_seq_len < cache_size:
+        if max_seq_len > cache_size:
             logger.warning(
                 f"The given max_seq_len ({max_seq_len}) is larger than the cache size "
                 f"and will be limited to {cache_size} tokens."

Original file line number	Diff line number	Diff line change
`@@ -264,7 +264,7 @@ async def create(cls, model_directory: pathlib.Path, hf_model: HFModel, **kwargs`
`264`	`264`	`)`
`265`	`265`	`cache_size = cache_size_default`
`266`	`266`
`267`		`- if max_seq_len < cache_size:`
	`267`	`+ if max_seq_len > cache_size:`
`268`	`268`	`logger.warning(`
`269`	`269`	`f"The given max_seq_len ({max_seq_len}) is larger than the cache size "`
`270`	`270`	`f"and will be limited to {cache_size} tokens."`