fix(mlx): accept file:// filesystem model paths (#7461)#10289
Open
localai-bot wants to merge 1 commit into
Open
fix(mlx): accept file:// filesystem model paths (#7461)#10289localai-bot wants to merge 1 commit into
localai-bot wants to merge 1 commit into
Conversation
…d models MLX backends passed request.Model verbatim to mlx_lm/mlx_vlm load(). For a model imported from the filesystem, LocalAI hands the backend a file:// URI (its LocalPrefix), which load() rejects: the scheme is neither a valid HF repo id nor an existing path (Path(model).exists() fails on the scheme), producing "Repo id must be in the form 'repo_name' or 'namespace/repo_name' ... Use repo_type argument if needed". Add a pure, unit-testable resolve_model_path(model, model_file) helper in the shared python_utils: it prefers the resolved ModelFile, strips a file:// scheme and percent-decodes the path, and leaves plain repo ids and local paths untouched. Wire it into the mlx, mlx-vlm and mlx-distributed backends (load, model_key, and the distributed broadcast all use the normalized path). Fixes #7461. Assisted-by: claude:claude-opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #7461
Problem
A filesystem-imported MLX model arrives with its
model:set to afile://URI (LocalAI'sLocalPrefix). The MLX backends passedrequest.Modelverbatim to mlx_lmload(), which rejected it ("Repo id must be in the form ... Use repo_type argument if needed") becausefile://...is neither a valid HF repo id nor an existing path.Fix
New pure helper
resolve_model_path(model, model_file="")inbackend/python/common/python_utils.py: prefers a non-emptyModelFile(the resolved local path, proto field 21), strips afile://scheme and percent-decodes it, and leaves plain HF repo ids / local paths unchanged. Wired into all three MLX backends (mlx,mlx-vlm,mlx-distributed) for theirload()/load_config()/model_key/broadcast paths.Test plan
TestSharedHelperscases inbackend/python/mlx/test.pycoveringfile://stripping, percent-decoding,ModelFileprecedence, and pass-through of repo ids/local paths.py_compile.Note
Could not load a real MLX model end-to-end (needs Apple Silicon); verified at helper/logic + syntax level.
Assisted-by: claude:claude-opus-4-8 [Claude Code]