MLX: expose topP / topK / minP / repetitionPenalty via CustomGenerati…#168
MLX: expose topP / topK / minP / repetitionPenalty via CustomGenerati…#168SpiraMira wants to merge 3 commits into
Conversation
…onOptions The MLX backend hardcoded sampling parameters in toGenerateParameters / toStructuredGenerateParameters (topP: 1.0, repetitionPenalty: nil, topK/minP at defaults) and never read GenerationOptions.sampling, so callers could only tune temperature and maximumResponseTokens. MLXLMCommon.GenerateParameters already supports the full set. Add topP / topK / minP / repetitionPenalty / repetitionContextSize to MLXLanguageModel.CustomGenerationOptions (all optional, default nil → existing behavior unchanged) and forward them in both parameter mappers, preserving each path's prior defaults via `custom?.field ?? <previous default>`. Implements huggingface#165. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reads the core GenerationOptions.sampling (SamplingMode) in toGenerateParameters / toStructuredGenerateParameters so top-p/top-k/greedy set via the standard sampling surface reach MLX, not only the custom block. Precedence: custom block wins, then sampling-derived, then existing default. Seed is not forwarded (no per-call seed in MLXLMCommon.GenerateParameters). Adds derivation + precedence tests. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ty gated) toFoundationModels() adopts the OS 27 GenerationOptions initializer (adds toolCallingMode) when built with the Xcode 27 SDK, gated by #if compiler(>=6.4) + #available(macOS/iOS/visionOS 27). Falls back to the existing 26 construction otherwise; deployment floor stays 26. Self-contained on this branch (no FM-parity mapping dependency). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
@pcuenca and @james-333i - my attempt at assuaging Pedro’s concerns... Additional changes1. Bridge
|
…onOptions
The MLX backend hardcoded sampling parameters in toGenerateParameters / toStructuredGenerateParameters (topP: 1.0, repetitionPenalty: nil, topK/minP at defaults) and never read GenerationOptions.sampling, so callers could only tune temperature and maximumResponseTokens. MLXLMCommon.GenerateParameters already supports the full set.
Add topP / topK / minP / repetitionPenalty / repetitionContextSize to MLXLanguageModel.CustomGenerationOptions (all optional, default nil → existing behavior unchanged) and forward them in both parameter mappers, preserving each path's prior defaults via
custom?.field ?? <previous default>.Implements #165.
per @pcuenca looking into bridging issues with (new) FoundationModels