add guides section and fine-tuning overview by iamleonie · Pull Request #115 · Liquid4All/docs

iamleonie · 2026-07-09T10:46:44Z

Added a new Guides section under the LFM sidebar, placed below Cloud Inference.
Added three new guide pages
- /guides/hardware-evaluation
- /guides/use-case-evaluation
- /guides/migration-guide
Added a new Fine-tuning Overview page at /lfm/fine-tuning/overview.
- Made Fine-tuning Overview the first page in the Fine-tuning sidebar section.
- Updated /lfm/fine-tuning to redirect to the new Overview page.
Updated SGLang migration guidance to match the existing SGLang docs command and lfm2 tool-call parser.
Standardized LFM2.5-8B-A1B wording to 1.5B active parameters.

mintlify · 2026-07-09T10:49:37Z

Preview deployment for your docs. Learn more about Mintlify Previews.

Project	Status	Preview	Updated (UTC)
liquid-docs	🟢 Ready	View Preview	Jul 9, 2026, 10:50 AM

💡 Tip: Enable Workflows to automatically generate PRs for you.

alay2shah

Looks good after the latest updates.

mlabonne

There are factual mistakes in the content we need to fix and re-deploy.

We also cannot stretch the maintenance surface that much, even if GTM maintains these pages.

mlabonne · 2026-07-09T18:59:34Z

+
+Migration notes:
+
+- **From Qwen:** ChatML is familiar, but token details differ. Use `apply_chat_template`; do not reuse literal Qwen strings.


I suggest adding a link to HF here like https://huggingface.co/docs/transformers/main/en/chat_templating#using-applychattemplate

mlabonne · 2026-07-09T19:02:06Z

+| Qwen3-0.6B, Llama-3.2-1B, Gemma-3-270M/1B | [LFM2.5-350M](https://huggingface.co/LiquidAI/LFM2.5-350M) | Classification, extraction, routing. Also consider LFM2.5-230M. |
+| Qwen3-1.7B, Llama-3.2-3B, Gemma-3-4B | [LFM2.5-1.2B-Instruct](https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct) | The default dense workhorse. |
+| Qwen3-4B/8B, Llama-3.1-8B, Gemma-3-12B | [LFM2.5-8B-A1B](https://huggingface.co/LiquidAI/LFM2.5-8B-A1B) | MoE with 8B total and 1.5B active parameters. |
+| Qwen3-14B/32B and larger dense models | [LFM2-24B-A2B](https://huggingface.co/LiquidAI/LFM2-24B-A2B) | 24B total and roughly 2.3B active parameters. |
+| Qwen3 thinking mode or DeepSeek distills | [LFM2.5-1.2B-Thinking](https://huggingface.co/LiquidAI/LFM2.5-1.2B-Thinking) | Reasoning traces at 1.2B. |


I'm not sure this is super accurate. I would bundle the 230M and 350M together. Same with 1.2B Instruct and Thinking (just depends on the acceptable latency). Same with the 2.6B and 8B-A1B.

mlabonne · 2026-07-09T19:02:21Z

+
+See the [Complete Model Library](/lfm/models/complete-library) for the full catalog.
+
+LFM2.5 dense models support 32K context, and LFM2.5-8B-A1B supports 128K. If you rely on 128K context in a small dense Qwen or Llama model, check your production context distribution before assuming this is a blocker. Most sub-8B deployments do not exceed 8K tokens.


Too model-specific, too hard to maintain.

mlabonne · 2026-07-09T19:03:06Z

+<Tabs>
+  <Tab title="Transformers">
+    Change the model ID and use the model's chat template:
+
+    ```python
+    from transformers import AutoModelForCausalLM, AutoTokenizer
+
+    model_id = "LiquidAI/LFM2.5-1.2B-Instruct"
+    model = AutoModelForCausalLM.from_pretrained(
+        model_id,
+        dtype="bfloat16",
+        device_map="auto",
+    )
+    tokenizer = AutoTokenizer.from_pretrained(model_id)
+
+    inputs = tokenizer.apply_chat_template(
+        [{"role": "user", "content": "What is C. elegans?"}],
+        add_generation_prompt=True,
+        return_tensors="pt",
+        return_dict=True,
+    ).to(model.device)
+
+    out = model.generate(
+        **inputs,
+        max_new_tokens=512,
+        do_sample=True,
+        temperature=0.1,
+        top_k=50,
+        repetition_penalty=1.05,
+    )
+    ```
+
+    Do not reuse literal Llama-style or Gemma-style prompt strings. They will usually run, but quality will degrade.
+
+    See [Transformers](/deployment/gpu-inference/transformers).
+  </Tab>
+
+  <Tab title="vLLM">
+    Serve the model behind an OpenAI-compatible endpoint:
+
+    ```bash
+    vllm serve LiquidAI/LFM2.5-1.2B-Instruct
+    ```
+
+    Your OpenAI-client code stays mostly unchanged:
+
+    ```python
+    client.chat.completions.create(
+        model="LiquidAI/LFM2.5-1.2B-Instruct",
+        messages=messages,
+        temperature=0.1,
+        extra_body={"top_k": 50, "repetition_penalty": 1.05},
+    )
+    ```
+
+    If you use OpenAI-style `tools=[...]`, configure the LFM tool-call parser before switching traffic. See [Tool calling](#tool-calling) below and the [vLLM guide](/deployment/gpu-inference/vllm).
+  </Tab>
+
+  <Tab title="SGLang">
+    Serve the model through SGLang's OpenAI-compatible API and keep your client integration mostly unchanged:
+
+    ```bash
+    sglang serve --model-path LiquidAI/LFM2.5-1.2B-Instruct --tool-call-parser lfm2
+    ```
+
+    Use LFM sampling defaults and configure the LFM tool-call parser if your evaluation includes tools.
+
+    See [SGLang](/deployment/gpu-inference/sglang).
+  </Tab>
+
+  <Tab title="llama.cpp">
+    Pull the official GGUF and choose the quantization level you plan to deploy:
+
+    ```bash
+    llama-server -hf LiquidAI/LFM2.5-1.2B-Instruct-GGUF:Q4_K_M
+    ```
+
+    The GGUF embeds the correct chat template. Do not override it with an old Qwen or Llama template.
+
+    See [llama.cpp](/deployment/on-device/llama-cpp).
+  </Tab>
+
+  <Tab title="Ollama / LM Studio">
+    Use the official GGUF artifact and keep the embedded chat template:
+
+    ```bash
+    ollama run hf.co/LiquidAI/LFM2.5-1.2B-Instruct-GGUF:Q4_K_M
+    ```
+
+    If you are using LM Studio, select the matching GGUF quantization and avoid overriding the prompt template with a previous Qwen, Llama, or Gemma template.
+
+    See [Ollama](/deployment/on-device/ollama) and [LM Studio](/deployment/on-device/lm-studio).
+  </Tab>
+
+  <Tab title="MLX / ONNX">
+    Per-precision repos exist for each model, including `-MLX-4bit` through `-MLX-8bit` and `-ONNX`. Swap the repo ID in `mlx-lm` or your ONNX Runtime pipeline.
+
+    See [MLX](/deployment/on-device/mlx) and [ONNX](/deployment/on-device/onnx).
+  </Tab>
+
+  <Tab title="LEAP SDK">
+    The [LEAP SDK](/deployment/on-device/sdk/quick-start) gives iOS and Android apps a supported runtime with function calling and constrained generation included.
+  </Tab>
+</Tabs>


We should really reuse existing code blocks instead of re-writing them here because it will be a nightmare to maintain

mlabonne · 2026-07-09T19:03:58Z

+| `temperature` | `0.1` | `0.6` to `0.7`, which can cause drift |
+| `top_k` | `50` | disabled |
+| `repetition_penalty` | `1.05` | `1.0` |


Our gen parameters are actually model-specific so it's not correct :(

mlabonne · 2026-07-09T19:04:58Z

+
+What to do:
+
+1. Pass tools through `tokenizer.apply_chat_template(messages, tools=[...])` or the serving runtime's supported tool mechanism.


We should also maybe link it to the relevant section in apply_chat_template()

mlabonne · 2026-07-09T19:05:20Z

+
+1. Pass tools through `tokenizer.apply_chat_template(messages, tools=[...])` or the serving runtime's supported tool mechanism.
+2. Parse the Pythonic call list between `<|tool_call_start|>` and `<|tool_call_end|>`.
+3. On vLLM or SGLang, configure the LFM tool-call parser instead of a generic JSON parser.


I think the correct name is "lfm2" (to verify)

mlabonne · 2026-07-09T19:05:45Z

+2. Parse the Pythonic call list between `<|tool_call_start|>` and `<|tool_call_end|>`.
+3. On vLLM or SGLang, configure the LFM tool-call parser instead of a generic JSON parser.
+4. Return results as `tool` role messages, with JSON content if useful.
+5. For strict schemas, use constrained decoding. See [Constrained Generation](/deployment/on-device/sdk/constrained-generation).


Not sure this is worth mentioning here

mlabonne · 2026-07-09T19:06:01Z

+3. On vLLM or SGLang, configure the LFM tool-call parser instead of a generic JSON parser.
+4. Return results as `tool` role messages, with JSON content if useful.
+5. For strict schemas, use constrained decoding. See [Constrained Generation](/deployment/on-device/sdk/constrained-generation).
+6. If you fine-tune for tool calling, train on the native Pythonic format and convert to any internal DSL after parsing.


DSL is never defined 👀

mlabonne · 2026-07-09T19:07:04Z

+- [ ] Chat template comes from the model
+- [ ] Sampling updated for LFM defaults
+- [ ] Tool parser configured if tool calling is in scope
+- [ ] Context length validated against production p95


We haven't talked about this, right?

iamleonie · 2026-07-10T15:24:14Z

Review findings addressed in #116

add guides section and fine-tuning overview

9a69930

iamleonie requested a review from alay2shah July 9, 2026 10:47

mintlify Bot deployed to staging July 9, 2026 10:50 View deployment

Merge remote-tracking branch 'origin/main' into add-guides-section

4a4cccf

mintlify Bot deployed to staging July 9, 2026 20:17 View deployment

Make table guide links purple

da52a86

mintlify Bot deployed to staging July 9, 2026 20:27 View deployment

Fix wrapped table link underlines

e4a7326

mintlify Bot deployed to staging July 9, 2026 20:30 View deployment

Match guide table links to model table style

4e27be5

mintlify Bot deployed to staging July 9, 2026 20:34 View deployment

Prevent guide table link overlap

393a0b2

mintlify Bot deployed to staging July 9, 2026 20:38 View deployment

Restore guide table link underline

9b351cd

mintlify Bot deployed to staging July 9, 2026 20:45 View deployment

Fix guide checklist styling

fccb2ad

mintlify Bot deployed to staging July 9, 2026 20:51 View deployment

Use bullets for guide checklists

5c23efc

mintlify Bot deployed to staging July 9, 2026 21:02 View deployment

Remove duplicate runtime docs list

36025e4

mintlify Bot deployed to staging July 9, 2026 21:06 View deployment

alay2shah approved these changes Jul 9, 2026

View reviewed changes

alay2shah merged commit 12a7d98 into main Jul 9, 2026
8 checks passed

alay2shah deleted the add-guides-section branch July 9, 2026 21:08

mlabonne reviewed Jul 10, 2026

View reviewed changes


		Migration notes:

		- From Qwen: ChatML is familiar, but token details differ. Use `apply_chat_template`; do not reuse literal Qwen strings.


		See the [Complete Model Library](/lfm/models/complete-library) for the full catalog.

		LFM2.5 dense models support 32K context, and LFM2.5-8B-A1B supports 128K. If you rely on 128K context in a small dense Qwen or Llama model, check your production context distribution before assuming this is a blocker. Most sub-8B deployments do not exceed 8K tokens.


		What to do:

		1. Pass tools through `tokenizer.apply_chat_template(messages, tools=[...])` or the serving runtime's supported tool mechanism.

Uh oh!

Conversation

iamleonie commented Jul 9, 2026

Uh oh!

mintlify Bot commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alay2shah left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mlabonne left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

iamleonie commented Jul 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

mintlify Bot commented Jul 9, 2026 •

edited

Loading