Skip to content

Add trainerv2 llm fine tuning #156

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
typhoonzero wants to merge 4 commits into
base: master
Choose a base branch
from
Open

Add trainerv2 llm fine tuning #156

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
c1e81ea
add trainerv2 llm fine tuning, WIP
typhoonzero Mar 17, 2026
f3eab95
update for distributed training
typhoonzero Mar 20, 2026
ec4fa2b
update link
typhoonzero Mar 21, 2026
6f872b6
support Kueue
typhoonzero Mar 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions docs/en/kubeflow/how_to/assets/build-train-image/fine_tune_with_llamafactory.Containerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
FROM nvidia/cuda:12.6.3-cudnn-devel-ubuntu24.04

RUN sed -i 's@//.*archive.ubuntu.com@//mirrors.ustc.edu.cn@g' /etc/apt/sources.list.d/ubuntu.sources && \
sed -i 's/security.ubuntu.com/mirrors.ustc.edu.cn/g' /etc/apt/sources.list.d/ubuntu.sources && \
apt-get update && \
apt-get install -yq software-properties-common && \
add-apt-repository ppa:deadsnakes/ppa && \
apt-get update && \
export DEBIAN_FRONTEND=noninteractive && \
apt-get install -yq --no-install-recommends \
python3.12 python3.12-venv python3.12-dev git git-lfs unzip curl ffmpeg default-libmysqlclient-dev build-essential pkg-config && \
apt clean && rm -rf /var/lib/apt/lists/*

RUN update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.12 1 && \
update-alternatives --install /usr/bin/python python /usr/bin/python3.12 1 && \
curl -LsSf https://astral.sh/uv/install.sh | UV_INSTALL_DIR=/usr/local/bin sh

ENV UV_SYSTEM_PYTHON=1

WORKDIR /opt
COPY pyproject.toml uv.lock ./
RUN HTTPS_PROXY=http://192.168.144.12:7890 uv sync --frozen
Copy link

@coderabbitai coderabbitai bot Mar 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Hardcoded internal proxy IP will cause build failures for external users.

The proxy 192.168.144.12:7890 is an internal resource that won't be accessible to users building this image. Consider parameterizing it or removing it entirely.

🔧 Suggested fix using build argument 
+ARG HTTPS_PROXY=""
 WORKDIR /opt
 COPY pyproject.toml uv.lock ./
-RUN HTTPS_PROXY=http://192.168.144.12:7890 uv sync --frozen
+RUN uv sync --frozen

If a proxy is required in certain environments, users can pass --build-arg HTTPS_PROXY=... when building.

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In
`@docs/en/kubeflow/how_to/assets/build-train-image/fine_tune_with_llamafactory.Containerfile`
at line 22, The Dockerfile contains a hardcoded proxy in the RUN line ("RUN
HTTPS_PROXY=http://192.168.144.12:7890 uv sync --frozen") which will break
external builds; change this to use a build-time argument or omit the proxy: add
an ARG HTTPS_PROXY (and optionally ENV HTTPS_PROXY) and replace the inline value
so the RUN uses the ARG (or remove the HTTPS_PROXY prefix entirely if not
required), and document that users can pass --build-arg HTTPS_PROXY=<value> when
building.


ENV PATH=$PATH:/opt/.venv/bin
24 changes: 24 additions & 0 deletions docs/en/kubeflow/how_to/assets/build-train-image/pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
[project]
name = "training-env"
version = "0.1.0"
requires-python = ">=3.11"
dependencies = [
"llamafactory[metrics,awq,modelscope]",
"torch==2.6.0",
"torchvision==0.21.0",
"torchaudio==2.6.0",
"transformers<=4.51.1",
"tokenizers>=0.21.1",
"sqlalchemy~=2.0.30",
"pymysql~=1.1.1",
"loguru~=0.7.2",
"mysqlclient~=2.2.7",
"deepspeed~=0.16.9",
"mlflow>=3.1",
]

[tool.uv]
index-url = "https://pypi.tuna.tsinghua.edu.cn/simple"

[tool.uv.sources]
llamafactory = { git = "https://github.com/hiyouga/LLaMA-Factory.git", tag = "v0.9.4" }
Loading