Skip to content

Cache Ollama manifests in macOS bootstrap #3

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
mrjf wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Cache Ollama manifests in macOS bootstrap #3

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions infra/macos/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -113,10 +113,19 @@ SET_GITHUB_VARIABLES=0 infra/macos/setup-local-qwen-ollama.sh
RUN_SMOKE=0 infra/macos/setup-local-qwen-ollama.sh
RESTART_OLLAMA=0 infra/macos/setup-local-qwen-ollama.sh
PULL_MODEL=0 infra/macos/setup-local-qwen-ollama.sh
CACHE_OLLAMA_MANIFEST=0 infra/macos/setup-local-qwen-ollama.sh
INSTALL_HOMEBREW=1 infra/macos/setup-local-qwen-ollama.sh
INSTALL_TAILSCALE=1 infra/macos/setup-local-qwen-ollama.sh
```

The helper caches the Ollama model manifest by default under:

```text
${XDG_CACHE_HOME:-$HOME/.cache}/gh-aw/ollama-manifests
```

That cache is intentionally metadata-only. The large model blobs stay in Ollama's own model directory, usually `$HOME/.ollama/models/blobs`. A restored manifest can help Ollama recognize existing blobs, but the helper still verifies the model with `ollama show` before skipping a pull.

If a nearby Linux `gh-aw` runner needs to call Ollama over the network, expose Ollama deliberately:

```bash
Expand Down
87 changes: 86 additions & 1 deletion infra/macos/setup-local-qwen-ollama.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,82 @@ ollama_port_from_bind() {
fi
}

ollama_models_dir() {
printf '%s\n' "${OLLAMA_MODELS:-${HOME}/.ollama/models}"
}

ollama_manifest_path() {
local model_ref="$1"
local ref_without_tag
local tag
local first_component
local manifest_root

if [[ "${model_ref##*/}" == *":"* ]]; then
ref_without_tag="${model_ref%:*}"
tag="${model_ref##*:}"
else
ref_without_tag="$model_ref"
tag="latest"
fi

first_component="${ref_without_tag%%/*}"
manifest_root="$(ollama_models_dir)/manifests"

if [[ "$ref_without_tag" != */* ]]; then
printf '%s/registry.ollama.ai/library/%s/%s\n' "$manifest_root" "$ref_without_tag" "$tag"
elif [[ "$first_component" == *.* || "$first_component" == *:* || "$first_component" == "localhost" ]]; then
printf '%s/%s/%s\n' "$manifest_root" "$ref_without_tag" "$tag"
else
printf '%s/registry.ollama.ai/%s/%s\n' "$manifest_root" "$ref_without_tag" "$tag"
fi
}

cache_manifest_path() {
local manifest_path="$1"
local models_dir
local relative_path

models_dir="$(ollama_models_dir)"
relative_path="${manifest_path#"${models_dir}/manifests/"}"
printf '%s/manifests/%s\n' "$OLLAMA_MANIFEST_CACHE_DIR" "$relative_path"
}

restore_ollama_manifest_cache() {
local manifest_path="$1"
local cached_path

if [[ "$CACHE_OLLAMA_MANIFEST" != "1" ]]; then
return 0
fi

cached_path="$(cache_manifest_path "$manifest_path")"

if [[ -s "$manifest_path" ]]; then
say "ok: Ollama manifest already exists: ${manifest_path}"
elif [[ -s "$cached_path" ]]; then
mkdir -p "$(dirname "$manifest_path")"
cp "$cached_path" "$manifest_path"
say "Restored cached Ollama manifest: ${cached_path}"
else
say "No cached Ollama manifest found yet: ${cached_path}"
fi
}

save_ollama_manifest_cache() {
local manifest_path="$1"
local cached_path

if [[ "$CACHE_OLLAMA_MANIFEST" != "1" || ! -s "$manifest_path" ]]; then
return 0
fi

cached_path="$(cache_manifest_path "$manifest_path")"
mkdir -p "$(dirname "$cached_path")"
cp "$manifest_path" "$cached_path"
say "Cached Ollama manifest: ${cached_path}"
}

ollama_ready() {
curl -fsS --connect-timeout 2 --max-time 5 "${OLLAMA_HEALTH_URL}/api/version" >/dev/null 2>&1
}
Expand Down Expand Up @@ -188,7 +264,7 @@ start_ollama() {
}

model_installed() {
ollama list | awk 'NR > 1 {print $1}' | grep -Fx -- "$QWEN_OLLAMA_MODEL" >/dev/null 2>&1
ollama show "$QWEN_OLLAMA_MODEL" >/dev/null 2>&1
}

configure_ollama_host() {
Expand Down Expand Up @@ -250,6 +326,7 @@ fi
: "${INSTALL_RUNNER_SERVICE:=1}"
: "${RESTART_OLLAMA:=1}"
: "${PULL_MODEL:=1}"
: "${CACHE_OLLAMA_MANIFEST:=1}"
: "${RUN_SMOKE:=1}"
: "${LOCAL_OPENAI_API_KEY:=ollama}"

Expand All @@ -264,16 +341,20 @@ fi
ollama_port="$(ollama_port_from_bind "$OLLAMA_HOST_BIND")"
: "${OLLAMA_HEALTH_URL:=http://127.0.0.1:${ollama_port}}"
: "${LOCAL_OPENAI_BASE_URL:=http://127.0.0.1:${ollama_port}/v1}"
: "${OLLAMA_MANIFEST_CACHE_DIR:=${XDG_CACHE_HOME:-${HOME}/.cache}/gh-aw/ollama-manifests}"
: "${RUNNER_LABELS:=macos-local,local-model,${QWEN_RUNNER_LABEL}}"

if [[ -z "${RUNNER_NAME:-}" ]]; then
RUNNER_NAME="$(default_host_name)-${QWEN_RUNNER_LABEL}"
fi

qwen_manifest_path="$(ollama_manifest_path "$QWEN_OLLAMA_MODEL")"

say "== macOS Qwen/Ollama setup =="
say "model: ${QWEN_OLLAMA_MODEL}"
say "local OpenAI-compatible URL: ${LOCAL_OPENAI_BASE_URL}"
say "Ollama bind: ${OLLAMA_HOST_BIND}"
say "Ollama manifest: ${qwen_manifest_path}"
say "register runner: ${REGISTER_MACOS_RUNNER}"
say "runner name: ${RUNNER_NAME}"
say
Expand All @@ -292,14 +373,18 @@ fi

configure_ollama_host
start_ollama
restore_ollama_manifest_cache "$qwen_manifest_path"

if model_installed; then
say "ok: Qwen model is already installed: ${QWEN_OLLAMA_MODEL}"
save_ollama_manifest_cache "$qwen_manifest_path"
elif [[ "$PULL_MODEL" == "1" ]]; then
say "Pulling Qwen model: ${QWEN_OLLAMA_MODEL}"
run ollama pull "$QWEN_OLLAMA_MODEL"
save_ollama_manifest_cache "$qwen_manifest_path"
else
warn "Qwen model is not installed and PULL_MODEL=0, so the model pull was skipped: ${QWEN_OLLAMA_MODEL}"
save_ollama_manifest_cache "$qwen_manifest_path"
fi

if [[ "$RUN_SMOKE" == "1" ]]; then
Expand Down