fix: block tar path traversal in decompress_to_cache

[serhiizghama]

Fixes #626. decompress_to_cache called tarfile.extractall() with no member sanitization, so a tar archive with a ../ member could write files anywhere the process can write — outside cache_dir. This is reachable through add_custom_model() pointing at an attacker-controlled URL, or a compromised HF/GCS source.

Switched extraction to the data filter (PEP 706), which rejects members that escape the destination. For interpreters that predate the filter argument, there's a small fallback that resolves each member path and refuses anything landing outside cache_dir. A malicious member now raises instead of extracting; normal nested paths extract as before.

Added a test that a ../ member is blocked and never written, plus one that a well-formed archive still extracts. On unpatched code the traversal test fails (the file lands outside the cache). Pinned ruff and ruff-format are clean.

[coderabbitai]

📝 Walkthrough

Walkthrough

decompress_to_cache now validates tar member paths before extraction. It uses the data filter when supported and falls back to manual validation on older runtimes. Tests cover rejection of parent-directory traversal and successful extraction of safe nested files.

Estimated code review effort: 2 (Simple) | ~10 minutes

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly identifies the main change: blocking tar path traversal in decompress_to_cache.
Description check	✅ Passed	The description explains the vulnerability, mitigation, compatibility fallback, and tests related to the changeset.
Linked Issues check	✅ Passed	The changes address #626 by blocking unsafe tar paths, supporting older Python versions, and preserving valid extraction.
Out of Scope Changes check	✅ Passed	The changes are limited to tar extraction safety and focused tests required by #626.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@fastembed/common/model_management.py`:
- Around line 317-322: Update the extraction failure cleanup in
decompress_to_cache so a tarfile.FilterError or other extraction failure cannot
delete a caller-owned cache_dir. Remove the path-based deletion or restrict
cleanup to directories explicitly created and owned by this method, preserving
safe extraction and fallback behavior around tar.extractall and
_safe_extractall.
- Around line 284-292: Update _safe_extractall in
fastembed/common/model_management.py to reject symbolic links, hard links, and
other special tar members before any extraction, while preserving the existing
path-traversal validation for regular members. In tests/test_common.py lines
40-53, force execution through the legacy fallback, add coverage for a
link-based archive targeting outside the cache, and assert that the external
target is not created.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 74fbe4a9-d866-46bb-b09e-69b9e25ae4bc

📥 Commits

Reviewing files that changed from the base of the PR and between f613647 and 6fdc066.

📒 Files selected for processing (2)

• fastembed/common/model_management.py
• tests/test_common.py

[coderabbitai]

🔒 Security & Privacy | 🔴 Critical | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -eu

printf '%s\n' '--- target implementation ---'
sed -n '240,320p' fastembed/common/model_management.py

printf '%s\n' '--- related extraction and test references ---'
rg -n -C 4 'safe_extractall|extractall|decompress_to_cache|tarfile|data_filter' \
  fastembed/common/model_management.py tests/test_common.py

printf '%s\n' '--- runtime versions ---'
python3 - <<'PY'
import sys, tarfile
print(sys.version)
print("tarfile has extraction_filter:", hasattr(tarfile.TarFile, "extraction_filter"))
print("tarfile has data_filter:", hasattr(tarfile, "data_filter"))
PY

Repository: qdrant/fastembed

Length of output: 10053

---

🏁 Script executed:

#!/bin/bash
set -eu

sed -n '240,320p' fastembed/common/model_management.py
rg -n -C 4 'safe_extractall|extractall|decompress_to_cache|tarfile|data_filter' \
  fastembed/common/model_management.py tests/test_common.py

python3 - <<'PY'
import sys, tarfile
print(sys.version)
print("extraction_filter:", hasattr(tarfile.TarFile, "extraction_filter"))
print("data_filter:", hasattr(tarfile, "data_filter"))
PY

Repository: qdrant/fastembed

Length of output: 9927

---

🏁 Script executed:

#!/bin/bash
set -eu

python3 - <<'PY'
import io
import os
import shutil
import tarfile
import tempfile
from pathlib import Path

def make_archive(path, members):
    with tarfile.open(path, "w:gz") as tar:
        for member, payload in members:
            tar.addfile(member, io.BytesIO(payload) if payload is not None else None)

def legacy_safe_extract(tar, cache_dir):
    base = os.path.realpath(cache_dir)
    for member in tar.getmembers():
        target = os.path.realpath(os.path.join(cache_dir, member.name))
        if os.path.commonpath([base, target]) != base:
            raise ValueError(member.name)
    tar.extractall(path=cache_dir)

with tempfile.TemporaryDirectory() as td:
    root = Path(td)
    cache = root / "cache"
    cache.mkdir()
    outside = root / "outside"
    outside.mkdir()

    link = tarfile.TarInfo("link")
    link.type = tarfile.SYMTYPE
    link.linkname = str(outside)

    child = tarfile.TarInfo("link/payload")
    child.size = 5

    archive = root / "links.tar.gz"
    make_archive(archive, [(link, None), (child, b"PWNED")])

    with tarfile.open(archive, "r:gz") as tar:
        try:
            legacy_safe_extract(tar, str(cache))
            result = "completed"
        except Exception as exc:
            result = f"{type(exc).__name__}: {exc}"

    print("legacy_result:", result)
    print("link_is_symlink:", (cache / "link").is_symlink())
    print("external_payload_exists:", (outside / "payload").exists())
    if (outside / "payload").exists():
        print("external_payload:", (outside / "payload").read_bytes())

    filtered_cache = root / "filtered-cache"
    filtered_cache.mkdir()
    with tarfile.open(archive, "r:gz") as tar:
        try:
            tar.extractall(path=filtered_cache, filter="data")
            filtered_result = "completed"
        except Exception as exc:
            filtered_result = f"{type(exc).__name__}: {exc}"
    print("data_filter_result:", filtered_result)
    print("filtered_external_payload_exists:", (outside / "payload").exists())

print("extractall_signature:", __import__("inspect").signature(tarfile.TarFile.extractall))
PY

Repository: qdrant/fastembed

Length of output: 463

---

🏁 Script executed:

#!/bin/bash
set -eu

python3 - <<'PY'
import io, os, tarfile, tempfile
from pathlib import Path

def make_archive(path, members):
    with tarfile.open(path, "w:gz") as tar:
        for member, payload in members:
            tar.addfile(member, io.BytesIO(payload) if payload is not None else None)

def legacy_safe_extract(tar, cache_dir):
    base = os.path.realpath(cache_dir)
    for member in tar.getmembers():
        target = os.path.realpath(os.path.join(cache_dir, member.name))
        if os.path.commonpath([base, target]) != base:
            raise ValueError(member.name)
    tar.extractall(path=cache_dir)

with tempfile.TemporaryDirectory() as td:
    root = Path(td)
    cache = root / "cache"
    cache.mkdir()
    outside = root / "outside"
    outside.mkdir()

    link = tarfile.TarInfo("link")
    link.type = tarfile.SYMTYPE
    link.linkname = str(outside)

    child = tarfile.TarInfo("link/payload")
    child.size = 5

    archive = root / "links.tar.gz"
    make_archive(archive, [(link, None), (child, b"PWNED")])

    with tarfile.open(archive, "r:gz") as tar:
        try:
            legacy_safe_extract(tar, str(cache))
            result = "completed"
        except Exception as exc:
            result = f"{type(exc).__name__}: {exc}"

    print("legacy_result:", result)
    print("link_is_symlink:", (cache / "link").is_symlink())
    print("external_payload_exists:", (outside / "payload").exists())
    if (outside / "payload").exists():
        print("external_payload:", (outside / "payload").read_bytes())

    filtered_cache = root / "filtered-cache"
    filtered_cache.mkdir()
    with tarfile.open(archive, "r:gz") as tar:
        try:
            tar.extractall(path=filtered_cache, filter="data")
            filtered_result = "completed"
        except Exception as exc:
            filtered_result = f"{type(exc).__name__}: {exc}"
    print("data_filter_result:", filtered_result)
    print("filtered_external_payload_exists:", (outside / "payload").exists())

print("extractall_signature:", __import__("inspect").signature(tarfile.TarFile.extractall))
PY

Repository: qdrant/fastembed

Length of output: 463

---

Reject link members in the legacy tar fallback.

When filter="data" is unavailable, _safe_extractall checks only member.name. A tar can create a symlink inside cache_dir, then extract link/payload outside cache_dir. Reject symbolic links, hard links, and special members before extraction. Force this fallback in tests/test_common.py and assert that the external target remains absent.

🧰 Tools

🪛 ast-grep (0.45.1)

[error] 291-291: Calling extractall() on a zipfile.ZipFile or tarfile archive without validating member paths lets a crafted entry (e.g. "../../etc/passwd") write outside the destination directory (Zip Slip). Validate each member resolves inside the target directory, or pass a safe filter (tarfile: filter="data" / tarfile.data_filter).
Context: tar.extractall(path=cache_dir)
Note: [CWE-22] Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal').

(archive-extractall-path-traversal-python)

🪛 Ruff (0.16.1)

[error] 292-292: Uses of tarfile.extractall()

(S202)

📍 Affects 2 files

• fastembed/common/model_management.py#L284-L292 (this comment)
• tests/test_common.py#L40-L53

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@fastembed/common/model_management.py` around lines 284 - 292, Update
_safe_extractall in fastembed/common/model_management.py to reject symbolic
links, hard links, and other special tar members before any extraction, while
preserving the existing path-traversal validation for regular members. In
tests/test_common.py lines 40-53, force execution through the legacy fallback,
add coverage for a link-based archive targeting outside the cache, and assert
that the external target is not created.

[coderabbitai]

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -e
printf '%s\n' '--- candidate files ---'
fd -i 'model_management' .
printf '%s\n' '--- relevant symbols and call sites ---'
rg -n -C 4 'def (decompress_to_cache|_safe_extractall)|decompress_to_cache|_safe_extractall|except tarfile\.TarError|rmtree\(cache_dir\)|"tmp" in cache_dir' .
printf '%s\n' '--- file outline ---'
ast-grep outline fastembed/common/model_management.py
printf '%s\n' '--- target implementation ---'
sed -n '240,350p' fastembed/common/model_management.py

Repository: qdrant/fastembed

Length of output: 10425

---

🏁 Script executed:

#!/bin/bash
set -e
python3 - <<'PY'
import io
import os
import shutil
import sys
import tarfile
import tempfile
from pathlib import Path

print("python:", sys.version.split()[0])
print("FilterError MRO:", [c.__name__ for c in tarfile.FilterError.__mro__])
print("FilterError is TarError:", issubclass(tarfile.FilterError, tarfile.TarError))

with tempfile.TemporaryDirectory(prefix="tar-review-") as root:
    root = Path(root)
    archive = root / "evil.tar.gz"
    cache = root / "existing-tmp-cache"
    cache.mkdir()
    sentinel = cache / "caller-owned.txt"
    sentinel.write_text("keep")

    with tarfile.open(archive, "w:gz") as tar:
        info = tarfile.TarInfo("../escaped.txt")
        payload = b"PWNED"
        info.size = len(payload)
        tar.addfile(info, io.BytesIO(payload))

    caught = None
    try:
        with tarfile.open(archive, "r:gz") as tar:
            tar.extractall(path=cache, filter="data")
    except tarfile.TarError as exc:
        caught = exc
        if "tmp" in str(cache):
            shutil.rmtree(cache)

    print("caught_type:", type(caught).__name__ if caught else None)
    print("caught_message:", str(caught) if caught else None)
    print("cache_exists_after_cleanup:", cache.exists())
    print("caller_file_exists_after_cleanup:", sentinel.exists())
PY

Repository: qdrant/fastembed

Length of output: 523

---

Do not delete caller-owned cache directories on extraction failures.

tarfile.FilterError is a tarfile.TarError. A rejected member therefore reaches the cleanup block, which removes any cache_dir whose path contains "tmp", including directories not created by decompress_to_cache. Remove this cleanup or track explicit ownership.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@fastembed/common/model_management.py` around lines 317 - 322, Update the
extraction failure cleanup in decompress_to_cache so a tarfile.FilterError or
other extraction failure cannot delete a caller-owned cache_dir. Remove the
path-based deletion or restrict cleanup to directories explicitly created and
owned by this method, preserving safe extraction and fallback behavior around
tar.extractall and _safe_extractall.