Skip to content

Fix model-engine image vulnerabilities #854

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
scale-ballen wants to merge 2 commits into
base: main
Choose a base branch
from
+57 −34
Open

Fix model-engine image vulnerabilities #854

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
609cc5e
fix model-engine image vulnerabilities
scale-ballen Jul 14, 2026
e500bf1
Merge branch 'main' into sec/model-engine-current-cves
scale-ballen Jul 14, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions model-engine/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# syntax = docker/dockerfile:1

FROM cgr.dev/chainguard/kubectl:latest@sha256:e305bc9ffe65a62c20e3bad506a4c42e2d6d79fbe624cd8dcab13a3687f979a6 AS kubectl

@greptile-apps greptile-apps Bot Jul 14, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 kubectl digest may be single-arch, breaking arm64 builds

The original kubectl build used KUBE_BUILD_PLATFORMS=linux/${TARGETARCH} to produce the correct architecture binary. The replacement COPY --from=kubectl does not reference TARGETARCH, so its correctness depends entirely on whether this digest resolves to a multi-arch manifest list or a single-platform (amd64) manifest. If the digest is for the amd64-specific manifest rather than the OCI index, an arm64 buildx invocation will silently copy the wrong-arch binary into the image. The PR description validates only linux/amd64, but ARG TARGETARCH on line 9 and the cross-compiled aws-iam-authenticator on line 49 indicate multi-arch intent. It is worth verifying with docker manifest inspect cgr.dev/chainguard/kubectl:latest@sha256:e305bc9ffe65a62c20e3bad506a4c42e2d6d79fbe624cd8dcab13a3687f979a6 that the digest refers to a multi-arch index before arm64 builds are attempted.

Prompt To Fix With AI 
This is a comment left during a code review.
Path: model-engine/Dockerfile
Line: 3

Comment:
**kubectl digest may be single-arch, breaking arm64 builds**

The original kubectl build used `KUBE_BUILD_PLATFORMS=linux/${TARGETARCH}` to produce the correct architecture binary. The replacement `COPY --from=kubectl` does not reference `TARGETARCH`, so its correctness depends entirely on whether this digest resolves to a multi-arch manifest list or a single-platform (amd64) manifest. If the digest is for the amd64-specific manifest rather than the OCI index, an arm64 buildx invocation will silently copy the wrong-arch binary into the image. The PR description validates only `linux/amd64`, but `ARG TARGETARCH` on line 9 and the cross-compiled aws-iam-authenticator on line 49 indicate multi-arch intent. It is worth verifying with `docker manifest inspect cgr.dev/chainguard/kubectl:latest@sha256:e305bc9ffe65a62c20e3bad506a4c42e2d6d79fbe624cd8dcab13a3687f979a6` that the digest refers to a multi-arch index before arm64 builds are attempted.

How can I resolve this? If you propose a fix, please make it concise.

Fix in Cursor Fix in Claude Code Fix in Codex


FROM cgr.dev/chainguard/python:latest-dev AS builder

USER root
Expand Down Expand Up @@ -42,14 +44,12 @@ RUN mkdir -p /tmp/runtime-bin /tmp/runtime-libs && \
cp /usr/lib/libreadline.so.8* /tmp/runtime-libs/ && \
cp /usr/lib/libtinfo.so.6* /tmp/runtime-libs/ && \
cp /usr/lib/libz.so.1* /tmp/runtime-libs/ && \
git clone --depth 1 --branch v1.35.4 https://github.com/kubernetes/kubernetes.git /tmp/k8s && \
cd /tmp/k8s && \
GOTOOLCHAIN=local KUBE_BUILD_PLATFORMS=linux/${TARGETARCH} make WHAT=cmd/kubectl && \
cp _output/local/bin/linux/${TARGETARCH}/kubectl /tmp/runtime-bin/kubectl && \
git clone --depth 1 --branch v0.7.15 https://github.com/kubernetes-sigs/aws-iam-authenticator.git /tmp/aws-iam-authenticator && \
git clone --depth 1 --branch v0.7.18 https://github.com/kubernetes-sigs/aws-iam-authenticator.git /tmp/aws-iam-authenticator && \
cd /tmp/aws-iam-authenticator && \
GOOS=linux GOARCH=${TARGETARCH} go build -o /tmp/runtime-bin/aws-iam-authenticator ./cmd/aws-iam-authenticator

COPY --from=kubectl /usr/bin/kubectl /tmp/runtime-bin/kubectl

FROM cgr.dev/chainguard/python:latest AS model-engine

USER root
Expand Down
18 changes: 10 additions & 8 deletions model-engine/requirements.in
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
GitPython>=3.1.49,<4.0
Jinja2>=3.1.5 # CVE-2024-56326; 3.1.0 bug was fixed in 3.1.1+
aiohttp>=3.13.4,<4.0
aiohttp>=3.14.1,<4.0 # CVE-2026-34993, CVE-2026-47265, CVE-2026-50269, CVE-2026-54273-54280
redis>=4.6.0 # replaces aioredis~=2.0; aioredis 2.x broken on Python 3.12 (duplicate base class TimeoutError); asyncio support built-in to redis>=4.2
alembic==1.8.1
asyncpg>=0.29.0 # 0.27.0 has no cp312/cp313 wheel; Python 3.12/3.13 support added in 0.29.0
Expand Down Expand Up @@ -31,15 +31,15 @@ gevent>=26.5.0 # celery gevent worker pool for the celery-forwarder (MLI-7328);
click~=8.1
cloudpickle==2.1.0
croniter==1.4.1
cryptography>=46.0.7 # not used directly, but needs to be pinned for Microsoft security scan
cryptography>=48.0.1 # GHSA-537c-gmf6-5ccf fixed in 48.0.1
dataclasses-json>=0.5.7
datadog-api-client==2.11.0
datadog~=0.47.0
ddtrace>=4.7.1,<5.0 # 4.7.1 publishes CPython 3.14 wheels; needed for public Chainguard latest
numpy>=2.4.4,<2.5 # 2.4.4 publishes CPython 3.14 wheels
deprecation~=2.1
docker~=5.0
fastapi>=0.115.8 # bumped to allow starlette>=0.49.1 (CVE-2025-62727 fix)
fastapi>=0.138.0 # compatible with starlette>=1.3.1 security fixes
gitdb2~=2.0
gunicorn~=23.0
httptools>=0.6.0 # 0.5.0 has no cp312/cp313 wheel
Expand All @@ -54,7 +54,7 @@ py-xid==0.3.0
pycurl~=7.44 # For celery[sqs]
pytz>=2024.1
pydantic==2.12.5
python-multipart>=0.0.26
python-multipart>=0.0.30 # CVE-2026-53539 fixed in 0.0.30
quart>=0.20.0,<0.21.0
werkzeug>=3.0.6 # CVE-2024-34069, CVE-2024-49766, CVE-2024-49767
requests-auth-aws-sigv4~=0.7
Expand All @@ -66,7 +66,7 @@ smart-open~=5.2
sqlalchemy[asyncio]>=2.0.36 # 2.0.36+ required for Python 3.13 (__static_attributes__ / __firstlineno__ compat)
sse-starlette==1.6.1
sseclient-py==1.7.2
starlette[full]>=0.40.0 # not used directly, but needs to be pinned for Microsoft security scan
starlette[full]>=1.3.1 # CVE-2026-48818 and CVE-2026-54283 fixed by 1.3.1
stringcase==1.2.0
tenacity>=6.0.0,<=6.2.0
testing-postgresql==1.3.0
Expand All @@ -78,16 +78,18 @@ uvicorn==0.30.6
uvloop>=0.19.0 # 0.17.0 has no cp312/cp313 wheel; Python 3.12/3.13 support added in 0.19.0
yarl~=1.4
certifi>=2024.7.4 # CVE-2024-39689
PyJWT>=2.9.0 # CVE-2024-33663
urllib3>=2.0.7 # CVE-2023-43804, CVE-2023-45803, CVE-2024-37891
PyJWT>=2.13.0 # CVE-2024-33663 and CVE-2026-48526
urllib3>=2.7.0 # CVE-2026-44431 and CVE-2026-44432
idna>=3.15 # CVE-2026-45409
setuptools>=70.0.0 # CVE-2024-6345
wheel>=0.46.2 # CVE-2026-24049 (GHSA-8rrh-rw8j-w5fx)
h11>=0.16.0 # CVE-2025-43859 (CRITICAL)
greenlet>=3.0.0 # 2.x has no cp312/cp313 wheel; Python 3.12/3.13 support added in 3.0
frozenlist>=1.4.0 # 1.3.x has no cp312/cp313 wheel
multidict>=6.1.0 # 6.0.x has no cp312/cp313 wheel
azure-core>=1.38.0 # CVE-2026-21226 (HIGH, deserialization) fixed in 1.38.0
pyasn1>=0.6.3 # CVE-2026-30922 (HIGH, DoS) fixed in 0.6.3
starlette>=0.49.1 # CVE-2025-62727 (HIGH, DoS) fixed in 0.49.1
starlette>=1.3.1 # CVE-2025-62727, CVE-2026-48818, CVE-2026-54283
pg8000>=1.31.5 # CVE-2025-61385 (HIGH, SQL injection) fixed in 1.31.5
flask>=3.1.3,<4.0 # CVE-2026-27205
mako>=1.3.11 # GHSA-v92g-xgxw-vvmm
Expand Down
49 changes: 28 additions & 21 deletions model-engine/requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ aiofiles==25.1.0
# quart
aiohappyeyeballs==2.6.1
# via aiohttp
aiohttp==3.13.5
aiohttp==3.14.1
# via
# -r requirements.in
# aiobotocore
Expand All @@ -34,6 +34,7 @@ annotated-types==0.7.0
anyio==3.7.1
# via
# httpx
# httpx2
# starlette
asn1crypto==1.5.1
# via scramp
Expand Down Expand Up @@ -65,8 +66,6 @@ backoff==2.2.1
# via gcloud-aio-auth
billiard==4.2.4
# via celery
bleach==6.0.0
# via readme-renderer
blinker==1.9.0
# via
# flask
Expand Down Expand Up @@ -135,7 +134,7 @@ commonmark==0.9.1
# via rich
croniter==1.4.1
# via -r requirements.in
cryptography==47.0.0
cryptography==48.0.1
# via
# -r requirements.in
# azure-identity
Expand All @@ -155,11 +154,11 @@ deprecation==2.1.0
# via -r requirements.in
docker==5.0.3
# via -r requirements.in
docutils==0.20.1
docutils==0.23
# via readme-renderer
envier==0.6.1
# via ddtrace
fastapi==0.135.1
fastapi==0.139.0
# via -r requirements.in
filelock==3.29.0
# via
Expand All @@ -180,6 +179,8 @@ gcloud-aio-auth==5.4.4
# via gcloud-aio-storage
gcloud-aio-storage==9.6.1
# via -r requirements.in
gevent==26.5.0
# via -r requirements.in
gitdb==4.0.10
# via gitpython
gitdb2==2.0.6
Expand Down Expand Up @@ -225,8 +226,6 @@ googleapis-common-protos==1.72.0
# grpc-google-iam-v1
# grpcio-status
# opentelemetry-exporter-otlp-proto-grpc
gevent==26.5.0
# via -r requirements.in
greenlet==3.3.2
# via
# -r requirements.in
Expand Down Expand Up @@ -256,6 +255,7 @@ h11==0.16.0
# via
# -r requirements.in
# httpcore
# httpcore2
# hypercorn
# uvicorn
# wsproto
Expand All @@ -269,12 +269,16 @@ hpack==4.1.0
# via h2
httpcore==1.0.9
# via httpx
httpcore2==2.3.0
# via httpx2
httptools==0.7.1
# via -r requirements.in
httpx==0.27.0
# via
# huggingface-hub
# starlette
httpx2==2.3.0
# via starlette
huggingface-hub==1.13.0
# via
# tokenizers
Expand All @@ -283,10 +287,12 @@ hypercorn==0.14.4
# via quart
hyperframe==6.1.0
# via h2
idna==3.7
idna==3.18
# via
# -r requirements.in
# anyio
# httpx
# httpx2
# requests
# yarl
importlib-metadata==6.8.0
Expand Down Expand Up @@ -329,8 +335,6 @@ kubernetes==25.3.0
# via -r requirements.in
kubernetes-asyncio==25.11.0
# via -r requirements.in
legacy-cgi==2.6.4
# via ddtrace
mako==1.3.12
# via
# -r requirements.in
Expand Down Expand Up @@ -379,6 +383,8 @@ mypy-boto3-sqs==1.40.61
# via boto3-stubs
mypy-extensions==1.0.0
# via typing-inspect
nh3==0.3.6
# via readme-renderer
numpy==2.4.4
# via
# -r requirements.in
Expand Down Expand Up @@ -478,7 +484,7 @@ pygments==2.20.0
# -r requirements.in
# readme-renderer
# rich
pyjwt==2.12.1
pyjwt==2.13.0
# via
# -r requirements.in
# gcloud-aio-auth
Expand All @@ -495,7 +501,7 @@ python-dateutil==2.8.2
# kubernetes
# kubernetes-asyncio
# pg8000
python-multipart==0.0.27
python-multipart==0.0.32
# via
# -r requirements.in
# starlette
Expand All @@ -511,7 +517,7 @@ pyyaml==6.0.1
# transformers
quart==0.20.0
# via -r requirements.in
readme-renderer==40.0
readme-renderer==45.0
# via twine
redis==4.6.0
# via
Expand Down Expand Up @@ -562,15 +568,12 @@ setuptools==82.0.1
# -r requirements.in
# kubernetes
# kubernetes-asyncio
wheel==0.46.2
# via -r requirements.in # CVE-2026-24049 (GHSA-8rrh-rw8j-w5fx); patched in 0.46.2
sh==1.14.3
# via -r requirements.in
shellingham==1.5.4
# via typer
six==1.16.0
# via
# bleach
# isodate
# kubernetes
# kubernetes-asyncio
Expand All @@ -596,7 +599,7 @@ sse-starlette==1.6.1
# via -r requirements.in
sseclient-py==1.7.2
# via -r requirements.in
starlette==0.52.1
starlette==1.3.1
# via
# -r requirements.in
# fastapi
Expand All @@ -623,6 +626,10 @@ tqdm==4.67.3
# twine
transformers==5.7.0
# via -r requirements.in
truststore==0.10.4
# via
# httpcore2
# httpx2
twine==3.7.1
# via -r requirements.in
typer==0.23.1
Expand Down Expand Up @@ -663,7 +670,7 @@ typing-inspection==0.4.2
# pydantic
tzdata==2025.3
# via kombu
urllib3==2.6.3
urllib3==2.7.0
# via
# -r requirements.in
# botocore
Expand All @@ -684,8 +691,6 @@ vine==5.1.0
# kombu
wcwidth==0.2.6
# via prompt-toolkit
webencodings==0.5.1
# via bleach
websocket-client==1.6.1
# via
# docker
Expand All @@ -695,6 +700,8 @@ werkzeug==3.1.6
# -r requirements.in
# flask
# quart
wheel==0.47.0
# via -r requirements.in
wrapt==1.17.3
# via
# aiobotocore
Expand Down
14 changes: 14 additions & 0 deletions model-engine/tests/unit/inference/test_http_forwarder.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@

import pytest
import requests_mock
from aiohttp import ClientResponse
from aioresponses import aioresponses
from fastapi import BackgroundTasks, FastAPI
from fastapi.responses import JSONResponse
Expand All @@ -32,6 +33,19 @@
PAYLOAD: Mapping[str, str] = {"hello": "world"}


@pytest.fixture(autouse=True)
def aioresponses_aiohttp_314_compat(monkeypatch):
"""Allow aioresponses 0.7.9 to construct aiohttp 3.14 responses."""
original_init = ClientResponse.__init__

def compatible_init(self, *args, stream_writer=None, **kwargs):
if stream_writer is None:
stream_writer = mock.Mock(output_size=0)
original_init(self, *args, stream_writer=stream_writer, **kwargs)

monkeypatch.setattr(ClientResponse, "__init__", compatible_init)


class ExceptionCapturedThread(threading.Thread):
def __init__(self, target, args):
super().__init__(target=target, args=args)
Expand Down