Skip to content

Commit 71370f7

Browse files
ObadaSObada Haddad
ObadaS
and
Obada Haddad
authored
Compute Worker Use docker-py instead of the current subprocess way of doing things (for podman and docker) (#2065)
* Using docker-py and podman-py instead of the current subprocess way of doing things * Removed useless line * Started work on Podman; re-added log stream to codabench instance * Corrected typo in variable name; small changes for image pulls for podman * remove the podman package and use only docker API calls for both podman and docker containers * add better progress bar when downloading and some debug logger output * add github workflow * workflow fixes * format with ruff; add cleanup * format with ruff; fix websockets connection infinite waiting * change Dockerfile.compute_worker to remove docker; make GPU selection compatible with podman and docker * fix a typo * ruff formatting; update documentation * delete useless workflow * update documentation, remove useless files, add docker in pyproject for pytests * try to fix pytest errors * fix typo * remove unused folder for podman * add more logs for the compute worker; update documentation * fix poetry lock after rebase * rebase + remove failing tests. We can test this in e2e tests * fix flake8 errors * fix poetry lock after rebase * remove poetry at the end to fix vulnerabilities and make image lighter * remove docker and rich from main pyproject.toml * fix missing f * add timeout for websocket connections * add more information about which container engine we are using and if GPU will be used * tentative fix for send_detailed_results websocket connection attempt * make error reporting better; handle websocket conenctions failures better * fix hostname inside celery worker to use container hostname * fix compute worker startup in compute worker service since we use entrypoint in Dockerfile.compute worker * add workflow to build and push compute worker image automatically on file changes for every branch * change Dockerfile.compute_worker and test workflow * fix missing checkout in new workflows * fix missing needs: to make the push job wait for the build to finish first * further fixes in workflow * rebase branch and update documentation about the new docker image workflow * update documentation to be more clear * fix typo * update compute worker image name * raise timeout to make it less likely to fail when instance is slow to load * fix compute worker name in github workflow --------- Co-authored-by: Obada Haddad <obada.haddad@lisn.fr>
1 parent 6f0af0b commit 71370f7

20 files changed

Lines changed: 865 additions & 545 deletions

‎.github/workflows/build_cw_image-branch.yml‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
name: build_CW_docker_image_branch
2+
on:
3+
push:
4+
# On push for every branch except develop and master since they have their own workflows
5+
branches:
6+
- '*'
7+
- '!develop'
8+
- '!master'
9+
paths:
10+
- Dockerfile.compute_worker
11+
- compute_worker/**
12+
jobs:
13+
build_push_image:
14+
name: Build Docker Image then Push it to Docker.io
15+
runs-on: ubuntu-latest
16+
steps:
17+
- name: Check out repository code
18+
uses: actions/checkout@v5
19+
- name: Build Image
20+
run: docker build -t codalab/codabench-compute-worker:${{ github.ref_name }} -f Dockerfile.compute_worker .
21+
- name: Login to Docker Hub
22+
uses: docker/login-action@v3
23+
with:
24+
username: ${{ vars.DOCKERHUB_USERNAME }}
25+
password: ${{ secrets.DOCKERHUB_TOKEN }}
26+
- name: Push image with branch name as tag
27+
run: docker push codalab/codabench-compute-worker:${{ github.ref_name }}

‎.github/workflows/build_cw_image-dev.yml‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
name: build_CW_docker_image_develop
2+
on:
3+
push:
4+
branches:
5+
- develop
6+
paths:
7+
- Dockerfile.compute_worker
8+
- compute_worker/**
9+
jobs:
10+
build:
11+
name: Build Docker Images
12+
runs-on: ubuntu-latest
13+
steps:
14+
- name: Check out repository code
15+
uses: actions/checkout@v5
16+
- name: Build Image
17+
run: docker build -t codalab/codabench-compute-worker:test -f Dockerfile.compute_worker .
18+
- name: Login to Docker Hub
19+
uses: docker/login-action@v3
20+
with:
21+
username: ${{ vars.DOCKERHUB_USERNAME }}
22+
password: ${{ secrets.DOCKERHUB_TOKEN }}
23+
- name: Push image with test tag
24+
run: docker push codalab/codabench-compute-worker:test

‎.github/workflows/build_cw_image-prod.yml‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
name: build_CW_docker_image_branch
2+
on:
3+
push:
4+
tags:
5+
- '*'
6+
paths:
7+
- Dockerfile.compute_worker
8+
- compute_worker/**
9+
jobs:
10+
build:
11+
name: Build Docker Images
12+
runs-on: ubuntu-latest
13+
steps:
14+
- name: Check out repository code
15+
uses: actions/checkout@v5
16+
- name: Build Image
17+
run: docker build -t codalab/codabench-compute-worker:${{ github.ref_name }} -f Dockerfile.compute_worker .
18+
- name: Login to Docker Hub
19+
uses: docker/login-action@v3
20+
with:
21+
username: ${{ vars.DOCKERHUB_USERNAME }}
22+
password: ${{ secrets.DOCKERHUB_TOKEN }}
23+
- name: Push image with prod release tag
24+
run: docker push codalab/codabench-compute-worker:${{ github.ref_name }}
25+
- name: Change tag to latest
26+
run: |
27+
docker tag codalab/codabench-compute-worker:${{ github.ref_name }} codalab/codabench-compute-worker:latest
28+
docker push codalab/codabench-compute-worker:latest

‎Containerfile.compute_worker_podman‎

Lines changed: 0 additions & 62 deletions
This file was deleted.

‎Containerfile.compute_worker_podman_gpu‎

Lines changed: 0 additions & 8 deletions
This file was deleted.

‎Dockerfile.compute_worker‎

Lines changed: 13 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,34 +1,34 @@
1-
FROM --platform=linux/amd64 fedora:42
1+
FROM --platform=linux/amd64 fedora:43
22

33
# This makes output not buffer and return immediately, nice for seeing results in stdout
4-
ENV PYTHONUNBUFFERED 1
4+
ENV PYTHONUNBUFFERED=1
55

6-
# Install Docker
7-
RUN dnf -y install dnf-plugins-core && \
8-
dnf-3 config-manager --add-repo https://download.docker.com/linux/fedora/docker-ce.repo && \
9-
dnf -y update && \
10-
dnf install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin && \
6+
# Install Python
7+
RUN dnf -y update && \
118
dnf install -y python3.9 && \
129
dnf clean all && \
1310
rm -rf /var/cache /var/log/dnf* /var/log/yum.*
1411

1512

1613
RUN curl -sSL https://install.python-poetry.org | python3.9 - --version 1.8.3
1714
# Poetry location so future commands (below) work
18-
ENV PATH $PATH:/root/.local/bin
15+
ENV PATH=$PATH:/root/.local/bin
1916
# Want poetry to use system python of docker container
2017
RUN poetry config virtualenvs.create false
2118
RUN poetry config virtualenvs.in-project false
19+
20+
2221
COPY ./compute_worker/pyproject.toml ./
2322
COPY ./compute_worker/poetry.lock ./
2423
# To use python3.9 instead of system python
24+
2525
RUN poetry config virtualenvs.prefer-active-python true && poetry install
2626

2727
ADD compute_worker .
2828
COPY ./src/settings/logs_loguru.py /usr/bin
2929

30-
CMD celery -A compute_worker worker \
31-
-l info \
32-
-Q compute-worker \
33-
-n compute-worker@%n \
34-
--concurrency=1
30+
# Uninstall Poetry since we don't need it anymore and it can introduce CVEs
31+
RUN curl -sSL https://install.python-poetry.org | python3.9 - --uninstall
32+
33+
ENTRYPOINT ["/bin/bash", "-c"]
34+
CMD ["celery -A compute_worker worker -l info -Q compute-worker -n compute-worker@$HOSTNAME --concurrency=1"]

‎Dockerfile.compute_worker_gpu‎

Lines changed: 0 additions & 12 deletions
This file was deleted.

0 commit comments

Comments
 (0)