Skip to content

Commit 395bc5e

Browse files
parmentelatparmentelat
committed
add README and service file
1 parent 967f46d commit 395bc5e

2 files changed

Lines changed: 246 additions & 0 deletions

File tree

‎.github/docker/README-setup-runner.md‎

Lines changed: 227 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,227 @@
1+
# self-hosted runner setup
2+
3+
## dockerhub
4+
5+
our image is at
6+
7+
`dockerhub.pl.sophia.inria.fr/python-nodejs:latest`
8+
9+
### publish the image
10+
11+
```
12+
# as boxtops@root
13+
14+
cd
15+
# if needed
16+
git clone https://github.com/flotpython/jupyterlab-examples
17+
18+
cd ~/jupyterlab-examples/.github/docker
19+
20+
# will build & push the image to dockerhub.pl.sophia.inria.fr
21+
# the docker login as admin has been made once and for good as the root user there
22+
./rebuild.sh
23+
24+
# check available tags
25+
curl -u admin:$password https://dockerhub.pl.sophia.inria.fr/v2/_catalog
26+
curl -u admin:$password https://dockerhub.pl.sophia.inria.fr/v2/python-nodejs/tags/list
27+
```
28+
29+
## general comments
30+
31+
### right in the host context
32+
33+
- the self-hosted runner is required to run as a regular user (not root), this
34+
is enforced by the GH runner code
35+
- as this needs a minimal amount of isolation between runs, we've tried using
36+
`conda` or `pixi` to provide that isolation; however, the bottom line is
37+
**it's not a good idea**
38+
- running right inside the host context is **doable**, and in a first stage
39+
we've used that for the `frontend` course on `bigjack`
40+
- however this is both **fragile** (not sure how several concurrent runs would
41+
behave) and also - very - **slow**
42+
43+
### in a container
44+
45+
- for that reason we have explored containerization options
46+
- ***BUT*** using **podman** turns out to be **overcomplex**, due to the need to
47+
run the runner as a non-root user - don't do that !
48+
- for that reason we have switched to using **regular docker**
49+
- btw: that is a bit unfortunate because we had `thermals` that could have done the
50+
job pretty well, but well, c'est la vie...
51+
- using `boxtops` then instead
52+
- also, it is important that the container itself must not run as root either,
53+
because `myst --execute` will then have its `jupyter-server` fail silently in
54+
that case (the logs would report that the `--allow-root` flag is missing)
55+
56+
### need to use a publicly available image
57+
58+
- the GH action that sets up the container insists on doing a `docker pull`
59+
first, there does not seem to be a way to skip that
60+
- hence the need to push the image to a publicly available docker registry
61+
- we have setup dockerhub on `dockerhub.pl.sophia.inria.fr` for that purpose
62+
it is requires login for publication but is configured to allow anonymous
63+
pulls
64+
65+
### the image
66+
67+
- this is expected to be run on boxtops
68+
does not really matter much, but it does require the `docker login` step
69+
- is published as `dockerhub.pl.sophia.inria.fr/python-nodejs:latest`
70+
- rebuild with
71+
```bash
72+
cd ~/jupyterlab-examples/.github/docker
73+
git pull
74+
./rebuild.sh
75+
```
76+
- this uses `Dockerfile.python-nodejs`
77+
- note that the user `github-actions` is created with uid/gid 1001 (see also
78+
below)
79+
80+
## setup steps on the GH side
81+
82+
### allow public repos to use self-hosted runners
83+
84+
- ***IMPORTANT*** - do it once for each orga !
85+
you also need to allow public repos to use self-hosted runners in the
86+
organization settings on GitHub
87+
88+
`github.com/$ORGA` -> Settings -> Actions -> Runner groups -> Default Group -> Allow public repositories
89+
90+
## setup on the runner host
91+
92+
***IMPORTANT***: make sure you run genuine docker and not podman
93+
94+
```bash
95+
dnf install -y docker-cli
96+
systemctl enable --now docker
97+
```
98+
99+
### runner cleanup steps
100+
101+
if needed: on the GH site, go to `github.com/$ORGA`
102+
> -> Settings -> Actions -> Runners
103+
104+
and remove the runners you don't need anymore
105+
106+
then on the host machine, as root:
107+
108+
```bash
109+
# spot running services
110+
systemctl | grep github-actions-runner
111+
112+
# stop them all - or be specific if only for one orga
113+
systemctl stop github-actions-runner@*
114+
115+
# disable them all - same
116+
systemctl disable github-actions-runner@*
117+
118+
# remove the service files - same
119+
rm /etc/systemd/system/github-actions-runner@*.service
120+
121+
# reload systemd
122+
systemctl daemon-reload
123+
124+
# remove the runner folders - if it's the last orga that had a runner on this box
125+
userdel -rf github-actions
126+
```
127+
128+
## runner setup steps
129+
130+
### once at least as root
131+
132+
it's important that `github-actions` user is created with **the same uid/gid** as the image !
133+
otherwise there will be permission issues and the container won't even start properly !
134+
135+
```bash
136+
137+
# create a user `github-actions` on the machine
138+
groupadd -g 1001 github-actions
139+
sudo useradd -u 1001 -g 1001 -d /containers/github-actions -s /bin/bash github-actions
140+
141+
# add it to the `docker` group
142+
usermod -aG docker github-actions
143+
```
144+
145+
### as github-actions - for one orga
146+
147+
```bash
148+
# login as `github-actions`
149+
su - github-actions
150+
151+
# create a folder for the orga runner
152+
ORGA=ue22-p25
153+
mkdir github-actions-runner-${ORGA}
154+
cd github-actions-runner-${ORGA}
155+
```
156+
157+
### create the runner
158+
159+
- go to github orga page
160+
-> Settings -> Actions -> Runners -> New self-hosted runner
161+
- select Linux / x64
162+
- Download:
163+
- copy-&-paste **but WITHOUT the creation of `actions-runner` folder**
164+
turns out in our setup this is replaced by `github-actions-runner-${ORGA}`
165+
- Configure:
166+
- you may add the 'docker' tag to the runner just in case
167+
168+
### back as root
169+
170+
- install the runner as a service
171+
172+
```bash
173+
# if needed
174+
cp ~/jupyterlab-examples/.github/scripts/github-actions-runner@.service /etc/systemd/system/github-actions-runner@.service
175+
systemctl daemon-reload
176+
177+
ORGA=ue22-p25
178+
systemctl enable --now github-actions-runner@${ORGA}
179+
```
180+
181+
- double check the logs - esp. wrt to the extra folder level if you did create
182+
`actions-runner` folder
183+
in that case rename the folder as appropriate
184+
```bash
185+
journalctl -u github-actions-runner@${ORGA} -f
186+
```
187+
188+
### how to check for current runners
189+
190+
one possible caveat is - done that already - is to attach the runner to a repo
191+
instead of the orga...
192+
193+
to check the current runners for the orga (run on your laptop for authenticated
194+
access):
195+
196+
```
197+
ORGA=ue22-p25
198+
gh api /orgs/$ORGA/actions/runners
199+
```
200+
201+
in case of a mistake (if you attached the runner to a repo instead of the orga):
202+
203+
- you'll need to remove the token in the downloaded folder before you can re-run
204+
config.sh again; this requires to have the old token though
205+
- another approach probably, remove the runner from the GH website, remove the
206+
folder and start over again
207+
208+
## historical notes
209+
210+
### first successful build with frontend (but without containerization)
211+
212+
- was done:
213+
- with orga=ue22-p25
214+
- using host context (no container)
215+
- relies on conda for isolation
216+
- runner box is bigjack
217+
- takes in the 12 minutes
218+
- and would probably be fragile if several runs were to be launched concurrently
219+
220+
### forget about pixi
221+
222+
too many issues with this idea of using pixi for isolation... just don't do that !
223+
the `pixi` branch in `flotpython/course` has some leftovers of that attempt
224+
225+
### remember to not try and run myst as root
226+
227+
one lesson though, it can't run as root (jupyter as spawned by myst would just fail)

‎.github/docker/github-actions-runner@.service‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
[Unit]
2+
Description=GitHub Actions Runner (%i)
3+
After=network.target
4+
5+
[Service]
6+
Type=simple
7+
User=github-actions
8+
# %i will be replaced by the name you provide (e.g., ue22-p25)
9+
WorkingDirectory=/containers/github-actions/github-actions-runner-%i
10+
ExecStart=/containers/github-actions/github-actions-runner-%i/run.sh
11+
Restart=always
12+
RestartSec=10
13+
14+
# Security hardening
15+
PrivateTmp=yes
16+
NoNewPrivileges=yes
17+
18+
[Install]
19+
WantedBy=multi-user.target

0 commit comments

Comments
 (0)