Skip to content

Commit 0ed4cf5

Browse files
eapeap
authored
Merge pull request #1915 from eap/feature/tier1_container
Add new tier-1 container site (configs/sites/tier1/container/) that will replace the legacy `configs/containers/` spack-based container definitions for the 2.1 release. * Provides Dockerfiles for GCC and OneAPI builds that use common configs and a shared site definition and aligned module files. * Build context is the repo root; local config/spack-ext changes are automatically reflected in the image ### Motivation The legacy container builds were a separate path from standard site configs and did not load modules or provide developer tools (without extensive post-build config modifications stretching and sometimes breaking the spack-container design intent). This brings containers under the tier-1 site model so they more closely match sites with loadable modules and pre-installed developer tools.
2 parents 50b32be + a34e51c commit 0ed4cf5

8 files changed

Lines changed: 483 additions & 0 deletions

File tree

‎configs/sites/tier1/container/Dockerfile.gcc‎

Lines changed: 138 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,138 @@
1+
# Spack-Stack Container Build for tier1/container GCC site
2+
# See the README.md for building instructions.
3+
4+
## ---------- build base container used by builder and runtime ----------
5+
FROM ubuntu:24.04 AS ubuntu_base
6+
7+
ARG BUILD_JOBS=4
8+
ARG SPACK_STACK_TEMPLATE=unified-dev
9+
10+
SHELL ["/bin/bash", "-c"]
11+
12+
13+
ENV DEBIAN_FRONTEND=noninteractive \
14+
TZ=Etc/UTC \
15+
LANGUAGE=en_US.UTF-8 \
16+
LANG=en_US.UTF-8 \
17+
LC_ALL=en_US.UTF-8 \
18+
COMPILER=gcc
19+
20+
# Install core packages.
21+
RUN set -euo pipefail; \
22+
apt-get -yqq update && \
23+
apt-get -yqq upgrade && \
24+
apt-get -yqq install --no-install-recommends \
25+
# Core build tools.
26+
build-essential \
27+
gcc-13 \
28+
gcc++-13 \
29+
gfortran-13 \
30+
cpp-13 \
31+
make \
32+
llvm-14 \
33+
autoconf \
34+
# External dependencies.
35+
libcurl4-openssl-dev \
36+
libmysqlclient-dev \
37+
libqt5svg5-dev \
38+
qt5-qmake \
39+
qt5dxcb-plugin \
40+
qtbase5-dev \
41+
zstd \
42+
# Shell tools, source retrieval and networking.
43+
sed \
44+
file \
45+
less \
46+
bzip2 \
47+
unzip \
48+
ca-certificates \
49+
curl \
50+
git \
51+
git-lfs \
52+
gpg \
53+
environment-modules \
54+
tcl \
55+
tcl-dev \
56+
vim \
57+
wget \
58+
nano \
59+
locales && \
60+
locale-gen en_US.UTF-8 && \
61+
# Update alternatives for compilers and tools
62+
update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-13 100 && \
63+
update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-13 100 && \
64+
update-alternatives --install /usr/bin/gfortran gfortran /usr/bin/gfortran-13 100 && \
65+
update-alternatives --install /usr/bin/llvm-config llvm-config /usr/bin/llvm-config-14 100 && \
66+
rm -rf /var/lib/apt/lists/*
67+
68+
69+
## ---------- build spack-stack environment ----------
70+
FROM ubuntu_base AS builder
71+
72+
ENV SPACK_STACK_DIR=/opt/spack-stack \
73+
SPACK_ROOT=/opt/spack-stack/spack
74+
75+
76+
# Copy spack-stack from build context. Note the build context must be the
77+
# root of the spack-stack repository (see the README.md for details).
78+
COPY . ${SPACK_STACK_DIR}
79+
80+
# Create spack-stack environment
81+
WORKDIR /opt/spack-stack
82+
RUN mkdir -p /tmp/spack-stack && \
83+
source setup.sh && \
84+
spack stack create env \
85+
--site container \
86+
--template ${SPACK_STACK_TEMPLATE} \
87+
--name container \
88+
--prefix /opt/spack-software/ \
89+
--compiler $COMPILER && \
90+
cd ${SPACK_STACK_DIR}/envs/container && \
91+
spack env activate . && \
92+
spack concretize 2>&1 | tee log.concretize && \
93+
spack install --fail-fast -j ${BUILD_JOBS} 2>&1 | tee log.install && \
94+
spack module tcl refresh -y && \
95+
spack stack setup-meta-modules && \
96+
# Save output of spack find.
97+
spack find 2>&1 | tee /opt/spack-software/spack_find.out && \
98+
spack clean --all
99+
100+
## ---------- Create runtime container ----------
101+
FROM ubuntu_base AS runtime
102+
103+
COPY --from=builder /opt/spack-software/ /opt/spack-software/
104+
105+
ENV CC=gcc \
106+
CXX=g++ \
107+
FC=gfortran
108+
109+
# Container-wide rc script with compiler defaults, MPI policy, module paths.
110+
RUN echo "ulimit -s unlimited" > /etc/spack_container_rc.sh \
111+
&& echo "ulimit -v unlimited" >> /etc/spack_container_rc.sh \
112+
&& echo "export CC=gcc" >> /etc/spack_container_rc.sh \
113+
&& echo "export CXX=g++" >> /etc/spack_container_rc.sh \
114+
&& echo "export FC=gfortran" >> /etc/spack_container_rc.sh \
115+
&& echo "# OpenMPI settings for running as root and oversubscription." >> /etc/spack_container_rc.sh \
116+
&& echo "export OMPI_ALLOW_RUN_AS_ROOT=1" >> /etc/spack_container_rc.sh \
117+
&& echo "export OMPI_ALLOW_RUN_AS_ROOT_CONFIRM=1" >> /etc/spack_container_rc.sh \
118+
&& echo "export PRTE_MCA_rmaps_default_mapping_policy=:oversubscribe" >> /etc/spack_container_rc.sh \
119+
&& echo "# TCL module path for the spack-stack environment." >> /etc/spack_container_rc.sh \
120+
&& echo "source /etc/profile.d/modules.sh" >> /etc/spack_container_rc.sh \
121+
&& echo "module use /opt/spack-software/modules/Core" >> /etc/spack_container_rc.sh \
122+
&& echo "source /etc/spack_container_rc.sh" >> /etc/bash.bashrc \
123+
&& printf "[credential]\n helper = cache --timeout=7200\n" >> /root/.gitconfig \
124+
&& mkdir /root/.pmix \
125+
&& echo "rmaps_default_mapping_policy=:oversubscribe" >> /root/.pmix/mca-params.conf
126+
127+
# Nonroot user for MPI
128+
RUN useradd -U -k /etc/skel -s /bin/bash -d /home/nonroot -m nonroot --uid 43891 \
129+
&& echo "ulimit -s unlimited" >> /home/nonroot/.bashrc \
130+
&& echo "ulimit -v unlimited" >> /home/nonroot/.bashrc \
131+
&& printf "[credential]\n helper = cache --timeout=7200\n" >> /home/nonroot/.gitconfig \
132+
&& mkdir /home/nonroot/.pmix \
133+
&& chown -R nonroot:nonroot /home/nonroot/.gitconfig /home/nonroot/.pmix
134+
135+
# Ensure the container rc is run by non-login shells too.
136+
ENV BASH_ENV=/etc/spack_container_rc.sh
137+
138+
CMD ["/bin/bash"]

‎configs/sites/tier1/container/Dockerfile.oneapi‎

Lines changed: 151 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,151 @@
1+
# Spack-Stack Container Build for tier1/container Intel oneapi site
2+
# See the README.md for building instructions.
3+
4+
## ---------- build base container used by builder and runtime ----------
5+
FROM ubuntu:24.04 AS ubuntu_base
6+
7+
ARG BUILD_JOBS=4
8+
ARG SPACK_STACK_TEMPLATE=unified-dev
9+
10+
SHELL ["/bin/bash", "-c"]
11+
12+
13+
ENV DEBIAN_FRONTEND=noninteractive \
14+
TZ=Etc/UTC \
15+
LANGUAGE=en_US.UTF-8 \
16+
LANG=en_US.UTF-8 \
17+
LC_ALL=en_US.UTF-8 \
18+
COMPILER=intel
19+
20+
# Install core packages.
21+
RUN set -euo pipefail; \
22+
apt-get -yqq update && \
23+
apt-get -yqq upgrade && \
24+
apt-get -yqq install --no-install-recommends \
25+
# Core build tools.
26+
build-essential \
27+
gcc-13 \
28+
gcc++-13 \
29+
gfortran-13 \
30+
cpp-13 \
31+
make \
32+
llvm-14 \
33+
autoconf \
34+
# External dependencies.
35+
libcurl4-openssl-dev \
36+
libmysqlclient-dev \
37+
libqt5svg5-dev \
38+
qt5-qmake \
39+
qt5dxcb-plugin \
40+
qtbase5-dev \
41+
zstd \
42+
# Shell tools, source retrieval and networking.
43+
sed \
44+
file \
45+
less \
46+
bzip2 \
47+
unzip \
48+
ca-certificates \
49+
curl \
50+
git \
51+
git-lfs \
52+
gpg \
53+
environment-modules \
54+
tcl \
55+
tcl-dev \
56+
vim \
57+
wget \
58+
nano \
59+
locales && \
60+
locale-gen en_US.UTF-8 && \
61+
# Update alternatives for compilers and tools
62+
update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-13 100 && \
63+
update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-13 100 && \
64+
update-alternatives --install /usr/bin/gfortran gfortran /usr/bin/gfortran-13 100 && \
65+
update-alternatives --install /usr/bin/llvm-config llvm-config /usr/bin/llvm-config-14 100 && \
66+
# Install the intel
67+
apt install -y apt-utils gpg && \
68+
wget -O- https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB \
69+
| gpg --dearmor \
70+
| tee /usr/share/keyrings/oneapi-archive-keyring.gpg > /dev/null && \
71+
echo "deb [signed-by=/usr/share/keyrings/oneapi-archive-keyring.gpg] https://apt.repos.intel.com/oneapi all main" \
72+
| tee /etc/apt/sources.list.d/oneAPI.list && \
73+
apt update && \
74+
apt install -y \
75+
intel-oneapi-compiler-dpcpp-cpp-2025.3 \
76+
intel-oneapi-compiler-fortran-2025.3 \
77+
intel-oneapi-mpi-devel-2021.17 \
78+
intel-oneapi-tbb-devel-2022.3 \
79+
intel-oneapi-mkl-devel-2025.3 && \
80+
/opt/intel/oneapi/modulefiles-setup.sh --output-dir=/opt/intel/oneapi/modulefiles && \
81+
echo "source /etc/profile.d/modules.sh && module use /opt/intel/oneapi/modulefiles" > /etc/profile.d/z01_oneapi_modules.sh && \
82+
rm -rf /var/lib/apt/lists/*
83+
84+
85+
## ---------- build spack-stack environment ----------
86+
FROM ubuntu_base AS builder
87+
88+
ENV SPACK_STACK_DIR=/opt/spack-stack \
89+
SPACK_ROOT=/opt/spack-stack/spack
90+
91+
92+
# Copy spack-stack from build context. Note the build context must be the
93+
# root of the spack-stack repository (see the README.md for details).
94+
COPY . ${SPACK_STACK_DIR}
95+
96+
# Create spack-stack environment
97+
WORKDIR /opt/spack-stack
98+
RUN mkdir -p /tmp/spack-stack && \
99+
source setup.sh && \
100+
spack stack create env \
101+
--site container \
102+
--template ${SPACK_STACK_TEMPLATE} \
103+
--name container \
104+
--prefix /opt/spack-software/ \
105+
--compiler oneapi && \
106+
cd ${SPACK_STACK_DIR}/envs/container && \
107+
spack env activate . && \
108+
spack concretize 2>&1 | tee log.concretize && \
109+
spack install --fail-fast -j ${BUILD_JOBS} 2>&1 | tee log.install && \
110+
spack module tcl refresh -y && \
111+
spack stack setup-meta-modules && \
112+
# Save output of spack find.
113+
spack find 2>&1 | tee /opt/spack-software/spack_find.out && \
114+
spack clean --all
115+
116+
## ---------- Create runtime container ----------
117+
FROM ubuntu_base AS runtime
118+
119+
COPY --from=builder /opt/spack-software/ /opt/spack-software/
120+
121+
ENV CC=icx \
122+
CXX=icpx \
123+
FC=ifx
124+
125+
# Container-wide rc script with compiler defaults, MPI policy, module paths.
126+
RUN echo "ulimit -s unlimited" > /etc/spack_container_rc.sh \
127+
&& echo "ulimit -v unlimited" >> /etc/spack_container_rc.sh \
128+
&& echo "export CC=icx" >> /etc/spack_container_rc.sh \
129+
&& echo "export CXX=icpx" >> /etc/spack_container_rc.sh \
130+
&& echo "export FC=ifx" >> /etc/spack_container_rc.sh \
131+
&& echo "# TCL module path for the spack-stack environment." >> /etc/spack_container_rc.sh \
132+
&& echo "export MODULEPATH=/opt/spack-software/modules/Core" >> /etc/spack_container_rc.sh \
133+
&& echo "source /etc/profile.d/modules.sh" >> /etc/spack_container_rc.sh \
134+
&& echo "module use /opt/intel/oneapi/modulefiles" >> /etc/spack_container_rc.sh \
135+
&& echo "module use /opt/spack-software/modules/Core" >> /etc/spack_container_rc.sh \
136+
&& echo "source /etc/spack_container_rc.sh" >> /etc/bash.bashrc \
137+
&& printf "[credential]\n helper = cache --timeout=7200\n" >> /root/.gitconfig \
138+
&& mkdir /root/.pmix
139+
140+
# Nonroot user for MPI
141+
RUN useradd -U -k /etc/skel -s /bin/bash -d /home/nonroot -m nonroot --uid 43891 \
142+
&& echo "ulimit -s unlimited" >> /home/nonroot/.bashrc \
143+
&& echo "ulimit -v unlimited" >> /home/nonroot/.bashrc \
144+
&& printf "[credential]\n helper = cache --timeout=7200\n" >> /home/nonroot/.gitconfig \
145+
&& mkdir /home/nonroot/.pmix \
146+
&& chown -R nonroot:nonroot /home/nonroot/.gitconfig /home/nonroot/.pmix
147+
148+
# Ensure the container rc is run by non-login shells too.
149+
ENV BASH_ENV=/etc/spack_container_rc.sh
150+
151+
CMD ["/bin/bash"]

‎configs/sites/tier1/container/README.md‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
# Container Site
2+
3+
This site config is used to build the official Spack-stack containers. This new tier-1 container site superceedes the legacy spack-based container builds and orients the container builder around common configs and a common site definition. The motivation for this site is to have our shared containers more closely match sites with loadable modules and developer tools-pre installed.
4+
5+
6+
# Building
7+
8+
This container is not a typical site config and is installed slightly differently to accomodate the use of a dockerfile. Please follow the instructions here to build it.
9+
10+
## Prerequisites
11+
- Docker
12+
- At least 100 GB of disk space for the build
13+
- Sufficient RAM (8GB+ recommended)
14+
15+
## Quick Build
16+
17+
The build context is the spack-stack repository root. The Dockerfile
18+
copies the entire local checkout (including the submodules) into
19+
the image, so any local changes to configs, templates, or spack-ext
20+
are automatically reflected in the container.
21+
22+
```bash
23+
docker build \
24+
-t spack-stack-gcc:local \
25+
-f "$(git rev-parse --show-toplevel)/configs/sites/tier1/container/Dockerfile.gcc" \
26+
--build-arg SPACK_STACK_TEMPLATE=unified-dev \
27+
--build-arg BUILD_JOBS=10 \
28+
"$(git rev-parse --show-toplevel)" 2>&1 | tee ${HOME}/log.docker_gcc
29+
30+
31+
docker build \
32+
-t spack-stack-oneapi:local \
33+
-f "$(git rev-parse --show-toplevel)/configs/sites/tier1/container/Dockerfile.oneapi" \
34+
--build-arg SPACK_STACK_TEMPLATE=unified-dev \
35+
--build-arg BUILD_JOBS=10 \
36+
"$(git rev-parse --show-toplevel)" 2>&1 | tee ${HOME}/log.docker_oneapi
37+
```

‎configs/sites/tier1/container/config.yaml‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
config:
2+
build_jobs: 10
3+
4+
# Container-specific paths for build and staging areas.
5+
# Use /tmp for transient build data (not preserved in final image).
6+
build_stage: /tmp/spack-stack/cache/build_stage
7+
test_stage: /tmp/spack-stack/cache/test_stage
8+
source_cache: /tmp/spack-stack/cache/source_cache
9+
misc_cache: /tmp/spack-stack/cache/misc_cache

‎configs/sites/tier1/container/modules.yaml‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
modules:
2+
default:
3+
enable::
4+
- tcl
5+
tcl:
6+
include:
7+
- python

0 commit comments

Comments
 (0)