Skip to content

Make sure that the TMPDIR is used as STORAGE if it is set #911

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
trz42 merged 2 commits into from
Feb 13, 2025
Merged

Make sure that the TMPDIR is used as STORAGE if it is set #911

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
174354a
Make sure that the TMPDIR is used as STORAGE if it is set
Feb 12, 2025
bc119ee
Do bind-mount of TMPDIR in eessi_container instead, and check that th…
Feb 12, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 16 additions & 5 deletions bot/build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -90,15 +90,23 @@ if [[ ! -z ${SINGULARITY_CACHEDIR} ]]; then
export SINGULARITY_CACHEDIR
fi

echo -n "setting \$STORAGE by replacing any var in '${LOCAL_TMP}' -> "
# replace any env variable in ${LOCAL_TMP} with its
# current value (e.g., a value that is local to the job)
STORAGE=$(envsubst <<< ${LOCAL_TMP})
echo "'${STORAGE}'"
if [[ -z "${TMPDIR}" ]]; then
echo -n "setting \$STORAGE by replacing any var in '${LOCAL_TMP}' -> "
# replace any env variable in ${LOCAL_TMP} with its
# current value (e.g., a value that is local to the job)
STORAGE=$(envsubst <<< ${LOCAL_TMP})
else
STORAGE=${TMPDIR}
fi
echo "bot/build.sh: STORAGE='${STORAGE}'"

# make sure ${STORAGE} exists
mkdir -p ${STORAGE}

# Make sure ${STORAGE} gets bind-mounted
# This will make sure that any subsequent jobs that create dirs or files under STORAGE have access to it in the container
export SINGULARITY_BIND="${SINGULARITY_BIND},${STORAGE}"
Comment thread
casparvl marked this conversation as resolved.
Outdated

# make sure the base tmp storage is unique
JOB_STORAGE=$(mktemp --directory --tmpdir=${STORAGE} bot_job_tmp_XXX)
echo "bot/build.sh: created unique base tmp storage directory at ${JOB_STORAGE}"
Expand Down Expand Up @@ -287,6 +295,9 @@ else
TARBALL_STEP_ARGS+=("--resume" "${REMOVAL_TMPDIR}")
fi

# Make sure we define storage, so that the TMPDIR is set to this in eessi_container.sh
TARBALL_STEP_ARGS+=("--storage" "${STORAGE}")
Comment thread
casparvl marked this conversation as resolved.
Outdated

timestamp=$(date +%s)
# to set EESSI_VERSION we need to source init/eessi_defaults now
source $software_layer_dir/init/eessi_defaults
Expand Down