Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -219,5 +219,8 @@ rfc.md
*.snap
*.comp

# Spread reuse files
.spread-reuse.*.yaml

# Markdown/mermaid lint tooling deps
scripts/lint-mermaid/node_modules/
45 changes: 45 additions & 0 deletions .image-garden.mk
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how is this used?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When image-garden make or image-garden allocate is used, it loads this file to compute the cloud-init profile to give to the VM. In practice this defines what is in the image that is perhaps cached by the CI stack. Locally it plays a larger role as the cached image is simply in the workspace (no separate caching step). This effectively allows iteration with one-off preparation costs paid once, regardless of how many test cycles you have.

Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

default_sandbox_image=ghcr.io/nvidia/openshell-community/sandboxes/base:latest

define install_and_setup_docker
- addgroup --system docker
- snap install docker
- while [ ! -S /run/docker.sock ]; do sleep 3; done
- "snap run docker pull $(default_sandbox_image)"
endef

define UBUNTU_CLOUD_INIT_USER_DATA_TEMPLATE
$(CLOUD_INIT_USER_DATA_TEMPLATE)
- snap wait system seed.loaded
$(install_and_setup_docker)
endef

define DEBIAN_CLOUD_INIT_USER_DATA_TEMPLATE
$(CLOUD_INIT_USER_DATA_TEMPLATE)
- systemctl enable --now snapd.socket snapd.service snapd.apparmor.service
- snap wait system seed.loaded
$(install_and_setup_docker)
packages:
- snapd
endef

define FEDORA_CLOUD_INIT_USER_DATA_TEMPLATE
$(CLOUD_INIT_USER_DATA_TEMPLATE)
- dnf install -y snapd
- systemctl enable --now snapd.socket
- snap wait system seed.loaded
- sudo ln -s /var/lib/snapd/snap /snap
$(install_and_setup_docker)
endef

define CENTOS_CLOUD_INIT_USER_DATA_TEMPLATE
$(CLOUD_INIT_USER_DATA_TEMPLATE)
- yum install -y epel-release
- yum install -y snapd
- systemctl enable --now snapd.socket snapd.service
- snap wait system seed.loaded
- sudo ln -s /var/lib/snapd/snap /snap
$(install_and_setup_docker)
endef
11 changes: 11 additions & 0 deletions .image-garden/.gitignore
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we instead add .image-garden/* to the root .gitignore?

Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

*.img
*.iso
*.lock
*.log
*.meta-data
*.qcow2
*.run
*.user-data
77 changes: 77 additions & 0 deletions spread.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

project: openshell

backends:
garden:
type: adhoc
allocate: |
if [ -n "${SPREAD_HOST_PATH-}" ]; then
PATH="${SPREAD_HOST_PATH}"
fi
exec image-garden allocate --spread "$SPREAD_SYSTEM"."$(uname -m)"
discard: |
if [ -n "${SPREAD_HOST_PATH-}" ]; then
PATH="${SPREAD_HOST_PATH}"
fi
image-garden discard "$SPREAD_SYSTEM_ADDRESS"
systems:
- ubuntu-cloud-24.04:
username: ubuntu
password: ubuntu
- ubuntu-cloud-26.04:
username: ubuntu
password: ubuntu
- debian-cloud-13:
username: debian
password: debian
- fedora-cloud-44:
username: fedora
password: fedora

exclude:
- ".cache/*"
- ".image-garden/*"
- "target/*"

path: /root/openshell

prepare: |
# Install the openshell snap that was copied into the test environment.
snap install $(ls ./openshell_*.snap | sort -r | head -n 1) --dangerous

# Connect snap interfaces. When installing from the store this
# is auto-connected by the snap declaration assertion, but locally
# we need to do it manually.
snap connect openshell:docker docker:docker-daemon
snap connect openshell:log-observe
snap connect openshell:system-observe
snap connect openshell:ssh-keys

# Add the local gateway to user configuration.
openshell gateway add http://127.0.0.1:17670 --local --name snap-docker
openshell gateway select snap-docker
openshell status

debug-each: |
echo "Kernel and architecture:"
uname -a

echo "OS release info:"
cat /etc/os-release

echo "Installed snaps:"
snap list

set +e

echo "Snap connections of the openshell snap:"
snap connections openshell

echo "Openshell version:"
snap run openshell --version

suites:
tests/:
summary: Smoke tests for OpenShell snap
20 changes: 20 additions & 0 deletions tests/smoke/task.yaml
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how is this used? do you see this getting expandded for non snap use cases? i'd prefer to keep snap specific infra in deploy/snap for now. we have an existing install canary that we run in .github/workflows/release-canary.yml.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is one of the files that spread loads to understand test cases and execute them.

I would love to expand this to cover the non snap case (note that in prepare we can simply install the pre-build binaries from target/release). Spread is used heavily across many of our products as it allows to perform realistic full system testing on many releases of ubuntu (with the real kernel, with the real bugs people would face) as well as to extend this testing to other distributions (this is what garden provides: distribution images).

Here we only have one test case but in our products we aim to have full coverage of all interactions expressed as spread tests. They provide invaluable support in ensuring quality and in preventing regressions in complex products.

Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

summary: Smoke test for creating a sandbox
details: |
This test verifies that the openshell snap is properly installed
and that the openshell command is available and functional.
A test sandbox is created on the selected gateway. A hello world program is
then invoked in the sandbox.

prepare: |
snap run openshell sandbox list | MATCH "No sandboxes found."

execute: |
snap run openshell sandbox create -- echo "Hello, OpenShell!" | MATCH "Hello, OpenShell!"
snap run openshell sandbox list | MATCH ".*Ready"

restore: |
snap run openshell sandbox delete --all | MATCH ".*Deleted.*"
snap run openshell sandbox list | MATCH "No sandboxes found."
Loading