diff --git a/build b/build index aebd69b..f2682de 100755 --- a/build +++ b/build @@ -126,7 +126,7 @@ if [ "$use_kms" = 1 ]; then done fi -# Default values which can be overriden via 'build.config' file +# Default values which can be overridden via 'build.config' file tempfs_size=2G if [[ -f "$PWD"/build.config ]]; then @@ -163,7 +163,7 @@ if [ "$container_engine" = "docker" ] \ apparmor_profile=builder fi -# Apply apparmor profile if seleceted +# Apply apparmor profile if selected if [ "$apparmor_profile" ]; then replaced=false for i in "${!container_run_opts[@]}"; do diff --git a/builder/make_get_image_dependencies b/builder/make_get_image_dependencies index f8789a0..cbb06c0 100755 --- a/builder/make_get_image_dependencies +++ b/builder/make_get_image_dependencies @@ -57,9 +57,9 @@ for feature in "${features[@]}"; do # Advanced convert scripts are scripts of the form convert.extA~extB. # Unlike regular convert scripts these don't always get the image.raw as input, - # instead they recieve image.extB as their input. + # instead they receive image.extB as their input. # This allows for convert scripts to operate on image scripts producing non .raw output - # or for covert scripts to operate on outputs of prior convert scripts. + # or for convert scripts to operate on outputs of prior convert scripts. for i in "${advanced_convert_scripts[@]}"; do if [ "$is_feature_script" = 1 ]; then diff --git a/docs/explanation/builder.md b/docs/explanation/builder.md new file mode 100644 index 0000000..df42449 --- /dev/null +++ b/docs/explanation/builder.md @@ -0,0 +1,105 @@ +--- +title: "Builder" +description: "Conceptual overview of the Garden Linux Builder build system, architecture and design philosophy" +related_topics: + - /explanation/builder + - /explanation/flavors + - /explanation/boot-modes + - /explanation/secure-boot + - /reference/builder + - /how-to/secure-boot + - /how-to/choosing-flavors + - /how-to/getting-images +migration_status: "done" +migration_issue: "https://github.com/gardenlinux/gardenlinux/issues/4627" +migration_stakeholder: "@tmang0ld, @yeoldegrove, @ByteOtter" +migration_approved: false +github_org: gardenlinux +github_repo: builder +github_source_path: docs/explanation/builder.md +github_target_path: docs/explanation/builder.md +--- + +# How the Build System Works + +## The Builder + +Garden Linux images are produced by [gardenlinux/builder](https://github.com/gardenlinux/builder), +a dedicated build tool maintained separately from the main +[gardenlinux/gardenlinux](https://github.com/gardenlinux/gardenlinux) repository. This separation +means the build infrastructure can evolve independently of the distribution content. + +The `./build` script in the `gardenlinux/gardenlinux` repository is the primary entry point. It +automatically fetches the correct builder container image, then delegates all internal build steps +to it. As a result, the only hard dependency on the host system is a working container engine — +no specific Linux distribution, compiler toolchain, or package set is required on the build host. + +## Design Philosophy + +The build system is designed around three principles: + +- **Minimal host dependencies** — The build runs entirely inside a container. Apart from the + container engine itself, the host needs no build tools. +- **Composability** — Images are assembled from reusable [features](/explanation/flavors#feature-based-design) + rather than hand-crafted for each target. The same feature definition is reused across all + platforms that include it. +- **Reproducibility** — The builder container is versioned and pinned, so a given combination + of source code and feature set should produce the same image regardless of when or where it + is built. + +## Flavor Names as Build Inputs + +The build command takes a *flavor name* as its argument. A flavor name directly encodes what +will be built: + +``` +--_- +``` + +For example, `aws-gardener_prod-amd64` tells the builder to produce an AWS image (`aws`) with +the [`gardener`](/reference/features/gardener) and [`_prod`](/reference/features/_prod) features for the `amd64` architecture. The build script parses this +name and assembles the image by combining the specified platform and features. + +For a full explanation of how flavors and features compose, including the [CNAME system](/explanation/flavors#the-cname-system) and how +features are joined, see [Flavors](/explanation/flavors). + +## Cross-Architecture Builds + +By default, the builder targets the native architecture of the build host. Building for a +different architecture (for example, building `arm64` images on an `amd64` host) requires +the host to be able to execute foreign binaries. + +The standard mechanism for this on Linux is +[binfmt_misc](https://docs.kernel.org/admin-guide/binfmt-misc.html), a kernel feature that +registers handlers for non-native executable formats. When combined with QEMU user-mode +emulation (`qemu-user-static`), the kernel transparently invokes the correct QEMU binary +whenever the builder attempts to run an `arm64` binary inside the container. + +Cross-architecture builds are slower than native builds because every foreign-architecture +instruction goes through QEMU emulation. + +## Certificates for Secure Boot and Trusted Boot + +Images that use the [`_trustedboot`](/reference/features/_trustedboot), +[`_tpm2`](/reference/features/_tpm2), or [`_secureboot`](/reference/features/_secureboot) +features must be signed with a custom certificate chain. This is because UEFI Secure Boot +validates the bootloader and kernel against enrolled certificates before execution — without +a valid signature, the firmware refuses to boot the image. + +The `./cert/build` script generates this certificate chain (Platform Key, Key Exchange Key, +and Signature Database) inside a container, keeping the same minimal-dependency model as the +main build. The private keys are stored locally in the `cert/` directory by default, or in +AWS Key Management Service (KMS) when the `--kms` flag is used. + +For the conceptual background on why signing is required and how Secure Boot and Trusted Boot +interact with the USI boot mode, see +[Boot Modes](/explanation/boot-modes) and +[Secure Boot and Trusted Boot](/explanation/secure-boot). + +For step-by-step build and deployment instructions, see +[Building Images](/how-to/building-images) and +[Deploying Secure Boot Images](/how-to/secure-boot). + +## Related Topics + + diff --git a/docs/getting_started.md b/docs/getting_started.md deleted file mode 100644 index 8d36296..0000000 --- a/docs/getting_started.md +++ /dev/null @@ -1,148 +0,0 @@ -# Getting Started: Creating a Custom Linux Image with Builder - -This tutorial will walk you through the process of creating a custom Linux image using the Builder tool. -We will start with the Builder example repository and build a *feature* to add an [`nginx` HTTP server](https://nginx.org/en/) to our image. - -Let's begin by creating a new GitHub repository based on the Builder example repository using this link: - -https://github.com/new?template_name=builder_example&template_owner=gardenlinux - -This repo has a GitHub Actions workflow enabled, so it will already start building the image on GitHub's hosted runners. - -To customize the image, clone the repo locally: - -```shell -git clone https://github.com/your_username/my_linux_image -cd my_linux_image -``` - -To ensure that your local Podman installation is working correctly, you can test it by running the following command: - -```shell -./build base -``` - -This command will create a bootable Debian Forky disk image at `.build/base-amd64-forky-6f72b564.raw` (note that the commit may have changed since the time of writing). -You can test run the image using [QEMU](https://www.qemu.org): - -```shell -qemu-system-x86_64 -m 2048 -nodefaults -display none -serial mon:stdio -drive if=pflash,unit=0,readonly=on,format=raw,file=/usr/share/OVMF/OVMF_CODE.fd -drive if=virtio,format=raw,file=.build/base-amd64-forky-6f72b564.raw -``` - -Now that we have verified that everything is working correctly, let's proceed to build our own feature. - -## Creating the Nginx Feature - -1. Create a directory called `nginx` inside the `features` directory: - -```shell -mkdir features/nginx -``` - -> This is where our nginx feature will live. -Features are a concept of the builder that allows us to build variants of images. -For example, if we wanted to add an alternative HTTP server later, we could add an `apacheHttpd` feature. -At image build time, we could pick if we want the `nginx` or the `apacheHttpd` feature. - -2. Create a file named `info.yaml` inside `features/nginx` and edit it with the content below: - -```yaml -description: HTTP server using Nginx -type: element -features: - include: - - base -``` - -> The `info.yaml` file is required for each feature by the builder. -We'll specify that our `nginx` feature includes the `base` feature. -This makes sense because the `nginx` feature on its own does not contain a full operating system, so to get a bootable image we include the debian system as it is defined in `base`. -See [features.md](./features.md) for detailed information on the structure of features. - -1. Create a file named `pkg.include` inside `features/nginx` with the following content: - -``` -nginx -``` - - -> `pkg.include` is a list of packages this feature needs, each feature on a new line. - -4. Create a file named `exec.config` inside `features/nginx` with the following content: - -```shell -#!/usr/bin/env bash - -set -eufo pipefail - -systemctl enable nginx -``` - - -> `exec.config` is a shell script we can use to customize our image. -In this case, we [enable the systemd unit for nginx](https://www.freedesktop.org/software/systemd/man/latest/systemctl.html#enable%20UNIT…) which makes nginx start on boot. - -5. Make the `exec.config` file executable: - -```shell -chmod +x features/nginx/exec.config -``` - -6. Create a directory named `/var/www/html` inside the `file.include` directory of Nginx: - -```shell -mkdir -p features/nginx/file.include/var/www/html -``` - - -> The `file.include` directory allows us to merge files and directories into the root filesystem of our image. - -7. Create a dummy `index.html` file inside `features/nginx/file.include/var/www/html` with content like the following (or customize it as desired): - -```html - - - -

Hello World!

- - -``` - -To test your feature, build the image using the following command: - -```shell -./build nginx -``` - -You can then run the image with QEMU using the following command: - -```shell -qemu-system-x86_64 -m 2048 -nodefaults -display none -serial mon:stdio -drive if=pflash,unit=0,readonly=on,format=raw,file=/usr/share/OVMF/OVMF_CODE.fd -drive if=virtio,format=raw,file=.build/nginx-amd64-forky-local.raw -netdev user,id=net0,hostfwd=tcp::8080-:80 -device virtio-net-pci,netdev=net0 -``` - -If everything worked as intended, you should see the system boot up. Once the system is booted, opening http://localhost:8080 in a browser should display the "Hello World!" message. - -To also build the new image on GitHub Actions, we'll need to modify the `.github/workflows/build.yml` file. - -Let's change the *build* step to include the `nginx` feature we just created, and let's upload our built image to GitHub's artifact storage: - -```diff -diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml -index 181a646..9e4261e 100644 ---- a/.github/workflows/build.yml -+++ b/.github/workflows/build.yml -@@ -13,4 +13,8 @@ jobs: - steps: - - uses: actions/checkout@v3 - - name: Build the image -- run: ./build base -+ run: ./build nginx -+ - uses: actions/upload-artifact@v4 -+ with: -+ name: my-linux-image -+ path: .build/ -``` - -Now commit and push your changes and GitHub will build the image for you. - -Congratulations! You have successfully created your first feature for the Builder and setup a CI Pipeline to build the image. :tada: diff --git a/docs/how-to/building-images.md b/docs/how-to/building-images.md new file mode 100644 index 0000000..e3728ff --- /dev/null +++ b/docs/how-to/building-images.md @@ -0,0 +1,146 @@ +--- +title: "Building Images" +description: "Step-by-step guide to building Garden Linux images locally with the Builder" +related_topics: + - /explanation/builder + - /explanation/flavors + - /explanation/boot-modes + - /explanation/secure-boot + - /reference/builder + - /how-to/secure-boot + - /how-to/choosing-flavors + - /how-to/getting-images +migration_status: "done" +migration_issue: "https://github.com/gardenlinux/gardenlinux/issues/4627" +migration_stakeholder: "@tmang0ld, @yeoldegrove, @ByteOtter" +migration_approved: false +github_org: gardenlinux +github_repo: builder +github_source_path: "docs/how-to/building-images.md" +github_target_path: "docs/how-to/building-images.md" +--- + +# Building Images + +This guide walks you through building Garden Linux images locally. For conceptual background on +how the build system works, see [Builder Explanation](/explanation/builder). + +## Prerequisites + +:::warning +Provide at least **8 GiB of RAM** to the container runtime or the virtual machine hosting it +(in Podman Desktop or Docker Desktop). The build may fail silently if memory is insufficient. +::: + +You need a working container engine. Three options are supported: + +- **Rootless Podman** (recommended) — See the + [Podman rootless setup guide](https://github.com/containers/podman/blob/main/docs/tutorials/rootless_tutorial.md) + for setup instructions. +- **Sudo Podman** — Use Podman with `sudo` for elevated privileges. +- **Docker** — Use Docker. + +:::info +See [Builder Command-Line Reference](/reference/builder#options) how to pass the container engine. +::: + +## Build an Image + +Run the build script with the target flavor name: + +```bash +./build ${platform}-${feature1}-${feature2}-${arch} +``` + +Where: + +- `${platform}` — the target platform (e.g. `kvm`, `metal`, `aws`); must be the first component. +- `${featureX}` — one or more features from the `features/` directory, separated by `-` + (or `_` for features whose names begin with `_`). +- `${arch}` — optional target architecture (`amd64` or `arm64`); must be the last component. + +Examples: + +```bash +./build kvm-python_dev +./build aws-gardener_prod-amd64 +``` + +For a full list of build script options, see the +[Builder Command-Line Reference](/reference/builder). + +For help choosing a flavor name, see [Choosing Flavors](/how-to/choosing-flavors). + +## Run Parallel Builds + +To build multiple targets simultaneously, use the `-j` flag: + +```bash +./build -j 4 kvm-amd64 kvm-arm64 aws-gardener_prod-amd64 aws-gardener_prod-arm64 +``` + +:::warning +Parallel builds multiply memory usage. Allow at least **8 GiB per thread**. There are no +safeguards against memory exhaustion; builds may fail silently if memory runs out. +::: + +## Build for a Different Architecture + +Append the target architecture to the flavor name: + +```bash +./build kvm-amd64 kvm-arm64 +``` + +:::info +[Cross-architecture builds](/explanation/builder#cross-architecture-builds) require +[binfmt_misc](https://docs.kernel.org/admin-guide/binfmt-misc.html) handlers and QEMU +user-mode emulation (`qemu-user-static`) on the build host. + +```bash +apt install qemu-user-static +``` +::: + + +## Build Secureboot / Trustedboot / TPM2 Images + +Before building any image with the `_tpm2`, `_trustedboot`, or `_secureboot` feature, generate +the signing certificates: + +```bash +./cert/build +``` + +:::tip +Do not use the `Makefile` in `cert/` directly. Always use `./cert/build`. +If `./cert/build` fails, try running `./cert/build clean` first. +::: + +By default, private keys are stored locally in the `cert/` directory. To use AWS Key Management +Service instead, pass the `--kms` flag (valid AWS credentials must be configured via the standard +AWS environment variables): + +```bash +./cert/build --kms +``` + +After generating certificates, build the image as normal: + +```bash +./build kvm-trustedboot +./build aws-trustedboot_tpm2 +``` + +For conceptual background on these features, see [Boot Modes](/explanation/boot-modes) and +[Secure Boot and Trusted Boot](/explanation/secure-boot). For cloud deployment steps, see +[Deploying Secure Boot Images](/how-to/secure-boot). + +## Troubleshooting + +If you encounter build failures, refer to the +[Garden Linux builder troubleshooting section](https://github.com/gardenlinux/builder#troubleshooting). + +## Related Topics + + diff --git a/docs/how-to/custom-feature.md b/docs/how-to/custom-feature.md new file mode 100644 index 0000000..99618af --- /dev/null +++ b/docs/how-to/custom-feature.md @@ -0,0 +1,238 @@ +--- +title: "Building a custom Feature" +description: Create a custom Feature and build an image with the Builder +related_topics: + - /reference/supporting_tools/builder.md + - /reference/features/ + - /how-to/custom-feature +migration_status: "done" +migration_issue: "https://github.com/gardenlinux/gardenlinux/issues/4628" +migration_stakeholder: "@tmangold, @yeoldegrove, @ByteOtter" +migration_approved: false +github_org: gardenlinux +github_repo: builder +github_source_path: docs/how-to/custom-feature.md +github_target_path: docs/how-to/custom-feature.md +--- + +# Create a custom Feature and build an image with the Builder + +The [Builder](/explanation/builder) is a generic, containerized tool for building Linux system images from config directories. It is reusable by any project that needs to create customized Linux distributions. + +This guide covers two scenarios: + +1. **Creating a custom feature for a [Debian-based builder project](#debian-based-builder-project)** — using the Builder standalone with your own config directory. +2. **Creating a custom feature for the [Garden Linux Distribution](#garden-linux-distribution)** — adding a feature to the Garden Linux distribution. + +Both scenarios will work with the [Creating a custom Feature](#creating-a-custom-feature) guide. + +--- + +## Debian-based builder project + +This section walks you through creating a custom feature in a standalone Builder project, such as one created from the [builder_example](https://github.com/gardenlinux/builder_example) template. + +Let's begin by creating a new GitHub repository based on the Builder example repository using this link: + +https://github.com/new?template_name=builder_example&template_owner=gardenlinux + +This repo has a GitHub Actions workflow enabled, so it will already start building the image on GitHub's hosted runners. + +### Customizing the image + +To customize the image, clone the repo locally: + +```shell +git clone https://github.com/your_username/my_linux_image +cd my_linux_image +``` + +To ensure that your local Podman installation is working correctly, you can test it by running the following command: + +```shell +./build base +``` + +This command will create a bootable Debian Forky disk image at `.build/base-amd64-forky-6f72b564.raw` (note that the commit may have changed since the time of writing). +You can test run the image using [QEMU](https://www.qemu.org): + +```shell +qemu-system-x86_64 -m 2048 -nodefaults -display none -serial mon:stdio -drive if=pflash,unit=0,readonly=on,format=raw,file=/usr/share/OVMF/OVMF_CODE.fd -drive if=virtio,format=raw,file=.build/base-amd64-forky-6f72b564.raw +``` + +Now that you have verified that everything is working correctly, proceed to [creating a custom feature](#creating-a-custom-feature). + +## Garden Linux Distribution + +This section covers adding a custom feature to the [Garden Linux](https://github.com/gardenlinux/gardenlinux) distribution. Garden Linux uses the Builder as its build engine but adds a rich set of platform and element features. + +### Key difference: platform requirement + +Garden Linux enforces exactly one `platform` feature per build (see [ADR 0020](/reference/adr/0020-enforce-single-platform-by-default-in-builder.md)). This means: + +- Running `./build base` **fails** because `base` is an `element`, not a `platform`. +- Running `./build myfeature` **fails** unless `myfeature` is a platform. +- You must always specify a platform: `./build kvm-myfeature-amd64` + +:::warning +Garden Linux requires exactly one platform feature per build. Running `./build base` or `./build myfeature` without a platform fails. Always include a platform, for example: `./build kvm-myfeature-amd64`. +::: + +### Feature types in Garden Linux + +Garden Linux features follow the same structure as vanilla builder features, but use three types: + +- **`platform`** — target platform such as `kvm`, `aws`, `azure`, `metal`, `gcp`. Rarely created by users. +- **`element`** — significant feature that may include/exclude other features, e.g., `server`, `cloud`, `cis`, `fedramp`. +- **`flag`** — minor toggles, conventionally prefixed with `_`, e.g., `_prod`, `_slim`, `_selinux`. + +### Customizing the image + +To customize the image, clone the repo locally: + +1. Clone the Garden Linux repository: + +```shell +git clone https://github.com/gardenlinux/gardenlinux +cd gardenlinux +``` + +To ensure that your local Podman installation is working correctly, you can test it by running the following command: + +```shell +./build kvm-base +``` + +This command will create a bootable Garden Linux disk image at `.build/kvm-base-amd64-today-6f72b564.raw` (note that the commit may have changed since the time of writing). +You can test run the image using [QEMU](https://www.qemu.org): + +```shell +qemu-system-x86_64 -m 2048 -nodefaults -display none -serial mon:stdio -drive if=pflash,unit=0,readonly=on,format=raw,file=/usr/share/OVMF/OVMF_CODE.fd -drive if=virtio,format=raw,file=.build/kvm-base-amd64-today-6f72b564.raw +``` + +Now that you have verified that everything is working correctly, proceed to [creating a custom feature](#creating-a-custom-feature). + +## Creating a custom feature + +We will create a custom feature (an element in the Garden Linux structure) and deploy a Nginx Webserver as an example. + +1. Create a directory called `nginx` inside the `features` directory: + +```shell +mkdir features/nginx +``` + +This is where our nginx feature will live. +Features are a concept of the builder that allows us to build variants of images. +For example, if we wanted to add an alternative HTTP server later, we could add an `apacheHttpd` feature. +At image build time, we could pick if we want the `nginx` or the `apacheHttpd` feature. + +2. Create a file named `info.yaml` inside `features/nginx` and edit it with the content below: + +```yaml +description: HTTP server using Nginx +type: element +features: + include: + - base +``` + +The `info.yaml` file is required for each feature by the builder. +We specify that our `nginx` feature includes the `base` feature. +This makes sense because the `nginx` feature on its own does not contain a full operating system, so to get a bootable image we include the debian system as it is defined in `base`. +See the [features documentation](/reference/features/) for detailed information on the structure of features. + +3. Create a file named `pkg.include` inside `features/nginx` with the following content: + +``` +nginx +``` + +`pkg.include` is a list of packages this feature needs, each on a new line. + +4. Create a file named `exec.config` inside `features/nginx` with the following content: + +```shell +#!/usr/bin/env bash + +set -eufo pipefail + +systemctl enable nginx +``` + +`exec.config` is a shell script we can use to customize our image. +In this case, we [enable the systemd unit for nginx](https://www.freedesktop.org/software/systemd/man/latest/systemctl.html#enable%20UNIT…) which makes nginx start on boot. + +5. Make the `exec.config` file executable: + +```shell +chmod +x features/nginx/exec.config +``` + +6. Create a directory named `/var/www/html` inside the `file.include` directory of Nginx: + +```shell +mkdir -p features/nginx/file.include/var/www/html +``` + +The `file.include` directory allows us to merge files and directories into the root filesystem of our image. + +7. Create a dummy `index.html` file inside `features/nginx/file.include/var/www/html` with content like the following: + +```html + + + +

Hello World!

+ + +``` + +To test your feature, build the image using the following command: + +```shell +# vanilla builder +./build nginx +# builder inside gardenlinux repo +./build kvm-nginx +``` + +You can then run the image with QEMU using the following command: + +```shell +# vanilla builder +qemu-system-x86_64 -m 2048 -nodefaults -display none -serial mon:stdio -drive if=pflash,unit=0,readonly=on,format=raw,file=/usr/share/OVMF/OVMF_CODE.fd -drive if=virtio,format=raw,file=.build/nginx-amd64-forky-local.raw -netdev user,id=net0,hostfwd=tcp::8080-:80 -device virtio-net-pci,netdev=net0 +# builder inside gardenlinux repo +qemu-system-x86_64 -m 2048 -nodefaults -display none -serial mon:stdio -drive if=pflash,unit=0,readonly=on,format=raw,file=/usr/share/OVMF/OVMF_CODE.fd -drive if=virtio,format=raw,file=.build/kvm-nginx-amd64-today-local.raw -netdev user,id=net0,hostfwd=tcp::8080-:80 -device virtio-net-pci,netdev=net0 +``` + +If everything worked as intended, you should see the system boot up. Once the system is booted, opening http://localhost:8080 in a browser should display the "Hello World!" message. + +To also build the new image on GitHub Actions, modify the `.github/workflows/build.yml` file. + +Change the _build_ step to include the `nginx` feature you just created, and upload the built image to GitHub's artifact storage: + +```diff +diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml +index 181a646..9e4261e 100644 +--- a/.github/workflows/build.yml ++++ b/.github/workflows/build.yml +@@ -13,4 +13,8 @@ jobs: + steps: + - uses: actions/checkout@v3 + - name: Build the image +- run: ./build base ++ run: ./build nginx ++ - uses: actions/upload-artifact@v4 ++ with: ++ name: my-linux-image ++ path: .build/ +``` + +Commit and push your changes and GitHub will build the image for you. + +You have successfully created your first feature with the Builder and set up a CI pipeline to build the image. + +## Related Topics + + diff --git a/docs/overview/README.md b/docs/overview/README.md new file mode 100644 index 0000000..e6751e7 --- /dev/null +++ b/docs/overview/README.md @@ -0,0 +1,89 @@ +--- +title: "Builder" +description: An overview of the Builder Tool +related_topics: + - /reference/supporting_tools/builder.md + - /reference/features/ + - /how-to/custom-feature +migration_status: "done" +migration_issue: "https://github.com/gardenlinux/gardenlinux/issues/4627" +migration_stakeholder: "@tmangold, @yeoldegrove, @ByteOtter" +migration_approved: false +github_org: gardenlinux +github_repo: python-gardenlinux-lib +github_source_path: docs/overview/index.md +github_target_path: "docs/reference/supporting_tools/builder.md" +--- + +# Builder + +## Overview + +The Builder is a powerful tool for effortlessly building Linux system images based on config directories. It serves as the primary build tooling for the [gardenlinux](https://github.com/gardenlinux/gardenlinux) project. + +By default, the Builder runs inside rootless Podman, enabling building without requiring elevated permissions. + +## Requirements + +The Builder has minimal dependencies and only requires a working container engine. We recommend using rootless Podman. Please refer to the [Podman rootless setup guide](https://github.com/containers/podman/blob/main/docs/tutorials/rootless_tutorial.md) for instructions on setting it up. + +## Usage + +To utilize the Builder, follow these steps: + +1. Download the latest version of the [build script](https://github.com/gardenlinux/builder/releases/download/latest/build). +2. Run the build script within a config directory. + +```shell +wget https://github.com/gardenlinux/builder/releases/download/latest/build +./build ${target} +``` + +By default, the Builder uses `podman` as the container engine. If you prefer using a different container engine, you can specify it using the `--container-engine` option. + +If you decide to use `docker` on a system restricting unprivileged user namespaces with apparmor (e.g. Ubuntu 23.10 or newer) an apparmor profile allowing `userns` is required. This can be automatically created and selected by the Builder by opting in to the permanent system change. You can avoid this by: + +- Using `podman` +- Passing a custom profile using the `--apparmor-profile` option +- Using a system not restricting unprivileged user namespaces + +## Config Directory + +A config directory serves as the input for the Builder and is used to create a Linux system image. It consists of the following components: + +- **`features` directory**: Contains sub-directories for each feature. You can create your own features by referring to [features.md](/reference/features/). + +- **`cert` directory** (optional): If you plan to use secure boot, include a `cert` directory. + +In addition to the above components, your configuration directory must include the following configuration scripts: + +- `get_commit`: This script should output the Git commit used to tag the build artifacts. +- `get_repo`: This script should output the apt package repository to use. +- `get_timestamp`: This script should output the timestamp to be used instead of the real system time, ensuring reproducibility of builds. +- `get_version`: This script should output the version of the package repository to use. For example, use `forky` for Debian or `today` for Garden Linux. +- `keyring.gpg`: The PGP key used to validate the package repository. For Debian, you can obtain this key from the [debian-archive-keyring](https://packages.debian.org/forky/debian-archive-keyring) package. + +For a quick start guide on setting up your own config directory with your own features checkout [custom-feature.md](/how-to/custom-feature.md). + +### Example Config Directory + +If you're new to configuring the Builder, you can find a minimal example config directory at [gardenlinux/builder_example](https://github.com/gardenlinux/builder_example). For a more comprehensive example, refer to the main [gardenlinux](https://github.com/gardenlinux/gardenlinux) repository. + +Feel free to explore these examples to gain a better understanding of how to effectively structure your own config directory. + +## Local Development + +To test changes made to the builder locally you can simply create a symlink to the build script inside the builder directory inside a config directory. This will automatically be detected by the build script and the builder re-build iff necessary. + +e.g.: if you have the gardenlinux and builder repos both inside the same parent directory and you want to work on the builder you would do the following: + +``` +cd gardenlinux +ln -f -s ../builder/build build +``` + +Now you can make your modifications inside the builder directory and running `./build ${target}` inside the gardenlinux repo will use the local builder, rebuilding the build container if necessary. + +## Related Topics + + diff --git a/docs/reference/builder.md b/docs/reference/builder.md new file mode 100644 index 0000000..98be12b --- /dev/null +++ b/docs/reference/builder.md @@ -0,0 +1,163 @@ +--- +title: "Builder - Command-Line Interface" +description: "Reference for Builder's ./build and ./cert/build script options, supported architectures, and container engines" +related_topics: + - /explanation/builder + - /explanation/flavors + - /explanation/boot-modes + - /explanation/secure-boot + - /reference/builder + - /how-to/secure-boot + - /how-to/choosing-flavors + - /how-to/getting-images +migration_status: "done" +migration_issue: "https://github.com/gardenlinux/gardenlinux/issues/4627" +migration_stakeholder: "@tmang0ld, @yeoldegrove, @ByteOtter" +migration_approved: false +github_org: gardenlinux +github_repo: builder +github_source_path: docs/reference/builder.md +github_target_path: docs/reference/builder.md +--- + +# Build Script Reference + +This page is a reference for the options accepted by the `./build` and `./cert/build` scripts. +For step-by-step instructions, see [Building Images](/how-to/building-images). For conceptual +background, see [Builder](/explanation/builder). + +## `./build` + +### Synopsis + +``` +./build [OPTIONS] [ ...] +``` + +Remaining arguments after all options are consumed are forwarded directly to `make` inside the +builder container. This means standard `make` flags such as `-j` are valid here. + +### Options + +| Option | Argument | Default | Description | +|---|---|---|---| +| `--container-engine` | `podman` \| `sudo podman` \| `docker` | `podman` | Container engine used to launch the builder container. See [Supported Container Engines](#supported-container-engines). | +| `--container-image` | `` | Pinned `ghcr.io/gardenlinux/builder:` | Overrides the builder container image. Pass `localhost/builder` to build and use a local builder image from the repository. | +| `--container-run-opts` | `` | Security and memory defaults | Replaces the entire set of container run options with a custom shell-quoted string. This overrides the default memory limit, seccomp, AppArmor, and label settings. Use with care. | +| `--target` | `` | `.build` | Directory where build artifacts are written. Created automatically if it does not exist. | +| `--privileged` | *(none)* | Off | Runs the builder container with `--privileged` and passes `--second-stage` to the container entrypoint. Required for two-stage builds that need elevated privileges inside the container. | +| `--kms` | *(none)* | Off | Forwards AWS credential environment variables (`AWS_DEFAULT_REGION`, `AWS_REGION`, `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_SESSION_TOKEN`) into the build. Required when Secure Boot key operations during the build use AWS Key Management Service. | +| `--allow-frankenstein` | *(none)* | Off | Disables the [single-platform enforcement](/reference/adr/0020-enforce-single-platform-by-default-in-builder) by passing `ALLOW_FRANKENSTEIN=1` to the build. Allows composing a flavor from multiple platform features. See [ADR 0020](/reference/adr/0020-enforce-single-platform-by-default-in-builder) for the rationale. | +| `--resolve-cname` | *(none)* | Off | Resolves the given flavor argument to its full canonical name (CNAME) including the 8-character commit hash, prints it to stdout, and exits. Requires exactly one flavor argument. Useful for generating reproducible artifact names in scripts. | +| `--print-container-image` | *(none)* | Off | Prints the pinned builder container image reference to stdout and exits immediately. Useful for scripts that need to pull or inspect the builder image independently of running a build. | +| `--apparmor-profile` | `` | Auto-detected | Applies the named AppArmor profile to the builder container, replacing the default `apparmor=unconfined`. If not set and Docker is used on a host with `kernel.apparmor_restrict_unprivileged_userns = 1`, the script detects the restriction and offers to create a `builder` profile at `/etc/apparmor.d/builder` automatically. | + +### `make` flags + +Any argument not consumed by the option parser above is forwarded to `make` inside the builder +container. The most commonly used `make` flag is: + +| Flag | Argument | Description | +|---|---|---| +| `-j` | `` | Number of flavors to build in parallel. Each parallel thread requires at least 8 GiB of RAM. There are no safeguards against memory exhaustion. | + +### Flavor Syntax + +``` +[-...]-[] +``` + +| Component | Position | Required | Description | +|---|---|---|---| +| `` | First | Yes | Target platform. See [Supported Platforms](#supported-platforms). | +| `` | Middle | No | One or more features from the `features/` directory, separated by `-`. Features whose names begin with `_` are appended directly without a preceding `-`. | +| `` | Last | No | Target CPU architecture. Defaults to the native host architecture. See [Supported Architectures](#supported-architectures). | + +### Supported Architectures + +| Value | Description | +|---|---| +| `amd64` | 64-bit x86 | +| `arm64` | 64-bit Arm (AArch64) | + +:::info +[Cross-architecture builds](/explanation/builder#cross-architecture-builds) require +[binfmt_misc](https://docs.kernel.org/admin-guide/binfmt-misc.html) handlers and QEMU +user-mode emulation (`qemu-user-static`) on the build host. +::: + +### Supported Platforms + +| Platform | Category | Description | +|---|---|---| +| [`ali`](/reference/features/ali) | Public cloud | Alibaba Cloud | +| [`aws`](/reference/features/aws) | Public cloud | Amazon Web Services | +| [`azure`](/reference/features/azure) | Public cloud | Microsoft Azure | +| [`bare`](/reference/features/bare) | Container | Bare (distroless) container base image | +| [`container`](/reference/features/container) | Container | Full container base image | +| [`gcp`](/reference/features/gcp) | Public cloud | Google Cloud Platform | +| [`gdch`](/reference/features/gdch) | Public cloud | Google Distributed Cloud Hosted | +| [`kvm`](/reference/features/kvm) | Hypervisor | KVM/QEMU virtualization | +| [`baremetal`](/reference/features/baremetal) | Bare metal | Physical server deployment | +| [`openstack`](/reference/features/openstack) | Private cloud | OpenStack | +| [`vmware`](/reference/features/vmware) | Hypervisor | VMware vSphere/ESXi | +| [`lima`](/reference/features/lima) | Local dev | Lima for macOS and Linux | + +### Supported Container Engines + +| Value | Description | +|---|---| +| `podman` | Rootless Podman (recommended). Requires a working rootless Podman setup. See the [Podman rootless setup guide](https://github.com/containers/podman/blob/main/docs/tutorials/rootless_tutorial.md). | +| `sudo podman` | Podman with `sudo` for elevated privileges. | +| `docker` | Docker Engine. On systems where `kernel.apparmor_restrict_unprivileged_userns = 1`, an AppArmor profile is required (see `--apparmor-profile` above). | + +### `build.config` File + +If a `build.config` file exists in the working directory, it is sourced by `./build` before the +container is launched. It can override build variables that are otherwise set to their defaults. + +| Variable | Default | Description | +|---|---|---| +| `tempfs_size` | `2G` | Size of the temporary filesystem used during the build, passed to the builder as `TEMPFS_SIZE`. Increase this if builds fail due to insufficient temporary space. | + +--- + +## `./cert/build` + +The `./cert/build` script generates the Secure Boot certificate chain required for +[`_trustedboot`](/reference/features/_trustedboot), [`_tpm2`](/reference/features/_tpm2), +and [`_secureboot`](/reference/features/_secureboot) images. + +### Synopsis + +``` +./cert/build [OPTIONS] [clean] +``` + +### Options + +| Option / Argument | Description | +|---|---| +| `--kms` | Store private keys in AWS Key Management Service instead of local files. Requires valid AWS credentials via standard AWS environment variables. | +| `clean` | Remove previously generated certificate files before regenerating. Use this if a previous `./cert/build` run failed or produced incomplete output. | + +### Output Files + +After a successful run, the `cert/` directory contains: + +| File | Format | Purpose | +|---|---|---| +| `secureboot.pk.esl` | ESL | AWS external enrollment — Platform Key | +| `secureboot.kek.esl` | ESL | AWS external enrollment — Key Exchange Key | +| `secureboot.db.esl` | ESL | AWS external enrollment — Signature Database | +| `secureboot.pk.der` | DER | GCP, Azure — Platform Key | +| `secureboot.kek.der` | DER | GCP, Azure — Key Exchange Key | +| `secureboot.db.der` | DER | GCP, Azure — Signature Database | +| `secureboot.db.crt` | PEM | Image signing during build | +| `secureboot.aws-efivars` | Binary blob | Pre-filled AWS UEFI variable store for external Secure Boot enrollment | + +For cloud-provider-specific enrollment steps, see [Deploying Secure Boot Images](/how-to/secure-boot). + +## Related Topics + + diff --git a/docs/features.md b/docs/reference/features.md similarity index 80% rename from docs/features.md rename to docs/reference/features.md index e5eae9b..3d49d16 100644 --- a/docs/features.md +++ b/docs/reference/features.md @@ -1,18 +1,35 @@ +--- +title: "Features" +description: Detailed specification of Features +related_topics: + - /reference/supporting_tools/builder.md + - /reference/features/ + - /how-to/custom-feature +migration_status: "adapt" +migration_issue: "https://github.com/gardenlinux/gardenlinux/issues/4600" +migration_stakeholder: "@tmangold, @yeoldegrove, @ByteOtter" +migration_approved: false +github_org: gardenlinux +github_repo: builder +github_source_path: docs/reference/features.md +github_target_path: docs/reference/features/index.md +--- + # Features Each feature must contain an `info.yaml` file that adheres to the following structure: ## `info.yaml` file structure: -- `description`: (*optional*) A string explaining the purpose or functionality of the feature. +- `description`: (_optional_) A string explaining the purpose or functionality of the feature. - `type`: Can be one of the following: - `platform` - `element` - `flag` - While the builder does not make any technical distinctions between these feature types, it is recommended that each image uses only one `platform`, and `flag` should be used for minor changes without including other features. -- `features`: (*optional*) A sub-structure that contains related features. - - `include`: (*optional*) A list of features that will automatically be included if this feature is selected. - - `exclude`: (*optional*) A list of features that are incompatible with this feature. If any of these features were implicitly included from another feature, they will be removed from the feature set. If they were explicitly provided as part of the target, the build will fail. +- `features`: (_optional_) A sub-structure that contains related features. + - `include`: (_optional_) A list of features that will automatically be included if this feature is selected. + - `exclude`: (_optional_) A list of features that are incompatible with this feature. If any of these features were implicitly included from another feature, they will be removed from the feature set. If they were explicitly provided as part of the target, the build will fail. Here's an example of an `info.yaml` file: @@ -88,14 +105,14 @@ The format is: - ``: these are additional args parsed by `makepart`. supported options: - `type=`: overwrite the default GPT partition type - `size=`: instead of dynamically calculating the ideal size for the partition set it explicitly - - `syslinux`: mark this as a partition on whith to install syslinux to the FAT32 boot sectors + - `syslinux`: mark this as a partition on which to install syslinux to the FAT32 boot sectors - `final_partition`: ensure this partition is placed at the end of the partition table regardless of default sorting. if you don't know why you'd need this you likely shouldn't use it! The `fstab` can be defined with an equally named file in one and only one feature. Additionally, other features can apply modifications to this base `fstab`. For this features can define executable `fstab.mod` scripts. -These scripts are executed in the same order as regular config scripts, each recieving the output of the previous script as its input. -The first script in the series recieves the init `fstab` file. +These scripts are executed in the same order as regular config scripts, each receiving the output of the previous script as its input. +The first script in the series receives the init `fstab` file. The output of the final file will be used as the effective `fstab`. > [!IMPORTANT] @@ -115,3 +132,7 @@ They take the image artifact created by an image script and output a different f Scripts of the form `convert.` get the raw image as input and produce a `.` output. Scripts of the form `convert.~` get `.` as input and produces `.` as output. The second form is only useful for advanced use cases, if you are not aware of one, you'll probably never need it! + +## Related Topics + +