Document GCS, Azure, IBM, and Oracle patch storage backends

docs/.custom_wordlist.txt

@@ -19,6 +19,7 @@ Canonical('s)?
 Charmcraft
 charmstore
 cjk
+COS
 crypto
 cryptographically
 CVEs
@@ -29,19 +30,23 @@ Downloader
 dvipng
 enablement
 Enablement
+Entra
 filepath
 firewalled
 fonts
 freefont
 Furo
 gcp
+GCE
+GCS
 github
 GitHub
 gkeop
 glibc
 GPG
 gyre
 hostnames
+HMAC
 hotfix(es)?
 html
 https
@@ -75,6 +80,7 @@ MyST
 npm
 Numpy
 oem
+OCI
 Open Graph
 openapi
 otf
@@ -93,6 +99,7 @@ pymarkdown
 pymarkdownlnt
 QEMU
 Read the Docs
+rclone
 readthedocs
 redeclare
 rediraffe
@@ -104,6 +111,7 @@ Rockcraft
 rollout
 rollouts
 rst
+rsync
 rtd
 SCons
 sequenceDiagram

docs/Makefile

@@ -120,6 +120,7 @@ lint-md: pymarkdownlnt-install
 			--recurse \
 			--exclude=$(DEV_DIR)/** \
 			--exclude=$(DOCS_VENVDIR)/** \
+			--exclude=$(DOCS_BUILDDIR)/** \
 			$(DOCS_SOURCEDIR); \
 	status=$$?; \
 	if [ $$status -eq 1 ]; then \

docs/server/how-to-guides/security/harden-your-deployment.md

@@ -41,7 +41,7 @@ In [federated deployments](/server/how-to-guides/patch-management/chain-livepatc
 
 TLS for PostgreSQL connections is supported through the connection string. Include `sslmode=verify-full` (or at minimum `sslmode=require`) in the database connection string to encrypt traffic between the server and PostgreSQL.
 
-For charm deployments, database credentials are obtained through Juju relations (`database` interface with the [PostgreSQL K8s Charm](https://charmhub.io/postgresql-k8s)). TLS for the database connection is managed by the PostgreSQL charm's [TLS integration](https://canonical-charmed-postgresql-k8s.readthedocs-hosted.com/14/tutorial/index.html#enable-encryption-with-tls). The Livepatch charm does not expose separate database credential configuration -- all credentials are exchanged through the Juju relation protocol.
+For charm deployments, database credentials are obtained through Juju relations (`database` interface with the [PostgreSQL K8s Charm](https://charmhub.io/postgresql-k8s)). TLS for the database connection is managed by the PostgreSQL charm's [TLS integration](https://canonical.com/data/postgresql/docs/16/how-to/network-and-encryption/enable-tls/). The Livepatch charm does not expose separate database credential configuration -- all credentials are exchanged through the Juju relation protocol.
 
 ## Encrypt patch storage connections
 
@@ -180,4 +180,4 @@ To reset the charm to its default configuration:
 juju config canonical-livepatch-server-k8s --reset <key>
 ```
 
-To fully reset, remove the application and redeploy.
+To fully reset, remove the application and redeploy.

docs/server/reference/patch-storage/index.md

@@ -15,10 +15,14 @@ Livepatch Server supports several different drivers for storing patch files down
 2. Swift
 3. S3 (and compatible implementations, e.g. minio)
 4. Postgresql
+5. Google Cloud Storage (GCS)
+6. Azure Blob Storage
+7. IBM Cloud Object Storage
+8. Oracle Cloud Infrastructure (OCI) Object Storage
 
 The filesystem patch store is easiest to deploy and suits most configurations. However, if there is a need to scale out the Livepatch Server such as have multiple Livepatch Servers running to handle the load, the filesystem patch store should not be used.
 
-In case there is a need to scale out Livepatch on-prem, use the s3, postgresql or swift patch stores. Any patch store should have enough space for storing live kernel patches - currently at least 45GB for all patches, see [this guide](/server/reference/patch-management/patch-sync-filters.md) to filter patches sent to your on-prem instance to specific kernel variants/architectures and lower this requirement.
+In case there is a need to scale out Livepatch on-prem, use one of the object storage backends (s3, gcs, azure, ibm, oracle), postgresql, or swift patch stores. Any patch store should have enough space for storing live kernel patches - currently at least 45GB for all patches, see [this guide](/server/reference/patch-management/patch-sync-filters.md) to filter patches sent to your on-prem instance to specific kernel variants/architectures and lower this requirement.
 
 See the [patch storage](/server/reference/platform/configuration.md) config for all available parameters.
 
@@ -29,4 +33,9 @@ See the [patch storage](/server/reference/platform/configuration.md) config for
 :hidden:
 
 Use S3 for patch storage <use-s3-for-patch-storage.md>
+Use GCS for patch storage <use-gcs-for-patch-storage.md>
+Use Azure for patch storage <use-azure-for-patch-storage.md>
+Use IBM COS for patch storage <use-ibm-for-patch-storage.md>
+Use OCI Object Storage for patch storage <use-oracle-for-patch-storage.md>
+Migrate patches between storage backends <migrating-patch-storage.md>
 ```

docs/server/reference/patch-storage/migrating-patch-storage.md

@@ -0,0 +1,43 @@
+---
+myst:
+  html_meta:
+    description: "Migrate patches between storage backends - learn about this topic in Livepatch on-prem."
+---
+
+
+(server-reference-migrating-patch-storage)=
+
+# Migrating patches between storage backends
+
+This guide covers migrating between the file-based patch storage backends (filesystem, S3, GCS, Azure, IBM, Oracle, Swift). Each of these stores patches as a flat set of files, each keyed by its filename, so the same set of patch files works unmodified on any of them: migrating between backends is simply a matter of copying the patch files across, then updating [`patch-storage.type`](/server/reference/platform/configuration.md) and its associated options to point at the new backend.
+
+The Postgres backend (`patch-storage.postgres-connection-string`) is not covered here, since it stores patches as rows in a database rather than as files, and cannot be migrated with `rsync` in the same way.
+
+The recommended tool for copying patches is `rsync`, since it performs a checksum-verified copy. Disable patch synchronisation in the config first (so no new patches arrive, and avoid running any manual syncs), then run `rsync` once to copy the existing patches across.
+
+## Filesystem to filesystem
+
+If either the source or destination (or both) is the filesystem backend, `rsync` can be used directly, locally or over SSH:
+
+```bash
+rsync -avz --progress /var/snap/canonical-livepatch-server/common/patches/ user@remote-host:/path/to/new/patches/
+```
+
+## Migrating to or from object storage
+
+For the object storage backends (S3, GCS, Azure, IBM, Oracle), mount the bucket or container as a local directory with [rclone](https://rclone.org/), which supports all of these backends (including any S3-compatible endpoint, such as IBM COS or MinIO), then `rsync` into or out of the mount as if it were a normal directory.
+
+1. Disable [`patch-sync`](/server/reference/patch-management/patch-sync-filters.md) in the config so no new patches are synced to the on-prem server, and avoid running any manual syncs, while the migration is in progress.
+2. Install rclone and configure a remote for the bucket/container, following [rclone's documentation](https://rclone.org/docs/) for the relevant backend (`s3`, `google cloud storage`, `azureblob`, or `oracle-object-storage`).
+3. Mount the remote:
+   ```bash
+   mkdir -p /mnt/livepatch-patches
+   rclone mount <remote-name>:<bucket-or-container> /mnt/livepatch-patches --daemon
+   ```
+4. Run `rsync` to copy the existing patches across:
+   ```bash
+   rsync -avz --progress /var/snap/canonical-livepatch-server/common/patches/ /mnt/livepatch-patches/
+   ```
+5. Verify the file counts match between source and destination, then update the [patch storage config](/server/reference/platform/configuration.md) to the new backend and re-enable patch-sync.
+
+The same approach applies in reverse (object storage to filesystem, or between two object storage backends) by mounting both sides and running `rsync` between the two mount points.

docs/server/reference/patch-storage/use-azure-for-patch-storage.md

@@ -0,0 +1,20 @@
+---
+myst:
+  html_meta:
+    description: "Use Azure for patch storage - learn about this topic in Livepatch on-prem."
+---
+
+
+(server-reference-livepatch-on-prem-with-azure-patch-storage)=
+
+# Livepatch on-prem with Azure Blob Storage patch storage
+
+To configure this, follow these steps:
+
+- Create a Blob Storage container in the preferred region (best if the region is the same as the deployment's). Care needs to be taken to make the container not publicly writable, as this would pose a significant security risk.
+- Choose an authentication method: a storage account name/key pair, a connection string, an Entra ID (Azure AD) service principal, or a managed identity bound to the VM.
+- Configure the relevant Azure [config options](/server/reference/platform/configuration.md).
+
+Once this is configured, Livepatch will store and retrieve patch files from the Azure Blob Storage container.
+
+`patch-storage.azure-account-name` is not required when `patch-storage.azure-connection-string` is set, since the connection string already carries the account name and key. When account key, connection string, and service principal credentials are all omitted, a managed identity bound to the VM is used instead (the system-assigned identity by default, or a user-assigned identity selected via `patch-storage.azure-managed-identity-client-id`).

docs/server/reference/patch-storage/use-gcs-for-patch-storage.md

@@ -0,0 +1,20 @@
+---
+myst:
+  html_meta:
+    description: "Use GCS for patch storage - learn about this topic in Livepatch on-prem."
+---
+
+
+(server-reference-livepatch-on-prem-with-gcs-patch-storage)=
+
+# Livepatch on-prem with Google Cloud Storage patch storage
+
+To configure this, follow these steps:
+
+- Create a GCS bucket in the preferred region (best if the region is the same as the deployment's). Care needs to be taken to make the bucket not publicly writable, as this would pose a significant security risk.
+- Create a service account with permissions to read and write objects in that bucket, or rely on the credentials already bound to the VM (e.g. the GCE metadata service, or workload identity) if it is granted the required permissions.
+- Configure the relevant GCS [config options](/server/reference/platform/configuration.md).
+
+Once this is configured, Livepatch will store and retrieve patch files from the GCS bucket.
+
+`patch-storage.gcs-credentials-file` and `patch-storage.gcs-credentials-json` are both optional; when both are omitted, Application Default Credentials are used instead. `patch-storage.gcs-impersonate-service-account` is also optional; when set, the resolved credentials are used to impersonate that service account instead of being used directly.

docs/server/reference/patch-storage/use-ibm-for-patch-storage.md

@@ -0,0 +1,20 @@
+---
+myst:
+  html_meta:
+    description: "Use IBM Cloud Object Storage for patch storage - learn about this topic in Livepatch on-prem."
+---
+
+
+(server-reference-livepatch-on-prem-with-ibm-patch-storage)=
+
+# Livepatch on-prem with IBM Cloud Object Storage patch storage
+
+To configure this, follow these steps:
+
+- Create a COS bucket in the preferred region (best if the region is the same as the deployment's). Care needs to be taken to make the bucket not publicly writable, as this would pose a significant security risk.
+- Choose an authentication method: HMAC credentials, an IAM API key with a service instance ID, or the ambient VPC Instance Metadata Service if the VM is granted the required trusted profile.
+- Configure the relevant IBM [config options](/server/reference/platform/configuration.md).
+
+Once this is configured, Livepatch will store and retrieve patch files from the COS bucket.
+
+Credentials are resolved in the following order of preference: HMAC keys (`patch-storage.ibm-access-key`/`patch-storage.ibm-secret-key`), then an IAM API key (`patch-storage.ibm-api-key`/`patch-storage.ibm-service-instance-id`), then, if none are configured, the ambient VPC Instance Metadata Service (optionally selecting a trusted profile via `patch-storage.ibm-trusted-profile-id`).

docs/server/reference/patch-storage/use-oracle-for-patch-storage.md

@@ -0,0 +1,20 @@
+---
+myst:
+  html_meta:
+    description: "Use OCI Object Storage for patch storage - learn about this topic in Livepatch on-prem."
+---
+
+
+(server-reference-livepatch-on-prem-with-oracle-patch-storage)=
+
+# Livepatch on-prem with Oracle Cloud Infrastructure Object Storage patch storage
+
+To configure this, follow these steps:
+
+- Create an Object Storage bucket in the preferred region (best if the region is the same as the deployment's). Care needs to be taken to make the bucket not publicly writable, as this would pose a significant security risk.
+- Choose an authentication method: an OCI config file/profile, or instance principal authentication if the compute instance is granted the required IAM policies.
+- Configure the relevant OCI [config options](/server/reference/platform/configuration.md).
+
+Once this is configured, Livepatch will store and retrieve patch files from the OCI Object Storage bucket.
+
+`patch-storage.oracle-config-file` (and `patch-storage.oracle-profile`) are optional; when omitted, instance principal authentication is used instead, bound to the OCI compute instance the server runs on.

docs/server/reference/platform/configuration.md

@@ -143,7 +143,7 @@ The following values configure how the server manages patch storage. See the [pa
 
 | Name | Description | Values |
 | ---- | ----------- | ------ |
-| `patch-storage.type` | Storage backend type for on-premises patch synchronisation. | `oneof: filesystem, swift, postgres, s3` |
+| `patch-storage.type` | Storage backend type for on-premises patch synchronisation. | `oneof: filesystem, swift, postgres, s3, gcs, azure, ibm, oracle` |
 | `patch-storage.filesystem-path` | Directory path for filesystem storage. | `string` |
 | `patch-storage.swift-username` | Swift account username. | `string` |
 | `patch-storage.swift-api-key` | Swift API key. | `string` |
@@ -154,11 +154,38 @@ The following values configure how the server manages patch storage. See the [pa
 | `patch-storage.swift-region` | Swift region. | `string` |
 | `patch-storage.postgres-connection-string` | PostgreSQL connection string. Can be left blank in charmed deployments to use Juju relations. | `string` |
 | `patch-storage.s3-bucket` | S3 bucket for storing patches. | `string` |
-| `patch-storage.s3-endpoint` | S3 endpoint URL. | `string` |
+| `patch-storage.s3-endpoint` | S3 endpoint URL (only needed for non-AWS S3-compatible services, e.g. MinIO). | `string` |
 | `patch-storage.s3-region` | AWS region for S3. | `string` |
 | `patch-storage.s3-secure` | Whether to use secure transfers. | `bool` |
-| `patch-storage.s3-access-key` | AWS access key. | `string` |
-| `patch-storage.s3-secret-key` | AWS secret key. | `string` |
+| `patch-storage.s3-use-path-style` | Whether to use path-style bucket addressing. Defaults to `false` (virtual-hosted-style); typically needs to be `true` for non-AWS S3-compatible services (e.g. MinIO, Ceph). | `bool` |
+| `patch-storage.s3-access-key` | AWS access key (optional; defaults to the VM's ambient credentials, e.g. an EC2 instance role). | `string` |
+| `patch-storage.s3-secret-key` | AWS secret key (optional; see `patch-storage.s3-access-key`). | `string` |
+| `patch-storage.s3-assume-role-arn` | IAM role ARN to assume via STS, using the resolved credentials above (optional). | `string` |
+| `patch-storage.gcs-bucket` | Google Cloud Storage bucket for storing patches. | `string` |
+| `patch-storage.gcs-credentials-file` | Path to a service account JSON key file (optional; defaults to Application Default Credentials). | `string` |
+| `patch-storage.gcs-credentials-json` | Inline service account JSON key (optional; see `patch-storage.gcs-credentials-file`). | `string` |
+| `patch-storage.gcs-impersonate-service-account` | Email of a service account to impersonate, using the resolved credentials above (optional). | `string` |
+| `patch-storage.azure-container` | Azure Blob Storage container for storing patches. | `string` |
+| `patch-storage.azure-account-name` | Azure storage account name (not required when `azure-connection-string` is set, which already carries the account name). | `string` |
+| `patch-storage.azure-account-key` | Azure storage account key (optional; defaults to a managed identity bound to the VM). | `string` |
+| `patch-storage.azure-connection-string` | Azure storage connection string (optional alternative to account name/key). | `string` |
+| `patch-storage.azure-tenant-id` | Entra ID tenant ID, for explicit service principal auth (optional; requires `patch-storage.azure-client-id` and `patch-storage.azure-client-secret`). | `string` |
+| `patch-storage.azure-client-id` | Entra ID application (client) ID, for explicit service principal auth (optional). | `string` |
+| `patch-storage.azure-client-secret` | Entra ID client secret, for explicit service principal auth (optional). | `string` |
+| `patch-storage.azure-managed-identity-client-id` | Client ID of a user-assigned managed identity to use instead of the VM's system-assigned identity (optional). | `string` |
+| `patch-storage.ibm-bucket` | IBM Cloud Object Storage bucket for storing patches. | `string` |
+| `patch-storage.ibm-endpoint` | IBM Cloud Object Storage endpoint. | `string` |
+| `patch-storage.ibm-region` | IBM Cloud Object Storage region. | `string` |
+| `patch-storage.ibm-access-key` | IBM COS HMAC access key (optional; see `patch-storage.ibm-api-key`). | `string` |
+| `patch-storage.ibm-secret-key` | IBM COS HMAC secret key (optional; see `patch-storage.ibm-access-key`). | `string` |
+| `patch-storage.ibm-api-key` | IBM Cloud IAM API key (optional; defaults to the ambient VPC Instance Metadata Service). | `string` |
+| `patch-storage.ibm-service-instance-id` | IBM COS resource instance ID, required when using `patch-storage.ibm-api-key`. | `string` |
+| `patch-storage.ibm-trusted-profile-id` | Trusted profile ID to assume via the ambient VPC Instance Metadata Service (optional; defaults to the instance's default trusted profile). | `string` |
+| `patch-storage.oracle-bucket` | OCI Object Storage bucket for storing patches. | `string` |
+| `patch-storage.oracle-namespace` | OCI Object Storage namespace. | `string` |
+| `patch-storage.oracle-region` | OCI region. | `string` |
+| `patch-storage.oracle-config-file` | Path to an OCI config file (optional; defaults to instance principal authentication). | `string` |
+| `patch-storage.oracle-profile` | Profile to use within `patch-storage.oracle-config-file` (optional). | `string` |
 
 ### Patch cache