Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ docker_cmd ?= docker
docker_opts ?= --rm --tty --user "$$(id -u)"

vale_cmd ?= $(docker_cmd) run $(docker_opts) --volume "$${PWD}"/docs/modules/ROOT/pages:/pages --workdir /pages ghcr.io/vshn/vale:2.15.5 --minAlertLevel=error /pages
sdd_vale_cmd ?= $(docker_cmd) run $(docker_opts) --volume "$${PWD}"/docs/modules/SDDs/pages:/pages --workdir /pages ghcr.io/vshn/vale:2.15.5 --minAlertLevel=error /pages
preview_cmd ?= $(docker_cmd) run --rm --publish 35729:35729 --publish 2020:2020 --volume "${PWD}":/preview/antora ghcr.io/vshn/antora-preview:3.1.2.3 --antora=docs --style=syn

.PHONY: all
Expand All @@ -16,9 +17,13 @@ clean:
rm -rf $(out_dir) '?' .cache

.PHONY: check
check:
check: check-sdds
$(vale_cmd)

.PHONY: check-sdds
check-sdds:
$(sdd_vale_cmd)

.PHONY: preview
preview:
$(preview_cmd)
10 changes: 10 additions & 0 deletions docs/modules/SDDs/pages/.vale.ini
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
StylesPath = /styles
MinAlertLevel = warning # suggestion, warning or error

# Only check Asciidoc files
[*.adoc]
BasedOnStyles = Microsoft
Microsoft.GenderBias = suggestion
Openly.GenderBias = suggestion
Microsoft.Dashes = suggestion
Microsoft.Quotes = suggestion
2 changes: 1 addition & 1 deletion docs/modules/SDDs/pages/0030-argocd-multitenancy.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ However, if another team wants to deploy applications on the cluster through Pro
Each team's root application is managed independently from the default root application (`root`).

The bootstrap process for the additional root applications is initiated by defining the team in the configuration hierarchy in parameter `syn`.
This parameter is not associated with a specific component and becomes a reserved parameter name which components can't use.
This parameter isn't associated with a specific component and becomes a reserved parameter name which components can't use.
The parameter has two fields `owner` and `teams`.
Field `owner` makes the team who owns the cluster explicit.
This allows the implementation to ensure that the owning team's applications are always managed through the default `root` application, even if that team has some explicit assignments for applications in `syn.teams`.
Expand Down
12 changes: 6 additions & 6 deletions docs/modules/SDDs/pages/0031-component-version-tracking.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -17,17 +17,17 @@ This describes how we want to extend the Lieutenant API, CRD, and operator to al
== Motivation

Currently, component versions used in a Commodore compile are tracked in the cluster catalog repository.
They are stored in the commit message.
They're stored in the commit message.

This approach has several drawbacks:
* The version information is not easily accessible programmatically.
* The version information isn't easily accessible programmatically.
* To get an overview one needs to find every repository and check the commit history.

To improve this situation, we want to introduce a central component version tracking system.
The system should be centralized and accessible programmatically.

Lieutenant is the central component for managing the cluster catalogs and already has a REST API.
It is backed by a CRD which already stores some state information and cluster metadata.
This API is backed by a CRD which already stores some state information and cluster metadata.

We want to extend the Lieutenant API, CRD, and operator to store version information for each Commodore compile.

Expand Down Expand Up @@ -66,7 +66,7 @@ Each version entry contains the following fields:
Can be a branch name, tag, or commit hash.
The `version` can point to different commits depending on when the compile was done.
* `gitSha`: The commit hash of the commit referenced by the `version` field
With the `gitSha` and the `url` it is possible to uniquely identify the commit.
With the `gitSha` and the `url` it's possible to uniquely identify the commit.

The `packages` and `instances` fields additionally contain:

Expand Down Expand Up @@ -143,7 +143,7 @@ syn_lieutenant_cluster_compile_meta_tenant{cluster="my-cluster", url="https://..
=== Commodore

Commodore will be extended to send updated compilation information for catalog compilations which actually push a new catalog commit.
Any other catalog compilations will not update the compile metadata on the Lieutenant API.
Any other catalog compilations won't update the compile metadata on the Lieutenant API.

Commodore will use its existing API token to push the compilation metadata to Lieutenant.

Expand All @@ -154,7 +154,7 @@ https://github.com/projectsyn/commodore/issues/563[projectsyn/commodore#563] wan
The current design with instance name as the top key allows for this.

Configuration packages::
Configuration package versions are not yet added to the commit message.
Configuration package versions aren't yet added to the commit message.
We need to add them to the commit message and the Lieutenant API.

== Alternatives
Expand Down
22 changes: 11 additions & 11 deletions docs/modules/SDDs/pages/0032-compile-pipeline.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ While we only aim to support GitLab at the moment, the architecture should be su
Having a continuous integration solution unlocks a number of benefits:
It solves the problem of configuration drift, where changes to the tenant repository might not be reflected in every cluster catalog because not all of them have since been compiled, and it lessens the burden on catalog maintainers who otherwise would need to locally compile each cluster individually.

It is already fairly straightforward to manually set up basic auto-compilation for individual tenant repositories without special support from Project Syn itself.
It's already fairly straightforward to manually set up basic automated compilation for individual tenant repositories without special support from Project Syn itself.
At VSHN, such a solution has been in use for several years.

However, certain features (such as automatic configuration of the compile pipeline) are hard to implement in a standalone fashion.
Expand All @@ -47,7 +47,7 @@ This will go hand-in-hand with the existing repository management features in Li
=== Requirements for Pipeline Configuration

Lieutenant imposes certain assumptions on the configuration of the pipeline:
Namely, the pipeline has to be set up on the tenant repository by way of adding (arbitrary) files to the repository, and it is configured through setting CI/CD variables on the repository.
Namely, the pipeline has to be set up on the tenant repository by way of adding (arbitrary) files to the repository, and it's configured through setting CI/CD variables on the repository.

In particular, Lieutenant configures the following CI/CD variables:

Expand All @@ -62,7 +62,7 @@ In particular, Lieutenant configures the following CI/CD variables:
We add two new fields to the `GitRepoTemplate` (and, by extension, the `GitRepo`) CRD, under the `.spec` key, called `accessTokenSecretName` and `ciVariables`.

The `accessTokenSecretName` field contains a reference to a secret.
If it is set, the Lieutenant operator will store an access token into this secret, which can be used to access the Git repository.
If the field is set, the Lieutenant operator will store an access token into this secret, which can be used to access the Git repository.
In the case of GitLab, this would be a Project Access Token with read-write access to the repository.

The `ciVariables` field contains a list of objects describing variable names and corresponding values.
Expand Down Expand Up @@ -94,7 +94,7 @@ We add a new field to the `Cluster` CRD, under the `.spec` key, called `enableCo

The field contains a boolean flag, which controls whether the compile pipeline should be enabled or disabled for this cluster.

It is optional; not specifying it is equivalent to setting it to `false`.
This field is optional; not specifying it defaults it to `false`.

[source,yaml]
----
Expand All @@ -111,11 +111,11 @@ spec:
We add new fields to the `Tenant` CRD: `spec.compilePipeline` and `status.compilePipeline`

The `spec.compilePipeline` field contains configuration pertaining to the automatic setup of the compile pipeline on the tenant repository.
It is optional.
This field is optional.

The `spec.compilePipeline` field contains a dict with the following fields:

* `enabled`: Boolean field which enables or disables automatic setup of compile pipelines for this tenant (regardless of whether it is enabled on the tenant's clusters).
* `enabled`: Boolean field which enables or disables automatic setup of compile pipelines for this tenant (regardless of whether automatic setup enabled on the tenant's clusters).
* `pipelineFiles`: Dictionary containing file paths as keys, and file contents as values.
These files will be added to the tenant's `gitRepoTemplate.templateFiles` by the Lieutenant operator.
This field is optional; if absent, no new template files are added to the `gitRepoTemplate`.
Expand Down Expand Up @@ -152,15 +152,15 @@ status:
The Lieutenant Operator will be extended to automatically manage the compile pipeline for repositories where this is enabled (by way of deploying the CI config file in the tenant and the `enableCompilePipeline` field on the cluster).

Since the compile pipeline has to interact with both the tenant repository as well as the cluster catalog repositories, it must be enabled on both corresponding resources for the configuration to be functional.
This way, it is possible to enable auto-compilation for some, but not all clusters on a tenant.
This way, it's possible to enable automated compilation for some, but not all clusters on a tenant.

The operator will reconcile *GitRepos* as follows:

* When `spec.accessTokenSecretName` is set, the operator generates an access token for the corresponding repository (via the repository host's API, using the API secret in `.spec.apiSecretRef`), and writes this token into a secret with the given name.
In the case of GitLab, this is a Project Access Token.
The operator also runs a scheduled job which refreshes these tokens when they are close to expiring, or when they no longer exist on the repository host.
The operator also runs a scheduled job which refreshes these tokens when they're close to expiring, or when they no longer exist on the repository host.
* The content of `.spec.ciVariables` is written to the repository's configuration on the Git host.
In the case of GitLab, it is written as CI/CD variables.
In the case of GitLab, each entry is written as a CI/CD variable.
If the content of `.spec.ciVariables` changes, the corresponding configuration on the Git host should be updated.
A scheduled job in the ooperator regularly checks for drift between `.spec.ciVariables` and the configuration on the Git host, and updates the latter if necessary.

Expand Down Expand Up @@ -196,11 +196,11 @@ If a setup already includes a bunch of tenant repositories with manually configu
+
In particular, these repositories would already have a working `.gitlab-ci.yml` that probably could be left as-is, but can also be replaced by a lieutenant-managed one.
+
Any existing manually created Project Access Tokens will be superseded by new auto-generated ones.
Any existing manually created Project Access Tokens will be superseded by new automatically generated ones.
This will lead to a bunch of now-unused tokens needing to be cleaned up, but should otherwise work without requiring extra effort.

External Catalog Repositories::
There may be cases where the catalog repositories are not hosted on the same repository host as the tenant repository, in which case API access for the purpose of creating Project Access Tokens is unavailable.
There may be cases where the catalog repositories aren't hosted on the same repository host as the tenant repository, in which case API access for the purpose of creating Project Access Tokens is unavailable.
The Commodore Compile Pipeline can still be used against such catalog repositories by specifying an SSH key to access them.
+
This can still be configured manually, and the automated configuration would not interfere.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ It additionally details the options we chose to implement.

With xref:0025-commodore-component-instantiation.adoc[SDD 0025], Commodore can instantiate a component multiple times per cluster if multiple aliases are specified.
However, all aliases currently use version information from the base component.
Commodore does not currently support specifying different component versions for each alias.
Commodore doesn't currently support specifying different component versions for each alias.

=== Goals

Expand All @@ -40,7 +40,7 @@ Support for providing different versions of Jsonnet dependencies is currently no
Jsonnet dependencies are always provided from the main (non-instantiated) version of the component.
If a component is to support multi-versioning, Jsonnet dependencies should therefore be compatible within a reasonable span of versions.

Similarly, if the component includes a Jsonnet library that is made available to other components, the version for the library is taken from the main (non-instantiated) version of the component.
Similarly, if the component includes a Jsonnet library that's made available to other components, the version for the library is taken from the main (non-instantiated) version of the component.

Finally, if a component reads another component's default values, it will always see the default values of the main (non-instantiated) version of the component.

Expand All @@ -62,7 +62,7 @@ This might force a component author to remove multi-version support in higher co
=== Automatically detecting multi-version support

We could try to automatically determine whether a component supports multi-versioning, by way of checking whether the `${_base_directory}` parameter is used in the component's compile configuration and for Jsonnet expressions that refer to downloaded data.
However, automatically declaring components to be "safe for multi-versioning" carries some risk - even if a component is configured accordingly, it is not guaranteed that its provided Jsonnet libraries and/or its Jsonnet dependencies work if used across versions.
However, automatically declaring components to be "safe for multi-versioning" carries some risk - even if a component is configured accordingly, there's no guarantee that its provided Jsonnet libraries and/or its Jsonnet dependencies work if used across versions.
Additionally, so far we've defined explicit flags for components to indicate that they support optional features, such as multiple instances.
For all of those reasons, we've decided that component authors must explicitly declare whether multi-versioning is safe to use for their component.

Expand All @@ -77,7 +77,7 @@ Making this change has no direct impact on the implementation of the multi-versi

It might be possible to allow multiple versions of a component's Jsonnet libraries to be made available to other components.
Other components would have to refer to the libraries using a specific instance's name.
This would change how components refer to libraries, and would cause components to be somewhat aware of the context in which they are used in order to refer to the correct instances.
This would change how components refer to libraries, and would cause components to be somewhat aware of the context in which they're used in order to refer to the correct instances.
Since components should be modular and reusable, introducing this sort of coupling goes against the design principles of Project Syn.

== References
Expand Down
Loading
Loading