Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
3fd5756
feat(ci): add gitlab ci core pipeline structure
universal-itengineer Jun 22, 2026
8594d30
feat(ci): add gitlab validation and scanning jobs
universal-itengineer Jun 22, 2026
a31a23a
feat(ci): add gitlab automation and changelog jobs
universal-itengineer Jun 22, 2026
807c183
fix(ci): correct gitlab ci migration config
universal-itengineer Jun 22, 2026
b12e066
fix(ci): add scripts to precache jobs
universal-itengineer Jun 23, 2026
27e55fc
fix(ci): adapt gitlab jobs for shell executor
universal-itengineer Jun 23, 2026
832b155
fix(ci): keep gitlab api logs off stdout
universal-itengineer Jun 23, 2026
493366f
fix(ci): harden gitlab helper scripts
universal-itengineer Jun 23, 2026
94ee292
fix(ci): fetch diff base for validation jobs
universal-itengineer Jun 23, 2026
59c06eb
fix(ci): correct auto-assign rules and dedupe pipelines
universal-itengineer Jun 23, 2026
69a6970
fix(ci): use gohooks task name and enable Go toolchain auto-download
universal-itengineer Jun 23, 2026
14cfe3b
fix(ci): stop svace dotenv leaking svace tag into MR build jobs
universal-itengineer Jun 23, 2026
7a12d77
fix(ci): auto-install shellcheck on runner when missing
universal-itengineer Jun 23, 2026
b2c6f93
fix(ci): fix shellcheck download URL and tarball extraction
universal-itengineer Jun 23, 2026
af9f482
fix(ci): use docker for lint:shellcheck, drop dead install-shellcheck
universal-itengineer Jun 23, 2026
1e4159d
chore: formatting yaml
universal-itengineer Jun 23, 2026
861cca3
ci(shellcheck): scan .gitlab/ci/scripts and make lint logs informative
universal-itengineer Jun 23, 2026
0e47b4a
ci(shellcheck): bump koalaman/shellcheck-alpine v0.10.0 -> v0.11.0
universal-itengineer Jun 23, 2026
fd06db2
refactor(ci): split scripts into bash/ and python/ subfolders
universal-itengineer Jun 23, 2026
bb7a6cb
fix(ci): install helm/jq in lint:helm-templates and migrate lint_go t…
universal-itengineer Jun 23, 2026
7cc2535
fix(ci): fix check:gens-files matrix for moq, bpf2go and api generate
universal-itengineer Jun 23, 2026
b5c9a75
fix(ci): run vm-route-forge bpf2go generation in docker image
universal-itengineer Jun 23, 2026
8fcae0a
fix(ci): use clang-14 in vm-route-forge bpf2go generation
universal-itengineer Jun 23, 2026
313c6ba
chore(core, vm-route-forge): regenerate bpf2go output for cilium/ebpf…
universal-itengineer Jun 23, 2026
2d81b9a
ci(gitlab): remove TODO_RUNNER_TAG placeholders, confirm deckhouse tag
universal-itengineer Jun 24, 2026
42992cf
fix(ci): wire edition/delve into dev builds, restore werf cleanup con…
universal-itengineer Jun 24, 2026
c60e84c
fix(ci): port remaining GH jobs, fix translate-changelog direction, a…
universal-itengineer Jun 24, 2026
c308d4f
update README.md
universal-itengineer Jun 24, 2026
424472b
fix(ci): prod release parity — MODULE_EDITION, se-plus, release-channels
universal-itengineer Jun 24, 2026
f97b9a9
fix(ci): make prod:check-requirements an optional need for prod:build…
universal-itengineer Jun 24, 2026
305f822
fix(ci): move prod_check stage before build for prod:build needs DAG
universal-itengineer Jun 24, 2026
7401c0f
fix(ci): add license headers and prettier-format lint:dmt/lint:yaml f…
universal-itengineer Jun 24, 2026
b1015b0
fix(ci): run unit tests on every MR push via needs DAG
universal-itengineer Jun 24, 2026
573ebde
fix(ci): make manual changelog/backport jobs non-blocking on MR
universal-itengineer Jun 24, 2026
7211ea4
fix(ci): build dev image on merge into release-* branches
universal-itengineer Jun 24, 2026
04c53f8
fix(ci): run lint:yaml/helm-templates/gens-files/gitlab-ci on release…
universal-itengineer Jun 25, 2026
42797c8
fix(ci): run test:scripts:js inside node:24 docker container
universal-itengineer Jun 25, 2026
de1c96f
fix(ci): run npm tests as runner-user to avoid root-owned workspace p…
universal-itengineer Jun 25, 2026
40a18f1
fix(ci): isolate npm tests in container tmpdir to avoid workspace pol…
universal-itengineer Jun 25, 2026
776de10
refactor(ci): single deploy_prod stage, drop dead gitleaks stage
universal-itengineer Jun 25, 2026
77d3160
refactor(ci): rename .local_build/.local_deploy to .base_build/.base_…
universal-itengineer Jun 25, 2026
dcde794
refactor(ci): group templates into base/ dev/ release/ subfolders
universal-itengineer Jun 25, 2026
0f64937
chore(ci): drop .gitlab/README.md and its references in CI files
universal-itengineer Jun 25, 2026
d67f4bb
chore(ci): drop bd issue references from CI comments
universal-itengineer Jun 25, 2026
d4249e6
fix(ci): add schedule discriminator for scheduled jobs
universal-itengineer Jun 26, 2026
1f8b2bb
fix(ci): correct scheduled changelog generation
universal-itengineer Jun 26, 2026
b4fa884
fix(ci): classify stuck merge requests by discussion age
universal-itengineer Jun 26, 2026
a15a1bc
fix(ci): restore milestone-based backport flow
universal-itengineer Jun 26, 2026
635f0cc
test(ci): port real JS/Python script unit tests, drop smoke-only jobs
universal-itengineer Jun 26, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
418 changes: 17 additions & 401 deletions .gitlab-ci.yml

Large diffs are not rendered by default.

55 changes: 55 additions & 0 deletions .gitlab/ci/changelog-sections.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# Copyright 2026 Flant JSC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# Allowed `section` values for ```changes fenced blocks in MR descriptions.
#
# This list is shared between:
# - .gitlab/ci/scripts/bash/check-changelog-entry.sh (validates MR description blocks)
# - .gitlab/ci/scripts/python/changelog_collect.py (groups changes by section)
# - .github/actions/milestone-changelog/action.yml (kept in sync during migration)
#
# Suffix `:low` pins the section to low impact_level (impact_level field becomes optional).
#
# Keep this file in byte-for-byte sync with the equivalent list in:
# .github/actions/milestone-changelog/action.yml
# .github/workflows/check-changelog-entry.yml
api
vm
vmop
vmbda
vmclass
vmip
vmipl
vdsnapshot
vmsnapshot
vmrestore
disks
vd
images
vi
cvi
core
api-service:low
vm-route-forge:low
kubevirt:low
kube-api-rewriter:low
cdi:low
dvcr:low
module
observability
ci:low
test:low
docs
network
cli
15 changes: 15 additions & 0 deletions .gitlab/ci/defaults.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Default settings applied to every job in the pipeline.
#
# This file replaces the previous root-level `default:` block. It mirrors
# the same defaults: the `deckhouse` runner tag is used everywhere, and
# resources are not pinned to a specific tag set (we leave GitLab's
# scheduling to runner tags only).
#
# Per-job `interruptible` is intentionally NOT set here; long-running jobs
# (build_prod and the deploy chain) need `interruptible: false`, while
# short-lived lint/test/build_dev jobs benefit from `interruptible: true`.
# Each job file opts in explicitly.

default:
tags:
- deckhouse
110 changes: 110 additions & 0 deletions .gitlab/ci/includes.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
# Top-level include fan-out for the deckhouse/virtualization GitLab pipeline.
#
# Order is significant: stages/variables/defaults must be loaded before any
# job file because GitLab CI merges variables and stages across all included
# documents. The Setup template from modules-gitlab-ci provides its own
# stages: [build, deploy]; we explicitly extend stages via .gitlab/ci/stages.yml
# so the final list is the union of both.
#
# Upstream template names (verified in
# /Users/korolevn/repos/Virtualization-tasks/github/3p-deckhouse/modules-gitlab-ci,
# branch v13.0, HEAD 006d51c):
# - Setup.gitlab-ci.yml (trdl + werf + ci-env + dual-registry login)
# - Build.gitlab-ci.yml (extends .build — werf build + bundle copy +
# release-channel copy + module registration)
# - Deploy.gitlab-ci.yml (extends .deploy — release-channel crane copy)
# - CVE_Scan.gitlab-ci.yml (extends .cve_scan — Trivy scan via Vault)
# - gitleaks.gitlab-ci.yml (extends .gitleaks_scan — secrets scanning)
# - Go_Checks.gitlab-ci.yml (Go lint/test helpers)
# - Translate_Changelog.gitlab-ci.yml (CHANGELOG/*.ru.yml -> EN MR)
# - Merge_Release.gitlab-ci.yml (release-label driven merge+tag+release)
# - Svace_Analayze.gitlab-ci.yml (note: typo in upstream file name)
# - Antivirus_Scan.gitlab-ci.yml (antivirus scan over recent releases)
# - Semgrep.gitlab-ci.yml (semgrep SAST)
#
# We only include the templates whose behavior is consumed by jobs owned by
# this issue (Setup, Build, Deploy). The remaining upstream templates are
# surfaced here for visibility — child issues that own cve-scan.yml,
# gitleaks.yml, etc. add the matching `include: local:` entries for the
# corresponding job files.

include:
# --- Upstream modules-gitlab-ci (deckhouse/3p, ref v13.0) ---
# TODO: pin to SHA after first green pipeline on virt-test; see migration
# plan §11.2. Until then, branch ref v13.0 keeps fixes flowing.
- project: "deckhouse/3p/deckhouse/modules-gitlab-ci"
ref: "v13.0"
file:
# Setup.gitlab-ci.yml provides trdl + werf setup + dual-registry
# `werf cr login` in before_script. See templates/Setup.gitlab-ci.yml.
- "/templates/Setup.gitlab-ci.yml"
# Validation/scanning jobs extend these upstream templates.
- "/templates/CVE_Scan.gitlab-ci.yml"
- "/templates/gitleaks.gitlab-ci.yml"
- "/templates/Svace_Analayze.gitlab-ci.yml"
# Build.gitlab-ci.yml and Deploy.gitlab-ci.yml are intentionally NOT
# included directly. Their hidden jobs (.build, .deploy) ship with
# `rules:` baked in (`if: $CI_COMMIT_BRANCH`, `if: $CI_COMMIT_TAG`,
# `when: manual`) that override our strict gating via
# .dev / .dev_tags / .main / .prod_manual. We mirror the upstream
# script bodies in .gitlab/ci/templates/base/{build,deploy}.yml as
# `.base_build` and `.base_deploy` and extend those instead.

# --- Local structural fragments (order: stages, vars, defaults, then jobs) ---
- local: ".gitlab/ci/stages.yml"
- local: ".gitlab/ci/variables.yml"
- local: ".gitlab/ci/defaults.yml"
- local: ".gitlab/ci/workflow.yml"

# --- Local shared templates (extends: ...) ---
# templates/ is grouped by the two global processes plus shared base:
# base/ - process-agnostic base job bodies + helpers
# dev/ - development process (DEV registry): MR / main / release / dev-tag
# release/ - release process (PROD registry): tag-push + manual dispatch
# Anchor names are unchanged (.base_build, .dev, .prod_vars, ...), so no job
# `extends:` edits are needed.
- local: ".gitlab/ci/templates/dev/dev_vars.yml"
- local: ".gitlab/ci/templates/release/prod_vars.yml"
- local: ".gitlab/ci/templates/dev/dev.yml"
- local: ".gitlab/ci/templates/dev/dev_tags.yml"
- local: ".gitlab/ci/templates/dev/main.yml"
- local: ".gitlab/ci/templates/dev/release.yml"
- local: ".gitlab/ci/templates/release/prod_manual.yml"
- local: ".gitlab/ci/templates/release/prod_always.yml"
- local: ".gitlab/ci/templates/base/info.yml"
- local: ".gitlab/ci/templates/base/dual_registry_login.yml"
- local: ".gitlab/ci/templates/base/build.yml"
- local: ".gitlab/ci/templates/base/deploy.yml"

# --- Local job files owned by this issue ---
- local: ".gitlab/ci/jobs/info.yml"
- local: ".gitlab/ci/jobs/test.yml"
- local: ".gitlab/ci/jobs/test-scripts-js.yml"
- local: ".gitlab/ci/jobs/test-scripts-python.yml"
- local: ".gitlab/ci/jobs/test-d8v-cli.yml"
- local: ".gitlab/ci/jobs/build-dev.yml"
- local: ".gitlab/ci/jobs/build-prod.yml"
- local: ".gitlab/ci/jobs/deploy-dev.yml"
- local: ".gitlab/ci/jobs/deploy-prod.yml"
# Prod release-channels parity flow (manual single-channel dispatch):
# requirements check, build/deploy per edition, version verification,
# GitLab release creation, Loop notification.
- local: ".gitlab/ci/jobs/release-channels.yml"
- local: ".gitlab/ci/jobs/cleanup.yml"

# --- Local validation and scanning jobs ---
- local: ".gitlab/ci/jobs/lint-dmt.yml"
- local: ".gitlab/ci/jobs/lint-validate.yml"
- local: ".gitlab/ci/jobs/precache.yml"
- local: ".gitlab/ci/jobs/svace.yml"
- local: ".gitlab/ci/jobs/cve-scan.yml"
- local: ".gitlab/ci/jobs/gitleaks.yml"

# --- Local GitLab API automation, changelog, backport, and manual tools ---
- local: ".gitlab/ci/jobs/auto-assign-author.yml"
- local: ".gitlab/ci/jobs/backport.yml"
- local: ".gitlab/ci/jobs/changelog.yml"
- local: ".gitlab/ci/jobs/check-changelog.yml"
- local: ".gitlab/ci/jobs/check-milestone.yml"
- local: ".gitlab/ci/jobs/manual-tools.yml"
- local: ".gitlab/ci/jobs/translate-changelog.yml"
35 changes: 35 additions & 0 deletions .gitlab/ci/jobs/auto-assign-author.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Copyright 2026 Flant JSC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# Auto-assign MR author as MR assignee (GitLab API).
#
# Migration of .github/workflows/dev_auto-pr-author-assign.yml.
# Behaviour per migration plan §0(4): if MR already has an assignee, skip.
# Required CI/CD variable: GITLAB_API_TOKEN (Project Access Token, scope api).

auto-assign-author:
stage: info
tags:
- deckhouse
before_script:
- bash .gitlab/ci/scripts/bash/check-runner-tools.sh bash curl jq
script:
- bash .gitlab/ci/scripts/bash/auto-assign-author.sh
rules:
# Run on every MR pipeline. The script is idempotent: it skips when the MR
# already has an assignee, so re-running on each push is harmless. Gating on
# changes: to this job's own files (the previous behaviour) was wrong — it
# meant the author was only auto-assigned when the auto-assign files changed,
# which never happens on normal MRs.
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
68 changes: 68 additions & 0 deletions .gitlab/ci/jobs/backport.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# Copyright 2026 Flant JSC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# Backport a merged MR to a release branch (clone + cherry-pick + push + MR).
#
# Migration of .github/workflows/on-pull-request-backport.yml which used
# deckhouse/backport-action@v1.0.0 with automerge=true and direct push to
# the release branch. Per migration plan §0(6) and §11.9 we open a
# reviewable backport MR instead of pushing directly.
#
# Triggers (per plan §11.9.4):
# 1. Manual pipeline (Run pipeline UI) with variable TARGET_BRANCH=release-1.21.
# 2. Manual pipeline on an MR carrying the `status/backport` label.
# GitLab does NOT auto-trigger pipelines on label change; the user must
# press "Run pipeline" on the MR (TODO: webhook-listener per plan §7).
# The target branch is derived from the source MR milestone title
# (vX.Y.Z or X.Y.Z -> release-X.Y); TARGET_BRANCH overrides this.
#
# After the backport job finishes, backport.sh updates the source MR:
# - always removes `status/backport`;
# - on success adds `status/backport/success` and comments with the
# backport MR link;
# - on failure adds `status/backport/failed` and comments with the job link.
#
# Required CI/CD variable: GITLAB_API_TOKEN (Project Access Token, scope api).

backport:
stage: lint
tags:
- deckhouse
before_script:
- bash .gitlab/ci/scripts/bash/check-runner-tools.sh bash git curl jq ssh-agent ssh-add
script:
- bash .gitlab/ci/scripts/bash/backport.sh
# allow_failure: true on every rule: `when: manual` inside `rules:` defaults
# to allow_failure: false, so on an MR carrying the `status/backport` label
# this unplayed manual job would block the whole MR pipeline (test/build/...).
# Backport is an opt-in side action, so it must never gate the pipeline.
rules:
# Mode 1: explicit manual run with TARGET_BRANCH provided via UI (overrides
# milestone-based inference).
- if: $TARGET_BRANCH
when: manual
allow_failure: true
# Mode 2: MR with the `status/backport` label. GitLab does NOT auto-run
# pipelines on label change; user has to press "Run pipeline" on the MR
# (TODO: webhook-listener per migration plan §7). The target branch is
# derived from the source MR milestone by backport.sh.
- if: $CI_PIPELINE_SOURCE == "merge_request_event" && $CI_MERGE_REQUEST_LABELS =~ /(^|,\s*)status\/backport(,|$)/
when: manual
allow_failure: true
# Mode 3: scheduled backport sweep (TODO: future automation), under the
# dedicated backport-sweep pipeline schedule
# ($SCHEDULE_TYPE == "backport-sweep").
- if: $CI_PIPELINE_SOURCE == "schedule" && $SCHEDULE_TYPE == "backport-sweep"
when: manual
allow_failure: true
87 changes: 87 additions & 0 deletions .gitlab/ci/jobs/build-dev.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
# DEV build jobs.
#
# Carries forward build_dev, build_dev_tags and build_main from the
# previous root .gitlab-ci.yml. All three extend:
# - .base_build (this repo) — same body as upstream modules-gitlab-ci
# Build.gitlab-ci.yml `.build` (werf build + bundle crane
# copy + release-channel crane copy + crane append to
# register module), but without the upstream `rules:`
# that would override our strict gating via .dev /
# .dev_tags / .main.
# - .dev / .dev_tags / .main (this repo) for registry + tag context.
#
# The previous root .gitlab-ci.yml had its own `.build` template with the
# same body; it is replaced here by `.base_build` (verified against
# /Users/korolevn/repos/Virtualization-tasks/github/3p-deckhouse/modules-gitlab-ci
# templates/Build.gitlab-ci.yml — werf build with --save-build-report,
# bundle / release-channel-version crane copy, and crane append to
# register the module). The upstream version uses WERF_REPO env instead of
# an explicit --repo flag, which is equivalent because MODULES_MODULE_SOURCE
# is the same in both cases.
#
# build_main keeps `interruptible: true` so a new main push cancels an
# older main pipeline. build_dev and build_dev_tags inherit the project
# default (interruptible not set, so they can be cancelled manually).

# `needs: [set_vars]` (with artifacts) is intentional: it opts these jobs
# into DAG mode and pulls ONLY the set_vars dotenv (MODULE_EDITION,
# DEBUG_COMPONENT, RELEASE_IN_DEV). It replaces the previous bare `needs: []`,
# which existed to stop GitLab's legacy behavior of downloading ALL prior-stage
# artifacts and leaking unrelated dotenv artifacts (e.g. svace:set-vars writes
# MODULES_MODULE_TAG=<branch>-svace) that override the per-template
# MODULES_MODULE_TAG from .dev / .dev_tags / .main. Listing set_vars explicitly
# keeps that protection: svace.env is still not downloaded. set_vars does NOT
# emit MODULES_MODULE_TAG, so the per-template tag is preserved.
#
# WERF_VIRTUAL_MERGE=0 mirrors the GitHub dev_setup_build job env.

build_dev:
stage: build
needs:
- job: set_vars
artifacts: true
variables:
WERF_VIRTUAL_MERGE: "0"
extends:
- .base_build
- .dev

build_dev_tags:
stage: build
needs:
- job: set_vars
artifacts: true
variables:
WERF_VIRTUAL_MERGE: "0"
extends:
- .base_build
- .dev_tags

build_main:
stage: build
interruptible: true
needs:
- job: set_vars
artifacts: true
variables:
WERF_VIRTUAL_MERGE: "0"
extends:
- .base_build
- .main

# build_release mirrors GitHub dev_module_build.yml `push: [release-*]`: a
# squash-merge into a release-X.Y branch rebuilds the dev image tagged with the
# branch name. Without this job GitLab created a pipeline on the merge push but
# built nothing for release branches (only .dev/.main/.dev_tags had build jobs).
# interruptible: true so a newer release-branch push cancels an older build.
build_release:
stage: build
interruptible: true
needs:
- job: set_vars
artifacts: true
variables:
WERF_VIRTUAL_MERGE: "0"
extends:
- .base_build
- .release
Loading
Loading