Skip to content

Commit 6a708fb

Browse files
ChristophShyperChristophShyper
committed
feat: Add centralized workflows and configuration files for automation
1 parent 00ef781 commit 6a708fb

28 files changed

Lines changed: 2196 additions & 172 deletions

‎.github/MIGRATION.md‎

Lines changed: 245 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,245 @@
1+
# Migration to Centralized Reusable Workflows
2+
3+
## Summary
4+
5+
This setup establishes the `.github` repository as a centralized meta repository for all devops-infra workflows, eliminating duplication across ~14 repositories.
6+
7+
## What Was Created
8+
9+
### 1. Reusable Workflows (`.github/workflows/`)
10+
11+
Five centralized workflows:
12+
13+
| Workflow | Purpose | Replaces |
14+
|--------------------------------|-----------------------------------------|--------------------------------|
15+
| `auto-create-pull-request.yml` | Auto-create PRs for feature branches | `auto-create-pull-request.yml` |
16+
| `auto-create-release.yml` | Create releases from merged PRs | `auto-create-release.yml` |
17+
| `cron-check-dependencies.yml` | Scheduled dependency testing | `cron-check-dependencies.yml` |
18+
| `manual-update-version.yml` | Manual version bumps | `manual-update-version.yml` |
19+
| `manual-sync-common-files.yml` | Sync common files from template sources | `manual-sync-common-files.yml` |
20+
21+
### 2. Example Caller Workflows (`.github/workflows/examples/`)
22+
23+
Ready-to-use examples showing how repositories call the centralized workflows:
24+
- `auto-create-pull-request.yml`
25+
- `auto-create-release.yml`
26+
- `cron-check-dependencies.yml`
27+
- `manual-update-version.yml`
28+
- `manual-sync-common-files.yml`
29+
30+
### 3. Reusable Taskfiles (`.github/taskfiles/`)
31+
32+
Shared Taskfile templates used by the sync-files workflow:
33+
- `Taskfile.yml`
34+
- `Taskfile.cicd.yml`
35+
- `Taskfile.docker.yml`
36+
- `Taskfile.variables.yml`
37+
38+
### 4. Reusable Configs (`.github/configs/`)
39+
40+
Shared config files synced by `sync:configs` and `sync:ignores`:
41+
- `.editorconfig`
42+
- `.hadolint.yaml`
43+
- `.pre-commit-config.yaml`
44+
- `.shellcheckrc`
45+
- `.yamllint.yml`
46+
- `.gitignore`
47+
- `.dockerignore`
48+
49+
### 5. Documentation
50+
51+
- `README.md` - Complete guide to centralized workflows, migration process, and troubleshooting
52+
- Updated `copilot-instructions.md` - Added centralized workflow section and Task runner info
53+
54+
### 6. Migration Script
55+
56+
`migrate-to-reusable.sh` - Automated script to migrate repositories with:
57+
- Dry-run mode for testing
58+
- Automatic backup creation
59+
- Selective workflow migration
60+
- Clear next-steps guidance
61+
62+
## Migration Process
63+
64+
### Step 1: Commit and Push to .github Repository
65+
66+
```bash
67+
cd /Users/christoph/IdeaProjects/devops-infra/.github
68+
git add .github/workflows/*.yml
69+
git add .github/workflows/examples/
70+
git add .github/workflows/README.md
71+
git add .github/workflows/migrate-to-reusable.sh
72+
git add .github/configs/
73+
git add .github/taskfiles/
74+
git add copilot-instructions.md
75+
git commit -m "feat: Add centralized workflows for organization"
76+
git push origin master
77+
```
78+
79+
### Step 2: Migrate Individual Repositories
80+
81+
Choose one of these approaches:
82+
83+
#### Option A: Manual Migration (Recommended for first repo)
84+
85+
1. **Pick a test repository** (suggest `action-format-hcl` as it's simpler)
86+
87+
2. **Copy example workflows:**
88+
```bash
89+
cd /Users/christoph/IdeaProjects/devops-infra/action-format-hcl
90+
91+
# Backup existing workflows
92+
cp -r .github/workflows .github/workflows.backup
93+
94+
# Copy examples
95+
cp ../. github/.github/workflows/examples/*.yml .github/workflows/
96+
```
97+
98+
3. **Remove old workflows:**
99+
```bash
100+
# Keep only the new caller workflows
101+
ls .github/workflows/
102+
```
103+
104+
4. **Test:**
105+
```bash
106+
git checkout -b test/central-workflows
107+
git add .github/workflows/
108+
git commit -m "test: Migrate to centralized workflows"
109+
git push origin test/central-workflows
110+
```
111+
112+
5. **Verify in GitHub Actions UI** that the workflow runs correctly
113+
114+
#### Option B: Automated Migration
115+
116+
```bash
117+
cd /Users/christoph/IdeaProjects/devops-infra/.github/.github/workflows
118+
119+
# Dry-run first
120+
./migrate-to-reusable.sh ../../action-format-hcl --dry-run
121+
122+
# If looks good, run for real
123+
./migrate-to-reusable.sh ../../action-format-hcl
124+
125+
# Migrate specific workflows only
126+
./migrate-to-reusable.sh ../../docker-terragrunt --workflows pr,release
127+
```
128+
129+
### Step 3: Rollout to All Repositories
130+
131+
Once verified with one repository, migrate the rest:
132+
133+
**Repositories to migrate:**
134+
- action-commit-push
135+
- action-format-hcl ✓ (use as test)
136+
- action-pull-request
137+
- action-terraform-copy-vars
138+
- action-terraform-validate
139+
- action-tflint
140+
- docker-simple-runner
141+
- docker-terragrunt
142+
- template-action
143+
144+
**Repositories that may need customization:**
145+
- `velez` - Uses Python/pytest, not Docker (may need different workflow)
146+
147+
**Archived repositories (excluded from automation):**
148+
- `docker-okta-aws-sso`
149+
150+
### Step 4: Cleanup
151+
152+
After successful migration and testing:
153+
154+
```bash
155+
# In each repository
156+
rm -rf .github/workflows.backup*
157+
```
158+
159+
## Benefits After Migration
160+
161+
### Before
162+
- ~70 duplicate workflow files across 14 repositories
163+
- Bug fixes require updating 14 repos individually
164+
- Inconsistent workflow versions between repos
165+
- High maintenance burden
166+
167+
### After
168+
- 5 centralized workflows + ~14 small caller workflows
169+
- Update logic once in `.github`, applies everywhere
170+
- Guaranteed consistency across all repos
171+
- Minimal maintenance
172+
173+
## Customization Examples
174+
175+
### Disable Docker for non-Docker repos
176+
```yaml
177+
jobs:
178+
call-workflow:
179+
uses: devops-infra/.github/.github/workflows/auto-create-pull-request.yml@master
180+
with:
181+
enable-docker: false
182+
enable-lint: true
183+
```
184+
185+
### Use different runner
186+
```yaml
187+
jobs:
188+
call-workflow:
189+
uses: devops-infra/.github/.github/workflows/auto-create-pull-request.yml@master
190+
with:
191+
runs-on: ubuntu-latest
192+
```
193+
194+
### Single-arch builds
195+
```yaml
196+
jobs:
197+
call-workflow:
198+
uses: devops-infra/.github/.github/workflows/auto-create-pull-request.yml@master
199+
with:
200+
docker-platforms: amd64
201+
```
202+
203+
## Rollback Plan
204+
205+
If issues occur:
206+
207+
1. **Individual repository:** Restore from `.github/workflows.backup`
208+
2. **Organization-wide:** Temporarily pin to old workflow files until fixed
209+
3. **Revert .github changes:** `git revert <commit>` in `.github` repository
210+
211+
## Future Enhancements
212+
213+
1. **Version tagging:** Pin workflows to tags instead of `@master` for stability
214+
```yaml
215+
uses: devops-infra/.github/.github/workflows/auto-create-pull-request.yml@v1.0.0
216+
```
217+
218+
2. **Python-specific workflow:** Create centralized workflow for `velez` and similar Python projects
219+
220+
3. **Makefile-specific workflow:** Create centralized workflow for repos still using Make
221+
222+
4. **Auto-sync workflow:** Create workflow that automatically updates caller workflows in all repos when examples change
223+
224+
5. **Workflow testing:** Add tests to validate centralized workflows before deployment
225+
226+
## Support
227+
228+
For issues or questions:
229+
- Check `README.md` in `.github/.github/workflows/`
230+
- Review examples in `examples/` directory
231+
- Test with `--dry-run` flag first
232+
- Start with one simple repository before migrating all
233+
234+
## Checklist
235+
236+
- [ ] Commit and push centralized workflows to `.github` repository
237+
- [ ] Test migration with `action-format-hcl` (or similar simple repo)
238+
- [ ] Verify GitHub Actions run successfully
239+
- [ ] Migrate remaining action-* repositories
240+
- [ ] Migrate docker-* repositories (may need customization)
241+
- [ ] Handle special cases (velez)
242+
- [ ] Skip archived repositories (docker-okta-aws-sso)
243+
- [ ] Clean up backup directories
244+
- [ ] Update any documentation referencing old workflows
245+
- [ ] Consider implementing version tagging for workflows

‎.github/configs/.dockerignore‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
# Exclude
2+
*
3+
4+
# Include
5+
!Dockerfile
6+
!entrypoint.sh
7+
!LICENSE
8+
!README.md

‎.github/configs/.editorconfig‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
# EditorConfig helps developers define and maintain consistent coding styles
2+
root = true
3+
4+
[*]
5+
charset = utf-8
6+
end_of_line = lf
7+
insert_final_newline = true
8+
indent_style = space
9+
indent_size = 2
10+
trim_trailing_whitespace = true

‎.github/configs/.gitignore‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
# Intellij
2+
/.idea/
3+
*.iml
4+
5+
# Custom
6+
.tmp/
7+
.venv
8+
.venv/
9+
.envrc
10+
.env
11+
.tmp

‎.github/configs/.hadolint.yaml‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
failure-threshold: error
2+
format: tty
3+
strict-labels: false
4+
no-color: false
5+
no-fail: false
6+
disable-ignore-pragma: false
7+
trustedRegistries:
8+
- docker.io
9+
- ghcr.io
10+
11+
# ignored: [string]
12+
# label-schema:
13+
# author: text
14+
# contact: email
15+
# created: rfc3339
16+
# version: semver
17+
# documentation: url
18+
# git-revision: hash
19+
# license: spdx
20+
# override:
21+
# error: [string]
22+
# warning: [string]
23+
# info: [string]
24+
# style: [string]

‎.github/configs/.pre-commit-config.yaml‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
repos:
2+
- repo: https://github.com/pre-commit/pre-commit-hooks
3+
rev: v6.0.0
4+
hooks:
5+
- id: check-added-large-files
6+
- id: check-case-conflict
7+
- id: check-executables-have-shebangs
8+
- id: check-illegal-windows-names
9+
- id: check-json
10+
- id: check-merge-conflict
11+
- id: check-shebang-scripts-are-executable
12+
- id: check-symlinks
13+
- id: check-xml
14+
- id: check-yaml
15+
- id: destroyed-symlinks
16+
- id: detect-private-key
17+
- id: end-of-file-fixer
18+
- id: mixed-line-ending
19+
args: [--fix=lf]
20+
- id: no-commit-to-branch
21+
args: [--branch, master, --branch, main]
22+
- id: pretty-format-json
23+
args: [--autofix]
24+
- id: trailing-whitespace
25+
- repo: local
26+
hooks:
27+
- id: actionlint
28+
name: actionlint
29+
entry: bash -lc 'docker run --rm -v "$PWD:/work" -w /work rhysd/actionlint:latest -color'
30+
language: system
31+
pass_filenames: false
32+
- id: hadolint
33+
name: hadolint
34+
entry: bash -lc 'docker run --rm -v "$PWD:/work" -w /work hadolint/hadolint:latest-debian hadolint Dockerfile'
35+
language: system
36+
pass_filenames: false
37+
- id: shellcheck
38+
name: shellcheck
39+
entry: bash -lc 'docker run --rm -v "$PWD:/work" -w /work koalaman/shellcheck:stable -x -S style entrypoint.sh'
40+
language: system
41+
pass_filenames: false
42+
- id: yamllint
43+
name: yamllint
44+
entry: bash -lc 'docker run --rm -v "$PWD:/work" -w /work cytopia/yamllint -c .yamllint.yml .'
45+
language: system
46+
pass_filenames: false

‎.github/configs/.shellcheckrc‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
# shellcheck configuration
2+
shell=bash
3+
check-sourced=true
4+
external-sources=true
5+
source-path=SCRIPTDIR

‎.github/configs/.yamllint.yml‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
extends: default
2+
rules:
3+
empty-lines:
4+
max: 2
5+
document-end:
6+
present: false
7+
document-start:
8+
present: false
9+
indentation:
10+
spaces: 2
11+
indent-sequences: true
12+
check-multi-line-strings: false
13+
line-length:
14+
max: 140
15+
allow-non-breakable-inline-mappings: true
16+
new-line-at-end-of-file: enable
17+
new-lines:
18+
type: unix
19+
quoted-strings:
20+
required: only-when-needed
21+
extra-allowed: ['true', 'false']
22+
trailing-spaces: {}
23+
truthy:
24+
allowed-values: ['true', 'false', 'yes', 'no']
25+
check-keys: false

0 commit comments

Comments
 (0)