Skip to content

Commit a6ea751

Browse files
authored
Merge pull request #5 from dev-ankit/claude/build-config-tool-bldp3
Add YAML set operations to config-utils
2 parents 01da5cb + ea7732a commit a6ea751

6 files changed

Lines changed: 1023 additions & 9 deletions

File tree

.github/workflows/test.yml

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,12 @@ on:
99
jobs:
1010
test:
1111
runs-on: ubuntu-latest
12+
strategy:
13+
matrix:
14+
tool:
15+
- locust-compare
16+
- config-utils
17+
fail-fast: false
1218
steps:
1319
- uses: actions/checkout@v4
1420

@@ -17,9 +23,10 @@ jobs:
1723
with:
1824
python-version: 3.12
1925

20-
- name: Install and test locust-compare
21-
working-directory: tools/locust-compare
26+
- name: Install and test ${{ matrix.tool }}
27+
working-directory: tools/${{ matrix.tool }}
2228
run: |
2329
python -m pip install --upgrade pip
2430
pip install -e .[dev]
25-
pytest
31+
python -m pytest -v
32+

tools/config-utils/README.md

Lines changed: 152 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,12 @@
11
# config-utils
22

3-
A CLI tool for capturing environment variables and Django settings in YAML format.
3+
A CLI tool for capturing environment variables and Django settings in YAML format, plus performing set operations on YAML configuration files.
44

55
## Features
66

77
- **capture-env**: Capture all environment variables and export them to YAML
88
- **capture-django-settings**: Capture Django project settings and export them to YAML
9+
- **Set Operations**: Compare and merge YAML files using set operations (union, intersect, diff, rdiff, symdiff)
910

1011
## Installation
1112

@@ -116,6 +117,124 @@ config-utils capture-django-settings
116117

117118
**Note**: This command must be run from your Django project directory or you must specify the path to `manage.py` using the `--manage-py` option.
118119

120+
### YAML Set Operations
121+
122+
Perform set operations on two YAML configuration files. All operations output valid YAML to stdout.
123+
124+
#### Commands
125+
126+
- `union`: Returns all keys (or key-value pairs) present in either file
127+
- `intersect`: Returns only keys (or key-value pairs) present in both files
128+
- `diff`: Returns keys (or key-value pairs) in file1 but not in file2 (A - B)
129+
- `rdiff`: Returns keys (or key-value pairs) in file2 but not in file1 (B - A)
130+
- `symdiff`: Returns keys (or key-value pairs) in either file but not in both (symmetric difference)
131+
132+
#### Options
133+
134+
- `--compare <mode>`: Comparison mode - `keys` or `kv` (key-values). Default: `kv`
135+
- `kv`: Compares key-value pairs. Two entries match only if both key AND value are identical
136+
- `keys`: Compares only keys. Values are ignored when determining matches
137+
- `--depth <n>`: How many levels deep to compare. Default: `1`
138+
- `1`: Root keys only
139+
- `2`: Compare up to 2 levels (root and one level of nesting)
140+
- `0`: Unlimited depth (fully flatten using dot notation)
141+
142+
#### Examples
143+
144+
**file1.yml:**
145+
```yaml
146+
database: postgres
147+
port: 5432
148+
debug: true
149+
```
150+
151+
**file2.yml:**
152+
```yaml
153+
database: postgres
154+
port: 3306
155+
logging: verbose
156+
```
157+
158+
**Find common configuration (intersect with key-values):**
159+
```bash
160+
config-utils intersect file1.yml file2.yml
161+
# Output:
162+
# database: postgres
163+
```
164+
165+
**Find common keys regardless of values:**
166+
```bash
167+
config-utils intersect file1.yml file2.yml --compare keys
168+
# Output:
169+
# database: postgres
170+
# port: 5432
171+
```
172+
173+
**Find all unique configuration (union):**
174+
```bash
175+
config-utils union file1.yml file2.yml
176+
# Output:
177+
# database: postgres
178+
# port: 5432
179+
# debug: true
180+
# logging: verbose
181+
```
182+
183+
**Find what's in file1 but not file2 (diff):**
184+
```bash
185+
config-utils diff file1.yml file2.yml
186+
# Output:
187+
# port: 5432
188+
# debug: true
189+
```
190+
191+
**Find what's in file2 but not file1 (rdiff):**
192+
```bash
193+
config-utils rdiff file1.yml file2.yml
194+
# Output:
195+
# port: 3306
196+
# logging: verbose
197+
```
198+
199+
**Nested comparison with depth:**
200+
201+
**nested1.yml:**
202+
```yaml
203+
database:
204+
host: localhost
205+
port: 5432
206+
app:
207+
name: myapp
208+
```
209+
210+
**nested2.yml:**
211+
```yaml
212+
database:
213+
host: localhost
214+
port: 3306
215+
app:
216+
name: myapp
217+
```
218+
219+
```bash
220+
# Depth 1 - root keys only
221+
config-utils intersect nested1.yml nested2.yml --depth 1
222+
# Output:
223+
# database:
224+
# host: localhost
225+
# port: 5432
226+
# app:
227+
# name: myapp
228+
229+
# Depth 2 - compare nested keys
230+
config-utils intersect nested1.yml nested2.yml --depth 2
231+
# Output:
232+
# database:
233+
# host: localhost
234+
# app:
235+
# name: myapp
236+
```
237+
119238
### Using with uvx
120239

121240
You can run the tool directly without installation:
@@ -152,15 +271,45 @@ uvx --from /path/to/config-utils config-utils capture-django-settings -m /path/t
152271
cd config-utils
153272

154273
# Install in editable mode with development dependencies
155-
pip install -e .
274+
pip install -e ".[dev]"
156275
```
157276

277+
### Running Tests
278+
279+
The project includes comprehensive pytest tests for all set operations functionality.
280+
281+
```bash
282+
# Run all tests
283+
python -m pytest
284+
285+
# Run with coverage report
286+
python -m pytest --cov=cli --cov-report=term-missing
287+
288+
# Run specific test class
289+
python -m pytest tests/test_set_operations.py::TestUnionCommand -v
290+
291+
# Run specific test
292+
python -m pytest tests/test_set_operations.py::TestUnionCommand::test_union_kv_mode -v
293+
```
294+
295+
### Test Coverage
296+
297+
The test suite covers:
298+
- All 5 set operations (union, intersect, diff, rdiff, symdiff)
299+
- Both comparison modes (keys and kv)
300+
- All depth levels (0, 1, 2+)
301+
- Error handling (missing files, invalid YAML, non-dict roots)
302+
- Edge cases (empty files, identical files, no matches)
303+
- Helper functions (flatten_dict, unflatten_dict, make_hashable, perform_set_operation, load_yaml_file)
304+
158305
### Project Structure
159306

160307
```
161308
config-utils/
162309
├── cli.py
163-
├── config_utils/
310+
├── tests/
311+
│ ├── __init__.py
312+
│ └── test_set_operations.py
164313
├── pyproject.toml
165314
└── README.md
166315
```

0 commit comments

Comments
 (0)