Skip to content
This repository was archived by the owner on Apr 23, 2025. It is now read-only.

Commit 616d8d7

Browse files
committed
Add comprehensive code coverage infrastructure and new tests
1 parent f1694cb commit 616d8d7

15 files changed

Lines changed: 2309 additions & 256 deletions

.coveragerc

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
[run]
2+
source = cli_code
3+
omit =
4+
*/.rules/*
5+
*/.venv/*
6+
*/docs/*
7+
*/test_*
8+
*/__pycache__/*
9+
*/venv/*
10+
*/.pytest_cache/*
11+
*/site-packages/*
12+
13+
[report]
14+
exclude_lines =
15+
pragma: no cover
16+
def __repr__
17+
raise NotImplementedError
18+
if __name__ == .__main__.:
19+
pass
20+
raise ImportError
21+
except ImportError
22+
raise AssertionError
23+
if TYPE_CHECKING
24+
25+
precision = 2
26+
show_missing = true
27+
28+
[html]
29+
directory = coverage_html
30+
title = CLI Code Coverage Report
31+
32+
[xml]
33+
output = coverage.xml
Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
---
2+
name: Coverage Improvement
3+
about: Template for adding tests to improve code coverage
4+
title: 'Coverage: [Module Name]'
5+
labels: 'test, enhancement'
6+
assignees: ''
7+
---
8+
9+
## Coverage Issue
10+
11+
**Module:** [Module Name]
12+
**Current Coverage:** [XX.XX]%
13+
**Target Coverage:** At least 60%
14+
15+
## Description
16+
17+
This issue tracks the need to improve test coverage for the specified module.
18+
19+
## Areas Needing Tests
20+
21+
- [ ] [Function/Method/Area 1]
22+
- [ ] [Function/Method/Area 2]
23+
- [ ] [Function/Method/Area 3]
24+
25+
## Approach
26+
27+
1. Analyze the current test coverage to identify untested code paths
28+
2. Write unit tests for the untested areas
29+
3. Focus on testing both normal operation and edge cases
30+
4. Run coverage to verify improvement
31+
32+
## Related Issues/PRs
33+
34+
- [List any related issues or PRs]
35+
36+
## Additional Context
37+
38+
[Any additional information that might be helpful]

.github/workflows/python-ci.yml

Lines changed: 14 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -57,13 +57,13 @@ jobs:
5757
echo "Available test files:"
5858
ls -la test_dir/
5959
60-
# Run pytest with only specific test files that we know work in CI
61-
# Generate XML report for SonarCloud
62-
python -m pytest --cov=src/cli_code --cov-report=term --cov-report=xml test_dir/test_config.py test_dir/test_main.py
63-
60+
# Run pytest with more comprehensive coverage
61+
# Generate both XML and HTML reports
62+
python -m pytest --cov=src/cli_code --cov-report=term --cov-report=xml --cov-report=html --verbose test_dir/
63+
6464
# Display the overall coverage percentage
6565
echo "Overall coverage percentage:"
66-
python -c "import xml.etree.ElementTree as ET; tree = ET.parse('coverage.xml'); root = tree.getroot(); print(f\"Coverage: {root.attrib['line-rate']}%\")"
66+
python -c "import xml.etree.ElementTree as ET; tree = ET.parse('coverage.xml'); root = tree.getroot(); print(f\"Coverage: {float(root.attrib['line-rate'])*100:.2f}%\")"
6767
6868
- name: SonarCloud Scan
6969
uses: SonarSource/sonarcloud-github-action@master # Or use a specific version like @v2.1
@@ -88,6 +88,15 @@ jobs:
8888
path: dist/
8989
retention-days: 7
9090

91+
- name: Upload coverage reports as artifacts
92+
uses: actions/upload-artifact@v4
93+
with:
94+
name: coverage-reports
95+
path: |
96+
coverage.xml
97+
coverage_html/
98+
retention-days: 7
99+
91100
publish:
92101
name: Publish to PyPI
93102
needs: build-and-test

CODE_COVERAGE.md

Lines changed: 140 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,140 @@
1+
# Code Coverage Guide
2+
3+
This document provides information on how to run and analyze code coverage for the CLI Code Agent project.
4+
5+
## Overview
6+
7+
Code coverage is a metric that helps identify which parts of your codebase are being executed during your tests. It helps in determining the effectiveness of your tests and identifying areas that need additional testing.
8+
9+
## Running Code Coverage
10+
11+
### Using the Shell Script
12+
13+
We provide a convenient shell script that runs the tests and generates coverage reports:
14+
15+
```bash
16+
./run_coverage.sh
17+
```
18+
19+
This script will:
20+
1. Run all tests with coverage enabled
21+
2. Generate coverage reports in XML and HTML formats
22+
3. Display the overall coverage percentage
23+
4. Open the HTML report (if on macOS)
24+
25+
### Manual Execution
26+
27+
If you prefer to run coverage manually, use the following commands:
28+
29+
```bash
30+
# Run pytest with coverage
31+
python -m pytest --cov=src/cli_code --cov-report=term --cov-report=xml --cov-report=html test_dir/
32+
33+
# To run specific test files
34+
python -m pytest --cov=src/cli_code --cov-report=term test_dir/test_file.py
35+
```
36+
37+
## Analyzing Coverage Results
38+
39+
### HTML Report
40+
41+
The HTML report provides a user-friendly interface to explore your coverage results. After running the coverage, open:
42+
43+
```
44+
coverage_html/index.html
45+
```
46+
47+
### Using the Coverage Analyzer
48+
49+
We provide a Python script to analyze coverage data and identify modules with low coverage:
50+
51+
```bash
52+
./find_low_coverage.py
53+
```
54+
55+
This script will:
56+
1. Parse the coverage XML file
57+
2. Calculate coverage for each module
58+
3. Display a table of module coverage
59+
4. Highlight modules that fall below the minimum threshold (60%)
60+
5. Provide a summary of modules needing improvement
61+
62+
### Using SonarCloud Locally
63+
64+
For the most accurate analysis that matches what runs in the CI pipeline, you can use the SonarCloud scanner locally:
65+
66+
```bash
67+
./run_sonar_scan.sh
68+
```
69+
70+
This script will:
71+
1. Check if you have sonar-scanner installed
72+
2. Verify that you have set the SONAR_TOKEN environment variable
73+
3. Run coverage if coverage.xml doesn't exist
74+
4. Run sonar-scanner with the same configuration as the CI pipeline
75+
5. Provide a link to view results on SonarCloud
76+
77+
To install the sonar-scanner CLI:
78+
1. Follow the instructions at [SonarScanner CLI](https://docs.sonarcloud.io/advanced-setup/ci-based-analysis/sonarscanner-cli/)
79+
2. Set up your SONAR_TOKEN environment variable:
80+
```bash
81+
export SONAR_TOKEN=your-token
82+
```
83+
84+
The SonarCloud analysis provides more advanced metrics beyond just coverage, including:
85+
- Code quality issues
86+
- Code smells
87+
- Security vulnerabilities
88+
- Maintainability rating
89+
- Reliability rating
90+
- Security rating
91+
92+
## Coverage Configuration
93+
94+
The coverage configuration is defined in two places:
95+
96+
1. `.coveragerc` file - Controls how coverage is measured and reported
97+
2. `pyproject.toml` - Contains the pytest-cov configuration
98+
99+
Key settings include:
100+
- `source` - Specifies which packages to measure
101+
- `omit` - Patterns for files to exclude from coverage
102+
- `exclude_lines` - Patterns for lines to exclude from coverage calculation
103+
- `fail_under` - Minimum required coverage percentage
104+
105+
## Continuous Integration
106+
107+
Coverage is automatically run as part of our CI/CD pipeline. The GitHub workflow:
108+
109+
1. Runs tests with coverage enabled
110+
2. Generates coverage reports
111+
3. Uploads coverage reports as artifacts
112+
4. Sends coverage data to SonarCloud for analysis
113+
114+
The pipeline uses the exact same SonarCloud configuration that you can run locally with `./run_sonar_scan.sh`.
115+
116+
## Improving Coverage
117+
118+
To improve code coverage:
119+
120+
1. Identify modules with low coverage using the analysis tools
121+
2. Focus on writing tests for uncovered code paths
122+
3. Pay special attention to error handling and edge cases
123+
4. Use parameterized tests for code with many variations
124+
125+
Remember that 100% coverage doesn't guarantee bug-free code, but it does help ensure that most code paths have been executed at least once during testing.
126+
127+
## Minimum Coverage Threshold
128+
129+
The minimum acceptable coverage for this project is **60%**. This threshold is enforced in:
130+
- The coverage configuration
131+
- The analysis scripts
132+
- (Optionally) The CI pipeline
133+
134+
## Tips for Writing Testable Code
135+
136+
- Keep functions small and focused on a single responsibility
137+
- Extract complex logic into separate testable functions
138+
- Use dependency injection to make external dependencies mockable
139+
- Avoid global state when possible
140+
- Use clear error handling patterns

README.md

Lines changed: 23 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@
22

33
[![Python CI](https://github.com/BlueCentre/cli-code/actions/workflows/python-ci.yml/badge.svg)](https://github.com/BlueCentre/cli-code/actions/workflows/python-ci.yml)
44
[![Coverage](https://sonarcloud.io/api/project_badges/measure?project=BlueCentre_cli-code&metric=coverage)](https://sonarcloud.io/summary/new_code?id=BlueCentre_cli-code)
5+
[![Maintainability Rating](https://sonarcloud.io/api/project_badges/measure?project=BlueCentre_cli-code&metric=sqale_rating)](https://sonarcloud.io/summary/new_code?id=BlueCentre_cli-code)
56

67
An AI coding assistant for your terminal, powered by multiple LLM providers (Gemini and Ollama).
78

@@ -166,6 +167,7 @@ For more detailed information, please refer to the following documentation:
166167

167168
- [Installation Guide](docs/install.md) - Detailed instructions for installing and configuring CLI-Code
168169
- [Contributing Guide](docs/contributing.md) - Guidelines for contributing to the project
170+
- [Code Coverage Guide](CODE_COVERAGE.md) - Information on code coverage tools and practices
169171
- [Changelog](docs/changelog.md) - History of changes and releases
170172
- [Architecture](docs/architecture.md) - Technical overview of the system architecture
171173
- [Context Guidelines](docs/context.md) - Guidelines for the CLI Code Assistant
@@ -220,7 +222,27 @@ Example: Create a file `.rules/python_standards.md` with your team's Python codi
220222
221223
## Development
222224
223-
To contribute to CLI Code, please see the [Contributing Guide](docs/contributing.md).
225+
To set up a development environment:
226+
227+
```bash
228+
# Clone the repository
229+
git clone https://github.com/BlueCentre/cli-code.git
230+
cd cli-code
231+
232+
# Install in development mode with dev dependencies
233+
pip install -e ".[dev]"
234+
235+
# Run tests
236+
python -m pytest
237+
238+
# Run coverage analysis
239+
./run_coverage.sh
240+
241+
# Run SonarCloud analysis locally (requires sonar-scanner CLI)
242+
./run_sonar_scan.sh
243+
```
244+
245+
The project uses [pytest](https://docs.pytest.org/) for testing and [SonarCloud](https://sonarcloud.io/) for code quality and coverage analysis.
224246

225247
## Contributing
226248

0 commit comments

Comments
 (0)