|
| 1 | +# Code Coverage Guide |
| 2 | + |
| 3 | +This document provides information on how to run and analyze code coverage for the CLI Code Agent project. |
| 4 | + |
| 5 | +## Overview |
| 6 | + |
| 7 | +Code coverage is a metric that helps identify which parts of your codebase are being executed during your tests. It helps in determining the effectiveness of your tests and identifying areas that need additional testing. |
| 8 | + |
| 9 | +## Running Code Coverage |
| 10 | + |
| 11 | +### Using the Shell Script |
| 12 | + |
| 13 | +We provide a convenient shell script that runs the tests and generates coverage reports: |
| 14 | + |
| 15 | +```bash |
| 16 | +./run_coverage.sh |
| 17 | +``` |
| 18 | + |
| 19 | +This script will: |
| 20 | +1. Run all tests with coverage enabled |
| 21 | +2. Generate coverage reports in XML and HTML formats |
| 22 | +3. Display the overall coverage percentage |
| 23 | +4. Open the HTML report (if on macOS) |
| 24 | + |
| 25 | +### Manual Execution |
| 26 | + |
| 27 | +If you prefer to run coverage manually, use the following commands: |
| 28 | + |
| 29 | +```bash |
| 30 | +# Run pytest with coverage |
| 31 | +python -m pytest --cov=src/cli_code --cov-report=term --cov-report=xml --cov-report=html test_dir/ |
| 32 | + |
| 33 | +# To run specific test files |
| 34 | +python -m pytest --cov=src/cli_code --cov-report=term test_dir/test_file.py |
| 35 | +``` |
| 36 | + |
| 37 | +## Analyzing Coverage Results |
| 38 | + |
| 39 | +### HTML Report |
| 40 | + |
| 41 | +The HTML report provides a user-friendly interface to explore your coverage results. After running the coverage, open: |
| 42 | + |
| 43 | +``` |
| 44 | +coverage_html/index.html |
| 45 | +``` |
| 46 | + |
| 47 | +### Using the Coverage Analyzer |
| 48 | + |
| 49 | +We provide a Python script to analyze coverage data and identify modules with low coverage: |
| 50 | + |
| 51 | +```bash |
| 52 | +./find_low_coverage.py |
| 53 | +``` |
| 54 | + |
| 55 | +This script will: |
| 56 | +1. Parse the coverage XML file |
| 57 | +2. Calculate coverage for each module |
| 58 | +3. Display a table of module coverage |
| 59 | +4. Highlight modules that fall below the minimum threshold (60%) |
| 60 | +5. Provide a summary of modules needing improvement |
| 61 | + |
| 62 | +### Using SonarCloud Locally |
| 63 | + |
| 64 | +For the most accurate analysis that matches what runs in the CI pipeline, you can use the SonarCloud scanner locally: |
| 65 | + |
| 66 | +```bash |
| 67 | +./run_sonar_scan.sh |
| 68 | +``` |
| 69 | + |
| 70 | +This script will: |
| 71 | +1. Check if you have sonar-scanner installed |
| 72 | +2. Verify that you have set the SONAR_TOKEN environment variable |
| 73 | +3. Run coverage if coverage.xml doesn't exist |
| 74 | +4. Run sonar-scanner with the same configuration as the CI pipeline |
| 75 | +5. Provide a link to view results on SonarCloud |
| 76 | + |
| 77 | +To install the sonar-scanner CLI: |
| 78 | +1. Follow the instructions at [SonarScanner CLI](https://docs.sonarcloud.io/advanced-setup/ci-based-analysis/sonarscanner-cli/) |
| 79 | +2. Set up your SONAR_TOKEN environment variable: |
| 80 | + ```bash |
| 81 | + export SONAR_TOKEN=your-token |
| 82 | + ``` |
| 83 | + |
| 84 | +The SonarCloud analysis provides more advanced metrics beyond just coverage, including: |
| 85 | +- Code quality issues |
| 86 | +- Code smells |
| 87 | +- Security vulnerabilities |
| 88 | +- Maintainability rating |
| 89 | +- Reliability rating |
| 90 | +- Security rating |
| 91 | + |
| 92 | +## Coverage Configuration |
| 93 | + |
| 94 | +The coverage configuration is defined in two places: |
| 95 | + |
| 96 | +1. `.coveragerc` file - Controls how coverage is measured and reported |
| 97 | +2. `pyproject.toml` - Contains the pytest-cov configuration |
| 98 | + |
| 99 | +Key settings include: |
| 100 | +- `source` - Specifies which packages to measure |
| 101 | +- `omit` - Patterns for files to exclude from coverage |
| 102 | +- `exclude_lines` - Patterns for lines to exclude from coverage calculation |
| 103 | +- `fail_under` - Minimum required coverage percentage |
| 104 | + |
| 105 | +## Continuous Integration |
| 106 | + |
| 107 | +Coverage is automatically run as part of our CI/CD pipeline. The GitHub workflow: |
| 108 | + |
| 109 | +1. Runs tests with coverage enabled |
| 110 | +2. Generates coverage reports |
| 111 | +3. Uploads coverage reports as artifacts |
| 112 | +4. Sends coverage data to SonarCloud for analysis |
| 113 | + |
| 114 | +The pipeline uses the exact same SonarCloud configuration that you can run locally with `./run_sonar_scan.sh`. |
| 115 | + |
| 116 | +## Improving Coverage |
| 117 | + |
| 118 | +To improve code coverage: |
| 119 | + |
| 120 | +1. Identify modules with low coverage using the analysis tools |
| 121 | +2. Focus on writing tests for uncovered code paths |
| 122 | +3. Pay special attention to error handling and edge cases |
| 123 | +4. Use parameterized tests for code with many variations |
| 124 | + |
| 125 | +Remember that 100% coverage doesn't guarantee bug-free code, but it does help ensure that most code paths have been executed at least once during testing. |
| 126 | + |
| 127 | +## Minimum Coverage Threshold |
| 128 | + |
| 129 | +The minimum acceptable coverage for this project is **60%**. This threshold is enforced in: |
| 130 | +- The coverage configuration |
| 131 | +- The analysis scripts |
| 132 | +- (Optionally) The CI pipeline |
| 133 | + |
| 134 | +## Tips for Writing Testable Code |
| 135 | + |
| 136 | +- Keep functions small and focused on a single responsibility |
| 137 | +- Extract complex logic into separate testable functions |
| 138 | +- Use dependency injection to make external dependencies mockable |
| 139 | +- Avoid global state when possible |
| 140 | +- Use clear error handling patterns |
0 commit comments