You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
As mentioned on the page [When and why to run tests?](why_test.qmd), you should run tests regularly **after any code or data changes**, as catching errors earlier makes them easier to fix. This practice of re-running tests is called **regression testing**, and it ensures recent changes haven't introduced errors.
16
+
17
+
GitHub Actions can be a great tool to support this.
18
+
19
+
## GitHub Actions
20
+
21
+
GitHub is widely used for hosting research code and managing version control. We have a [tutorial on setting up a repository](https://pythonhealthdatascience.github.io/des_rap_book/pages/guide/setup/version.html) if you are new to GitHUb.
22
+
23
+
**GitHub Actions** is a built-in automation system that runs workflows directly in your repository. You can access it from the **Actions** tab on your GitHub repository page:
24
+
25
+
26
+
27
+
Workflows are defined using YAML files stored in `.github/workflows/` in your repository. Each workflow can be triggered by one or more events, with common triggers including:
28
+
29
+
*`push`: run tests on every push to a branch.
30
+
*`push: branches: ["main"]`: run tests on every push to the `main` branch.
31
+
*`pull_request`: run tests when a pull request is opened or updated.
32
+
*`workflow_dispatch`: allows manual runs from the "Actions" tab.
33
+
34
+
## Workflow to run tests
35
+
36
+
This workflow will run the tests from our case study via GitHub actions. We explain it step-by-step below.
37
+
38
+
```{bash}
39
+
#| eval: false
40
+
#| file: ../.github/workflows/python_tests.yaml
41
+
```
42
+
43
+
### Explaining the workflow
44
+
45
+
```
46
+
name: python_tests
47
+
run-name: Run python tests
48
+
```
49
+
50
+
The beginning of the YAML sets the workflow's name and how it appears in the Actions tab.
51
+
52
+
*`name` is the internal name of the workflow file.
53
+
*`run-name` is what is displayed when a run appears in the Actions history.
54
+
55
+
```
56
+
on:
57
+
push:
58
+
branches: [main]
59
+
workflow_dispatch:
60
+
```
61
+
62
+
Next, we define when the workflow is triggered. Here we have chosen:
63
+
64
+
*`push`: automatically run on pushes to the `main` branch.
65
+
*`workflow_dispatch`: allows you to trigger the workflow manually from the GitHub actions interface (*note: it only becomes available after first having been pushed to main*).
66
+
67
+
```
68
+
jobs:
69
+
tests:
70
+
runs-on: ${{ matrix.os }}
71
+
strategy:
72
+
fail-fast: false
73
+
matrix:
74
+
include:
75
+
- os: ubuntu-latest
76
+
python-version: '3.11'
77
+
- os: ubuntu-latest
78
+
python-version: '3.12'
79
+
- os: windows-latest
80
+
python-version: '3.12'
81
+
- os: macos-latest
82
+
python-version: '3.12'
83
+
```
84
+
85
+
Now we start to define the job that runs our tests. We are using matrix testing as this allows us to check our code across multiple operating systems and Python versions. In this case the tests will run on:
86
+
87
+
* Python 3.11 (Linux)
88
+
* Python 3.12 (Linux, Windows, macOS)
89
+
90
+
This is good as it allows you to spot any bugs/run issues related to specific operating systems or python versions. They will run in parallel, ensuring a single efficient workflow.
91
+
92
+
```
93
+
steps:
94
+
- name: Check out repository
95
+
uses: actions/checkout@v4
96
+
```
97
+
98
+
Now we start defining the steps executed within our test job. The first step is typically to check out your repository, so the workflow can access your code.
99
+
100
+
```
101
+
- name: Install python and dependencies
102
+
uses: actions/setup-python@v4
103
+
with:
104
+
python-version: ${{ matrix.python-version }}
105
+
cache: 'pip'
106
+
```
107
+
108
+
Next we install the version of Python specified in the matrix and enable pip caching to speed up future runs.
Depending on the operating system, the command syntax for installing dependencies differs slightly. We use `requirements-test.txt` instead of `environment.yaml` as we want to use different Python versions. To reduce runtime, our requirements file only contains packages needed for running tests (e.g., excludes our linting packages).
Finally, we run the tests! We call `pytest` on our case study (`examples/python_package/`). If all tests pass, you'll see green ticks for each environment - confirming that your code works consistently across Python versions and operating systems.
137
+
138
+
### See GitHub actions, in action!
139
+
140
+
The video below demonstrates this workflow running in GitHub Actions. For the demo, the workflow is triggered manually using `workflow_dispatch`, but it would also run automatically whenever you push changes to `main`.
<!-- Test coverage - what it means, how to generate, use, etc. -->
15
+
**Coverage** refers to the percentage of your code that is executed when you run your tests. It can help you spot parts of your code that are not included in any tests.
16
+
17
+
## `pytest-cov`
18
+
19
+
The [pytest-cov](https://github.com/pytest-dev/pytest-cov) package can be used to run coverage calculations easily alongside `pytest`. You can install it from PyPI or conda:
20
+
21
+
```{.bash}
22
+
pip install pytest-cov
23
+
```
24
+
25
+
```{.bash}
26
+
conda install pytest-cov
27
+
```
28
+
29
+
To calculate coverage, you can then simply run tests with the `--cov` flag:
You can see we get nearly 100% coverage. But what does this actually mean?
58
+
59
+
## Interpreting coverage
60
+
61
+
Coverage is telling you whether code was **executed** during testing - but not necessarily whether it has been tested well. A function could run as part of another test without its results or behaviour being properly checked by assertions.
62
+
63
+
::: {.box-grey}
64
+
65
+
**Coverage tells you what code ran, not whether it worked correctly**.
66
+
67
+
:::
68
+
69
+
It's mostly useful for finding code that **isn't covered by tests at all**. Having parts of your code with no/low coverage means:
70
+
71
+
* They're not imported or run by any tests.
72
+
* They're only used in rare branches or failure conditions.
73
+
* They were added recently but have not yet been incorporated into tests.
74
+
75
+
Rather than try to achieve 100% coverage, you should aim to meaningfully test all your code: every important path, decision and behaviour should be tested at least once.
0 commit comments