|
| 1 | +import Admonition from "@theme/Admonition"; |
| 2 | + |
| 3 | +# How to configure selectivity when running tests |
| 4 | + |
| 5 | +## Introduction |
| 6 | + |
| 7 | +Selectivity allows you to significantly speed up the testing process by running only relevant tests instead of the entire test suite. Testplane tracks each test's dependencies on project files — both the test code itself and the code executed in the browser — and, when a file changes, runs only those tests that depend on it. |
| 8 | + |
| 9 | +## How does it work? |
| 10 | + |
| 11 | +On the first run with selectivity enabled, Testplane collects dependency information for each test: |
| 12 | + |
| 13 | +- which Node.js modules were loaded while the test was running; |
| 14 | +- which source files were executed in the browser. |
| 15 | + |
| 16 | +After a file changes, on the next run only those tests that depend on the changed file will be executed. This saves a lot of time, especially in large projects with many tests. |
| 17 | + |
| 18 | +<Admonition type="info"> |
| 19 | + If at least one test fails, then on the next run all the same tests will be executed again — |
| 20 | + Testplane will "remember" the new state only after a fully successful run. |
| 21 | +</Admonition> |
| 22 | + |
| 23 | +## Setup |
| 24 | + |
| 25 | +To enable selectivity, just add a [`selectivity`](/docs/v8/config/browsers#selectivity) section with `enabled: true` to your Testplane configuration: |
| 26 | + |
| 27 | +```typescript |
| 28 | +// testplane.config.ts |
| 29 | +export default { |
| 30 | + // ... Other configs |
| 31 | + selectivity: { |
| 32 | + enabled: true, |
| 33 | + }, |
| 34 | +} satisfies import("testplane").ConfigInput; |
| 35 | +``` |
| 36 | + |
| 37 | +## Usage example |
| 38 | + |
| 39 | +Let’s enable selectivity in a project with `chrome` and `firefox` browsers. For convenience, we’ll also configure a [devServer][dev-server]: |
| 40 | + |
| 41 | +```ts |
| 42 | +// testplane.config.ts |
| 43 | +const DEV_SERVER_PORT = 3050; |
| 44 | + |
| 45 | +export default { |
| 46 | + baseUrl: `http://127.0.0.1:${DEV_SERVER_PORT}`, |
| 47 | + // ... Other settings |
| 48 | + devServer: { |
| 49 | + command: `npm run dev -- --port=${DEV_SERVER_PORT}`, |
| 50 | + reuseExisting: true, |
| 51 | + readinessProbe: { |
| 52 | + url: `http://127.0.0.1:${DEV_SERVER_PORT}`, |
| 53 | + }, |
| 54 | + }, |
| 55 | + selectivity: { |
| 56 | + enabled: true, |
| 57 | + }, |
| 58 | +} satisfies import("testplane").ConfigInput; |
| 59 | +``` |
| 60 | + |
| 61 | +We’ll also enable SourceMap generation in the Webpack config used by the project: |
| 62 | + |
| 63 | +```js |
| 64 | +// webpack.config.js |
| 65 | +module.exports = (env, argv) => { |
| 66 | + return { |
| 67 | + // ... |
| 68 | + // Use multiple entry points |
| 69 | + entry: { |
| 70 | + main: './js/home.js', |
| 71 | + about: './js/about.js', |
| 72 | + contact: './js/contact.js', |
| 73 | + shared: './js/shared.js' |
| 74 | + }, |
| 75 | + // Enable SourceMap generation |
| 76 | + // Both external (`source-map`) and inline (`inline-source-map`) are suitable |
| 77 | + devtool: 'source-map', |
| 78 | + output: { |
| 79 | + // Specify a template that Webpack will use to store paths to source files |
| 80 | + // `[resource-path]` or `[absolute-resource-path]` will also work |
| 81 | + devtoolModuleFilenameTemplate: "webpack://[resource-path]", |
| 82 | + }, |
| 83 | +``` |
| 84 | +
|
| 85 | +For the example we’ll use a plain JavaScript project without React, with the following file structure: |
| 86 | +
|
| 87 | +``` |
| 88 | +. |
| 89 | +|____css |
| 90 | +| |____home.css |
| 91 | +| |____shared.css |
| 92 | +| |____about.css |
| 93 | +|____js |
| 94 | +| |____about.js |
| 95 | +| |____home.js |
| 96 | +| |____main.js |
| 97 | +| |____contact.js |
| 98 | +| |____shared.js |
| 99 | +|____index.html |
| 100 | +|____about.html |
| 101 | +|____contact.html |
| 102 | +|____webpack.config.js |
| 103 | +|____testplane.config.ts |
| 104 | +|____testplane-tests |
| 105 | +| |____example.testplane.ts |
| 106 | +| |____tsconfig.json |
| 107 | +|____package.json |
| 108 | +|____package-lock.json |
| 109 | +``` |
| 110 | +
|
| 111 | +Let’s write the following tests: |
| 112 | +
|
| 113 | +```ts |
| 114 | +describe("test examples", () => { |
| 115 | + it("main page", async ({ browser }) => { |
| 116 | + await browser.url("/"); |
| 117 | + }); |
| 118 | + |
| 119 | + it("main page with navigation to about page", async ({ browser }) => { |
| 120 | + await browser.url("/"); |
| 121 | + await browser.$("#about-link").click(); |
| 122 | + }); |
| 123 | + |
| 124 | + it("main page with navigation to contact page", async ({ browser }) => { |
| 125 | + await browser.url("/"); |
| 126 | + await browser.$("#contact-link").click(); |
| 127 | + }); |
| 128 | + |
| 129 | + it("contact page", async ({ browser }) => { |
| 130 | + await browser.url("/contact"); |
| 131 | + }); |
| 132 | +}); |
| 133 | +``` |
| 134 | +
|
| 135 | +And run Testplane. For clarity, we’ll use the `DEBUG=testplane:selectivity` environment variable. |
| 136 | +
|
| 137 | +On the first launch Testplane will run all tests and recorded their dependencies: |
| 138 | +
|
| 139 | + |
| 140 | +
|
| 141 | +Let’s try running it again. As there are no changes, running all tests is skipped: |
| 142 | +
|
| 143 | + |
| 144 | +
|
| 145 | +Now let’s change a couple of source files that are executed in the browser, and only one test that depends on the changed files will be run. The rest will be skipped by selectivity. |
| 146 | +
|
| 147 | + |
| 148 | +
|
| 149 | +The same applies to test code selectivity: let’s create `./testplane-tests/my-module.ts`: |
| 150 | +
|
| 151 | +```ts |
| 152 | +export const foo = () => { |
| 153 | + throw new Error("bar"); |
| 154 | +}; |
| 155 | +``` |
| 156 | +
|
| 157 | +And use it in the test file: |
| 158 | +
|
| 159 | +```ts |
| 160 | +import { foo } from "./my-module"; |
| 161 | + |
| 162 | +describe("test examples", () => { |
| 163 | + it("main page", async ({ browser }) => { |
| 164 | + foo(); |
| 165 | + await browser.url("/"); |
| 166 | + }); |
| 167 | + // The rest of the tests |
| 168 | +}); |
| 169 | +``` |
| 170 | +
|
| 171 | +Now while processing the test file Testplane noticed that it now depends on another file, which means the functionality might have changed, so all tests in this file will be run: |
| 172 | +
|
| 173 | + |
| 174 | +
|
| 175 | +Since the tests failed, the new state was not saved, so re-running the tests will run the same set of tests again. |
| 176 | +
|
| 177 | +## Important details |
| 178 | +
|
| 179 | +### Collecting browser dependencies |
| 180 | +
|
| 181 | +Testplane records browser dependencies only in the Chrome browser, because dependency collection works via the [Chrome Devtools Protocol](/docs/v8/reference/webdriver-vs-cdp). However, if you have multiple browsers (for example, Chrome and Firefox), Firefox will still be able to use the list of browser dependencies for a test that was collected in Chrome. |
| 182 | +
|
| 183 | +### SourceMap requirement |
| 184 | +
|
| 185 | +For selectivity to work correctly, **SourceMap generation is required**. Thanks to SourceMaps, the browser can understand which source file the executed code belongs to. If a SourceMap is not generated for a code file, Testplane won’t be able to determine which source file the executed code comes from, and selectivity will not work correctly. |
| 186 | +
|
| 187 | +### Server-side code and disableSelectivityPatterns |
| 188 | +
|
| 189 | +Testplane cannot track code that runs on your server, regardless of the language it’s written in. Therefore, it’s recommended to add your server-side code to the `disableSelectivityPatterns` option — any change to files matching these patterns will disable selectivity and run all tests. |
| 190 | +
|
| 191 | +If you are using the [reactStrictMode](https://nextjs.org/docs/pages/api-reference/config/next-config-js/reactStrictMode) option of the [Next.js](https://nextjs.org/) framework, rendering pages in the browser as well will allow Testplane to account for these files as if this were a regular SPA application. However, code executed exclusively on the server (such as API request handlers) will not be considered by Testplane’s selectivity. |
| 192 | +
|
| 193 | +You should also add your Testplane configuration files (for example, `testplane.config.ts`) to `disableSelectivityPatterns`, since changing them can affect the behavior of any test. |
| 194 | +
|
| 195 | +<Admonition type="warning"> |
| 196 | + Do not add `node_modules` to `disableSelectivityPatterns` — this will significantly slow things |
| 197 | + down. Testplane tracks the used modules from `node_modules` on its own. |
| 198 | +</Admonition> |
| 199 | +
|
| 200 | +Example `disableSelectivityPatterns` configuration: |
| 201 | +
|
| 202 | +```typescript |
| 203 | +selectivity: { |
| 204 | + enabled: true, |
| 205 | + disableSelectivityPatterns: [ |
| 206 | + "testplane.config.ts", |
| 207 | + "backend/**/*.py", // server-side code in Python |
| 208 | + "server/**/*.go", // server-side code in Go |
| 209 | + ] |
| 210 | +} |
| 211 | +``` |
| 212 | +
|
| 213 | +### Manually disabling selectivity |
| 214 | +
|
| 215 | +When you use other filtering methods when running Testplane (such as specifying a path to a test file or using the `--grep` and `--set` options), selectivity is automatically disabled. |
| 216 | +
|
| 217 | +You can also disable selectivity manually using an environment variable: |
| 218 | +
|
| 219 | +```bash |
| 220 | +testplane_selectivity_enabled=false # disable |
| 221 | +``` |
| 222 | +
|
| 223 | +## Configuration |
| 224 | +
|
| 225 | +Additional information about selectivity configuration is provided in the [browsers-selectivity][selectivity-config] section of the documentation. |
| 226 | +
|
| 227 | +[dev-server]: ../../config/dev-server |
| 228 | +[selectivity-config]: ../../config/browsers#selectivity |
0 commit comments