Skip to content

Commit 6a4c5cd

Browse files
authored
doc(proposal): expectFailure label and/or matcher (nodejs#10)
1 parent c437a32 commit 6a4c5cd

1 file changed

Lines changed: 125 additions & 0 deletions

File tree

Lines changed: 125 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,125 @@
1+
# Feature proposal: `expectFailure` label & matcher
2+
3+
## Summary
4+
5+
Update the `expectFailure` option in `test()` to accept different types of values, enabling both **custom label** and **error matching**. This proposal integrates the requirements from [nodejs/node#61570](https://github.com/nodejs/node/issues/61570), ensuring consistency with `skip`/`todo` while adding robust validation capabilities.
6+
7+
## API & Behavior
8+
9+
The behavior of `expectFailure` is strictly determined by the type of value provided:
10+
11+
### String: Failure label
12+
13+
When a **non-empty** `string` is provided, it acts as an output label (reason) for that test-case (identical to `skip` and `todo` options).
14+
15+
```js
16+
test('fails with a specific reason', {
17+
expectFailure: 'Bug #123: Feature not implemented yet'
18+
}, () => {
19+
throw new Error('boom');
20+
});
21+
```
22+
- **Behavior**: The test is expected to fail. The string is treated as a label/reason.
23+
- **Validation**: None. It accepts _any_ error.
24+
- **Output**: The reporter displays the string literally, prefixed by a TAP comment character (e.g., `# EXPECTED FAILURE Bug #123…`).
25+
26+
### Matcher: Error constructor, Error-like object, RegExp, or validation function
27+
28+
When an `Error` constructor, `Error`-like object, `RegExp`, or validation function is provided, it is treated as validation to match against, mirroring the behaviour of [`assert.throws`](https://nodejs.org/docs/latest-v25.x/api/assert.html#assertthrowsfn-error-message) (possibly leveraging it under the hood).
29+
30+
```js
31+
test('fails with matching error (RegExp)', {
32+
expectFailure: /expected error message/
33+
}, () => {
34+
throw new Error('this is the expected error message');
35+
});
36+
37+
test('fails with matching error (Class)', {
38+
expectFailure: RangeError
39+
}, () => {
40+
throw new RangeError('Index out of bounds');
41+
});
42+
```
43+
44+
### Configuration Object: Reason & Validation
45+
46+
When a **Plain Object** with specific properties (`match`, `label`) is provided, it allows specifying both a failure reason and validation logic simultaneously.
47+
48+
```js
49+
test('fails with reason and specific error', {
50+
expectFailure: {
51+
label: 'Bug #123: Edge case behavior', // Reason
52+
match: /Index out of bounds/ // Validation
53+
}
54+
}, () => {
55+
throw new RangeError('Index out of bounds');
56+
});
57+
```
58+
- **Properties**:
59+
- `label` (String): The failure reason/label (displayed in reporter).
60+
- `match` (RegExp | Object | Function | Class): Validation logic. This is passed directly to `assert.throws` validation argument, supporting all its capabilities.
61+
- **Requirement**: The object must contain at least one of `label` or `match`.
62+
- **Behavior**: The test passes **only if** the error matches the `match` criteria.
63+
- **Output**: The reporter displays the `label`.
64+
65+
### Equivalence
66+
67+
The following configurations are equivalent in behavior:
68+
69+
**1. Reason only:**
70+
```js
71+
expectFailure: 'reason';
72+
```
73+
74+
**2. Validation only:**
75+
```js
76+
expectFailure: /error/
77+
expectFailure: { match: /error/ }
78+
```
79+
80+
**3. Catch-all (Any Error):**
81+
```js
82+
expectFailure: true
83+
```
84+
85+
## Ambiguity Resolution
86+
Potential ambiguity between a **Matcher Object** and a **Configuration Object** is resolved as follows:
87+
88+
1. **String** → Reason.
89+
2. **RegExp** or **Function** → Matcher (Validation).
90+
3. **Object**:
91+
* **Empty Object** (`{}`) → **Error**: throws `ERR_INVALID_ARG_VALUE`.
92+
```js
93+
// Uses Node.js standard error code
94+
throw new ERR_INVALID_ARG_VALUE(
95+
'expectFailure',
96+
expectFailure,
97+
'must not be an empty object'
98+
);
99+
```
100+
* If the object contains **only** `match` and/or `label` properties → **Configuration Object**.
101+
* Otherwise → **Matcher Object** (passed to `assert.throws` for property matching).
102+
103+
## Activation & Truthiness
104+
105+
To maintain strict consistency with `todo` and `skip` options:
106+
* The feature is **disabled** only if `expectFailure` is `undefined` or `false`.
107+
* **All other values** enable the feature (treat as truthy).
108+
* `expectFailure: ''` (Empty String) → **Enabled** (treats as generic failure expectation).
109+
* `expectFailure: 0`**Enabled** (treated as a Matcher Object unless specific logic excludes numbers, but per consistency it enables the feature).
110+
111+
### Flat Options (`expectFailureError`)
112+
113+
It was proposed to split the options into `expectFailure` (reason) and `expectFailureError` (validation).
114+
This was rejected in favor of the nested/polymorphic structure using `match` and `label` properties. This syntax was selected as the preferred choice for its readability and clarity:
115+
* `match`: Clearly indicates "fails **matching** this error" (Validation).
116+
* `label`: Clearly indicates the **label** or reason for the expected failure.
117+
This approach keeps related configuration grouped without polluting the top-level options namespace.
118+
119+
## Implementation Details
120+
121+
### Validation Logic
122+
123+
The implementation leverages `assert.throws` internally to perform error validation.
124+
- If `expectFailure` is a Matcher (RegExp, Class, Object), it is passed as the second argument to `assert.throws(fn, expectFailure)`.
125+
- If `expectFailure` is a Configuration Object, `expectFailure.match` is passed to `assert.throws`.

0 commit comments

Comments
 (0)