Skip to content

Commit d11850f

Browse files
committed
feat: add assert_stderr, refute_stderr, assert_stderr_line, refute_stderr_line
Refactor the `assert_output` function to improve clarity and maintainability. Introduce a new helper function `__assert_stream` to handle different stream types (output and stderr). Introduce the `assert_stderr` function to validate that the standard error output matches the expected value. Refactor the `refute_output` function to improve stream handling by introducing a new `__refute_stream` function. This change allows for more flexible handling of both `output` and `stderr` streams. This enhances the maintainability and usability of the code. Introduce the `refute_stderr` function to allow verification that a command or function does not produce unexpected stderr. This function complements `assert_stderr` and supports matching via literal, partial, or regular expression. Update documentation to reflect this addition and clarify usage of `assert_stderr` with `--separate-stderr` for proper stderr handling. Add a new assert_stderr_line function for checking expected lines in the stderr output. This improves the clarity and functionality of the assertion process. Refactor the `refute_line` function to handle both output and stderr streams uniformly. Introduce a new helper function `__refute_stream_line` to streamline the logic and reduce code duplication. Introduce the `refute_stderr_line` function to verify that an unexpected line does not appear in the stderr output. Update documentation to reflect usage and necessary conditions for correct operation, including the requirement to use `--separate-stderr` when running commands.
1 parent e2d855b commit d11850f

10 files changed

Lines changed: 1758 additions & 81 deletions

README.md

Lines changed: 242 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,8 @@ This project provides the following functions:
3939
- [assert_output](#assert_output) / [refute_output](#refute_output) Assert output does (or does not) contain given content.
4040
- [assert_line](#assert_line) / [refute_line](#refute_line) Assert a specific line of output does (or does not) contain given content.
4141
- [assert_regex](#assert_regex) / [refute_regex](#refute_regex) Assert a parameter does (or does not) match given pattern.
42+
- [assert_stderr](#assert_stderr) / [refute_stderr](#refute_stderr) Assert stderr does (or does not) contain given content.
43+
- [assert_stderr_line](#assert_stderr_line) / [refute_stderr_line](#refute_stderr_line) Assert a specific line of stderr does (or does not) contain given content.
4244

4345
These commands are described in more detail below.
4446

@@ -750,7 +752,7 @@ Fail if the value (first parameter) matches the pattern (second parameter).
750752

751753
On failure, the value, the pattern and the match are displayed.
752754

753-
```
755+
```bash
754756
@test 'refute_regex()' {
755757
refute_regex 'WhatsApp' 'What.'
756758
}
@@ -777,6 +779,245 @@ For description of the matching behavior, refer to the documentation of the
777779
> Thus, it's good practice to avoid using `BASH_REMATCH` in conjunction with `refute_regex()`.
778780
> The valuable information the array contains is the matching part of the value which is printed in the failing test log, as mentioned above._
779781
782+
### `assert_stderr`
783+
784+
> _**Note**:
785+
> `run` has to be called with `--separate-stderr` to separate stdout and stderr into `$output` and `$stderr`.
786+
> If not, `$stderr` will be empty, causing `assert_stderr` to always fail.
787+
788+
Similarly to `assert_output`, this function verifies that a command or function produces the expected stderr.
789+
The stderr matching can be literal (the default), partial or by regular expression.
790+
The expected stderr can be specified either by positional argument or read from STDIN by passing the `-`/`--stdin` flag.
791+
792+
#### Literal matching
793+
794+
By default, literal matching is performed.
795+
The assertion fails if `$stderr` does not equal the expected stderr.
796+
797+
```bash
798+
echo_err() {
799+
echo "$@" >&2
800+
}
801+
802+
@test 'assert_stderr()' {
803+
run --separate-stderr echo_err 'have'
804+
assert_stderr 'want'
805+
}
806+
807+
@test 'assert_stderr() with pipe' {
808+
run --separate-stderr echo_err 'hello'
809+
echo_err 'hello' | assert_stderr -
810+
}
811+
812+
@test 'assert_stderr() with herestring' {
813+
run --separate-stderr echo_err 'hello'
814+
assert_stderr - <<< hello
815+
}
816+
```
817+
818+
On failure, the expected and actual stderr are displayed.
819+
820+
```
821+
-- stderr differs --
822+
expected : want
823+
actual : have
824+
--
825+
```
826+
827+
#### Existence
828+
829+
To assert that any stderr exists at all, omit the `expected` argument.
830+
831+
```bash
832+
@test 'assert_stderr()' {
833+
run --separate-stderr echo_err 'have'
834+
assert_stderr
835+
}
836+
```
837+
838+
On failure, an error message is displayed.
839+
840+
```
841+
-- no stderr --
842+
expected non-empty stderr, but stderr was empty
843+
--
844+
```
845+
846+
#### Partial matching
847+
848+
Partial matching can be enabled with the `--partial` option (`-p` for short).
849+
When used, the assertion fails if the expected _substring_ is not found in `$stderr`.
850+
851+
```bash
852+
@test 'assert_stderr() partial matching' {
853+
run --separate-stderr echo_err 'ERROR: no such file or directory'
854+
assert_stderr --partial 'SUCCESS'
855+
}
856+
```
857+
858+
On failure, the substring and the stderr are displayed.
859+
860+
```
861+
-- stderr does not contain substring --
862+
substring : SUCCESS
863+
stderr : ERROR: no such file or directory
864+
--
865+
```
866+
867+
#### Regular expression matching
868+
869+
Regular expression matching can be enabled with the `--regexp` option (`-e` for short).
870+
When used, the assertion fails if the *extended regular expression* does not match `$stderr`.
871+
872+
*Note: The anchors `^` and `$` bind to the beginning and the end (respectively) of the entire stderr; not individual lines.*
873+
874+
```bash
875+
@test 'assert_stderr() regular expression matching' {
876+
run --separate-stderr echo_err 'Foobar 0.1.0'
877+
assert_stderr --regexp '^Foobar v[0-9]+\.[0-9]+\.[0-9]$'
878+
}
879+
```
880+
881+
On failure, the regular expression and the stderr are displayed.
882+
883+
```
884+
-- regular expression does not match stderr --
885+
regexp : ^Foobar v[0-9]+\.[0-9]+\.[0-9]$
886+
stderr : Foobar 0.1.0
887+
--
888+
```
889+
890+
### `refute_stderr`
891+
892+
> _**Note**:
893+
> `run` has to be called with `--separate-stderr` to separate stdout and stderr into `$output` and `$stderr`.
894+
> If not, `$stderr` will be empty, causing `refute_stderr` to always pass.
895+
896+
Similar to `refute_output`, this function verifies that a command or function does not produce the unexpected stderr.
897+
(It is the logical complement of `assert_stderr`.)
898+
The stderr matching can be literal (the default), partial or by regular expression.
899+
The unexpected stderr can be specified either by positional argument or read from STDIN by passing the `-`/`--stdin` flag.
900+
901+
### `assert_stderr_line`
902+
903+
> _**Note**:
904+
> `run` has to be called with `--separate-stderr` to separate stdout and stderr into `$output` and `$stderr`.
905+
> If not, `$stderr` will be empty, causing `assert_stderr_line` to always fail.
906+
907+
Similarly to `assert_stderr`, this function verifies that a command or function produces the expected stderr.
908+
It checks that the expected line appears in the stderr (default) or at a specific line number.
909+
Matching can be literal (default), partial or regular expression.
910+
This function is the logical complement of `refute_stderr_line`.
911+
912+
#### Looking for a line in the stderr
913+
914+
By default, the entire stderr is searched for the expected line.
915+
The assertion fails if the expected line is not found in `${stderr_lines[@]}`.
916+
917+
```bash
918+
echo_err() {
919+
echo "$@" >&2
920+
}
921+
922+
@test 'assert_stderr_line() looking for line' {
923+
run --separate-stderr echo_err $'have-0\nhave-1\nhave-2'
924+
assert_stderr_line 'want'
925+
}
926+
```
927+
928+
On failure, the expected line and the stderr are displayed.
929+
930+
```
931+
-- stderr does not contain line --
932+
line : want
933+
stderr (3 lines):
934+
have-0
935+
have-1
936+
have-2
937+
--
938+
```
939+
940+
#### Matching a specific line
941+
942+
When the `--index <idx>` option is used (`-n <idx>` for short), the expected line is matched only against the line identified by the given index.
943+
The assertion fails if the expected line does not equal `${stderr_lines[<idx>]}`.
944+
945+
```bash
946+
@test 'assert_stderr_line() specific line' {
947+
run --separate-stderr echo_err $'have-0\nhave-1\nhave-2'
948+
assert_stderr_line --index 1 'want-1'
949+
}
950+
```
951+
952+
On failure, the index and the compared stderr_lines are displayed.
953+
954+
```
955+
-- line differs --
956+
index : 1
957+
expected : want-1
958+
actual : have-1
959+
--
960+
```
961+
962+
#### Partial matching
963+
964+
Partial matching can be enabled with the `--partial` option (`-p` for short).
965+
When used, a match fails if the expected *substring* is not found in the matched line.
966+
967+
```bash
968+
@test 'assert_stderr_line() partial matching' {
969+
run --separate-stderr echo_err $'have 1\nhave 2\nhave 3'
970+
assert_stderr_line --partial 'want'
971+
}
972+
```
973+
974+
On failure, the same details are displayed as for literal matching, except that the substring replaces the expected line.
975+
976+
```
977+
-- no stderr line contains substring --
978+
substring : want
979+
stderr (3 lines):
980+
have 1
981+
have 2
982+
have 3
983+
--
984+
```
985+
986+
#### Regular expression matching
987+
988+
Regular expression matching can be enabled with the `--regexp` option (`-e` for short).
989+
When used, a match fails if the *extended regular expression* does not match the line being tested.
990+
991+
*Note: As expected, the anchors `^` and `$` bind to the beginning and the end (respectively) of the matched line.*
992+
993+
```bash
994+
@test 'assert_stderr_line() regular expression matching' {
995+
run --separate-stderr echo_err $'have-0\nhave-1\nhave-2'
996+
assert_stderr_line --index 1 --regexp '^want-[0-9]$'
997+
}
998+
```
999+
1000+
On failure, the same details are displayed as for literal matching, except that the regular expression replaces the expected line.
1001+
1002+
```
1003+
-- regular expression does not match line --
1004+
index : 1
1005+
regexp : ^want-[0-9]$
1006+
line : have-1
1007+
--
1008+
```
1009+
1010+
### `refute_stderr_line`
1011+
1012+
> _**Note**:
1013+
> `run` has to be called with `--separate-stderr` to separate stdout and stderr into `$output` and `$stderr`.
1014+
> If not, `$stderr` will be empty, causing `refute_stderr_line` to always pass.
1015+
1016+
Similarly to `refute_stderr`, this function helps to verify that a command or function produces the correct stderr.
1017+
It checks that the unexpected line does not appear in the stderr (default) or in a specific line of it.
1018+
Matching can be literal (default), partial or regular expression.
1019+
This function is the logical complement of `assert_stderr_line`.
1020+
7801021
<!-- REFERENCES -->
7811022
7821023
[bats]: https://github.com/bats-core/bats-core

0 commit comments

Comments
 (0)