Skip to content

Commit a10f6ce

Browse files
d-biehld-biehl
committed
feat(runner): add the standard result filters to results diff
`results diff` now accepts the same filter options as the other `results` subcommands: `--status`, `-i/--include`, `-e/--exclude`, `-s/--suite`, `-t/--test`, `-bl/--by-longname`, `-ebl/--exclude-by-longname`, plus `--search` / `--search-regex`. Each filter is applied identically to both baseline and current before the diff, so the comparison can be scoped to a single suite, tag pattern, name, or message snippet: ``` robotcode results diff baseline.xml -s "MyProject.Login" robotcode results diff baseline.xml -i smoke robotcode results diff baseline.xml --search TimeoutError robotcode results diff baseline.xml -bl "MyProject.Login.Bad Password" ``` `DiffResult` gains a `filtersApplied` field (additive, optional) so the JSON output records the filters in the same shape used by the other result models.
1 parent 4311747 commit a10f6ce

3 files changed

Lines changed: 143 additions & 28 deletions

File tree

‎docs/03_reference/cli.md‎

Lines changed: 73 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -833,12 +833,12 @@ robotcode debug [OPTIONS] [ROBOT_OPTIONS_AND_ARGS]...
833833

834834
- `--tcp [<ADDRESS>:]<PORT>`
835835

836-
Run in `tcp` server mode and listen at the given port. (Equivalent to `--mode tcp --port <port>`) *NOTE:* This option is mutually exclusive with options: pipe-name, mode, pipe-server, port.
836+
Run in `tcp` server mode and listen at the given port. (Equivalent to `--mode tcp --port <port>`) *NOTE:* This option is mutually exclusive with options: pipe-name, mode, port, pipe-server.
837837

838838

839839
- `--pipe-server NAME`
840840

841-
Run in `pipe-server` mode and listen at the given pipe name. (Equivalent to `--mode pipe-server --pipe-name <name>`) *NOTE:* This option is mutually exclusive with options: pipe-name, mode, tcp, port, bind.
841+
Run in `pipe-server` mode and listen at the given pipe name. (Equivalent to `--mode pipe-server --pipe-name <name>`) *NOTE:* This option is mutually exclusive with options: port, bind, pipe-name, tcp, mode.
842842

843843

844844
- `--mode [tcp|pipe_server]`
@@ -858,7 +858,7 @@ robotcode debug [OPTIONS] [ROBOT_OPTIONS_AND_ARGS]...
858858

859859
- `--pipe-name NAME`
860860

861-
The pipe to listen on or connect to. (Only valid in `pipe` and `pipe-server` mode) *NOTE:* This option is mutually exclusive with options: bind, tcp, pipe-server, port. [env var: ROBOTCODE_PIPE_NAME]
861+
The pipe to listen on or connect to. (Only valid in `pipe` and `pipe-server` mode) *NOTE:* This option is mutually exclusive with options: port, bind, tcp, pipe-server. [env var: ROBOTCODE_PIPE_NAME]
862862

863863

864864
- `--version`
@@ -885,47 +885,47 @@ robotcode debug-launch [OPTIONS]
885885
**Options:**
886886
- `--stdio`
887887

888-
Run in `stdio` mode. (Equivalent to `--mode stdio`) *NOTE:* This option is mutually exclusive with options: pipe-name, mode, pipe-server, tcp, port, socket, pipe, bind. [env var: ROBOTCODE_STDIO]
888+
Run in `stdio` mode. (Equivalent to `--mode stdio`) *NOTE:* This option is mutually exclusive with options: port, bind, socket, pipe, pipe-name, tcp, pipe-server, mode. [env var: ROBOTCODE_STDIO]
889889

890890

891891
- `--tcp [<ADDRESS>:]<PORT>`
892892

893-
Run in `tcp` server mode and listen at the given port. (Equivalent to `--mode tcp --port <port>`) *NOTE:* This option is mutually exclusive with options: pipe-name, mode, pipe-server, port, socket, pipe, stdio.
893+
Run in `tcp` server mode and listen at the given port. (Equivalent to `--mode tcp --port <port>`) *NOTE:* This option is mutually exclusive with options: port, socket, pipe, pipe-name, pipe-server, stdio, mode.
894894

895895

896896
- `--socket [<ADDRESS>:]<PORT>`
897897

898-
Run in `socket` mode and connect to the given port. (Equivalent to `--mode socket --port <port>`) *NOTE:* This option is mutually exclusive with options: pipe-name, mode, pipe-server, tcp, port, pipe, stdio.
898+
Run in `socket` mode and connect to the given port. (Equivalent to `--mode socket --port <port>`) *NOTE:* This option is mutually exclusive with options: port, pipe, pipe-name, tcp, pipe-server, stdio, mode.
899899

900900

901901
- `--pipe NAME`
902902

903-
Run in `pipe` mode and connect to the given pipe name. (Equivalent to `--mode pipe --pipe-name <name>`) *NOTE:* This option is mutually exclusive with options: pipe-name, mode, pipe-server, tcp, port, socket, bind, stdio.
903+
Run in `pipe` mode and connect to the given pipe name. (Equivalent to `--mode pipe --pipe-name <name>`) *NOTE:* This option is mutually exclusive with options: port, bind, socket, pipe-name, tcp, pipe-server, stdio, mode.
904904

905905

906906
- `--pipe-server NAME`
907907

908-
Run in `pipe-server` mode and listen at the given pipe name. (Equivalent to `--mode pipe-server --pipe-name <name>`) *NOTE:* This option is mutually exclusive with options: pipe-name, mode, tcp, port, socket, pipe, bind, stdio.
908+
Run in `pipe-server` mode and listen at the given pipe name. (Equivalent to `--mode pipe-server --pipe-name <name>`) *NOTE:* This option is mutually exclusive with options: port, bind, socket, pipe, pipe-name, tcp, stdio, mode.
909909

910910

911911
- `--mode [stdio|tcp|socket|pipe|pipe_server]`
912912

913-
The mode to use for the debug launch server. *NOTE:* This option is mutually exclusive with options: tcp, pipe-server, socket, pipe, stdio. [env var: ROBOTCODE_MODE; default: STDIO]
913+
The mode to use for the debug launch server. *NOTE:* This option is mutually exclusive with options: socket, pipe, tcp, pipe-server, stdio. [env var: ROBOTCODE_MODE; default: STDIO]
914914

915915

916916
- `--port PORT`
917917

918-
The port to listen on or connect to. (Only valid for `tcp` and `socket mode`) *NOTE:* This option is mutually exclusive with options: pipe-name, pipe-server, pipe. [env var: ROBOTCODE_PORT; default: 6611; 1<=x<=65535]
918+
The port to listen on or connect to. (Only valid for `tcp` and `socket mode`) *NOTE:* This option is mutually exclusive with options: pipe-name, pipe, pipe-server. [env var: ROBOTCODE_PORT; default: 6611; 1<=x<=65535]
919919

920920

921921
- `--bind ADDRESS *`
922922

923-
Specify alternate bind address. If no address is specified `localhost` is used. (Only valid for tcp and socket mode) *NOTE:* This option is mutually exclusive with options: pipe-name, pipe-server, pipe. [env var: ROBOTCODE_BIND; default: 127.0.0.1]
923+
Specify alternate bind address. If no address is specified `localhost` is used. (Only valid for tcp and socket mode) *NOTE:* This option is mutually exclusive with options: pipe-name, pipe, pipe-server. [env var: ROBOTCODE_BIND; default: 127.0.0.1]
924924

925925

926926
- `--pipe-name NAME`
927927

928-
The pipe to listen on or connect to. (Only valid in `pipe` and `pipe-server` mode) *NOTE:* This option is mutually exclusive with options: tcp, pipe-server, port, socket, pipe, bind, stdio. [env var: ROBOTCODE_PIPE_NAME]
928+
The pipe to listen on or connect to. (Only valid in `pipe` and `pipe-server` mode) *NOTE:* This option is mutually exclusive with options: port, bind, socket, pipe, tcp, pipe-server, stdio. [env var: ROBOTCODE_PIPE_NAME]
929929

930930

931931
- `--version`
@@ -1355,22 +1355,22 @@ robotcode language-server [OPTIONS] [PATHS]...
13551355
**Options:**
13561356
- `--stdio`
13571357

1358-
Run in `stdio` mode. (Equivalent to `--mode stdio`) *NOTE:* This option is mutually exclusive with options: pipe-name, mode, tcp, port, socket, pipe, bind. [env var: ROBOTCODE_STDIO]
1358+
Run in `stdio` mode. (Equivalent to `--mode stdio`) *NOTE:* This option is mutually exclusive with options: port, bind, socket, pipe, pipe-name, tcp, mode. [env var: ROBOTCODE_STDIO]
13591359

13601360

13611361
- `--tcp [<ADDRESS>:]<PORT>`
13621362

1363-
Run in `tcp` server mode and listen at the given port. (Equivalent to `--mode tcp --port <port>`) *NOTE:* This option is mutually exclusive with options: pipe-name, mode, port, socket, pipe, stdio.
1363+
Run in `tcp` server mode and listen at the given port. (Equivalent to `--mode tcp --port <port>`) *NOTE:* This option is mutually exclusive with options: port, socket, pipe, pipe-name, stdio, mode.
13641364

13651365

13661366
- `--socket [<ADDRESS>:]<PORT>`
13671367

1368-
Run in `socket` mode and connect to the given port. (Equivalent to `--mode socket --port <port>`) *NOTE:* This option is mutually exclusive with options: pipe-name, mode, tcp, port, pipe, stdio.
1368+
Run in `socket` mode and connect to the given port. (Equivalent to `--mode socket --port <port>`) *NOTE:* This option is mutually exclusive with options: port, pipe, pipe-name, tcp, stdio, mode.
13691369

13701370

13711371
- `--pipe NAME`
13721372

1373-
Run in `pipe` mode and connect to the given pipe name. (Equivalent to `--mode pipe --pipe-name <name>`) *NOTE:* This option is mutually exclusive with options: pipe-name, mode, tcp, port, socket, bind, stdio.
1373+
Run in `pipe` mode and connect to the given pipe name. (Equivalent to `--mode pipe --pipe-name <name>`) *NOTE:* This option is mutually exclusive with options: port, bind, socket, pipe-name, tcp, stdio, mode.
13741374

13751375

13761376
- `--mode [socket|tcp|pipe|stdio]`
@@ -1390,7 +1390,7 @@ robotcode language-server [OPTIONS] [PATHS]...
13901390

13911391
- `--pipe-name NAME`
13921392

1393-
The pipe to listen on or connect to. (Only valid in `pipe` and `pipe-server` mode) *NOTE:* This option is mutually exclusive with options: tcp, port, socket, pipe, bind, stdio. [env var: ROBOTCODE_PIPE_NAME]
1393+
The pipe to listen on or connect to. (Only valid in `pipe` and `pipe-server` mode) *NOTE:* This option is mutually exclusive with options: port, bind, socket, pipe, tcp, stdio. [env var: ROBOTCODE_PIPE_NAME]
13941394

13951395

13961396
- `--version`
@@ -1632,47 +1632,47 @@ robotcode repl-server [OPTIONS] [FILES]...
16321632
**Options:**
16331633
- `--stdio`
16341634

1635-
Run in `stdio` mode. (Equivalent to `--mode stdio`) *NOTE:* This option is mutually exclusive with options: pipe-name, mode, pipe-server, tcp, port, socket, pipe, bind. [env var: ROBOTCODE_STDIO]
1635+
Run in `stdio` mode. (Equivalent to `--mode stdio`) *NOTE:* This option is mutually exclusive with options: port, bind, socket, pipe, pipe-name, tcp, pipe-server, mode. [env var: ROBOTCODE_STDIO]
16361636

16371637

16381638
- `--tcp [<ADDRESS>:]<PORT>`
16391639

1640-
Run in `tcp` server mode and listen at the given port. (Equivalent to `--mode tcp --port <port>`) *NOTE:* This option is mutually exclusive with options: pipe-name, mode, pipe-server, port, socket, pipe, stdio.
1640+
Run in `tcp` server mode and listen at the given port. (Equivalent to `--mode tcp --port <port>`) *NOTE:* This option is mutually exclusive with options: port, socket, pipe, pipe-name, pipe-server, stdio, mode.
16411641

16421642

16431643
- `--socket [<ADDRESS>:]<PORT>`
16441644

1645-
Run in `socket` mode and connect to the given port. (Equivalent to `--mode socket --port <port>`) *NOTE:* This option is mutually exclusive with options: pipe-name, mode, pipe-server, tcp, port, pipe, stdio.
1645+
Run in `socket` mode and connect to the given port. (Equivalent to `--mode socket --port <port>`) *NOTE:* This option is mutually exclusive with options: port, pipe, pipe-name, tcp, pipe-server, stdio, mode.
16461646

16471647

16481648
- `--pipe NAME`
16491649

1650-
Run in `pipe` mode and connect to the given pipe name. (Equivalent to `--mode pipe --pipe-name <name>`) *NOTE:* This option is mutually exclusive with options: pipe-name, mode, pipe-server, tcp, port, socket, bind, stdio.
1650+
Run in `pipe` mode and connect to the given pipe name. (Equivalent to `--mode pipe --pipe-name <name>`) *NOTE:* This option is mutually exclusive with options: port, bind, socket, pipe-name, tcp, pipe-server, stdio, mode.
16511651

16521652

16531653
- `--pipe-server NAME`
16541654

1655-
Run in `pipe-server` mode and listen at the given pipe name. (Equivalent to `--mode pipe-server --pipe-name <name>`) *NOTE:* This option is mutually exclusive with options: pipe-name, mode, tcp, port, socket, pipe, bind, stdio.
1655+
Run in `pipe-server` mode and listen at the given pipe name. (Equivalent to `--mode pipe-server --pipe-name <name>`) *NOTE:* This option is mutually exclusive with options: port, bind, socket, pipe, pipe-name, tcp, stdio, mode.
16561656

16571657

16581658
- `--mode [stdio|tcp|socket|pipe|pipe_server]`
16591659

1660-
The mode to use for the debug launch server. *NOTE:* This option is mutually exclusive with options: tcp, pipe-server, socket, pipe, stdio. [env var: ROBOTCODE_MODE; default: STDIO]
1660+
The mode to use for the debug launch server. *NOTE:* This option is mutually exclusive with options: socket, pipe, tcp, pipe-server, stdio. [env var: ROBOTCODE_MODE; default: STDIO]
16611661

16621662

16631663
- `--port PORT`
16641664

1665-
The port to listen on or connect to. (Only valid for `tcp` and `socket mode`) *NOTE:* This option is mutually exclusive with options: pipe-name, pipe-server, pipe. [env var: ROBOTCODE_PORT; default: 6601; 1<=x<=65535]
1665+
The port to listen on or connect to. (Only valid for `tcp` and `socket mode`) *NOTE:* This option is mutually exclusive with options: pipe-name, pipe, pipe-server. [env var: ROBOTCODE_PORT; default: 6601; 1<=x<=65535]
16661666

16671667

16681668
- `--bind ADDRESS *`
16691669

1670-
Specify alternate bind address. If no address is specified `localhost` is used. (Only valid for tcp and socket mode) *NOTE:* This option is mutually exclusive with options: pipe-name, pipe-server, pipe. [env var: ROBOTCODE_BIND; default: 127.0.0.1]
1670+
Specify alternate bind address. If no address is specified `localhost` is used. (Only valid for tcp and socket mode) *NOTE:* This option is mutually exclusive with options: pipe-name, pipe, pipe-server. [env var: ROBOTCODE_BIND; default: 127.0.0.1]
16711671

16721672

16731673
- `--pipe-name NAME`
16741674

1675-
The pipe to listen on or connect to. (Only valid in `pipe` and `pipe-server` mode) *NOTE:* This option is mutually exclusive with options: tcp, pipe-server, port, socket, pipe, bind, stdio. [env var: ROBOTCODE_PIPE_NAME]
1675+
The pipe to listen on or connect to. (Only valid in `pipe` and `pipe-server` mode) *NOTE:* This option is mutually exclusive with options: port, bind, socket, pipe, tcp, pipe-server, stdio. [env var: ROBOTCODE_PIPE_NAME]
16761676

16771677

16781678
- `-v, --variable name:value *`
@@ -1796,6 +1796,8 @@ Examples:
17961796
robotcode results diff baseline.xml
17971797
robotcode results diff prev/output.xml curr/output.xml
17981798
robotcode results diff baseline.xml --only new-failures
1799+
robotcode results diff baseline.xml -i smoke
1800+
robotcode results diff baseline.xml --search TimeoutError
17991801
robotcode --format json results diff baseline.xml
18001802
```
18011803

@@ -1807,6 +1809,51 @@ robotcode results diff [OPTIONS] BASELINE [CURRENT]
18071809

18081810

18091811
**Options:**
1812+
- `--status [pass|fail|skip|not-run] *`
1813+
1814+
Only include tests with one of these statuses.
1815+
1816+
1817+
- `-i, --include TAG_PATTERN *`
1818+
1819+
Include tests matching the tag pattern. Supports Robot's tag pattern syntax (AND, OR, NOT, *, ?).
1820+
1821+
1822+
- `-e, --exclude TAG_PATTERN *`
1823+
1824+
Exclude tests matching the tag pattern. Same syntax as --include.
1825+
1826+
1827+
- `-s, --suite NAME *`
1828+
1829+
Only include tests inside the named suite (glob against full suite name).
1830+
1831+
1832+
- `-t, --test, --task NAME *`
1833+
1834+
Only include tests whose name matches (glob against full test name). `--task` is an alias for `--test` (Robot's RPA terminology).
1835+
1836+
1837+
- `-bl, --by-longname NAME *`
1838+
1839+
Select tests/tasks or suites by long name (exact match).
1840+
1841+
1842+
- `-ebl, --exclude-by-longname NAME *`
1843+
1844+
Exclude tests/tasks or suites by long name (exact match).
1845+
1846+
1847+
- `--search TEXT`
1848+
1849+
Only include tests with at least one case-insensitive substring match against TEXT. Searches test name, full name, failure message, tags, keyword names, keyword arguments, and log messages. Mutually exclusive with `--search-regex`.
1850+
1851+
1852+
- `--search-regex PATTERN`
1853+
1854+
Only include tests with at least one match against PATTERN (Python regular expression, case-sensitive — prefix with `(?i)` for case-insensitive). Same target fields as `--search`. Mutually exclusive with `--search`.
1855+
1856+
18101857
- `--full-paths / --no-full-paths`
18111858

18121859
Show absolute source paths instead of paths relative to cwd. [default: no-full-paths]

‎packages/runner/src/robotcode/runner/cli/results/_models.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -199,3 +199,4 @@ class DiffResult(CamelSnakeMixin):
199199
status_changes: Optional[List[DiffChange]] = None
200200
added: Optional[List[DiffChange]] = None
201201
removed: Optional[List[DiffChange]] = None
202+
filters_applied: Optional[Dict[str, List[str]]] = None

‎packages/runner/src/robotcode/runner/cli/results/results.py‎

Lines changed: 69 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -890,6 +890,7 @@ def _bump(name: str, status: str, secs: Optional[float]) -> None:
890890
type=click.Path(path_type=Path, exists=True, dir_okay=False, readable=True),
891891
required=False,
892892
)
893+
@add_options(*RESULT_FILTER_OPTIONS, *SEARCH_OPTIONS)
893894
@click.option(
894895
"--full-paths/--no-full-paths",
895896
default=False,
@@ -916,6 +917,15 @@ def diff(
916917
app: Application,
917918
baseline: Path,
918919
current: Optional[Path],
920+
status_filters: Tuple[str, ...],
921+
include_tags: Tuple[str, ...],
922+
exclude_tags: Tuple[str, ...],
923+
suite_globs: Tuple[str, ...],
924+
test_globs: Tuple[str, ...],
925+
by_longname: Tuple[str, ...],
926+
exclude_by_longname: Tuple[str, ...],
927+
search_substring: Optional[str],
928+
search_regex: Optional[str],
919929
full_paths: bool,
920930
message_chars: int,
921931
only_categories: Tuple[str, ...],
@@ -932,6 +942,8 @@ def diff(
932942
robotcode results diff baseline.xml
933943
robotcode results diff prev/output.xml curr/output.xml
934944
robotcode results diff baseline.xml --only new-failures
945+
robotcode results diff baseline.xml -i smoke
946+
robotcode results diff baseline.xml --search TimeoutError
935947
robotcode --format json results diff baseline.xml
936948
```
937949
"""
@@ -946,8 +958,39 @@ def diff(
946958
baseline_exec = _load_execution_result(baseline_path)
947959
current_exec = _load_execution_result(current_path)
948960

949-
baseline_tests = {_get_full_name(t): t for t in _iter_all_tests(baseline_exec.suite)}
950-
current_tests = {_get_full_name(t): t for t in _iter_all_tests(current_exec.suite)}
961+
tree_filter = any([include_tags, exclude_tags, suite_globs, test_globs, by_longname, exclude_by_longname])
962+
if tree_filter:
963+
_apply_tree_filters(
964+
baseline_exec.suite,
965+
include_tags,
966+
exclude_tags,
967+
suite_globs,
968+
test_globs,
969+
by_longname,
970+
exclude_by_longname,
971+
)
972+
_apply_tree_filters(
973+
current_exec.suite,
974+
include_tags,
975+
exclude_tags,
976+
suite_globs,
977+
test_globs,
978+
by_longname,
979+
exclude_by_longname,
980+
)
981+
982+
wanted = _normalise_statuses(status_filters)
983+
match = _make_matcher(search_substring, search_regex)
984+
985+
def _eligible(t: Any) -> bool:
986+
if wanted and t.status not in wanted:
987+
return False
988+
if match is not None and not _raw_test_search_matches(t, match):
989+
return False
990+
return True
991+
992+
baseline_tests = {_get_full_name(t): t for t in _iter_all_tests(baseline_exec.suite) if _eligible(t)}
993+
current_tests = {_get_full_name(t): t for t in _iter_all_tests(current_exec.suite) if _eligible(t)}
951994

952995
new_failures: List[DiffChange] = []
953996
new_passes: List[DiffChange] = []
@@ -976,6 +1019,17 @@ def diff(
9761019

9771020
selected = {c.lower() for c in only_categories} if only_categories else None
9781021

1022+
filters_active = bool(
1023+
status_filters
1024+
or include_tags
1025+
or exclude_tags
1026+
or suite_globs
1027+
or test_globs
1028+
or by_longname
1029+
or exclude_by_longname
1030+
or search_substring
1031+
or search_regex
1032+
)
9791033
data = DiffResult(
9801034
baseline=_make_file_info(baseline_path),
9811035
current=_make_file_info(current_path),
@@ -984,6 +1038,19 @@ def diff(
9841038
status_changes=status_changes if selected is None or "status-changes" in selected else None,
9851039
added=added if selected is None or "added" in selected else None,
9861040
removed=removed if selected is None or "removed" in selected else None,
1041+
filters_applied=_filters_dict(
1042+
status_filters,
1043+
include_tags,
1044+
exclude_tags,
1045+
suite_globs,
1046+
test_globs,
1047+
by_longname,
1048+
exclude_by_longname,
1049+
search_substring,
1050+
search_regex,
1051+
)
1052+
if filters_active
1053+
else None,
9871054
)
9881055

9891056
if app.config.output_format in (None, OutputFormat.TEXT):

0 commit comments

Comments
 (0)