Skip to content

Provide report options to separate benchmarks by groups #115

@sobolevn

Description

@sobolevn

Hi, I am using codspeed in my project: https://github.com/wemake-services/django-modern-rest / https://codspeed.io/wemake-services/django-modern-rest

And I really like the experience. Thanks a lot for building it!

I have several different benchmarks for different parts of the app: https://github.com/wemake-services/django-modern-rest/tree/master/benchmarks/tests

Each benchmark is in its own test module.
Currently the report looks like so:

Image

It is already hard to compare different benchmarks. But, we plan to add even more benches.

My proposal is to add an option to group the final report based on modules.

So it would be like so: --codspeed-group=module --codspeed-sort=slowest:

                                 Benchmark Results                                 
                                 test_router.py                                 
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━┓
┃                      Benchmark ┃ Time (best) ┃ Rel. StdDev ┃ Run time ┃   Iters ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━┩
│           test_negotiation_raw │        19ns │        2.5% │    2.95s │ 102,737 │
│        test_router_path_native │      13.1ms │        0.6% │    2.98s │     112 │
│           test_router_path_dmr │      10.9ms │        1.9% │    2.97s │     134 │
└────────────────────────────────┴─────────────┴─────────────┴──────────┴─────────┘

                                 test_negotiation.py                                 
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━┓
┃                      Benchmark ┃ Time (best) ┃ Rel. StdDev ┃ Run time ┃   Iters ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━┩
│ test_negotiation_django_native │       275ns │        2.0% │    2.93s │  27,051 │
│      test_negotiation_compiled │         8ns │        2.6% │    2.87s │ 153,786 │
└────────────────────────────────┴─────────────┴─────────────┴──────────┴─────────┘

Now - everything is clear:

  • Everything is groupped by module from --codspeed-group=module
  • Everything is sorted by the Time (best) from --codspeed-sort=slowest

What do you think? :)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions