Speed test

This folder contains configuration and expectations to run the OTP speed test. The test runs automatically after each merged PR and the results are visualised on a Grafana instance.

Each run is listed under the GitHub Actions.

If you need to run the test locally you need to download the fixed datasets (OSM, transit) listed below and build a graph from them first.

After the graph is built, copy it into ./test/performance/${location} and execute the following command to run the speed test (also make sure the tests are compiled with e.g. mvn test-compile):

mvn --projects application exec:java -Dexec.mainClass="org.opentripplanner.transit.speed_test.SpeedTest" -Dexec.classpathScope=test -Dexec.args="--dir=test/performance/${location} -p md -n 4 -i 3 -0"

The results will be displayed in the console.

CI

The test is run after every merge to dev-2.x. Its GitHub Actions workflow is defined in performance-test.yml.

Running the test manually

Sometimes it is desirable to run the test configuration in the CI environment manually, for example, when making changes to the tests. To do this, perform these steps:

Create a local branch in git that contains the desired performance test configuration.
In performance-test.yml add your test branch name to the configuration and change all those locations that you want to run on the branch to core:

name: Performance test

on:
  push:
    branches:
      - dev-2.x
+     - test-branch-name

jobs:
  perf-test:
    if: github.repository_owner == 'opentripplanner' && !startsWith(github.event.head_commit.message ,'Bump serialization version id for') && !startsWith(github.event.head_commit.message ,'Upgrade debug client to version')
    runs-on: performance-test
    strategy:
      fail-fast: false
      matrix:
        include:
          ...
          - location: helsinki
            iterations: 1
            jfr-delay: "50s"
-           profile: extended
+           profile: core
...

Commit the changes to performance-test.yml.
Push the changes to a branch in the upstream OpenTripPlanner repository with the same name you added to performance-test.yml.
The tests will run after the push.

Note In order to visualise before and after, it is helpful to push a commit that is the same as the current dev-2.x and run the speed test on that.

Instrumentation

Each run on CI is instrumented with Java Flight Recorder. The results are then saved as an artifact and can be downloaded. IntelliJ for example can display a very useful flame graph from those jrf files that shows bottlenecks in the code.

Configure the test

Pick a valid "testDate" for your data set and set it in the speed-test-config.json.
Make sure build-config "transitServiceStart" and "transitServiceEnd" include the "testDate".