README.md

lit - LLVM Integrated Tester Example

• lit docs
• Workflow for this example

This directory shows how to use github-action-benchmark with lit.

Run benchmarks

Install dependencies with venv package using Python3.

$ python -m venv venv
$ source venv/bin/activate
$ pip install lit

Setup your test suite. In this example it consists of a configuration file for lit lit.cfg and two tests a.txt and b.txt containing one RUN line each.

e.g

import lit.formats

config.name = "time-tests"
config.suffixes = [".txt"]
config.test_format = lit.formats.ShTest()
config.test_source_root = None
config.test_exec_root = None

# RUN: true

# RUN: sleep 1

And run the test suite with --resultdb-output in workflow. The JSON file will be an input to github-action-benchmark.

e.g.

- name: Run benchmark
  run: lit examples/lit --resultdb-output output.json

Process benchmark results

Store the benchmark results with step using the action. Please set tool to lit input and pass the path to the output file.

- name: Store benchmark result
  uses: benchmark-action/github-action-benchmark@v1
  with:
    tool: 'lit'
    output-file-path: output.json

Please read 'How to use' section for common usage.