Skip to content

Commit ff15ca3

Browse files
authored
Update index.md
1 parent 2e0f93f commit ff15ca3

1 file changed

Lines changed: 4 additions & 6 deletions

File tree

docs/index.md

Lines changed: 4 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,5 @@
11
# python-ragability
22

3-
*IMPORTANT*: this project is still under development and may change significantly in the near future!
4-
53
A library and corpus for checking/benchmarking LLMs with regard to properties relevant to their use in RAG
64
systems.
75

@@ -19,22 +17,22 @@ systems.
1917
* Activate conda environment: `conda activate ragability`
2018
* After installation, the following commands are available:
2119
* `ragability_info` : show the versions of all relevant Python packages installed
22-
" `ragability-cc-wc1` : convert the wiki-contradict based corpus to ragability format
20+
* `ragability_cc_wc1` : convert the wiki-contradict based corpus to ragability format
2321
* `ragability_query` : given an input file with facts and queries, a list of candidate LLMs and a prompt template, produce and output file that contains
2422
LLM answers to the queries
2523
* `ragability_check` : given a query result file and a judge LLM, evaluate the answers received against the pre-defined answers and create an output file that contains the evaluation scores and meta-information for each example
2624
* `ragability_eval` : given the data created with `ragability_check`, calculate detailled performance statistics
27-
* `ragability_hjson_info` : show some infor about the number of entries and keys present in a hjson, json, or jsonl file
25+
* `ragability_hjson_info` : provides an overview of the major information present in a hjson, json, or jsonl file
2826
* `ragability_hjson_cat` : concatenate several hjson, json or jsonl files into one hjson or json file
2927
* `llms_wrapper_test`: for the configured/specified LLMs, test if they are working and returning an answer. This is useful to check a config file and
3028
test if all the API keys are correctly set and working
3129
* All commands take the `--help` option to get usage information
3230

3331
LLMs currently supported: support is based on the [LiteLLM](https://github.com/BerriAI/litellm) backend via the [llms_wrapper](https://github.com/OFAI/python-llms-wrapper/) package. The supported LLMs are listed [here](https://docs.litellm.ai/docs/providers/)
3432

35-
### Current usage
33+
### Example usage
3634

37-
Example usage with the converted wiki-contradict dataset:
35+
with the converted wiki-contradict dataset:
3836

3937
* (optional conversion step, the current converted dataset is already part of the repo): convert the dataset tsv file to ragability format
4038
* `ragability_cc_wc1 --input corpus/wikicontradict1/Dataset_v0.2_short.tsv --output corpus/wikicontradict1/v0d2.hjson`

0 commit comments

Comments
 (0)