Skip to content

Commit 7509c4d

Browse files
committed
Added EXAMPLES.md with in-depth usage examples for evaluation functions and Result handling.
1 parent 713f13f commit 7509c4d

1 file changed

Lines changed: 120 additions & 0 deletions

File tree

EXAMPLES.md

Lines changed: 120 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,120 @@
1+
# Examples
2+
3+
## Basic correct/incorrect result
4+
5+
```python
6+
from lf_toolkit.evaluation import Result, Params
7+
8+
def eval_function(response: str, answer: str, params: Params) -> Result:
9+
is_correct = response.strip() == answer.strip()
10+
return Result(is_correct=is_correct)
11+
```
12+
13+
## Adding feedback
14+
15+
```python
16+
def eval_function(response: str, answer: str, params: Params) -> Result:
17+
is_correct = response.strip() == answer.strip()
18+
result = Result(is_correct=is_correct)
19+
20+
if not is_correct:
21+
result.add_feedback("hint", "Check your spelling.")
22+
result.add_feedback("hint", f'The correct answer starts with "{answer[0]}".')
23+
24+
return result
25+
```
26+
27+
Feedback items are grouped by tag. Common tags are `"hint"` and `"error"`. All messages for a tag are joined with `<br>` when displayed to the student.
28+
29+
## Feedback in the constructor
30+
31+
```python
32+
result = Result(
33+
is_correct=False,
34+
feedback_items=[
35+
("hint", "Try simplifying your fraction first."),
36+
("error", "Sign error in the second term."),
37+
],
38+
)
39+
```
40+
41+
## Comparing numbers
42+
43+
```python
44+
def eval_function(response: str, answer: str, params: Params) -> Result:
45+
try:
46+
is_correct = float(response.strip()) == float(answer.strip())
47+
except ValueError:
48+
result = Result(is_correct=False)
49+
result.add_feedback("error", "Your answer must be a number.")
50+
return result
51+
52+
result = Result(is_correct=is_correct)
53+
if not is_correct:
54+
result.add_feedback("hint", f"The expected answer is {answer}.")
55+
return result
56+
```
57+
58+
## Reading Params
59+
60+
`Params` carries configuration set by the question author. Always provide a default in case the key is not set.
61+
62+
```python
63+
def eval_function(response: str, answer: str, params: Params) -> Result:
64+
is_latex = params.get("is_latex", False)
65+
should_simplify = params.get("simplify", True)
66+
symbols = params.get("symbols", {})
67+
68+
# symbols looks like:
69+
# {"x": {"latex": "x", "aliases": []}, "alpha": {"latex": r"\alpha", "aliases": ["a"]}}
70+
...
71+
```
72+
73+
## Using symbols from Params
74+
75+
```python
76+
def eval_function(response: str, answer: str, params: Params) -> Result:
77+
symbols = params.get("symbols", {})
78+
79+
allowed_names = set(symbols.keys())
80+
for name, data in symbols.items():
81+
allowed_names.update(data.get("aliases", []))
82+
83+
# check response only uses allowed symbol names
84+
...
85+
```
86+
87+
## Including parsed representations in Result
88+
89+
If you parse the student's response, include the canonical forms so the platform can display them.
90+
91+
```python
92+
result = Result(
93+
is_correct=True,
94+
latex=r"\frac{1}{2}",
95+
simplified="1/2",
96+
)
97+
```
98+
99+
## Async eval function
100+
101+
```python
102+
async def eval_function(response: str, answer: str, params: Params) -> Result:
103+
result = await some_async_check(response, answer)
104+
return Result(is_correct=result)
105+
```
106+
107+
## Testing a Result
108+
109+
```python
110+
from lf_toolkit.evaluation import Result
111+
112+
def test_correct_answer():
113+
result = eval_function("42", "42", {})
114+
assert result.is_correct
115+
116+
def test_wrong_answer_has_hint():
117+
result = eval_function("41", "42", {})
118+
assert not result.is_correct
119+
assert result.get_feedback("hint") != []
120+
```

0 commit comments

Comments
 (0)