Wishlist: capture output of print statements in answer and test cells

[callegar]

I am just starting to experiment with nbgrader, so please take my apology if this issue is something that can be done already or inconsistent with the nbgrader workflow.

I wonder it it might be possible to assure that when you validate or autograde the python code is run with the stdout redirected, so that statements like print do not just print, but also save their output into some string that can be used in the assertions in hidden tests.

[lahwaacz]

You can just put this in a module and import it from a protected/locked cell in the assignment:

import builtins
from builtins import print as builtin_print

stdout = ""

def print(*values, sep=" ", end="\n", **kwargs):
    global stdout
    stdout += sep.join([str(value) for value in values]) + end
    builtin_print(*values, sep=sep, end=end, **kwargs)

builtins.print = print

if __name__ == "__main__":
    print("foo", 42, 3.14, sep="\t", end="\n\n")
    assert stdout == "foo\t42\t3.14\n\n"

[callegar]

Redefining the print function is a possibility I had indeed thought of. However, it ends up requiring an extra module (which might not need distribution, being needed only for grading, if one imports it in a previous invisible test). Yet I wonder:

1. whether it might be possible to read the printouts directly from the data structures of the notebook, being that they appear on it;
2. if there might be interest in nbgrader to incorporate a similar functionality in its autograding module.

[shreve]

nbgrader is built on top of a whole ecosystem, which includes some stuff that you may find helpful. For example, IPython's %%capture may do exactly what you want.

%%capture variable_name

print("Hello, world!")

assert variable_name.stdout == "Hello, world!\n"

[callegar]

Thanks! My point is however better integrating these parts into nbgrader. For instance can I put the %%capture inside a notebook cell so that the student version won't have it and the auto-grader version will?

[shreve]

There are a lot of different ways you could approach the design of this. Could you illustrate your desired use case a bit more?

A lot of the assignments in the program I'm supporting include solutions inside functions or accept answers as string variables rather than printing out.

def do_the_thing() -> float:
    ### BEGIN SOLUTION
    return 4.2
    ### END SOLUTION

yes_or_no: Literal["yes", "no"]
### BEGIN SOLUTION
yes_or_no = "no"
### END SOLUTION

assert np.isclose(do_the_thing(), 4.2)
assert yes_or_no.lower()[0] == "n"

Could the print statements you're looking to capture be done inside a function you can control the call of?

For example:

def student_answer():
    ### BEGIN SOLUTION
    print("the answer")
    ### END SOLUTION

import io
from contextlib import redirect_stdout

f = io.StringIO()
with redirect_stdout(f):
    student_answer()
out = f.getvalue()
assert out == "the answer"