Pytest Capsys: Checking Output And Getting It Reported?

Python 3.4.1, pytest 2.6.2. When a test fails, pytest will routinely report what was printed to stdout by the test. For instance this code: def method_under_test(): print('Hall

Solution 1:

You're seeing the correct behaviour, when using capsys.readouterr() you're consuming the captured output. Hence any output to stdout and stderr will no longer show up in the test report. But any new output which you create after this and do not consume will still be reported, so you can get the full output back in the report by simply writing it to the output streams once more:

def test_result_and_stdout(capsys):
    result = method_under_test()
    out, err = capsys.readouterr()
    sys.stdout.write(out)
    sys.stderr.write(err)
    assert out.startswith("Hello")
    assert result == 42

Solution 2:

From the documentation the behavior seems correct: only after the test function (test_result_and_stdout) finishes the output streams will be restored, not after each readouterr call. I don't think that currently capsys supports also redirecting to the original streams besides capturing them, which seems to be what you want.

I would suggest to create an issue in the official repository and see what people have to say.

Solution 3:

Asides using "startswith".You can also use the "in" keyword, for example:

assert"Hello"inoutput

This is great if you have a huge amount of data being passed to stdout, you can use "in" to check for different lines in your stdout.

def test_result_and_stdout(capsys):
    result = method_under_test()
    out, err = capsys.readouterr()
    sys.stdout.write(out)
    sys.stderr.write(err)
    assert"Hello"in out 
    assert result == 42

You can also assert what's passed into stderr instead of stdout by using:

assert"What you are expecting" in err

Also note that line:

out, err = capsys.readouterr()

creates a snapshot of the output to both stdout and stderr so far so you can assert what you are expecting for that test.

Solution 4:

As @flub describes, when using capsys, outputs to stdout and stderr are captured and consumed, which is the intended behaviour of capsys. You can write the messages back to stdout and stderr after capturing them if you want to see the messages in the test error capture.

However, I find that a common pattern is to run a couple of commands in series as part of the same test, typically because of state changes after one command affecting the behaviour of the next. When testing verbose outputs, you'll want to test the against the contents of the output of a specific command in the test, not the whole history of outputs which you've running during the test. In order to isolate the output from a single command, you need to capture stdout/stderr before calling the command as well as afterwards. That way you capture and discard the previous stdout contents before printing to stdout for an isolated command.

Unfortunately, this doesn't sit well with sending the messages back out to stdout and stderr after capturing them, since if you do that they will pollute the output from the next command within the test.

My solution is to make a helper function, recapsys, that plays the role of capturing and immediately regurgitating the contents of stdout/stderr (like how @flub describes). In addition, recapsys accepts arbitrarily many previous captures as arguments and will regurgitate these before the current capture. With this, you can capture previous stdout/stderr content before calling the command under test, and then print them out after running the command.

The definition for recapsys, and an example usage is given below. (For a class-based implementation of recapsys, check out the class methods in my python template repository.)

# Contents of test_fn.pyimport sys


defrecapsys(capsys, *captures):
    capture_now = capsys.readouterr()
    for capture in captures + (capture_now,):
        sys.stdout.write(capture.out)
        sys.stderr.write(capture.err)
    return capture_now


deftest_stdout(capsys):
    capture_pre = capsys.readouterr()  # Clear stdout
    msg = "To be, or not to be, that is the question:"print(msg)  # Execute method (verbose)
    capture_post = recapsys(capsys, capture_pre)  # Capture and then re-outputassert capture_post.out == msg + "\n"# Compare output to target

    capture_pre = capsys.readouterr()  # Clear stdout
    msg = "Whether 'tis nobler in the mind to suffer"print(msg)  # Execute method (verbose)
    capture_post = recapsys(capsys, capture_pre)  # Capture and then re-outputassert capture_post.out.lower().startswith("whether")  # Test

    capture_pre = capsys.readouterr()  # Clear stdout
    msg = "The slings and arrows of outrageous fortune,"print(msg)  # Execute method (verbose)
    capture_post = recapsys(capsys, capture_pre)  # Capture and then re-outputassert capture_post.out == "We're no strangers to love\n"# Test

Running pytest on test_fn.py produces the following output:

===================================== FAILURES ======================================
____________________________________ test_stdout ____________________________________

capsys = <_pytest.capture.CaptureFixture object at 0x7f49015ce278>

    def test_stdout(capsys):
        capture_pre = capsys.readouterr()  # Clear stdout
        msg = "To be, or not to be, that is the question:"print(msg)  # Execute method (verbose)
        capture_post = recapsys(capsys, capture_pre)  # Capture and then re-output
        assert capture_post.out == msg + "\n"# Compare output to target
    
        capture_pre = capsys.readouterr()  # Clear stdout
        msg = "Whether 'tis nobler in the mind to suffer"print(msg)  # Execute method (verbose)
        capture_post = recapsys(capsys, capture_pre)  # Capture and then re-output
        assert capture_post.out.lower().startswith("whether")  # Test
    
        capture_pre = capsys.readouterr()  # Clear stdout
        msg = "The slings and arrows of outrageous fortune,"print(msg)  # Execute method (verbose)
        capture_post = recapsys(capsys, capture_pre)  # Capture and then re-output
>       assert capture_post.out == "We're no strangers to love\n"# Test
E       assert 'The slings a...us fortune,\n' == "We're no strangers to love\n"
E         - We're no strangers to love
E         + The slings and arrows of outrageous fortune,

test_fn.py:30: AssertionError
------------------------------- Captured stdout call --------------------------------
To be, or not to be, that is the question:
Whether 'tis nobler in the mind to suffer
The slings and arrows of outrageous fortune,
============================== short test summary info ==============================
FAILED test_fn.py::test_stdout - assert 'The slings a...us fortune,\n' == "We're n...
================================= 1 failed in 0.26s =================================