httpx.RequestNotRead: Attempted to access request content, without having called read()

[max-iov42]

I would like to test if a client uses the REST interface in a proper way. I wanted to use respx to mock the HTTP requests. I found no easy way to test the content of the HTTP request sent to the service.

What I would expect:

• I can use the content attribute on the Request object returned by the calls to verify correctness of the request.

What happens:

• Accessing the content attribute throws an httpx.RequestNotRead exception
• The workaround I have found is to access the stream._body attribute which is kind of a hack. Or maybe I don not use the library in a proper way and the documentation should show this kind of example.

Used Environment:

• Python 3.8.5
• pytest 6.0.2
• httpx 0.15.5
• respx 0.13.0

Here the minimal example code:

import json
import respx
import httpx


def my_client():
    httpx.post("http://www.example.com/entries", content=json.dumps({"id": "1234567"}))
    return True


@respx.mock
def test_rest_call():
    """If the client sends the correct HTTP request - url and content."""
    respx.post(
        "http://www.example.com/entries",
        alias="example",
    )
    my_client()
    assert respx.aliases["example"].called
    assert respx.aliases["example"].call_count == 1
    request,_ = respx.aliases["example"].calls[0]
    
    # This works but is kinda ugly
    assert request.stream._body == '{"id": "1234567"}'.encode()
    # Would expect to work, but throws exception 'httpx.RequestNotRead'
    assert request.content == '{"id": "1234567"}'.encode()

Calling this with pytest:

$ pytest tests/test_request.py 
============================================================ test session starts =============================================================
platform linux -- Python 3.8.5, pytest-6.0.2, py-1.9.0, pluggy-0.13.1
rootdir: /home/max/work/iov42/python/core-sdk-python
plugins: xdoctest-0.15.0, typeguard-2.9.1
collected 1 item                                                                                                                             

tests/test_request.py F                                                                                                                [100%]

================================================================== FAILURES ==================================================================
_______________________________________________________________ test_rest_call _______________________________________________________________

    @respx.mock
    def test_rest_call():
        respx.post(
            "http://www.example.com/entries",
            alias="example",
        )
        my_client()
        assert respx.aliases["example"].called
        assert respx.aliases["example"].call_count == 1
        request,_ = respx.aliases["example"].calls[0]
    
        # This works but is kinda ugly
        assert request.stream._body == '{"id": "1234567"}'.encode()
        # Would expect to work, but does not work
>       assert request.content == '{"id": "1234567"}'.encode()

tests/test_request.py:26: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Request('POST', 'http://www.example.com/entries')>

    @property
    def content(self) -> bytes:
        if not hasattr(self, "_content"):
>           raise RequestNotRead()
E           httpx.RequestNotRead: Attempted to access request content, without having called `read()`.

/home/max/.cache/pypoetry/virtualenvs/iov42-core-python-4oIM64sE-py3.8/lib/python3.8/site-packages/httpx/_models.py:823: RequestNotRead
========================================================== short test summary info ===========================================================
FAILED tests/test_request.py::test_rest_call - httpx.RequestNotRead: Attempted to access request content, without having called `read()`.
============================================================= 1 failed in 0.17s ==============================================================

[lundberg]

Thanks for finding and raising this issue!

You should be able to solve it non-hacky in your test by calling request.read() before asserting request.content.

That being said, I don't think it should be needed, since both currently open PR #82 and closed #68 deals with pre-reading stream/content of the Response, for the same reason and to simplify usage of the stats/calls history.

It's quite easy to add a request pre-read to #82 that should solve your issue. Will try to add this before merging it.

FYI, the RESPX pass-through feature is what makes this "hard", since if we always pre-read request and response for a pattern marked as pass-through, then the sent request body and returned response by the HTTPX client will be "exhausted".

[max-iov42]

An alternative solution would be to enhance the request "method" call with a matching functionality on content ("content matcher" parameter - similar to what pytest_httpx is providing).

[lundberg]

Do you mean that if we extend RequestPattern to be able to match on certain request content, then reading of the request content in call history is not that important?

Sounds like a great feature anyhow, will add a separate issue for that. I think we should support both.

[max-iov42]

Yes, I meant matching the RequestPattern with content. And I agree with you, supporting both would be nice.

You have just to be careful, to not overwhelm users of respx with a plethora of parameters. That's why I find respx "feels better" compared to pytest_httpx.

[lundberg]

FYI @max-iov42, #106 adds support for content pattern matching, etc. 😉