Provide a proper way to define expected response data

[theseion]

Currently, go-ftw only provides a way to send requests. Because httpbin reflects the request body in the response body, we can use a request specification to specify the expected response. However, from the perspective of a test author that is not the correct way to describe the behaviour. Instead, there should be a way to specify both the request and the expected response.

[floyd-fuh]

it would also be nice to have a response_not_contains as well as OR logic. Imagine a WAF that has a true negative (no blocking, e.g. a request is fine), I would like to specify something like "this should be a 404 OR a redirect to /home", because depending on the used URI, the response either says this is an unknown location or does the default behavior of redirecting to the start page

[theseion]

Negative response check will be added.

Not sure about disjunction logic though. The way you describe it, you would be checking two different URLs and thus should use two different tests. Could you give a concrete example?

[floyd-fuh]

Isn't the status: [200, 300, 305]syntax already a disjunction logic?

For the disjunction logic: In cloud mode, we often know the "block" answer (e.g. an HTTP 400 is returned), which in most cases is very static.

The question is how do we formulate a "no-block" test? It depends very much on the URI the test case is sending, usually the backend will send back some 200, 3XX or 404. In this particular case the block case is a 302 to /400.html whereas the no-block case is a 200 or a 302 to /404.html.

Of course for most cases it would be enough to have a negative response check, saying response_not_contains: 400.html.

[theseion]

Isn't the status: [200, 300, 305] syntax already a disjunction logic?

Yes, you're right. But I am in the process of dropping that to a single status code. The reason we had the option to supply more than one status code was for running the same test against different backends that behave differently. For example, nginx might return 400 for some invalid requests, while httpd would accept them (or the other way around). We will have a better approach in the future.

The question is how do we formulate a "no-block" test? It depends very much on the URI the test case is sending, usually the backend will send back some 200, 3XX or 404. In this particular case the block case is a 302 to /400.html whereas the no-block case is a 200 or a 302 to /404.html.

If it depends on the URL, then I still don't see how you would use a single test for that, since you can only specify a single target URL per test. For the single URL you would know whether the no-block case is 200 or 302 to /404.html, wouldn't you?

I'm not trying to prove you wrong, I just want to understand the use case before we implement something like disjunction logic (I like to avoid work ;)).

[theseion]

We now have albedo.