Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

httpbin does not reflect requests perfectly #198

Closed
theseion opened this issue Nov 6, 2023 · 4 comments
Closed

httpbin does not reflect requests perfectly #198

theseion opened this issue Nov 6, 2023 · 4 comments

Comments

@theseion
Copy link
Collaborator

theseion commented Nov 6, 2023

httpbin usually returns the request data in the response exactly as sent. At least in case of Unicode that is not true, unfortunately. Example:

curl -X POST "https://httpbin.org/anything" --json '{"über": "bort"}'
{
  "args": {},
  "data": "{\"\u00fcber\": \"bort\"}",
  "files": {},
  "form": {},
  "headers": {
    "Accept": "application/json",
    "Content-Length": "17",
    "Content-Type": "application/json",
    "Host": "httpbin.org",
    "User-Agent": "curl/8.1.2",
    "X-Amzn-Trace-Id": "Root=1-6548859d-09da901d70e79647362f7b83"
  },
  "json": {
    "\u00fcber": "bort"
  },
  "method": "POST",
  "origin": "89.133.9.17",
  "url": "https://httpbin.org/anything"
}

The Unicode sequence is returned as a backslash escape sequence instead. That is semantically correct but bad for testing. We need to find a way around this.

@M4tteoP
Copy link
Member

M4tteoP commented Dec 30, 2023

The httpbin go port (go-httpbin) is behaving in a better way:

▶ go run github.com/mccutchen/go-httpbin/v2/cmd/go-httpbin@v2.9 -port 8080 & 
▶ curl -X POST localhost:8080/anything --json '{"über": "bort"}'
{
  "args": {},
  "headers": {
    "Accept": [
      "application/json"
    ],
    "Content-Length": [
      "17"
    ],
    "Content-Type": [
      "application/json"
    ],
    "Host": [
      "localhost:8080"
    ],
    "User-Agent": [
      "curl/8.1.2"
    ]
  },
  "method": "POST",
  "origin": "127.0.0.1:54971",
  "url": "http://localhost:8080/anything",
  "data": "{\"über\": \"bort\"}",
  "files": null,
  "form": null,
  "json": {
    "über": "bort"
  }
}

Around Coraza we are already using it in some places as the backend to test the CRS, moving to it should be pretty smooth

@theseion
Copy link
Collaborator Author

theseion commented Jan 5, 2024

Great! I'll take a look.

theseion added a commit to theseion/coreruleset that referenced this issue Jan 7, 2024
httpbin does not properly reflect Unicode sequences but returns JSON
Unicode escape sequences instead. This can break response tests that
rely on the data in the response to be an exact copy of the request
data. See coreruleset/go-ftw#198.

This PR switches the backend for tests from httpbin to the
go-httpbin port, as the port behaves correctly.
theseion added a commit to theseion/coreruleset that referenced this issue Jan 7, 2024
httpbin does not properly reflect Unicode sequences but returns JSON
Unicode escape sequences instead. This can break response tests that
rely on the data in the response to be an exact copy of the request
data. See coreruleset/go-ftw#198.

This PR switches the backend for tests from httpbin to the
go-httpbin port, as the port behaves correctly.
@theseion
Copy link
Collaborator Author

theseion commented Jan 7, 2024

coreruleset/coreruleset#3464 fixes the issue for the CRS test suite but is not a generic solution for go-ftw.

theseion added a commit to coreruleset/coreruleset that referenced this issue Jan 8, 2024
httpbin does not properly reflect Unicode sequences but returns JSON
Unicode escape sequences instead. This can break response tests that
rely on the data in the response to be an exact copy of the request
data. See coreruleset/go-ftw#198.

This PR switches the backend for tests from httpbin to the
go-httpbin port, as the port behaves correctly.
@theseion
Copy link
Collaborator Author

theseion commented Jun 5, 2024

We now have albedo.

@theseion theseion closed this as completed Jun 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants