Flaky http tests tyrian.HttpTests

[davesmith00000]

They are pretty slow and fail intermittently. Here for example the HEAD test passed, but I've seen it fail too, despite no code changes in that area.

tyrian.HttpTests:
  + HEAD 1.33s
  + POST 1.09s
  + PUT 3.28s
  + PATCH 0.40s
  + DELETE 1.65s
==> X tyrian.HttpTests.GET with headers 0.93s munit.ComparisonFailException: 

[davesmith00000]

Annoyingly I can't replicate this now. Will leave the issue open a bit longer...

[JPonte]

Seems like the tests I added 😞 I wonder if httpbin was having issues at the time? That's the downside of depending on an external service for the unit tests...

[davesmith00000]

Yeah it's being flaky again this morning - no worries though, it's still a massive improvement to have tests for this stuff!

I think maybe I'll just have to switch httpbin out for some sort of mock when I get a chance.

[davesmith00000]

I've disable the tests for now and left the following comment in the code while I have a think about what to do next:

    // These tests work great, but http://httpbin.org is unreliable, it seems.
    // So we either need an HTTP mock server compatible with Scala.js, or to use
    // containers (I'd rather not, but...). In the meantime these failing tests
    // are lowering build confidence, so I've disabled them.

I do want to bring them back though! Just not quite sure how best to proceed. httpbin is available as a Docker image, but I really dislike docker. 😋

[JPonte]

Understandable, I also thought about some sbt magic that would start the httpbin container just for the tests but I didn't like the idea and my sbt knowledge is also questionable. I think @armanbilge had some fancy setup in their project with some simple http4s server running only for the tests or something? Maybe we can make an http4sbin 😅

[davesmith00000]

http4sbin

The thought crossed my mind... 😄

[armanbilge]

httpbin is available as a Docker image, but I really dislike docker.

httpbin is actually a python program, so its available like that as well.

https://pypi.org/project/httpbin/

[davesmith00000]

I happened to be manually testing an http call to httpbin this morning and it was a bit slow. I wonder if we could get away with just having retries with a backoff policy...

[JPonte]

Were you thinking of adding the retry policy to the API itself or just for the tests?