Is docker-compose-rule a dead project?

[jannis-baratheon]

Looking at commit history and lack of responses to pull requests/issues it seems that development stopped. Is this official? Should we all move to test-containers? Can someone from Palantir's team share an official statement?

[dotCipher]

As of right now, we haven't been able to invest any resources into continuing to build docker-compose-rule, and since a suitable alternative currently exists (testcontainers' docker compose integration), we advise all current users of docker-compose-rule to migrate to testcontainers going forward.

We plan to update the repo with a more formal statement with a guide for migration soon ™️ . Apologies for any inconvenience this might cause.

[jannis-baratheon]

That's a pity. I hope Testcontainers improve Windows support.

[bsideup]

Hi @jannis-baratheon!

Testcontainers team member here :) Do you have a concrete issue with Windows support, or? We invested a significant time into Windows support, did tests on our machines and the current state of it is quite stable (at least on Windows 10 with Docker for Windows, which is the recommended combination anyways)

[jannis-baratheon]

I've revisited testcontainers recently and I must say it looks much better than 6 months back in regard to Windows support. We have testcontainers migration in our short-term backlog. will let you know how it went. BR

[k9m]

Hi guys, just wanted to thank you for this tool this is exactly what I have been looking for. Saves so much effort not having need to set everything up at build time and then passing in the random port numbers to the test suite.

[wesleyk]

Hey folks! Wondering if you've gotten around to adding a more formal statement. I recently started integrating with docker-compose-rule but happened to notice this question. Figured it would be valuable to others if the project stated its position re: being unmaintained.

[jannis-baratheon]

@bsideup We've finished migrating a few months back. Testcontainers has been working really well throughout this time on both Windows and Linux machines (I'm pretty sure we've got folks working on OSXes as well). To all the future readers of this comment: I highly recommend migrating to TC. Guys there are doing a great job delivering a fine product and what's more important - they have a great community providing quality support. BR

[jimfcarroll]

It seems calling testcontainers a "suitable replacement" is an extreme overstatement. It doesn't support anywhere near the features supported by this plugin. It runs docker-from-docker and starts multiple extraneous containers to proxy, making debugging completely opaque. When a container wont start there's no reason given, no way to check (e.g. you need to maintain the service name in both the docker-compose.yml file as well as your code - try figuring out what the problem is when they don't match). Can't even set the container name, can't use network hosts, etc.

EDIT:
A few more:

• can't run on well defined ports and so you're required to programmatically determine the service URI/URL/port rather than be able to test configuration as part of your test.
• can't supply a shutdown strategy and so you can't leave the container up after the test for debugging and analysis
• No JUnit5 support

[jannis-baratheon]

@jimfcarroll Are we talking about the same Testcontainers? I don't really have time now to verify all objections you've raised, but I'm pretty sure at least some of them seem to be misplaced. For example on the subject of JUnit 5: https://www.testcontainers.org/test_framework_integration/junit_5/

[jimfcarroll]

Thanks. Corrected. Plus slightly toned down. Was really frustrating experience. Ended up sticking with plantir and fixing the problem w/Java 16.

EDIT: Your link was to junit 5 integration in general and I guess their custom annotations are supported in both JUnit 4 and 5 but it's the docker-compose module that replaces what plantir has here. THAT doesn't seem to have junit 5 support. So I removed the strikthrough

[bsideup]

@jimfcarroll I am sorry to hear that you had a frustrating experience. To be honest, I am surprised that, for so much frustration, you did not a single issue to Testcontainers:
https://github.com/testcontainers/testcontainers-java/issues?q=is%3Aissue+author%3Ajimfcarroll++
Consider reporting next time, so that we know about your problems and can provide help when it is needed :)

Let me iterate over it and maybe clarify some things:

It runs docker-from-docker

Testcontainers does not run any docker-from-docker (or docker-in-docker). For convenience, we do provide an option to run the docker-compose binary in a container, just so that you don't need to install it on your machine, but you can always use the local one as described here: https://www.testcontainers.org/modules/docker_compose/#local-compose-mode

starts multiple extraneous containers to proxy

We only start one additional container, so called "Ambassador container". You can read more about it here:
https://github.com/testcontainers/testcontainers-java/blob/26be859459a8596a58791f4b7101fe7026a3d5ba/core/src/main/java/org/testcontainers/containers/DockerComposeContainer.java#L370-L382
I haven't seen any issue reports from you saying that this causes any troubles (also, I'd say you're the first to say that). Consider creating an issue in Testcontainers, and we will be happy to help with it (e.g. add a special mode that removes the ambassador and relies on exposed ports only). Perhaps you could also explain in the issue what makes it hard to debug.

When a container wont start there's no reason given, no way to check

Testcontainers will print the output of docker-compose. Also, if wait strategy fails, it will say what condition wasn't satisfied. If you need more debug information, consider reporting an issue.

you need to maintain the service name in both the docker-compose.yml file as well as your code - try figuring out what the problem is when they don't match

We could try to fail fast, yes, but I don't really get why "the name must match" isn't obvious. Also, DCR has the same requirement in waitingForService

Can't even set the container name
Testcontainers focuses on isolation, while container_name directive would prevent running two tests in parallel on CIs. So yes, it is not supported (although there is an open PR to ignore it), but there is a good reason why.

can't use network hosts, etc.

You can use networks the same way as you do it with Docker Compose, so I am not sure I understand this one (also "etc" here is a little bit weird, would love to see it expanded, as I am not aware of any other limitation).

can't run on well defined ports and so you're required to programmatically determine the service URI/URL/port rather than be able to test configuration as part of your test.

Please see this article: https://bsideup.github.io/posts/testcontainers_fixed_ports/
tl;dr: you really should be programmatically determining the URLs, and not just port, but also the host. The reason for that is because, if you hardcode localhost:5432, then your tests will fail on Jenkins, for example, if you're running them on a container already. Or fail due to the port conflicts.
That's the same with DCR, btw: https://github.com/palantir/docker-compose-rule#accessing-services-in-containers-from-outside-a-container

can't supply a shutdown strategy and so you can't leave the container up after the test for debugging and analysis

One of the top priorities for Testcontainers is to cleanup after the test execution, so that there are no leftovers. Have you considered debugging your tests as usual, and inspecting the container while you're on a breakpoint? Nothing will be stopped while your testing session is still running.

No JUnit5 support

As mentioned by @jannis-baratheon, Testcontainers supports JUnit 5, and you can use @Container annotation with DockerComposeContainer as well. Have you tried?

[jimfcarroll]

Testcontainers does not run any docker-from-docker (or docker-in-docker). For convenience, we do provide an option to run the docker-compose binary in a container, just so that you don't need to install it on your machine, but you can always use the local one as described here: https://www.testcontainers.org/modules/docker_compose/#local-compose-mode

starts multiple extraneous containers to proxy

We only start one additional container, so called "Ambassador container".

When using the default and not running local docker compose it starts the ambassador container and also a docker-compose container. But okay. When you run it using a local docker compose it only starts the ambassador container.

I haven't seen any issue reports from you saying that this causes any troubles (also, I'd say you're the first to say that). Consider creating an issue in Testcontainers, and we will be happy to help with it (e.g. add a special mode that removes the ambassador and relies on exposed ports only). Perhaps you could also explain in the issue what makes it hard to debug.

I mentioned "you need to maintain the service name in both the docker-compose.yml file as well as your code - try figuring out what the problem is when they don't match."

We could try to fail fast, yes, but I don't really get why "the name must match" isn't obvious. Also, DCR has the same requirement in waitingForService

The problem isn't that it's not obvious that the names must match, it's the fact that there's a piece of configuration information defined in 2 places. Also, I used that as an example of how to generate sparse debug information.

Can't even set the container name
Testcontainers focuses on isolation, while container_name directive would prevent running two tests in parallel on CIs. So yes, it is not supported (although there is an open PR to ignore it), but there is a good reason why.

can't use network hosts, etc.

You can use networks the same way as you do it with Docker Compose, so I am not sure I understand this one (also "etc" here is a little bit weird, would love to see it expanded, as I am not aware of any other limitation).

Are you saying setting the network mode to host in docker-compose works? It might be I misdiagnosed the problem but if you don't support the container_name for the stated reason, you wouldn't support setting the network_mode to host for the same reason.

The "EDIT" filled in the "etc"

can't run on well defined ports and so you're required to programmatically determine the service URI/URL/port rather than be able to test configuration as part of your test.
Please see this article: https://bsideup.github.io/posts/testcontainers_fixed_ports/
tl;dr: you really should be programmatically determining the URLs, and not just port, but also the host.

The ports are part of our environment configuration and even control the deployment environment in production. The configuration system is part of what's tested in our integration tests as well as the actual configuration. We programatically drive the URL/URI/host/port setting from the configuration. But with testcontainers we are required to skip our configuration and drive the test from whatever testcontainers decides the ports and services will be.

can't supply a shutdown strategy and so you can't leave the container up after the test for debugging and analysis

One of the top priorities for Testcontainers is to cleanup after the test execution, so that there are no leftovers. Have you considered debugging your tests as usual, and inspecting the container while you're on a breakpoint? Nothing will be stopped while your testing session is still running.

Yes. Doable.

No JUnit5 support

As mentioned by @jannis-baratheon, Testcontainers supports JUnit 5, and you can use @Container annotation with DockerComposeContainer as well. Have you tried?

No, I didn't try. It wasn't obvious. Retracted again with strikethrough.

If it's not obvious, we have a large suite of test tools written to support our tests including out own container rules driven from our configuration system. I made an attempt to port our tools (because porting all the tests individually would be monumental). The problems I listed, or I should say, the design assumptions made by test containers, conflict with the assumptions our test tools are built on which is what makes the porting so frustrating. It's possible if we started with testcontainers rather than DCR then it would have been built on different assumptions but in any case, porting is problematic and currently unnecessary since I resolved the current issue ( #577 )