Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kafka cluster example #3758

Merged
merged 8 commits into from Feb 6, 2021
Merged

Kafka cluster example #3758

merged 8 commits into from Feb 6, 2021

Conversation

bsideup
Copy link
Member

@bsideup bsideup commented Feb 6, 2021

Follow-up from #1984

seglo and others added 5 commits February 4, 2020 15:50
# Conflicts:
#	modules/kafka/src/main/java/org/testcontainers/containers/KafkaContainer.java
#	modules/kafka/src/test/java/org/testcontainers/containers/KafkaContainerTest.java
* Simplify `KafkaContainerCluster#start`

* When an image version is not specified, use `latest` as the default tag (#3313)

* Add workflow for Update Gradle Wrapper Action. (#3297)

Co-authored-by: Richard North <rich.north@gmail.com>

* Always continue on error for examples CI (#3339)

* Bump snakeyaml from 1.25 to 1.27 in /core (#3252)

Bumps [snakeyaml](https://bitbucket.org/asomov/snakeyaml) from 1.25 to 1.27.
- [Commits](https://bitbucket.org/asomov/snakeyaml/branches/compare/snakeyaml-1.27..snakeyaml-1.25)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump lombok from 1.18.12 to 1.18.14 in /examples (#3322)

Bumps [lombok](https://github.com/rzwitserloot/lombok) from 1.18.12 to 1.18.14.
- [Release notes](https://github.com/rzwitserloot/lombok/releases)
- [Changelog](https://github.com/rzwitserloot/lombok/blob/master/doc/changelog.markdown)
- [Commits](projectlombok/lombok@v1.18.12...v1.18.14)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump r2dbc-mariadb from 0.8.3-beta1 to 0.8.4-rc in /modules/mariadb (#3300)

Bumps [r2dbc-mariadb](https://github.com/mariadb-corporation/mariadb-connector-r2dbc) from 0.8.3-beta1 to 0.8.4-rc.
- [Release notes](https://github.com/mariadb-corporation/mariadb-connector-r2dbc/releases)
- [Changelog](https://github.com/mariadb-corporation/mariadb-connector-r2dbc/blob/master/CHANGELOG.md)
- [Commits](https://github.com/mariadb-corporation/mariadb-connector-r2dbc/commits)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump testng from 7.2.0 to 7.3.0 in /examples (#3068)

Bumps [testng](https://github.com/cbeust/testng) from 7.2.0 to 7.3.0.
- [Release notes](https://github.com/cbeust/testng/releases)
- [Changelog](https://github.com/cbeust/testng/blob/master/CHANGES.txt)
- [Commits](https://github.com/cbeust/testng/commits)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump assertj-core from 3.17.1 to 3.17.2 in /core (#3251)

Bumps [assertj-core](https://github.com/joel-costigliola/assertj-core) from 3.17.1 to 3.17.2.
- [Release notes](https://github.com/joel-costigliola/assertj-core/releases)
- [Commits](assertj/assertj@assertj-core-3.17.1...assertj-core-3.17.2)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump aws-java-sdk-dynamodb from 1.11.865 to 1.11.880 in /modules/dynalite (#3332)

Bumps [aws-java-sdk-dynamodb](https://github.com/aws/aws-sdk-java) from 1.11.865 to 1.11.880.
- [Release notes](https://github.com/aws/aws-sdk-java/releases)
- [Changelog](https://github.com/aws/aws-sdk-java/blob/master/CHANGELOG.md)
- [Commits](aws/aws-sdk-java@1.11.865...1.11.880)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump mockito-core from 3.5.11 to 3.5.13 in /core (#3275)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump elasticsearch-rest-client from 7.9.1 to 7.9.2 in /modules/elasticsearch (#3276)

Bumps [elasticsearch-rest-client](https://github.com/elastic/elasticsearch) from 7.9.1 to 7.9.2.
- [Release notes](https://github.com/elastic/elasticsearch/releases)
- [Commits](elastic/elasticsearch@v7.9.1...v7.9.2)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Increase memory limits used in example (#3340)

For improved test stability

* Bump mockito-core from 3.5.11 to 3.5.13 in /modules/junit-jupiter (#3283)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Richard North <rich.north@gmail.com>

* Bump tomcat-jdbc from 9.0.37 to 9.0.39 in /modules/jdbc-test (#3338)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Richard North <rich.north@gmail.com>

* Bump junit from 4.13 to 4.13.1 in /examples (#3328)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Richard North <rich.north@gmail.com>

* Bump mariadb-java-client from 2.6.2 to 2.7.0 in /modules/mariadb (#3278)

Bumps [mariadb-java-client](https://github.com/mariadb-corporation/mariadb-connector-j) from 2.6.2 to 2.7.0.
- [Release notes](https://github.com/mariadb-corporation/mariadb-connector-j/releases)
- [Changelog](https://github.com/mariadb-corporation/mariadb-connector-j/blob/master/CHANGELOG.md)
- [Commits](mariadb-corporation/mariadb-connector-j@2.6.2...2.7.0)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump solr-solrj from 8.6.2 to 8.6.3 in /examples (#3321)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Richard North <rich.north@gmail.com>

* Bump postgresql from 42.2.16 to 42.2.17 in /examples (#3323)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Richard North <rich.north@gmail.com>

* Bump postgresql from 42.2.16 to 42.2.17 in /modules/junit-jupiter (#3327)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Richard North <rich.north@gmail.com>

* Bump postgresql from 42.2.16 to 42.2.17 in /modules/spock (#3330)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Richard North <rich.north@gmail.com>

* Bump tomcat-jdbc from 9.0.37 to 9.0.39 in /modules/jdbc (#3333)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Richard North <rich.north@gmail.com>

* Bump postgresql from 42.2.16 to 42.2.17 in /modules/postgresql (#3334)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Richard North <rich.north@gmail.com>

* Bump aws-java-sdk-sqs from 1.11.860 to 1.11.880 in /modules/localstack (#3337)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Richard North <rich.north@gmail.com>

* Bump httpclient from 4.5.12 to 4.5.13 in /modules/junit-jupiter (#3326)

Bumps httpclient from 4.5.12 to 4.5.13.

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump httpclient from 4.5.12 to 4.5.13 in /modules/spock (#3329)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Richard North <rich.north@gmail.com>

* Bump cucumber-junit from 6.7.0 to 6.8.1 in /examples (#3325)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Richard North <rich.north@gmail.com>

* Bump org.springframework.boot from 2.3.3.RELEASE to 2.3.4.RELEASE in /examples (#3247)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Richard North <rich.north@gmail.com>

* Bump aws-java-sdk-s3 from 1.11.870 to 1.11.880 in /modules/localstack (#3336)

Bumps [aws-java-sdk-s3](https://github.com/aws/aws-sdk-java) from 1.11.870 to 1.11.880.
- [Release notes](https://github.com/aws/aws-sdk-java/releases)
- [Changelog](https://github.com/aws/aws-sdk-java/blob/master/CHANGELOG.md)
- [Commits](aws/aws-sdk-java@1.11.870...1.11.880)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump zt-exec from 1.10 to 1.12 in /core (#3253)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Richard North <rich.north@gmail.com>

* Bump s3 from 2.14.21 to 2.15.7 in /modules/localstack (#3335)

Bumps s3 from 2.14.21 to 2.15.7.

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump cucumber-java from 6.6.0 to 6.8.1 in /examples (#3324)

Bumps [cucumber-java](https://github.com/cucumber/cucumber-jvm) from 6.6.0 to 6.8.1.
- [Release notes](https://github.com/cucumber/cucumber-jvm/releases)
- [Changelog](https://github.com/cucumber/cucumber-jvm/blob/main/CHANGELOG.md)
- [Commits](cucumber/cucumber-jvm@v6.6.0...v6.8.1)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Remove GitHub Actions cache restore keys (#3342)

For more specific cache matching

* Allow users to specify a MongoDB database name (#2980)

Co-authored-by: Richard North <rich.north@gmail.com>

* Add GCloud module for Google Cloud Datastore, Firestore, PubSub, and Spanner emulators (#2690)

Co-authored-by: Richard North <rich.north@gmail.com>

* Use a lighter weight image for MultiplePortsExposedTest (#3343)

* Use a lighter weight image for MultiplePortsExposedTest

* Update helloworld container version

* docker-machine: get full remote daemon URL, to allow for use of custom daemon port (#2769) (#3237)

Co-authored-by: Vitalii Chura <c-vitalii.chura@hulu.com>

* Fix remote gradle cache 400 InvalidArgument error (#3346)

per hint given in https://www.digitalocean.com/community/questions/node-upload-file-to-s3-error-invalidargument-null

* Add gcloud endpoint accessors (#3344)

Co-authored-by: Sergei Egorov <bsideup@gmail.com>
Co-authored-by: Richard North <rich.north@gmail.com>
Co-authored-by: Cristian Greco <cristian@regolo.cc>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: silaev <silaev256@gmail.com>
Co-authored-by: Eddú Meléndez Gonzales <eddu.melendez@gmail.com>
Co-authored-by: vcvitaly <vcvitaly@yahoo.com>
Co-authored-by: Vitalii Chura <c-vitalii.chura@hulu.com>
# Conflicts:
#	.github/workflows/ci-examples.yml
#	.github/workflows/update-gradle-wrapper.yml
#	core/build.gradle
#	core/src/test/java/org/testcontainers/dockerclient/AmbiguousImagePullTest.java
#	docs/modules/gcloud.md
#	examples/cucumber/build.gradle
#	examples/disque-job-queue/build.gradle
#	examples/linked-container/build.gradle
#	examples/selenium-container/build.gradle
#	examples/solr-container/build.gradle
#	examples/spring-boot/build.gradle
#	modules/dynalite/build.gradle
#	modules/elasticsearch/build.gradle
#	modules/gcloud/build.gradle
#	modules/gcloud/src/test/java/org/testcontainers/containers/DatastoreEmulatorContainerTest.java
#	modules/gcloud/src/test/java/org/testcontainers/containers/FirestoreEmulatorContainerTest.java
#	modules/gcloud/src/test/java/org/testcontainers/containers/PubSubEmulatorContainerTest.java
#	modules/gcloud/src/test/java/org/testcontainers/containers/SpannerEmulatorContainerTest.java
#	modules/jdbc-test/build.gradle
#	modules/jdbc/build.gradle
#	modules/junit-jupiter/build.gradle
#	modules/localstack/build.gradle
#	modules/mariadb/build.gradle
#	modules/postgresql/build.gradle
#	modules/spock/build.gradle
@bsideup bsideup added this to the next milestone Feb 6, 2021
@bsideup bsideup merged commit 625ddc3 into master Feb 6, 2021
@delete-merged-branch delete-merged-branch bot deleted the seglo/multi-broker-kafka branch February 6, 2021 15:41
@lfgcampos
Copy link

lfgcampos commented Feb 12, 2021

Awesome work @bsideup and @seglo ... thank you very much for this!
Just one question: Shouldn't the KafkaContainerCluster goes next to the KafkaContainer on the testcontainers-java project/modules?

I felt like I had to 'copy and paste' the whole class on my local setup to have a cluster working... which by the way, works flawlessly!

If that is the case, I am more than happy to provide a PR.

@bsideup
Copy link
Member Author

bsideup commented Feb 12, 2021

@lfgcampos See the conversation in #1984. This example just demonstrates that the cluster can be created by using KafkaContainers + Network, and one may even decide to not create a dedicated class for it, but just 4 fields.

@seglo
Copy link
Contributor

seglo commented Feb 12, 2021

Yes, there's not much to this implementation and there's not much in the way of error handling so it was decided to keep it as an example instead.

I've iterated on this version of the implementation in Alpakka Kafka's TestKit. I may create a testcontainer PR to carry over some of that functionality to this example, but that might be beyond the example's scope.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants