Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: testcontainers/testcontainers-python
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: testcontainers-v4.5.1
Choose a base ref
...
head repository: testcontainers/testcontainers-python
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: testcontainers-v4.6.0
Choose a head ref
  • 10 commits
  • 31 files changed
  • 8 contributors

Commits on May 31, 2024

  1. feat(core): Image build (Dockerfile support) (#585)

    As part of the effort described, detailed and presented on #559 
    (Providing the implementation for #83 - Docker file support and more)
    This is the first PR (out of 4) that should provide all the groundwork
    to support image build.
    
    This would allow users to use custom images:
    ```python
      with DockerImage(path=".") as image:
                with DockerContainer(str(image)) as container:
                    # Test something with/on custom image
    ```
    
    Next in line is:
    `feat(core): Added SrvContainer`
    And later on:
    `feat(core): Added FastAPI module`
    `feat(core): Added AWS Lambda module`
    (all of the above can be overviewed on #559)
    Tranquility2 authored May 31, 2024

    Unverified

    This user has not yet uploaded their public signing key.
    Copy the full SHA
    54c88cf View commit details

Commits on Jun 5, 2024

  1. docs(main): Private registry (#598)

    Following #566 - Private registry, adding the relevant doc so the usage
    will be clear and (hopefully) reachable.
    Tranquility2 authored Jun 5, 2024

    Unverified

    This user has not yet uploaded their public signing key.
    Copy the full SHA
    9045c0a View commit details
  2. chore(core): Display status on readme (#589)

    Make the Readme display some more details regarding the project:
    - using ruff (just a nice reference)
    - pypi version (good visibility)
    - license (good visibility)
    - supported python versions (good visibility)
    - code coverage reporting (main driver for this PR)
    
    
    ![image](https://github.com/testcontainers/testcontainers-python/assets/7189138/1e8897b6-15eb-47eb-a1e7-42caa7db1eca)
    
    Relates to #544 - export code coverage (e.g. to codecov)
    Tranquility2 authored Jun 5, 2024
    Copy the full SHA
    54822de View commit details

Commits on Jun 7, 2024

  1. fix: move TESTCONTAINERS_HOST_OVERRIDE to config.py (#603)

    fix #602
    alexanderankin authored Jun 7, 2024
    Copy the full SHA
    2a5a190 View commit details

Commits on Jun 8, 2024

  1. docs: Update private registry instructions (#604)

    Fix some issues with the private registry instructions:
    - issue with the link to official documentation
    - convert all relevant blocks to code-block
    - fix some typos
    Tranquility2 authored Jun 8, 2024
    Copy the full SHA
    f5a019b View commit details

Commits on Jun 13, 2024

  1. fix: Container for Milvus database (#606)

    I use this wonderful package for writing tests, but I did not find a
    container for [Milvus vector database](https://milvus.io/docs)
    
    Please check, I'm ready to correct comments
    
    ---------
    
    Co-authored-by: ivan <ivan.belyaev@ailet.com>
    Co-authored-by: David Ankin <daveankin@gmail.com>
    3 people authored Jun 13, 2024
    Copy the full SHA
    ec76df2 View commit details

Commits on Jun 18, 2024

  1. fix(mqtt): Add mqtt.MosquittoContainer (#568) (#599)

    This PR is adding a new MosquittoContainer class that helps creating
    integration tests for MQTT clients.
    The MosquittoContainer class contains a bunch of methods to help with
    testing:
    * checking number of messages received 
    * watching topics
    * check last payload published on a particular topic
    * etc
    
    This PR lacks tests. I can add them if there is interest in this PR...
    
    ---------
    
    Co-authored-by: Dave Ankin <daveankin@gmail.com>
    f18m and alexanderankin authored Jun 18, 2024
    Copy the full SHA
    59cb6fc View commit details
  2. feat(core): Added ServerContainer (#595)

    As part of the effort described, detailed and presented on #559 
    This is the seconds PR (out of 4) that should provide all the groundwork
    to support containers running a server.
    
    
    This would allow users to use custom images:
    ```python
    with DockerImage(path=".", tag="test:latest") as image:
        with ServerContainer(port=9000, image=image) as srv:
            # Test something with/on the server using port 9000
    ```
    
    Next in line are:
    `feat(core): Added FastAPI module`
    `feat(core): Added AWS Lambda module`
    
    ---
    Based on the work done on #585
    Expended from issue #83
    
    ---------
    
    Co-authored-by: David Ankin <daveankin@gmail.com>
    Tranquility2 and alexanderankin authored Jun 18, 2024
    Copy the full SHA
    0768490 View commit details
  3. fix: Add Cockroach DB Module to Testcontainers (#608)

    Adds [Cockroach DB] (https://www.cockroachlabs.com/) module to use with
    Test containers
    
    I had done this previously under
    #281, but
    opted to just redo it rather than try to rebase all the things.
    
    
    - [x] Create a new feature directory and populate it with the package
    structure [described in the
    documentation](https://testcontainers-python.readthedocs.io/en/latest/#package-structure).
    Copying one of the existing features is likely the best way to get
    started.
    - [x] Implement the new feature (typically in `__init__.py`) and
    corresponding tests.
    - [x] Update the feature `README.rst` and add it to the table of
    contents (`toctree` directive) in the top-level `README.rst`.
    - [] Add a line `[feature name]` to the list of components in the GitHub
    Action workflow in `.github/workflows/main.yml` to run tests, build, and
    publish your package when pushed to the `main` branch.
    - [x] Rebase your development branch on `main` (or merge `main` into
    your development branch).
    - [x] Add Package to pyproject.toml
    - [ ] Add a line `-e file:[feature name]` to `requirements.in` and open
    a pull request. Opening a pull request will automatically generate lock
    files to ensure reproducible builds (see the [pip-tools
    documentation](https://pip-tools.readthedocs.io/en/latest/) for
    details). Finally, run `python get_requirements.py --pr=[your PR
    number]` to fetch the updated requirement files (the build needs to have
    succeeded).
    
    ---------
    
    Co-authored-by: joelhess <joelhess@flywheel.io>
    Co-authored-by: David Ankin <daveankin@gmail.com>
    3 people authored Jun 18, 2024
    Copy the full SHA
    4aff679 View commit details

Commits on Jun 20, 2024

  1. chore(main): release testcontainers 4.6.0 (#594)

    🤖 I have created a release *beep* *boop*
    ---
    
    
    ##
    [4.6.0](testcontainers-v4.5.1...testcontainers-v4.6.0)
    (2024-06-18)
    
    
    ### Features
    
    * **core:** Added ServerContainer
    ([#595](#595))
    ([0768490](0768490))
    * **core:** Image build (Dockerfile support)
    ([#585](#585))
    ([54c88cf](54c88cf))
    
    
    ### Bug Fixes
    
    * Add Cockroach DB Module to Testcontainers
    ([#608](#608))
    ([4aff679](4aff679))
    * Container for Milvus database
    ([#606](#606))
    ([ec76df2](ec76df2))
    * move TESTCONTAINERS_HOST_OVERRIDE to config.py
    ([#603](#603))
    ([2a5a190](2a5a190)),
    closes
    [#602](#602)
    * **mqtt:** Add mqtt.MosquittoContainer
    ([#568](#568))
    ([#599](#599))
    ([59cb6fc](59cb6fc))
    
    
    ### Documentation
    
    * **main:** Private registry
    ([#598](#598))
    ([9045c0a](9045c0a))
    * Update private registry instructions
    ([#604](#604))
    ([f5a019b](f5a019b))
    
    ---
    This PR was generated with [Release
    Please](https://github.com/googleapis/release-please). See
    [documentation](https://github.com/googleapis/release-please#release-please).
    
    Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
    github-actions[bot] authored Jun 20, 2024
    Copy the full SHA
    090bd0d View commit details
2 changes: 1 addition & 1 deletion .github/.release-please-manifest.json
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
{
".": "4.5.1"
".": "4.6.0"
}
31 changes: 30 additions & 1 deletion .github/workflows/ci-core.yml
Original file line number Diff line number Diff line change
@@ -9,7 +9,7 @@ on:
branches: [main]

jobs:
test:
run-tests-and-coverage:
runs-on: ubuntu-22.04
strategy:
fail-fast: false
@@ -27,5 +27,34 @@ jobs:
run: poetry build && poetry run twine check dist/*.tar.gz
- name: Run tests
run: make core/tests
- name: Rename coverage file
run: mv .coverage .coverage.${{ matrix.python-version}}
- name: "Save coverage artifact"
uses: actions/upload-artifact@v4
with:
name: "coverage-artifact-${{ matrix.python-version}}"
path: ".coverage.*"
retention-days: 1
- name: Run doctests
run: make core/doctests

coverage-compile:
needs: "run-tests-and-coverage"
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: ./.github/actions/setup-env
- name: Install Python dependencies
run: poetry install --all-extras
- name: "Download coverage artifacts"
uses: actions/download-artifact@v4
with:
pattern: "coverage-artifact-*"
merge-multiple: true
- name: Compile coverage
run: make coverage
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v4
with:
token: ${{ secrets.CODECOV_TOKEN }}
22 changes: 22 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,27 @@
# Changelog

## [4.6.0](https://github.com/testcontainers/testcontainers-python/compare/testcontainers-v4.5.1...testcontainers-v4.6.0) (2024-06-18)


### Features

* **core:** Added ServerContainer ([#595](https://github.com/testcontainers/testcontainers-python/issues/595)) ([0768490](https://github.com/testcontainers/testcontainers-python/commit/076849015ad3542384ecf8cf6c205d5d498e4986))
* **core:** Image build (Dockerfile support) ([#585](https://github.com/testcontainers/testcontainers-python/issues/585)) ([54c88cf](https://github.com/testcontainers/testcontainers-python/commit/54c88cf00ad7bb08eb7894c52bed7a9010fd7786))


### Bug Fixes

* Add Cockroach DB Module to Testcontainers ([#608](https://github.com/testcontainers/testcontainers-python/issues/608)) ([4aff679](https://github.com/testcontainers/testcontainers-python/commit/4aff6793f28fbeb8358adcc728283ea9a7b94e5f))
* Container for Milvus database ([#606](https://github.com/testcontainers/testcontainers-python/issues/606)) ([ec76df2](https://github.com/testcontainers/testcontainers-python/commit/ec76df27c3d95ac1b79df3a049b4e2c12539081d))
* move TESTCONTAINERS_HOST_OVERRIDE to config.py ([#603](https://github.com/testcontainers/testcontainers-python/issues/603)) ([2a5a190](https://github.com/testcontainers/testcontainers-python/commit/2a5a1904391020a9da4be17b32f23b36d9385c29)), closes [#602](https://github.com/testcontainers/testcontainers-python/issues/602)
* **mqtt:** Add mqtt.MosquittoContainer ([#568](https://github.com/testcontainers/testcontainers-python/issues/568)) ([#599](https://github.com/testcontainers/testcontainers-python/issues/599)) ([59cb6fc](https://github.com/testcontainers/testcontainers-python/commit/59cb6fc4e7d93870ff2d0d961d14ccd5142a8a05))


### Documentation

* **main:** Private registry ([#598](https://github.com/testcontainers/testcontainers-python/issues/598)) ([9045c0a](https://github.com/testcontainers/testcontainers-python/commit/9045c0aea6029283490c89aea985e625dcdfc7b9))
* Update private registry instructions ([#604](https://github.com/testcontainers/testcontainers-python/issues/604)) ([f5a019b](https://github.com/testcontainers/testcontainers-python/commit/f5a019b6d2552788478e4a10cd17f7a2b453abb9))

## [4.5.1](https://github.com/testcontainers/testcontainers-python/compare/testcontainers-v4.5.0...testcontainers-v4.5.1) (2024-05-31)


11 changes: 11 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -27,6 +27,13 @@ tests : ${TESTS}
${TESTS} : %/tests :
poetry run pytest -v --cov=testcontainers.$* $*/tests

# Target to combine and report coverage.
coverage:
poetry run coverage combine
poetry run coverage report
poetry run coverage xml
poetry run coverage html

# Target to lint the code.
lint:
pre-commit run -a
@@ -56,6 +63,10 @@ ${TESTS_DIND} : %/tests-dind : image
docs :
poetry run sphinx-build -nW . docs/_build

# Target to build docs watching for changes as per https://stackoverflow.com/a/21389615
docs-watch :
poetry run sphinx-autobuild . docs/_build # requires 'pip install sphinx-autobuild'

doctests : ${DOCTESTS}
poetry run sphinx-build -b doctest . docs/_build

7 changes: 7 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,10 @@
[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
[![image](https://img.shields.io/pypi/v/testcontainers.svg)](https://pypi.python.org/pypi/testcontainers)
[![image](https://img.shields.io/pypi/l/testcontainers.svg)](https://github.com/testcontainers/testcontainers-python/blob/main/LICENSE)
[![image](https://img.shields.io/pypi/pyversions/testcontainers.svg)](https://pypi.python.org/pypi/testcontainers)
[![codecov](https://codecov.io/gh/testcontainers/testcontainers-python/branch/master/graph/badge.svg)](https://codecov.io/gh/testcontainers/testcontainers-python)


# Testcontainers Python

`testcontainers-python` facilitates the use of Docker containers for functional and integration testing.
26 changes: 25 additions & 1 deletion core/README.rst
Original file line number Diff line number Diff line change
@@ -1,6 +1,30 @@
testcontainers-core
Testcontainers Core
===================

:code:`testcontainers-core` is the core functionality for spinning up Docker containers in test environments.

.. autoclass:: testcontainers.core.container.DockerContainer

Using `DockerContainer` and `DockerImage` directly:

.. doctest::

>>> from testcontainers.core.container import DockerContainer
>>> from testcontainers.core.waiting_utils import wait_for_logs
>>> from testcontainers.core.image import DockerImage

>>> with DockerImage(path="./core/tests/image_fixtures/sample/", tag="test-sample:latest") as image:
... with DockerContainer(str(image)) as container:
... delay = wait_for_logs(container, "Test Sample Image")

---

.. autoclass:: testcontainers.core.image.DockerImage

---

.. autoclass:: testcontainers.core.generic.ServerContainer

---

.. autoclass:: testcontainers.core.generic.DbContainer
11 changes: 9 additions & 2 deletions core/testcontainers/core/config.py
Original file line number Diff line number Diff line change
@@ -14,6 +14,7 @@
RYUK_DISABLED: bool = environ.get("TESTCONTAINERS_RYUK_DISABLED", "false") == "true"
RYUK_DOCKER_SOCKET: str = environ.get("TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE", "/var/run/docker.sock")
RYUK_RECONNECTION_TIMEOUT: str = environ.get("RYUK_RECONNECTION_TIMEOUT", "10s")
TC_HOST_OVERRIDE: Optional[str] = environ.get("TC_HOST", environ.get("TESTCONTAINERS_HOST_OVERRIDE"))

TC_FILE = ".testcontainers.properties"
TC_GLOBAL = Path.home() / TC_FILE
@@ -52,12 +53,18 @@ class TestcontainersConfiguration:
ryuk_reconnection_timeout: str = RYUK_RECONNECTION_TIMEOUT
tc_properties: dict[str, str] = field(default_factory=read_tc_properties)
_docker_auth_config: Optional[str] = field(default_factory=lambda: environ.get("DOCKER_AUTH_CONFIG"))
tc_host_override: Optional[str] = TC_HOST_OVERRIDE
"""
https://github.com/testcontainers/testcontainers-go/blob/dd76d1e39c654433a3d80429690d07abcec04424/docker.go#L644
if os env TC_HOST is set, use it
"""

@property
def docker_auth_config(self):
if "DOCKER_AUTH_CONFIG" in _WARNINGS:
config = self._docker_auth_config
if config and "DOCKER_AUTH_CONFIG" in _WARNINGS:
warning(_WARNINGS.pop("DOCKER_AUTH_CONFIG"))
return self._docker_auth_config
return config

@docker_auth_config.setter
def docker_auth_config(self, value: str):
29 changes: 23 additions & 6 deletions core/testcontainers/core/docker_client.py
Original file line number Diff line number Diff line change
@@ -16,10 +16,12 @@
import os
import urllib
import urllib.parse
from collections.abc import Iterable
from typing import Callable, Optional, TypeVar, Union

import docker
from docker.models.containers import Container, ContainerCollection
from docker.models.images import Image, ImageCollection
from typing_extensions import ParamSpec

from testcontainers.core.config import testcontainers_config as c
@@ -40,6 +42,14 @@ def wrapper(*args: _P.args, **kwargs: _P.kwargs) -> _T:
return wrapper


def _wrapped_image_collection(function: Callable[_P, _T]) -> Callable[_P, _T]:
@ft.wraps(ImageCollection.build)
def wrapper(*args: _P.args, **kwargs: _P.kwargs) -> _T:
return function(*args, **kwargs)

return wrapper


class DockerClient:
"""
Thin wrapper around :class:`docker.DockerClient` for a more functional interface.
@@ -94,6 +104,17 @@ def run(
)
return container

@_wrapped_image_collection
def build(self, path: str, tag: str, rm: bool = True, **kwargs) -> tuple[Image, Iterable[dict]]:
"""
Build a Docker image from a directory containing the Dockerfile.
:return: A tuple containing the image object and the build logs.
"""
image_object, image_logs = self.client.images.build(path=path, tag=tag, rm=rm, **kwargs)

return image_object, image_logs

def find_host_network(self) -> Optional[str]:
"""
Try to find the docker host network.
@@ -166,18 +187,14 @@ def host(self) -> str:
"""
Get the hostname or ip address of the docker host.
"""
# https://github.com/testcontainers/testcontainers-go/blob/dd76d1e39c654433a3d80429690d07abcec04424/docker.go#L644
# if os env TC_HOST is set, use it
host = os.environ.get("TC_HOST")
if not host:
host = os.environ.get("TESTCONTAINERS_HOST_OVERRIDE")
host = c.tc_host_override
if host:
return host
try:
url = urllib.parse.urlparse(self.client.api.base_url)

except ValueError:
return None
return "localhost"
if "http" in url.scheme or "tcp" in url.scheme:
return url.hostname
if inside_container() and ("unix" in url.scheme or "npipe" in url.scheme):
73 changes: 72 additions & 1 deletion core/testcontainers/core/generic.py
Original file line number Diff line number Diff line change
@@ -10,11 +10,14 @@
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from typing import Optional
from typing import Optional, Union
from urllib.error import HTTPError
from urllib.parse import quote
from urllib.request import urlopen

from testcontainers.core.container import DockerContainer
from testcontainers.core.exceptions import ContainerStartException
from testcontainers.core.image import DockerImage
from testcontainers.core.utils import raise_for_deprecated_parameter
from testcontainers.core.waiting_utils import wait_container_is_ready

@@ -29,6 +32,8 @@

class DbContainer(DockerContainer):
"""
**DEPRECATED (for removal)**
Generic database container.
"""

@@ -79,3 +84,69 @@ def _configure(self) -> None:

def _transfer_seed(self) -> None:
pass


class ServerContainer(DockerContainer):
"""
**DEPRECATED - will be moved from core to a module (stay tuned for a final/stable import location)**
Container for a generic server that is based on a custom image.
Example:
.. doctest::
>>> import httpx
>>> from testcontainers.core.generic import ServerContainer
>>> from testcontainers.core.waiting_utils import wait_for_logs
>>> from testcontainers.core.image import DockerImage
>>> with DockerImage(path="./core/tests/image_fixtures/python_server", tag="test-srv:latest") as image:
... with ServerContainer(port=9000, image=image) as srv:
... url = srv._create_connection_url()
... response = httpx.get(f"{url}", timeout=5)
... assert response.status_code == 200, "Response status code is not 200"
... delay = wait_for_logs(srv, "GET / HTTP/1.1")
:param path: Path to the Dockerfile to build the image
:param tag: Tag for the image to be built (default: None)
"""

def __init__(self, port: int, image: Union[str, DockerImage]) -> None:
super().__init__(str(image))
self.internal_port = port
self.with_exposed_ports(self.internal_port)

@wait_container_is_ready(HTTPError)
def _connect(self) -> None:
# noinspection HttpUrlsUsage
url = self._create_connection_url()
try:
with urlopen(url) as r:
assert b"" in r.read()
except HTTPError as e:
# 404 is expected, as the server may not have the specific endpoint we are looking for
if e.code == 404:
pass
else:
raise

def get_api_url(self) -> str:
raise NotImplementedError

def _create_connection_url(self) -> str:
if self._container is None:
raise ContainerStartException("container has not been started")
host = self.get_container_host_ip()
exposed_port = self.get_exposed_port(self.internal_port)
url = f"http://{host}:{exposed_port}"
return url

def start(self) -> "ServerContainer":
super().start()
self._connect()
return self

def stop(self, force=True, delete_volume=True) -> None:
super().stop(force, delete_volume)
Loading