Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[25.0 backport] client: fix connection-errors being shadowed by API version errors #47470

Merged
merged 4 commits into from Feb 29, 2024

Conversation

neersighted
Copy link
Member

client: fix TestPingWithError

This test was added in 27ef09a (#33827), which changed
the Ping handling to ignore internal server errors. That case is tested in
TestPingFail, which verifies that we accept the Ping response if a 500
status code was received.

The TestPingWithError test was added to verify behavior if a protocol
(connection) error occurred; however the mock-client returned both a
response, and an error; the error returned would only happen if a connection
error occurred, which means that the server would not provide a reply.

Running the test also shows that returning a response is unexpected, and
ignored:

=== RUN   TestPingWithError
2024/02/23 14:16:49 RoundTripper returned a response & error; ignoring response
2024/02/23 14:16:49 RoundTripper returned a response & error; ignoring response
--- PASS: TestPingWithError (0.00s)
PASS

This patch updates the test to remove the response.

client: NegotiateAPIVersion: do not ignore (connection) errors from Ping

NegotiateAPIVersion was ignoring errors returned by Ping. The intent here
was to handle API responses from a daemon that may be in an unhealthy state,
however this case is already handled by Ping itself.

Ping only returns an error when either failing to connect to the API (daemon
not running or permissions errors), or when failing to parse the API response.

Neither of those should be ignored in this code, or considered a successful
"ping", so update the code to return

client: doRequest: make sure we return a connection-error

This function has various errors that are returned when failing to make a
connection (due to permission issues, TLS mis-configuration, or failing to
resolve the TCP address).

The errConnectionFailed error is currently used as a special case when
processing Ping responses. The current code did not consistently treat
connection errors, and because of that could either absorb the error,
or process the empty response.

client: fix connection-errors being shadowed by API version mismatch errors

Commit e690724 (#46463) applied a fix for situations
where the client was configured with API-version negotiation, but did not yet
negotiate a version.

However, the checkVersion() function that was implemented copied the semantics
of cli.NegotiateAPIVersion, which ignored connection failures with the
assumption that connection errors would still surface further down.

However, when using the result of a failed negotiation for NewVersionError,
an API version mismatch error would be produced, masking the actual connection
error.

This patch changes the signature of checkVersion to return unexpected errors,
including failures to connect to the API.

Before this patch:

docker -H unix:///no/such/socket.sock secret ls
"secret list" requires API version 1.25, but the Docker daemon API version is 1.24

With this patch applied:

docker -H unix:///no/such/socket.sock secret ls
Cannot connect to the Docker daemon at unix:///no/such/socket.sock. Is the docker daemon running?

- Description for the changelog

- Fix a regression that caused API socket connection failures to report an API version negotiation failure instead.

- A picture of a cute animal (not mandatory but encouraged)

This test was added in 27ef09a, which changed
the Ping handling to ignore internal server errors. That case is tested in
TestPingFail, which verifies that we accept the Ping response if a 500
status code was received.

The TestPingWithError test was added to verify behavior if a protocol
(connection) error occurred; however the mock-client returned both a
response, and an error; the error returned would only happen if a connection
error occurred, which means that the server would not provide a reply.

Running the test also shows that returning a response is unexpected, and
ignored:

    === RUN   TestPingWithError
    2024/02/23 14:16:49 RoundTripper returned a response & error; ignoring response
    2024/02/23 14:16:49 RoundTripper returned a response & error; ignoring response
    --- PASS: TestPingWithError (0.00s)
    PASS

This patch updates the test to remove the response.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 349abc6)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
NegotiateAPIVersion was ignoring errors returned by Ping. The intent here
was to handle API responses from a daemon that may be in an unhealthy state,
however this case is already handled by Ping itself.

Ping only returns an error when either failing to connect to the API (daemon
not running or permissions errors), or when failing to parse the API response.

Neither of those should be ignored in this code, or considered a successful
"ping", so update the code to return

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 901b905)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
This function has various errors that are returned when failing to make a
connection (due to permission issues, TLS mis-configuration, or failing to
resolve the TCP address).

The errConnectionFailed error is currently used as a special case when
processing Ping responses. The current code did not consistently treat
connection errors, and because of that could either absorb the error,
or process the empty response.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 913478b)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
@neersighted
Copy link
Member Author

Note that #47139 caused (trivial) conflicts; I have elected not to include it in the backport as the type movement introduced new deprecations, and pulling those deprecations into a patch release would constitute a breaking change.

@thaJeztah
Copy link
Member

Looks like you probably missed one;

client/build_cancel.go:1: : # github.com/docker/docker/client [github.com/docker/docker/client.test]
client/image_list_test.go:39:56: undefined: image.ListOptions (typecheck)
package client // import "github.com/docker/docker/client"

@neersighted
Copy link
Member Author

Drat, it cherry-picks cleanly but doesn't compile. Will take a look in a bit.

@thaJeztah
Copy link
Member

Yeah; looks also related to #47139, so should be an easy fix

…errors

Commit e690724 applied a fix for situations
where the client was configured with API-version negotiation, but did not yet
negotiate a version.

However, the checkVersion() function that was implemented copied the semantics
of cli.NegotiateAPIVersion, which ignored connection failures with the
assumption that connection errors would still surface further down.

However, when using the result of a failed negotiation for NewVersionError,
an API version mismatch error would be produced, masking the actual connection
error.

This patch changes the signature of checkVersion to return unexpected errors,
including failures to connect to the API.

Before this patch:

    docker -H unix:///no/such/socket.sock secret ls
    "secret list" requires API version 1.25, but the Docker daemon API version is 1.24

With this patch applied:

    docker -H unix:///no/such/socket.sock secret ls
    Cannot connect to the Docker daemon at unix:///no/such/socket.sock. Is the docker daemon running?

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6aea26b)
Conflicts: client/image_list.go
    client/image_list_test.go
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Copy link
Member

@thaJeztah thaJeztah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@vvoland
Copy link
Contributor

vvoland commented Feb 29, 2024

We really need to get #47471 in 😅

Run teststat -markdown $(find /tmp/reports -type f -name '*.json' -print0 | xargs -0) >> $GITHUB_STEP_SUMMARY
2024/02/29 14:33:39 json: cannot unmarshal string into Go value of type main.Line
2024/02/29 14:33:39 invalid character 'p' looking for beginning of value
...

@vvoland vvoland merged commit 9f1b47c into moby:25.0 Feb 29, 2024
126 checks passed
@neersighted neersighted deleted the backport/47440/25.0 branch February 29, 2024 14:57
idodod added a commit to earthly/dind that referenced this pull request Apr 22, 2024
[![Mend
Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [docker/docker](https://togithub.com/docker/docker) | patch | `25.0.1`
-> `25.0.5` |

---

### Release Notes

<details>
<summary>docker/docker (docker/docker)</summary>

### [`v25.0.5`](https://togithub.com/moby/moby/releases/tag/v25.0.5)

[Compare
Source](https://togithub.com/docker/docker/compare/v25.0.4...v25.0.5)

#### 25.0.5

For a full list of pull requests and changes in this release, refer to
the relevant GitHub milestones:

- [docker/cli, 25.0.5
milestone](https://togithub.com/docker/cli/issues?q=is%3Aclosed+milestone%3A25.0.5)
- [moby/moby, 25.0.5
milestone](https://togithub.com/moby/moby/issues?q=is%3Aclosed+milestone%3A25.0.5)
- Deprecated and removed features, see [Deprecated
Features](https://togithub.com/docker/cli/blob/v25.0.5/docs/deprecated.md).
- Changes to the Engine API, see [API version
history](https://togithub.com/moby/moby/blob/v25.0.5/docs/api/version-history.md).

##### Security

This release contains a security fix for [CVE-2024-29018], a potential
data exfiltration from 'internal' networks via authoritative DNS
servers.

##### Bug fixes and enhancements

- [CVE-2024-29018]: Do not forward requests to external DNS servers for
a container that is only connected to an 'internal' network. Previously,
requests were forwarded if the host's DNS server was running on a
loopback address, like systemd's 127.0.0.53.
[moby/moby#47589](https://togithub.com/moby/moby/pull/47589)
- plugin: fix mounting /etc/hosts when running in UserNS.
[moby/moby#47588](https://togithub.com/moby/moby/pull/47588)
- rootless: fix `open /etc/docker/plugins: permission denied`.
[moby/moby#47587](https://togithub.com/moby/moby/pull/47587)
- Fix multiple parallel `docker build` runs leaking disk space.
[moby/moby#47527](https://togithub.com/moby/moby/pull/47527)

[CVE-2024-29018]:
https://togithub.com/moby/moby/security/advisories/GHSA-mq39-4gv4-mvpx

### [`v25.0.4`](https://togithub.com/moby/moby/releases/tag/v25.0.4)

[Compare
Source](https://togithub.com/docker/docker/compare/v25.0.3...v25.0.4)

#### 25.0.4

For a full list of pull requests and changes in this release, refer to
the relevant GitHub milestones:

- [docker/cli, 25.0.4
milestone](https://togithub.com/docker/cli/issues?q=is%3Aclosed+milestone%3A25.0.4)
- [moby/moby, 25.0.4
milestone](https://togithub.com/moby/moby/issues?q=is%3Aclosed+milestone%3A25.0.4)
- Deprecated and removed features, see [Deprecated
Features](https://togithub.com/docker/cli/blob/v25.0.4/docs/deprecated.md).
- Changes to the Engine API, see [API version
history](https://togithub.com/moby/moby/blob/v25.0.4/docs/api/version-history.md).

##### Bug fixes and enhancements

- Restore DNS names for containers in the default "nat" network on
Windows. [moby/moby#47490](https://togithub.com/moby/moby/pull/47490)
- Fix `docker start` failing when used with `--checkpoint`
[moby/moby#47466](https://togithub.com/moby/moby/pull/47466)
- Don't enforce new validation rules for existing swarm networks
[moby/moby#47482](https://togithub.com/moby/moby/pull/47482)
- Restore IP connectivity between the host and containers on an internal
bridge network.
[moby/moby#47481](https://togithub.com/moby/moby/pull/47481)
- Fix a regression introduced in v25.0 that prevented the classic
builder from ADDing a tar archive with xattrs created on a non-Linux OS
[moby/moby#47483](https://togithub.com/moby/moby/pull/47483)
- containerd image store: Fix image pull not emitting `Pulling fs layer`
status [moby/moby#47484](https://togithub.com/moby/moby/pull/47484)

##### API

- To preserve backwards compatibility, make read-only mounts not
recursive by default when using older clients (API version < v1.44).
[moby/moby#47393](https://togithub.com/moby/moby/pull/47393)
- `GET /images/{id}/json` omits the `Created` field (previously it was
`0001-01-01T00:00:00Z`) if the `Created` field is missing from the image
config. [moby/moby#47451](https://togithub.com/moby/moby/pull/47451)
- Populate a missing `Created` field in `GET /images/{id}/json` with
`0001-01-01T00:00:00Z` for API version <= 1.43.
[moby/moby#47387](https://togithub.com/moby/moby/pull/47387)
- Fix a regression that caused API socket connection failures to report
an API version negotiation failure instead.
[moby/moby#47470](https://togithub.com/moby/moby/pull/47470)
- Preserve supplied endpoint configuration in a container-create API
request, when a container-wide MAC address is specified, but
`NetworkMode` name-or-id is not the same as the name-or-id used in
`NetworkSettings.Networks`.
[moby/moby#47510](https://togithub.com/moby/moby/pull/47510)

##### Packaging updates

- Upgrade Go runtime to
[1.21.8](https://go.dev/doc/devel/release#go1.21.8).
[moby/moby#47503](https://togithub.com/moby/moby/pull/47503)
- Upgrade RootlessKit to
[v2.0.2](https://togithub.com/rootless-containers/rootlesskit/releases/tag/v2.0.2).
[moby/moby#47508](https://togithub.com/moby/moby/pull/47508)
- Upgrade Compose to
[v2.24.7](https://togithub.com/docker/compose/releases/tag/v2.24.7).
[docker/docker-ce-packaging#998
- Upgrade Buildx to
[v0.13.0](https://togithub.com/docker/buildx/releases/tag/v0.13.0).
[docker/docker-ce-packaging#997

**Full Changelog**:
moby/moby@v25.0.3...v25.0.4

### [`v25.0.3`](https://togithub.com/moby/moby/releases/tag/v25.0.3)

[Compare
Source](https://togithub.com/docker/docker/compare/v25.0.2...v25.0.3)

#### 25.0.3

For a full list of pull requests and changes in this release, refer to
the relevant GitHub milestones:

- [docker/cli, 25.0.3
milestone](https://togithub.com/docker/cli/issues?q=is%3Aclosed+milestone%3A25.0.3)
- [moby/moby, 25.0.3
milestone](https://togithub.com/moby/moby/issues?q=is%3Aclosed+milestone%3A25.0.3)

##### Bug fixes and enhancements

- containerd image store: Fix a bug where `docker image history` would
fail if a manifest wasn't found in the content store.
[moby/moby#47348](https://togithub.com/moby/moby/pull/47348)
- Ensure that a generated MAC address is not restored when a container
is restarted, but a configured MAC address is preserved.
[moby/moby#47304](https://togithub.com/moby/moby/pull/47304)

    > **Note**
    >
> - Containers created with Docker Engine version 25.0.0 may have
duplicate MAC addresses.
    >     They must be re-created.
> - Containers with user-defined MAC addresses created with Docker
Engine versions 25.0.0 or 25.0.1
> receive new MAC addresses when started using Docker Engine version
25.0.2.
    >     They must also be re-created.

<!---->

- Fix `docker save <image>@&#8203;<digest>` producing an OCI archive
with index without manifests.
[moby/moby#47294](https://togithub.com/moby/moby/pull/47294)
- Fix a bug preventing bridge networks from being created with an MTU
higher than 1500 on RHEL and CentOS 7.
[moby/moby#47308](https://togithub.com/moby/moby/issues/47308),
[moby/moby#47311](https://togithub.com/moby/moby/pull/47311)
- Fix a bug where containers are unable to communicate over an
`internal` network.
[moby/moby#47303](https://togithub.com/moby/moby/pull/47303)
- Fix a bug where the value of the `ipv6` daemon option was ignored.
[moby/moby#47310](https://togithub.com/moby/moby/pull/47310)
- Fix a bug where trying to install a pulling using a digest revision
would cause a panic.
[moby/moby#47323](https://togithub.com/moby/moby/pull/47323)
- Fix a potential race condition in the managed containerd supervisor.
[moby/moby#47313](https://togithub.com/moby/moby/pull/47313)
- Fix an issue with the `journald` log driver preventing container logs
from being followed correctly with systemd version 255.
[moby/moby47243](https://togithub.com/moby/moby/pull/47243)
- seccomp: Update the builtin seccomp profile to include syscalls added
in kernel v5.17 - v6.7 to align the profile with the profile used by
containerd. [moby/moby#47341](https://togithub.com/moby/moby/pull/47341)
- Windows: Fix cache not being used when building images based on
Windows versions older than the host's version.
[moby/moby#47307](https://togithub.com/moby/moby/pull/47307),
[moby/moby#47337](https://togithub.com/moby/moby/pull/47337)

##### Packaging updates

- Removed support for Ubuntu Lunar (23.04).
[docker/ce-packaging#986](https://togithub.com/docker/docker-ce-packaging/pull/986)

### [`v25.0.2`](https://togithub.com/moby/moby/releases/tag/v25.0.2)

[Compare
Source](https://togithub.com/docker/docker/compare/v25.0.1...v25.0.2)

#### 25.0.2

For a full list of pull requests and changes in this release, refer to
the relevant GitHub milestones:

- [docker/cli, 25.0.2
milestone](https://togithub.com/docker/cli/issues?q=is%3Aclosed+milestone%3A25.0.2)
- [moby/moby, 25.0.2
milestone](https://togithub.com/moby/moby/issues?q=is%3Aclosed+milestone%3A25.0.2)

##### Security

This release contains security fixes for the following CVEs
affecting Docker Engine and its components.

| CVE | Component | Fix version | Severity |
| ----------------------------------------------------------- |
------------- | ----------- | ---------------- |
| [CVE-2024-21626](https://scout.docker.com/v/CVE-2024-21626) | runc |
1.1.12 | High, CVSS 8.6 |
| [CVE-2024-23651](https://scout.docker.com/v/CVE-2024-23651) | BuildKit
| 1.12.5 | High, CVSS 8.7 |
| [CVE-2024-23652](https://scout.docker.com/v/CVE-2024-23652) | BuildKit
| 1.12.5 | High, CVSS 8.7 |
| [CVE-2024-23653](https://scout.docker.com/v/CVE-2024-23653) | BuildKit
| 1.12.5 | High, CVSS 7.7 |
| [CVE-2024-23650](https://scout.docker.com/v/CVE-2024-23650) | BuildKit
| 1.12.5 | Medium, CVSS 5.5 |
| [CVE-2024-24557](https://scout.docker.com/v/CVE-2024-24557) | Docker
Engine | 25.0.2 | Medium, CVSS 6.9 |

The potential impacts of the above vulnerabilities include:

-   Unauthorized access to the host filesystem
-   Compromising the integrity of the build cache
- In the case of CVE-2024-21626, a scenario that could lead to full
container escape

For more information about the security issues addressed in this
release,
refer to the [blog
post](https://www.docker.com/blog/docker-security-advisory-multiple-vulnerabilities-in-runc-buildkit-and-moby/).
For details about each vulnerability, see the relevant security
advisory:

-
[CVE-2024-21626](https://togithub.com/opencontainers/runc/security/advisories/GHSA-xr7r-f8xq-vfvv)
-
[CVE-2024-23651](https://togithub.com/moby/buildkit/security/advisories/GHSA-m3r6-h7wv-7xxv)
-
[CVE-2024-23652](https://togithub.com/moby/buildkit/security/advisories/GHSA-4v98-7qmw-rqr8)
-
[CVE-2024-23653](https://togithub.com/moby/buildkit/security/advisories/GHSA-wr6v-9f75-vh2g)
-
[CVE-2024-23650](https://togithub.com/moby/buildkit/security/advisories/GHSA-9p26-698r-w4hx)
-
[CVE-2024-24557](https://togithub.com/moby/moby/security/advisories/GHSA-xw73-rw38-6vjc)

##### Packaging updates

- Upgrade containerd to
[v1.6.28](https://togithub.com/containerd/containerd/releases/tag/v1.6.28).
- Upgrade containerd to v1.7.13 (static binaries only).
[moby/moby#47280](https://togithub.com/moby/moby/pull/47280)
- Upgrade runc to v1.1.12.
[moby/moby#47269](https://togithub.com/moby/moby/pull/47269)
- Upgrade Compose to v2.24.5.
[docker/docker-ce-packaging#985](https://togithub.com/docker/docker-ce-packaging/pull/985)
- Upgrade BuildKit to v0.12.5.
[moby/moby#47273](https://togithub.com/moby/moby/pull/47273)

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "after 6am on monday" (UTC), Automerge
- At any time (no schedule defined).

🚦 **Automerge**: Enabled.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Mend
Renovate](https://www.mend.io/free-developer-tools/renovate/). View
repository job log
[here](https://developer.mend.io/github/earthly/dind).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4yOTMuMCIsInVwZGF0ZWRJblZlciI6IjM3LjI5My4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJyZW5vdmF0ZSJdfQ==-->

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: idodod <ido@earthly.dev>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants