spring-boot-maven-plugin:build-image fails on Docker Desktop / Windows due missing authentication

[eidottermihi]

When running the build-image goal of spring-boot-maven-plugin on Windows with Docker Desktop, the build fails when the plugin tries to pull the buildpacks builder image via Docker API.

[INFO] --- spring-boot-maven-plugin:3.1.2:build-image-no-fork (build-image) @ appswitcher-server ---
[INFO] Building image 'docker.io/itatm/appswitcher-server:1.0.3-SNAPSHOT'
[INFO]
[INFO]  > Pulling builder image 'docker.io/paketobuildpacks/builder:base' 100%
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  5.199 s
[INFO] Finished at: 2023-08-02T10:51:59+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.springframework.boot:spring-boot-maven-plugin:3.1.2:build-image-no-fork (build-image) on project appswitcher-server: 
  Execution build-image of goal org.springframework.boot:spring-boot-maven-plugin:3.1.2:build-image-no-fork failed: 
    Docker API call to 'localhost/v1.24/images/create?fromImage=docker.io%2Fpaketobuildpacks%2Fbuilder%3Abase' failed with status code 407 "Proxy Authentication Required" -> [Help 1]

(full maven debug log see attachment)

This seems to be related to an authentication issue, because the docker.log (located %LOCALAPPDATA%\Docker\log\vm) contains this error:

[2023-08-02T08:51:59.764710000Z][docker][I] [2023-08-02T08:51:59.764357000Z][lifecycle-server][I] missing X-Registry-Auth header
[2023-08-02T08:51:59.764852000Z][docker][I] [2023-08-02T08:51:59.764593100Z][lifecycle-server][I] must be logged in to call /v1.24/images/create?fromImage=docker.io%2Fpaketobuildpacks%2Fbuilder%3Abase

To debug this issue further I manually issued the problematic Docker API call with cURL:

$ curl -X POST -v http://localhost:2375/v1.24/images/create?fromImage=docker.io%2Fpaketobuildpacks%2Fbuilder%3Atiny
* Connected to localhost (::1) port 2375 (#0)
> POST /v1.24/images/create?fromImage=docker.io%2Fpaketobuildpacks%2Fbuilder%3Atiny HTTP/1.1
> Host: localhost:2375
> User-Agent: curl/7.73.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 407 Proxy Authentication Required
< Content-Type: text/plain; charset=utf-8
< Date: Wed, 02 Aug 2023 08:11:10 GMT
< X-Content-Type-Options: nosniff
< Transfer-Encoding: chunked
<
{ [32 bytes data]
100    26    0    26    0     0   1238      0 --:--:-- --:--:-- --:--:--  1300Please run 'docker login'

* Connection #0 to host localhost left intact

As the response body of the problematic call contains Please run 'docker login' I also checked that my Docker config.json contains the relevant data:

{
	"auths": {
		"ghcr.io": {},
		"https://index.docker.io/v1/": {}
	},
	"credsStore": "wincred"
}

So somehow this is also related or blocked by #25898

Pulling the image via Docker CLI works without a problem:

$ docker pull docker.io/paketobuildpacks/builder:base
base: Pulling from paketobuildpacks/builder
Digest: sha256:17ea21162ba8c7717d3ead3ee3836a368aced7f02f2e59658e52029bd6d149e7
Status: Downloaded newer image for paketobuildpacks/builder:base
docker.io/paketobuildpacks/builder:base

My environment:

• Windows 10 22H2 (Build 19045.3208)
• Docker Desktop v4.19.0
• Spring Boot 3.1.2

full_log.txt

[scottfrederick]

The Spring Boot plugin and the curl command both fail with 407 Proxy Authentication Required. It appears there is a network proxy between the Docker daemon and the docker.io host that requires authentication.

So somehow this is also related or blocked by #25898

The linked issue is related to Docker repositories that require authentication in order to pull images. That's not the case here, as the Paketo builder repository is a sponsored OSS repo that does not require authentication and is not subject to rate limiting. It's a proxy that is requiring authentication, not the repository.

Pulling the image via Docker CLI works without a problem:

Have some proxy configuration been applied to the Docker CLI?

There are no options on the Spring Boot build plugins to configure a proxy. There are configuration options that tell the plugins how to talk to the Docker daemon process, but that is different from the Docker daemon talking to a repository.

Can you try some of the options documented by Docker here and here to configure the daemon for proxy?

[eidottermihi]

@scottfrederick thank you for looking into this issue!

It appears there is a network proxy between the Docker daemon and the docker.io host that requires authentication.

That is correct, but our corporate proxy does not require authentication. I've configured the proxy for the daemon according to the docker docs in daemon.json:

{
  "proxies": {
    "http-proxy": "http://proxy.mycorp.com:80",
    "https-proxy": "http://proxy.mycorp.com:80",
    "no-proxy": "*.mycorp.com,mycorp.com"
  }
}

I still think it is related to missing authentification for docker.io:

If I'm using the Docker CLI and do a "docker logout" and then performing a docker pull, this then also fails:

$ docker logout
Removing login credentials for https://index.docker.io/v1/

$ docker pull docker.io/paketobuildpacks/builder:tiny
Error response from daemon: Please run 'docker login'

The error message for docker pull looks exactly the same as the error, when the maven plugin tries to pull this image:

[2023-08-03T06:19:35.610758000Z][docker][I] [2023-08-03T06:19:35.610417500Z][lifecycle-server][I] must be logged in to call /v1.42/images/create?fromImage=paketobuildpacks%2Fbuilder&tag=tiny

After performing the docker login for docker.io again, docker pull is working again.

If I'm configuring authentication for the builderRegistry directly in plugin configuration, it starts to work!

<plugin>
  <artifactId>spring-boot-maven-plugin</artifactId>
  <configuration>
    <docker>
      <builderRegistry>
        <username>my_dockerhub_user</username>
        <password>my_dockerhub_pat</password>
        <url>https://index.docker.io/v1/</url>
        <email>my_dockerhub_email</email>
      </builderRegistry>
    </docker>
<!-- ... --->

So It seems to me that docker.io requires auth - also for sponsored OSS images? Not sure if this is a maven plugin related issue then, but maybe this can be added to the docs?

Or #25898 comes back into play, as this would get the authentication from my local config.json?

[scottfrederick]

So It seems to me that docker.io requires auth - also for sponsored OSS images?

This is not the case. I have Docker Engine and Docker Desktop on Linux, and I can pull the Paketo builder images while logged out with both of those Docker daemons. Spring Boot has CI infrastructure that pulls Paketo images without authentication also.

The flow here is: Docker client (CLI or Boot plugins) using the Docker Engine API -> Docker deamon (Docker Desktop in your case) at localhost/v1.42 -> docker.io

I believe the message must be logged in to call /v1.42/images/create?fromImage=paketobuildpacks%2Fbuilder&tag=tiny is the Docker daemon telling client using the API at localhost/v1.42 (the Boot plugin in this case) that the client must be logged in or provide authentication before using the Docker Engine API. I do not think this is the docker.io host telling the daemon that authentication to the repository is required. The same thing goes for the Error response from daemon: Please run 'docker login' message from the Docker CLI.

You could test this theory by using the Docker CLI or the Boot plugins to pull an image from a different repository to see if that changes anything.

I don't know what would cause Docker Desktop on Windows to require any call to the Docker Engine API to be authenticated, but that appears to be what's going on.

[eidottermihi]

You could test this theory by using the Docker CLI or the Boot plugins to pull an image from a different repository to see if that changes anything.

Apparently Docker Desktop on Windows requires authentication when pulling from docker.io - all other public registries work fine without authentication:

with docker.io: "docker login" required

* Connected to localhost (::1) port 2375 (#0)
> POST /v1.24/images/create?fromImage=docker.io%2Fpaketobuildpacks%2Fbuilder%3Atiny HTTP/1.1
< HTTP/1.1 407 Proxy Authentication Required
100    26    0    26    0     0   1529      0 --:--:-- --:--:-- --:--:--  1625Please run 'docker login'

with quay.io registry: works fine

curl -X POST -v http://localhost:2375/v1.24/images/create?fromImage=quay.io%2Fopenshift%2Forigin-cli
* Connected to localhost (::1) port 2375 (#0)
> POST /v1.24/images/create?fromImage=quay.io%2Fopenshift%2Forigin-cli HTTP/1.1
< HTTP/1.1 200 OK
<
100  2244    0  2244    0     0    269      0 --:--:--  0:00:08 --:--:--   548{"status":"Pulling from openshift/origin-cli","id":"v3.10.0"}
{"status":"Pulling fs layer","progressDetail":{},"id":"7dc0dca2b151"}
[....]

This is also the case if I'm using the Docker CLI:

$ docker pull docker.io/paketobuildpacks/builder:tiny
Error response from daemon: Please run 'docker login'

$ docker pull quay.io/openshift/origin-cli
Using default tag: latest
latest: Pulling from openshift/origin-cli
[...]

I will open an case at Docker Desktop support and hope they can shine some light into why the Docker Engine API on Docker Desktop for Windows requires authentication to pull public/sponsored OSS image from docker.io.

In the meantime I will use the "workaround" and supply docker.io builderRegistry credentials to the spring-boot-maven-plugin (see #36671 (comment) ).

@scottfrederick Thanks for sticking with me! Closing the issue as it not related to Spring Boot and the Maven plugin itself.