Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unexpected javax.net.ssl.SSLHandshakeException in seemingly simple usage scenario with Jersey Client 2 (>= 2.41) #5647

Open
sergiy-kozak opened this issue May 17, 2024 · 8 comments

Comments

@sergiy-kozak
Copy link

sergiy-kozak commented May 17, 2024

Using Jersey Client 2.41-2.43, I have tried to execute the following sample program with openjdk 21 to figure out other issues encountered in the real application.

package org.jersey.issue;

import static java.util.Map.entry;

import java.io.IOException;
import java.io.InputStream;
import java.net.URI;
import java.nio.charset.StandardCharsets;
import java.time.Duration;
import java.time.LocalDateTime;
import java.util.Map;
import java.util.Objects;
import java.util.Optional;
import java.util.logging.Level;

import javax.ws.rs.client.Client;
import javax.ws.rs.client.ClientBuilder;
import javax.ws.rs.client.Entity;
import javax.ws.rs.client.WebTarget;
import javax.ws.rs.core.Response;

import org.glassfish.jersey.client.ClientConfig;
import org.glassfish.jersey.client.ClientProperties;
import org.glassfish.jersey.jackson.JacksonFeature;
import org.glassfish.jersey.logging.LoggingFeature;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;


public class Jersey241ConnectionsIssue
{

    private static final Logger LOG = LoggerFactory.getLogger(Jersey241ConnectionsIssue.class);

    private static final URI targetUri = URI.create("https://reqbin.com/echo/post/json");


    static Optional<String> getErrText(final Response response)
    {
        final Object errDetails = response.getEntity();
        String errStr = null;
        if (!Objects.isNull(errDetails) && errDetails instanceof InputStream)
        {
            try
            {
                errStr = new String(((InputStream) errDetails).readAllBytes(), StandardCharsets.UTF_8);
            }
            catch (IOException ex)
            {
                LOG.error("Failed to read error response (JSON) payload from the response entity: ", ex);
            }
        }
        return Optional.ofNullable(errStr);
    }


    public static void main(final String[] args)
    {
        final ClientConfig clientCfg = new ClientConfig(JacksonFeature.class);
        clientCfg.register(new LoggingFeature(java.util.logging.Logger.getLogger(LOG.getName()), Level.INFO, null, null))
                 .property(ClientProperties.FOLLOW_REDIRECTS, true)
                 .property(ClientProperties.READ_TIMEOUT, 60*1000)
                 .property(ClientProperties.CONNECT_TIMEOUT, 60*1000);
        final Client webClient = ClientBuilder.newBuilder().withConfig(clientCfg).build();

        LocalDateTime startTs = LocalDateTime.now();
        for (int c = 0; c < 200; c++)
        {
            var values = Map.ofEntries(entry("Id", "12345"),
                    entry("Customer", "Roger Moose"),
                    entry("Quantity", String.valueOf(c)),
                    entry("Price", "167.35"));
            final WebTarget t = webClient.target(targetUri);
            try (final Response response = t.request().post(Entity.json(values)))
            {
                if (Response.Status.OK.getStatusCode() != response.getStatus())
                {
                    final String errStr = getErrText(response).orElse("null");
                    LOG.error("Cannot POST payload to: {}. Error: {}", targetUri, errStr);
                    throw new RuntimeException("Cannot POST payload to: " + targetUri + ". Error: " + errStr);
                }
            }
        }
        final Duration elapsed = Duration.between(startTs, LocalDateTime.now());
        LOG.info("Elapsed time: {} sec", elapsed.toSeconds());
    }
}

The program fails to reach the end of requests loop:

May 16, 2024 10:25:43 PM org.glassfish.jersey.message.internal.MessagingBinders$EnabledProvidersBinder bindToBinder
WARNING: A class javax.activation.DataSource for a default provider MessageBodyWriter<javax.activation.DataSource> was not found. The provider is not available.
May 16, 2024 10:25:44 PM org.glassfish.jersey.logging.LoggingInterceptor log
INFO: 1 * Sending client request on thread main
1 > POST https://reqbin.com/echo/post/json
1 > Content-Type: application/json
{"Price":"167.35","Customer":"Roger Moose","Quantity":"0","Id":"12345"}

May 16, 2024 10:25:44 PM org.glassfish.jersey.logging.LoggingInterceptor log
INFO: 1 * Client response received on thread main
1 < 200
1 < Access-Control-Allow-Origin: https://reqbin.com
1 < alt-svc: h3=":443"; ma=86400
1 < CF-Cache-Status: DYNAMIC
1 < CF-RAY: 884e1945dce39ffa-AMS
1 < Connection: keep-alive
1 < Content-Length: 19
1 < Content-Type: application/json
1 < Date: Thu, 16 May 2024 20:25:44 GMT
1 < NEL: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
1 < referrer-policy: same-origin
1 < Report-To: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v4?s=T4GU%2FhByPH%2FoTJL41MluC%2F1N7%2FeqZSZDSEAXDXJC4JdfboXdyYAnYp5R6ZzZZfpVMdu%2BmYbCh0T7Q%2BmA1Yzu1ecDz404%2FpH%2B2liZJcQvj3Dz0zHwchhD0kBLwg0%3D"}],"group":"cf-nel","max_age":604800}
1 < Server: cloudflare
1 < x-content-type-options: nosniff
1 < x-frame-options: SAMEORIGIN
1 < x-xss-protection: 1; mode=block
{"success":"true"}

...........................................................................
clipped output here
............................................................................

May 16, 2024 10:26:01 PM org.glassfish.jersey.logging.LoggingInterceptor log
INFO: 93 * Sending client request on thread main
93 > POST https://reqbin.com/echo/post/json
93 > Content-Type: application/json
{"Price":"167.35","Customer":"Roger Moose","Quantity":"92","Id":"12345"}

May 16, 2024 10:26:01 PM org.glassfish.jersey.logging.LoggingInterceptor log
INFO: 93 * Client response received on thread main
93 < 200
93 < Access-Control-Allow-Origin: https://reqbin.com
93 < alt-svc: h3=":443"; ma=86400
93 < CF-Cache-Status: DYNAMIC
93 < CF-RAY: 884e19ae1a441c86-AMS
93 < Connection: keep-alive
93 < Content-Length: 19
93 < Content-Type: application/json
93 < Date: Thu, 16 May 2024 20:26:01 GMT
93 < NEL: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
93 < referrer-policy: same-origin
93 < Report-To: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v4?s=xUui60S%2F9gt3ACq1MbpyoTzoZ%2FMQXg9RNHkM%2F%2Fwa7jHAgKszEDzcna176gdtxoVq13m17cA%2Bvx7ziThKE0wBXvPKX2Aq9UB0kJgsVSGUr26dYZJvEklzXlM0ldc%3D"}],"group":"cf-nel","max_age":604800}
93 < Server: cloudflare
93 < x-content-type-options: nosniff
93 < x-frame-options: SAMEORIGIN
93 < x-xss-protection: 1; mode=block
{"success":"true"}


Exception in thread "main" javax.ws.rs.client.ResponseProcessingException: javax.net.ssl.SSLHandshakeException: Received fatal alert: decode_error
	at org.glassfish.jersey.client.JerseyInvocation.call(JerseyInvocation.java:712)
	at org.glassfish.jersey.client.JerseyInvocation.lambda$runInScope$3(JerseyInvocation.java:703)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:205)
	at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:390)
	at org.glassfish.jersey.client.JerseyInvocation.runInScope(JerseyInvocation.java:703)
	at org.glassfish.jersey.client.JerseyInvocation.invoke(JerseyInvocation.java:673)
	at org.glassfish.jersey.client.JerseyInvocation$Builder.method(JerseyInvocation.java:439)
	at org.glassfish.jersey.client.JerseyInvocation$Builder.post(JerseyInvocation.java:345)
	at com.irdeto.switcher.cli.Jersey241ConnectionsIssue.main(Jersey241ConnectionsIssue.java:79)
Caused by: javax.net.ssl.SSLHandshakeException: Received fatal alert: decode_error
	at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:130)
	at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:117)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:365)
	at java.base/sun.security.ssl.Alert$AlertConsumer.consume(Alert.java:287)
	at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:204)
	at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:172)
	at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1506)
	at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1421)
	at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:455)
	at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:426)
	at java.base/sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:586)
	at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:187)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1446)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1417)
	at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:219)
	at org.glassfish.jersey.client.internal.HttpUrlConnector.lambda$_apply$2(HttpUrlConnector.java:418)
	at org.glassfish.jersey.message.internal.CommittingOutputStream.commitStream(CommittingOutputStream.java:174)
	at org.glassfish.jersey.message.internal.CommittingOutputStream.commitStream(CommittingOutputStream.java:168)
	at org.glassfish.jersey.message.internal.CommittingOutputStream.write(CommittingOutputStream.java:202)
	at org.glassfish.jersey.logging.LoggingInterceptor$LoggingStream.write(LoggingInterceptor.java:331)
	at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$UnCloseableOutputStream.write(WriterInterceptorExecutor.java:276)
	at com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.java:2210)
	at com.fasterxml.jackson.core.json.UTF8JsonGenerator.flush(UTF8JsonGenerator.java:1204)
	at com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:1063)
	at org.glassfish.jersey.jackson.internal.jackson.jaxrs.base.ProviderBase.writeTo(ProviderBase.java:648)
	at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.invokeWriteTo(WriterInterceptorExecutor.java:242)
	at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.aroundWriteTo(WriterInterceptorExecutor.java:227)
	at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139)
	at org.glassfish.jersey.logging.LoggingInterceptor.aroundWriteTo(LoggingInterceptor.java:244)
	at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139)
	at org.glassfish.jersey.message.internal.MessageBodyFactory.writeTo(MessageBodyFactory.java:1116)
	at org.glassfish.jersey.client.ClientRequest.doWriteEntity(ClientRequest.java:468)
	at org.glassfish.jersey.client.ClientRequest.writeEntity(ClientRequest.java:450)
	at org.glassfish.jersey.client.internal.HttpUrlConnector._apply(HttpUrlConnector.java:420)
	at org.glassfish.jersey.client.internal.HttpUrlConnector.apply(HttpUrlConnector.java:273)
	at org.glassfish.jersey.client.ClientRuntime.invoke(ClientRuntime.java:300)
	at org.glassfish.jersey.client.JerseyInvocation.lambda$invoke$0(JerseyInvocation.java:674)
	at org.glassfish.jersey.client.JerseyInvocation.call(JerseyInvocation.java:709)
	... 10 more

During the above program execution, there is unexplainable lots of connections open by the process (obviously managed in some ways by client):

$ while true; do ./display_javapid_open_sockets.sh; sleep 1; done
PID: 69866
java process open files:
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 146 -> socket:[616908]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 147 -> socket:[616918]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 148 -> socket:[616919]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 149 -> socket:[616920]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 150 -> socket:[616921]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 151 -> socket:[616922]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 152 -> socket:[616923]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 153 -> socket:[614630]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 154 -> socket:[614631]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 155 -> socket:[614632]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 156 -> socket:[614633]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 157 -> socket:[614634]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 158 -> socket:[614635]
java open sockets count: 13
PID: 69866
java process open files:
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 146 -> socket:[616908]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 147 -> socket:[616918]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 148 -> socket:[616919]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 149 -> socket:[616920]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 150 -> socket:[616921]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 151 -> socket:[616922]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 152 -> socket:[616923]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 153 -> socket:[614630]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 154 -> socket:[614631]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 155 -> socket:[614632]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 156 -> socket:[614633]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 157 -> socket:[614634]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 158 -> socket:[614635]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 159 -> socket:[614646]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 160 -> socket:[614647]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 161 -> socket:[614648]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 162 -> socket:[614649]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 163 -> socket:[614650]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 164 -> socket:[614651]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 165 -> socket:[614652]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 166 -> socket:[614653]
java open sockets count: 21
PID: 69866
java process open files:
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 146 -> socket:[616908]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 147 -> socket:[616918]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 148 -> socket:[616919]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 149 -> socket:[616920]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 150 -> socket:[616921]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 151 -> socket:[616922]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 152 -> socket:[616923]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 153 -> socket:[614630]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 154 -> socket:[614631]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 155 -> socket:[614632]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 156 -> socket:[614633]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 157 -> socket:[614634]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 158 -> socket:[614635]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 159 -> socket:[614646]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 160 -> socket:[614647]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 161 -> socket:[614648]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 162 -> socket:[614649]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 163 -> socket:[614650]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 164 -> socket:[614651]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 165 -> socket:[614652]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 166 -> socket:[614653]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 167 -> socket:[614662]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 168 -> socket:[614663]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 169 -> socket:[614664]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 170 -> socket:[614665]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 171 -> socket:[614666]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 172 -> socket:[614667]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 173 -> socket:[614668]
java open sockets count: 28
PID: 69866
java process open files:
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 146 -> socket:[616908]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 147 -> socket:[616918]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 148 -> socket:[616919]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 149 -> socket:[616920]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 150 -> socket:[616921]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 151 -> socket:[616922]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 152 -> socket:[616923]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 153 -> socket:[614630]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 154 -> socket:[614631]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 155 -> socket:[614632]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 156 -> socket:[614633]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 157 -> socket:[614634]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 158 -> socket:[614635]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 159 -> socket:[614646]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 160 -> socket:[614647]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 161 -> socket:[614648]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 162 -> socket:[614649]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 163 -> socket:[614650]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 164 -> socket:[614651]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 165 -> socket:[614652]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 166 -> socket:[614653]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 167 -> socket:[614662]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 168 -> socket:[614663]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 169 -> socket:[614664]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 170 -> socket:[614665]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 171 -> socket:[614666]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 172 -> socket:[614667]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 173 -> socket:[614668]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 174 -> socket:[614677]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 175 -> socket:[614678]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 176 -> socket:[614679]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 177 -> socket:[614680]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 178 -> socket:[614681]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 179 -> socket:[618667]
java open sockets count: 34
PID: 69866
java process open files:
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 146 -> socket:[616908]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 147 -> socket:[616918]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 148 -> socket:[616919]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 149 -> socket:[616920]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 150 -> socket:[616921]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 151 -> socket:[616922]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 152 -> socket:[616923]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 153 -> socket:[614630]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 154 -> socket:[614631]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 155 -> socket:[614632]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 156 -> socket:[614633]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 157 -> socket:[614634]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 158 -> socket:[614635]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 159 -> socket:[614646]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 160 -> socket:[614647]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 161 -> socket:[614648]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 162 -> socket:[614649]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 163 -> socket:[614650]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 164 -> socket:[614651]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 165 -> socket:[614652]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 166 -> socket:[614653]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 167 -> socket:[614662]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 168 -> socket:[614663]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 169 -> socket:[614664]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 170 -> socket:[614665]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 171 -> socket:[614666]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 172 -> socket:[614667]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 173 -> socket:[614668]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 174 -> socket:[614677]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 175 -> socket:[614678]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 176 -> socket:[614679]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 177 -> socket:[614680]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 178 -> socket:[614681]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 179 -> socket:[618667]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 180 -> socket:[618668]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 181 -> socket:[618669]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 182 -> socket:[618670]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 183 -> socket:[618671]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 184 -> socket:[618672]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 185 -> socket:[618673]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 186 -> socket:[618674]
java open sockets count: 41
PID: 69866
java process open files:
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 146 -> socket:[616908]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 147 -> socket:[616918]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 148 -> socket:[616919]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 149 -> socket:[616920]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 150 -> socket:[616921]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 151 -> socket:[616922]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 152 -> socket:[616923]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 153 -> socket:[614630]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 154 -> socket:[614631]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 155 -> socket:[614632]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 156 -> socket:[614633]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 157 -> socket:[614634]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 158 -> socket:[614635]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 159 -> socket:[614646]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 160 -> socket:[614647]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 161 -> socket:[614648]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 162 -> socket:[614649]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 163 -> socket:[614650]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 164 -> socket:[614651]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 165 -> socket:[614652]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 166 -> socket:[614653]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 167 -> socket:[614662]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 168 -> socket:[614663]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 169 -> socket:[614664]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 170 -> socket:[614665]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 171 -> socket:[614666]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 172 -> socket:[614667]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 173 -> socket:[614668]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 174 -> socket:[614677]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 175 -> socket:[614678]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 176 -> socket:[614679]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 177 -> socket:[614680]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 178 -> socket:[614681]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 179 -> socket:[618667]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 180 -> socket:[618668]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 181 -> socket:[618669]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 182 -> socket:[618670]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 183 -> socket:[618671]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 184 -> socket:[618672]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 185 -> socket:[618673]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 186 -> socket:[618674]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 187 -> socket:[618682]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 188 -> socket:[618683]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 189 -> socket:[618684]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 190 -> socket:[618685]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 191 -> socket:[618686]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 192 -> socket:[618687]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 193 -> socket:[618688]
java open sockets count: 48
PID: 69866
java process open files:
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 146 -> socket:[616908]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 147 -> socket:[616918]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 148 -> socket:[616919]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 149 -> socket:[616920]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 150 -> socket:[616921]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 151 -> socket:[616922]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 152 -> socket:[616923]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 153 -> socket:[614630]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 154 -> socket:[614631]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 155 -> socket:[614632]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 156 -> socket:[614633]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 157 -> socket:[614634]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 158 -> socket:[614635]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 159 -> socket:[614646]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 160 -> socket:[614647]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 161 -> socket:[614648]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 162 -> socket:[614649]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 163 -> socket:[614650]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 164 -> socket:[614651]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 165 -> socket:[614652]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 166 -> socket:[614653]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 167 -> socket:[614662]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 168 -> socket:[614663]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 169 -> socket:[614664]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 170 -> socket:[614665]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 171 -> socket:[614666]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 172 -> socket:[614667]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 173 -> socket:[614668]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 174 -> socket:[614677]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 175 -> socket:[614678]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 176 -> socket:[614679]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 177 -> socket:[614680]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 178 -> socket:[614681]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 179 -> socket:[618667]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 180 -> socket:[618668]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 181 -> socket:[618669]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 182 -> socket:[618670]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 183 -> socket:[618671]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 184 -> socket:[618672]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 185 -> socket:[618673]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 186 -> socket:[618674]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 187 -> socket:[618682]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 188 -> socket:[618683]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 189 -> socket:[618684]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 190 -> socket:[618685]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 191 -> socket:[618686]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 192 -> socket:[618687]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 193 -> socket:[618688]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 194 -> socket:[618692]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 195 -> socket:[618693]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 196 -> socket:[618694]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 197 -> socket:[618695]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 198 -> socket:[618696]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 199 -> socket:[618697]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 200 -> socket:[618698]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 201 -> socket:[618699]
java open sockets count: 56
PID: 69866
java process open files:
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 146 -> socket:[616908]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 147 -> socket:[616918]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 148 -> socket:[616919]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 149 -> socket:[616920]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 150 -> socket:[616921]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 151 -> socket:[616922]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 152 -> socket:[616923]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 153 -> socket:[614630]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 154 -> socket:[614631]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 155 -> socket:[614632]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 156 -> socket:[614633]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 157 -> socket:[614634]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 158 -> socket:[614635]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 159 -> socket:[614646]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 160 -> socket:[614647]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 161 -> socket:[614648]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 162 -> socket:[614649]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 163 -> socket:[614650]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 164 -> socket:[614651]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 165 -> socket:[614652]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 166 -> socket:[614653]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 167 -> socket:[614662]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 168 -> socket:[614663]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 169 -> socket:[614664]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 170 -> socket:[614665]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 171 -> socket:[614666]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 172 -> socket:[614667]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 173 -> socket:[614668]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 174 -> socket:[614677]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 175 -> socket:[614678]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 176 -> socket:[614679]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 177 -> socket:[614680]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 178 -> socket:[614681]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 179 -> socket:[618667]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 180 -> socket:[618668]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 181 -> socket:[618669]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 182 -> socket:[618670]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 183 -> socket:[618671]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 184 -> socket:[618672]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 185 -> socket:[618673]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 186 -> socket:[618674]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 187 -> socket:[618682]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 188 -> socket:[618683]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 189 -> socket:[618684]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 190 -> socket:[618685]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 191 -> socket:[618686]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 192 -> socket:[618687]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 193 -> socket:[618688]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 194 -> socket:[618692]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 195 -> socket:[618693]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 196 -> socket:[618694]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 197 -> socket:[618695]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 198 -> socket:[618696]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 199 -> socket:[618697]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 200 -> socket:[618698]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 201 -> socket:[618699]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 202 -> socket:[618703]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 203 -> socket:[618704]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 204 -> socket:[618705]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 205 -> socket:[618706]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 206 -> socket:[617063]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 207 -> socket:[617064]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 208 -> socket:[617065]
java open sockets count: 63
PID: 69866
java process open files:
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 146 -> socket:[616908]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 147 -> socket:[616918]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 148 -> socket:[616919]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 149 -> socket:[616920]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 150 -> socket:[616921]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 151 -> socket:[616922]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 152 -> socket:[616923]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 153 -> socket:[614630]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 154 -> socket:[614631]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 155 -> socket:[614632]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 156 -> socket:[614633]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 157 -> socket:[614634]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 158 -> socket:[614635]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 159 -> socket:[614646]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 160 -> socket:[614647]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 161 -> socket:[614648]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 162 -> socket:[614649]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 163 -> socket:[614650]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 164 -> socket:[614651]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 165 -> socket:[614652]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 166 -> socket:[614653]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 167 -> socket:[614662]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 168 -> socket:[614663]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 169 -> socket:[614664]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 170 -> socket:[614665]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 171 -> socket:[614666]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 172 -> socket:[614667]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 173 -> socket:[614668]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 174 -> socket:[614677]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 175 -> socket:[614678]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 176 -> socket:[614679]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 177 -> socket:[614680]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 178 -> socket:[614681]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 179 -> socket:[618667]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 180 -> socket:[618668]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 181 -> socket:[618669]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 182 -> socket:[618670]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 183 -> socket:[618671]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 184 -> socket:[618672]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 185 -> socket:[618673]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 186 -> socket:[618674]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 187 -> socket:[618682]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 188 -> socket:[618683]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 189 -> socket:[618684]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 190 -> socket:[618685]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 191 -> socket:[618686]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 192 -> socket:[618687]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 193 -> socket:[618688]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 194 -> socket:[618692]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 195 -> socket:[618693]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 196 -> socket:[618694]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 197 -> socket:[618695]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 198 -> socket:[618696]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 199 -> socket:[618697]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 200 -> socket:[618698]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 201 -> socket:[618699]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 202 -> socket:[618703]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 203 -> socket:[618704]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 204 -> socket:[618705]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 205 -> socket:[618706]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 206 -> socket:[617063]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 207 -> socket:[617064]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 208 -> socket:[617065]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:23 209 -> socket:[617084]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:23 210 -> socket:[617085]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 211 -> socket:[617086]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 212 -> socket:[617087]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 213 -> socket:[617088]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 214 -> socket:[617089]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 215 -> socket:[617090]
java open sockets count: 70
PID: 69866
java process open files:
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 146 -> socket:[616908]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 147 -> socket:[617594]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 148 -> socket:[617595]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 149 -> socket:[617596]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 150 -> socket:[617597]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 181 -> socket:[618669]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 182 -> socket:[618670]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 183 -> socket:[618671]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 184 -> socket:[618672]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 185 -> socket:[618673]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 186 -> socket:[618674]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 187 -> socket:[618682]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 188 -> socket:[618683]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 189 -> socket:[618684]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 190 -> socket:[618685]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 191 -> socket:[618686]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 192 -> socket:[618687]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 193 -> socket:[618688]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 194 -> socket:[618692]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 195 -> socket:[618693]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 196 -> socket:[618694]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 197 -> socket:[618695]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 198 -> socket:[618696]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 199 -> socket:[618697]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 200 -> socket:[618698]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 201 -> socket:[618699]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 202 -> socket:[618703]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 203 -> socket:[618704]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 204 -> socket:[618705]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 205 -> socket:[618706]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 206 -> socket:[617063]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 207 -> socket:[617064]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 208 -> socket:[617065]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:23 209 -> socket:[617084]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:23 210 -> socket:[617085]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 211 -> socket:[617086]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 212 -> socket:[617087]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 213 -> socket:[617088]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 214 -> socket:[617089]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 215 -> socket:[617090]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 216 -> socket:[617592]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 217 -> socket:[617593]
java open sockets count: 42

<clipped output>

PID: 69866
java process open files:
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 146 -> socket:[616908]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 147 -> socket:[617671]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 148 -> socket:[617672]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 149 -> socket:[617673]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 150 -> socket:[600923]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 151 -> socket:[600924]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 152 -> socket:[613957]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 153 -> socket:[613958]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 154 -> socket:[613959]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 155 -> socket:[613960]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 156 -> socket:[613961]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 157 -> socket:[613962]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 158 -> socket:[613963]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 159 -> socket:[613964]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 160 -> socket:[613965]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 161 -> socket:[613966]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 162 -> socket:[613967]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 163 -> socket:[613968]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 164 -> socket:[613969]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 165 -> socket:[613970]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 166 -> socket:[613971]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 167 -> socket:[613974]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 168 -> socket:[613975]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 169 -> socket:[613976]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 170 -> socket:[613977]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 171 -> socket:[613978]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 172 -> socket:[613979]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 173 -> socket:[613980]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 174 -> socket:[614000]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 175 -> socket:[614001]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 176 -> socket:[614002]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 177 -> socket:[614003]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 178 -> socket:[614004]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 179 -> socket:[614005]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 180 -> socket:[619570]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 181 -> socket:[619571]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:23 209 -> socket:[617670]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:23 210 -> socket:[613981]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 211 -> socket:[613984]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 212 -> socket:[613985]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 213 -> socket:[613986]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 214 -> socket:[613987]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 215 -> socket:[613988]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 216 -> socket:[613989]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 217 -> socket:[613990]
java open sockets count: 45
PID: 69866
java process open files:
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 146 -> socket:[616908]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 147 -> socket:[617671]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 148 -> socket:[617672]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 149 -> socket:[617673]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 150 -> socket:[600923]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 151 -> socket:[600924]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 152 -> socket:[613957]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 153 -> socket:[613958]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 154 -> socket:[613959]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 155 -> socket:[613960]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 156 -> socket:[613961]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 157 -> socket:[613962]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 158 -> socket:[613963]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 159 -> socket:[613964]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 160 -> socket:[613965]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 161 -> socket:[613966]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 162 -> socket:[613967]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 163 -> socket:[613968]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 164 -> socket:[613969]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 165 -> socket:[613970]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 166 -> socket:[613971]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 167 -> socket:[613974]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 168 -> socket:[613975]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 169 -> socket:[613976]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 170 -> socket:[613977]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 171 -> socket:[613978]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 172 -> socket:[613979]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 173 -> socket:[613980]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 174 -> socket:[614000]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 175 -> socket:[614001]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 176 -> socket:[614002]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 177 -> socket:[614003]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 178 -> socket:[614004]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 179 -> socket:[614005]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 180 -> socket:[619570]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 181 -> socket:[619571]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 182 -> socket:[619581]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 183 -> socket:[619582]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 184 -> socket:[619583]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 185 -> socket:[619584]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 186 -> socket:[619585]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:23 209 -> socket:[617670]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:23 210 -> socket:[613981]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 211 -> socket:[613984]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 212 -> socket:[613985]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 213 -> socket:[613986]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 214 -> socket:[613987]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 215 -> socket:[613988]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 216 -> socket:[613989]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 217 -> socket:[613990]
java open sockets count: 50
PID: 69866
java process open files:
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 146 -> socket:[616908]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 147 -> socket:[617671]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 148 -> socket:[617672]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 149 -> socket:[617673]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 150 -> socket:[600923]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 151 -> socket:[600924]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 152 -> socket:[613957]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 153 -> socket:[613958]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 154 -> socket:[613959]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 155 -> socket:[613960]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 156 -> socket:[613961]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 157 -> socket:[613962]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 158 -> socket:[613963]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 159 -> socket:[613964]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 160 -> socket:[613965]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 161 -> socket:[613966]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 162 -> socket:[613967]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 163 -> socket:[613968]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 164 -> socket:[613969]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 165 -> socket:[613970]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 166 -> socket:[613971]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 167 -> socket:[613974]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 168 -> socket:[613975]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 169 -> socket:[613976]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 170 -> socket:[613977]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 171 -> socket:[613978]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 172 -> socket:[613979]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 173 -> socket:[613980]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 174 -> socket:[614000]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 175 -> socket:[614001]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 176 -> socket:[614002]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 177 -> socket:[614003]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 178 -> socket:[614004]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 179 -> socket:[614005]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 180 -> socket:[619570]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 181 -> socket:[619571]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 182 -> socket:[619581]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 183 -> socket:[619582]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 184 -> socket:[619583]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 185 -> socket:[619584]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 186 -> socket:[619585]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 187 -> socket:[619598]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 188 -> socket:[619599]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 189 -> socket:[619600]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 190 -> socket:[619601]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:23 191 -> socket:[619602]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 192 -> socket:[619603]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:23 209 -> socket:[617670]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:23 210 -> socket:[613981]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 211 -> socket:[613984]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 212 -> socket:[613985]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 213 -> socket:[613986]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 214 -> socket:[613987]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 215 -> socket:[613988]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 216 -> socket:[613989]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 217 -> socket:[613990]
java open sockets count: 56
PID: 69866
java process open files:
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 146 -> socket:[616908]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 147 -> socket:[617671]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 148 -> socket:[617672]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 149 -> socket:[617673]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 150 -> socket:[600923]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 151 -> socket:[600924]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 152 -> socket:[613957]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 153 -> socket:[613958]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 154 -> socket:[613959]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 155 -> socket:[613960]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 156 -> socket:[613961]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 157 -> socket:[613962]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 158 -> socket:[613963]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 159 -> socket:[613964]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 160 -> socket:[613965]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 161 -> socket:[613966]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 162 -> socket:[613967]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 163 -> socket:[613968]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 164 -> socket:[613969]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 165 -> socket:[613970]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 166 -> socket:[613971]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 167 -> socket:[613974]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 168 -> socket:[613975]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 169 -> socket:[613976]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 170 -> socket:[613977]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 171 -> socket:[613978]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 172 -> socket:[613979]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 173 -> socket:[613980]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 174 -> socket:[614000]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 175 -> socket:[614001]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 176 -> socket:[614002]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 177 -> socket:[614003]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 178 -> socket:[614004]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 179 -> socket:[614005]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 180 -> socket:[619570]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 181 -> socket:[619571]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 182 -> socket:[619581]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 183 -> socket:[619582]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 184 -> socket:[619583]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 185 -> socket:[619584]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 186 -> socket:[619585]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 187 -> socket:[619598]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 188 -> socket:[619599]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 189 -> socket:[619600]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 190 -> socket:[619601]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:23 191 -> socket:[619602]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 192 -> socket:[619603]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 193 -> socket:[619607]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 194 -> socket:[600990]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 195 -> socket:[600991]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 196 -> socket:[600992]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 197 -> socket:[600993]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 198 -> socket:[600994]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 199 -> socket:[600995]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 200 -> socket:[600996]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:23 209 -> socket:[617670]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:23 210 -> socket:[613981]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 211 -> socket:[613984]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 212 -> socket:[613985]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 213 -> socket:[613986]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 214 -> socket:[613987]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 215 -> socket:[613988]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 216 -> socket:[613989]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:22 217 -> socket:[613990]
java open sockets count: 64

However, when switching to Jersey 2.40 that issue is not anymore. All requests are completed in expected normal way, and for program's java process only 2 sockets are kept open (and I assume just reused) through the entire run.

$ while true; do ./display_javapid_open_sockets.sh; sleep 1; done
PID: 70875
java process open files:
lrwx------ 1 ubuntu ubuntu 64 May 17 08:29 146 -> socket:[601083]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:29 147 -> socket:[624646]
java open sockets count: 2
PID: 70875
java process open files:
lrwx------ 1 ubuntu ubuntu 64 May 17 08:29 146 -> socket:[601083]
lrwx------ 1 ubuntu ubuntu 64 May 17 08:29 147 -> socket:[624646]
java open sockets count: 2
.................................................................
same output repeats till the end
.................................................................

System:

$ uname -a
Linux primary 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

$ cat /etc/os-release
PRETTY_NAME="Ubuntu 22.04.4 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.4 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

$ java --version
openjdk 21.0.2 2024-01-16
OpenJDK Runtime Environment (build 21.0.2+13-Ubuntu-122.04.1)
OpenJDK 64-Bit Server VM (build 21.0.2+13-Ubuntu-122.04.1, mixed mode, sharing)

Is this a regression of some kind? The same might be applicable for latest 3.x, but I cannot test it right ahead. For real application I will roll back to 2.40 as latest working one; in real application running in Debian 12 environment other exception is thrown, javax.ws.rs.ProcessingException: java.net.SocketTimeoutException: Connect timed out while observing the same pattern of TCP connections usage.

Update:
Jackson dependencies version used 2.17.0 (if relevant).
The artifact com.fasterxml.jackson.jaxrs:jackson-jaxrs-json-provider was present on the classpath when running program with the above observations. When using 2.43 Jersey and removing jackson-jaxrs-json-provider from classpath makes program reach the end of the requests loop, however the usage of TCP connection shows same pattern. Also, there is major difference in the execution time (reproduced fairly consistent):

  • 2.40: "Elapsed time: 24 sec"
  • 2.43: "Elapsed time: 32 sec"
@sergiy-kozak
Copy link
Author

Captured also list of client connections with netstat:

$ while true; do netstat -anp | grep "$(pgrep java | tail -n1)/java"; sleep 1; done
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
tcp6       0      0 172.25.75.3:40586       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40346       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40316       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40450       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:50142       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40658       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40498       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40512       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40404       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40308       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:50156       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:50108       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40466       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40294       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40318       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:50122       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40602       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40330       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40630       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40416       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40344       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40520       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40434       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40612       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40456       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40418       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40280       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40544       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40600       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40626       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40288       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40650       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40354       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40302       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40482       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40474       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40432       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40360       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:50120       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40324       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40572       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40412       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40368       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40238       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40560       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40400       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40378       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40642       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40532       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:50164       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40252       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:50134       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40264       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40646       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40392       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:50094       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40574       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40230       172.67.72.249:443       ESTABLISHED 109768/java
unix  2      [ ]         STREAM     CONNECTED     798347   109768/java
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
tcp6       0      0 172.25.75.3:40586       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40346       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40680       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40316       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40450       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:50142       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40658       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40498       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40512       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40404       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40308       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:50156       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:50108       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40466       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40294       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40318       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:50122       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40602       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40330       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40630       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40416       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40344       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40520       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40434       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40612       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40676       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40456       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40418       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40280       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40544       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40682       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40600       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40626       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40288       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40650       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40694       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40354       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40664       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40302       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40482       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40474       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40432       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40360       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:50120       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40324       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40572       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40688       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40412       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40368       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40238       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40560       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40400       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40378       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40642       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40532       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:50164       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40252       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:50134       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40264       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40646       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40392       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:50094       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40686       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40574       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40230       172.67.72.249:443       ESTABLISHED 109768/java
unix  2      [ ]         STREAM     CONNECTED     798347   109768/java
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
tcp6       0      0 172.25.75.3:40586       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40680       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40450       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40658       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40712       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40498       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40512       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40466       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40602       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40630       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40520       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40612       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40676       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40456       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40740       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40544       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40682       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40600       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40626       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40742       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40710       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40650       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40694       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40664       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40728       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40482       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40474       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0    111 172.25.75.3:40760       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40572       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40688       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40750       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40738       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40560       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40642       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40532       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40646       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40686       172.67.72.249:443       ESTABLISHED 109768/java
tcp6       0      0 172.25.75.3:40574       172.67.72.249:443       ESTABLISHED 109768/java
...................................................................................
...................................................................................

@sergiy-kozak sergiy-kozak changed the title Unexpected javax.net.ssl.SSLHandshakeException in seemingly simply usage scenario with Jersey Client 2 (>= 2.41) Unexpected javax.net.ssl.SSLHandshakeException in seemingly simple usage scenario with Jersey Client 2 (>= 2.41) May 17, 2024
@jansupol
Copy link
Contributor

This sounds like a problem with the certificate, or the SSL version. The JDK 21 release notes says:

security-libs/javax.net.ssl
The Default TLS Diffie-Hellman Group Size Has Been Increased from 1024-bit to 2048-bit (JDK-8301700)
The JDK implementation of TLS 1.2 now uses a default Diffie Hellman keysize of 2048 bits when a TLS_DHE cipher suite is negotiated and either the client or server does not support FFDHE, which can negotiate a stronger keysize. The JDK TLS implementation supports FFDHE and it is enabled by default.

As a workaround, users can revert to the previous size by setting the jdk.tls.ephemeralDHKeySize system property to 1024 (at their own risk).

This change does not affect TLS 1.3 as the minimum DH group size is already 2048 bits.

@jansupol
Copy link
Contributor

The issue happens during the SSL connection handshake, there could be some difference between the server and client settings.

You can use -Djavax.net.debug=all -Dssl.debug=true Java options to debug.

@sergiy-kozak
Copy link
Author

I have the impression that the error I get from running the program is just part of a bigger issue. I'm more worried what I observe about open connections by the process. Also note, clearly this has little to do with JDK itself, issue starts manifesting when changing Jersey libraries from 2.40 to >=2.41.

@sergiy-kozak
Copy link
Author

For Jersey 2.43 when I enabled -Djavax.net.debug=all to test, I can see that for each of the POST requests from the program's loop new connection is getting open and SSL protocol negotiation occurs, which explains why execution takes significantly more time. Initially open connections get "abandoned" and would stale after a while, producing this kind of output:

javax.net.ssl|WARNING|02|Keep-Alive-Timer|2024-05-23 16:51:29.122 CEST|SSLSocketImpl.java:1220|input stream close depletion failed (
"throwable" : {
  java.net.SocketTimeoutException: Read timed out
  	at java.base/sun.nio.ch.NioSocketImpl.timedRead(NioSocketImpl.java:278)
  	at java.base/sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:304)
  	at java.base/sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:346)
  	at java.base/sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:796)
  	at java.base/java.net.Socket$SocketInputStream.read(Socket.java:1099)
  	at java.base/java.net.Socket$SocketInputStream.read(Socket.java:1093)
  	at java.base/sun.security.ssl.SSLSocketInputRecord.deplete(SSLSocketInputRecord.java:509)
  	at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.readLockedDeplete(SSLSocketImpl.java:1216)
  	at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.deplete(SSLSocketImpl.java:1191)
  	at java.base/sun.security.ssl.SSLSocketImpl.bruteForceCloseInput(SSLSocketImpl.java:808)
  	at java.base/sun.security.ssl.SSLSocketImpl.duplexCloseOutput(SSLSocketImpl.java:664)
  	at java.base/sun.security.ssl.SSLSocketImpl.close(SSLSocketImpl.java:584)
  	at java.base/sun.net.www.http.HttpClient.closeServer(HttpClient.java:1139)
  	at java.base/sun.net.www.protocol.https.HttpsClient.closeServer(HttpsClient.java:442)
  	at java.base/sun.net.www.http.KeepAliveCache.run(KeepAliveCache.java:282)
  	at java.base/java.lang.Thread.run(Thread.java:1583)
  	at java.base/jdk.internal.misc.InnocuousThread.run(InnocuousThread.java:186)}

)
javax.net.ssl|DEBUG|02|Keep-Alive-Timer|2024-05-23 16:51:29.122 CEST|SSLSocketImpl.java:1775|close the SSL connection (passive)
javax.net.ssl|DEBUG|02|Keep-Alive-Timer|2024-05-23 16:51:29.122 CEST|SSLSocketImpl.java:577|duplex close of SSLSocket

With Jersey 2.40 this doesn't happen. For program one connection is open and reused for consequent requests execution, protocol negotiation occurs once before first request as expected. Is there anything changed in connections management with HttpUrlConnection after 2.40? Why would every time to make a request the new connection is needed, is that anticipated by the implementation for e.g. pooling?

@sergiy-kozak
Copy link
Author

There was quite some changes around SSL configuration and HttpUrlConnector happening in this commit: 73bdf5f
Not sure how to validate if that changeset introduced a problem with connections and open sockets that I observe.

@johannesherr
Copy link

With Jersey 2.40 this doesn't happen. For program one connection is open and reused for consequent requests execution, protocol negotiation occurs once before first request as expected. Is there anything changed in connections management with HttpUrlConnection after 2.40? Why would every time to make a request the new connection is needed, is that anticipated by the implementation for e.g. pooling?

KeepAlive caching of connections seems to be broken from 2.41 on. This would explain what you observe. See this ticket for details:

#5671

@sergiy-kozak
Copy link
Author

With Jersey 2.40 this doesn't happen. For program one connection is open and reused for consequent requests execution, protocol negotiation occurs once before first request as expected. Is there anything changed in connections management with HttpUrlConnection after 2.40? Why would every time to make a request the new connection is needed, is that anticipated by the implementation for e.g. pooling?

KeepAlive caching of connections seems to be broken from 2.41 on. This would explain what you observe. See this ticket for details:

#5671

Yes, that explains, thanks for linking it here. Currently forced to use old version, as Jersey client as of 2.41 is effectively broken.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants