Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Spring WebFlux "ResourceLeakDetector:LEAK" #25213

Closed
xuep2006 opened this issue Jun 9, 2020 · 10 comments
Closed

Spring WebFlux "ResourceLeakDetector:LEAK" #25213

xuep2006 opened this issue Jun 9, 2020 · 10 comments

Comments

@xuep2006
Copy link

xuep2006 commented Jun 9, 2020

Spring Boot Version

spring-boot-starter-webflux:2.1.14.RELEASE
spring-boot-starter-reactor-netty:2.1.14.RELEASE
reactor-netty:0.8.18.RELEASE
jdk 1.8.0_162

Problem

Hi,
I am using jmeter to test the performance of a microservice, the test case is:

  • Running the application with -Dio.netty.leakDetectionLevel=paranoid .
  • Jmeter uses 50 threads to start performance testing.
  • Force stop jmeter in the testing.
  • Start the performance testing again.

An exception occurs: io.netty.util.ResourceLeakDetector:LEAK: ByteBuf.release() was not called before it's garbage-collected.
When I remove the subscribeOn in the code, no exception occurs.

Code

  • MyApplication
@SpringBootApplication
@EnableWebFlux
@Configuration
public class MyApplication {
    public static void main(String[] args) {
        SpringApplication app = new SpringApplication(MyApplication.class);
        app.run(args);
    }

    @Bean
    public Scheduler requestScheduler() {
        ExecutorService executorService =
            new ThreadPoolExecutor(1024, 1024, 60, TimeUnit.SECONDS, new SynchronousQueue<Runnable>());
        return Schedulers.fromExecutorService(executorService);
    }
}
  • MyController
@RestController
public class MyController {

    @Autowired
    @Qualifier("requestScheduler")
    private Scheduler requestScheduler;

    @RequestMapping(value = "/test")
    public Mono<ResponseEntity<String>> process(@RequestBody String reqMsg, @RequestHeader HttpHeaders reqHttpHeaders,
        ServerHttpRequest serverHttpRequest) {
        return Mono.just(reqMsg).subscribeOn(requestScheduler).onErrorResume(e -> {
            return Mono.just(e.getMessage());
        }).flatMap(responseMsg -> {
            return Mono.just(new ResponseEntity<>(responseMsg, HttpStatus.OK));
        });
    }
}

Exception stack

2020-06-09T15:57:00.976+0800|ERROR|pool-2-thread-1021|320|io.netty.util.ResourceLeakDetector:LEAK: ByteBuf.release() was not called before it's garbage-collected. See https://netty.io/wiki/reference-counted-objects.html for more information.
Recent access records: 
Created at:
	io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:363)
	io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:187)
	io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:178)
	io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:115)
	org.springframework.core.io.buffer.NettyDataBufferFactory.allocateBuffer(NettyDataBufferFactory.java:71)
	org.springframework.core.io.buffer.NettyDataBufferFactory.allocateBuffer(NettyDataBufferFactory.java:39)
	org.springframework.core.codec.CharSequenceEncoder.lambda$encode$1(CharSequenceEncoder.java:85)
	reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:107)
	reactor.core.publisher.FluxJust$WeakScalarSubscription.request(FluxJust.java:99)
	reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:162)
	reactor.core.publisher.MonoSingle$SingleSubscriber.request(MonoSingle.java:94)
	reactor.core.publisher.Operators$MultiSubscriptionSubscriber.set(Operators.java:1993)
	reactor.core.publisher.Operators$MultiSubscriptionSubscriber.onSubscribe(Operators.java:1867)
	reactor.core.publisher.MonoSingle$SingleSubscriber.onSubscribe(MonoSingle.java:114)
	reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:90)
	reactor.core.publisher.FluxJust.subscribe(FluxJust.java:70)
	reactor.core.publisher.FluxMapFuseable.subscribe(FluxMapFuseable.java:63)
	reactor.core.publisher.MonoSingle.subscribe(MonoSingle.java:58)
	reactor.core.publisher.MonoSwitchIfEmpty.subscribe(MonoSwitchIfEmpty.java:44)
	reactor.core.publisher.MonoFlatMap.subscribe(MonoFlatMap.java:60)
	reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:150)
	reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1623)
	reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:144)
	reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onNext(FluxOnErrorResume.java:73)
	reactor.core.publisher.FluxSubscribeOnValue$ScheduledScalar.run(FluxSubscribeOnValue.java:178)
	reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:50)
	reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:27)
	java.util.concurrent.FutureTask.run(FutureTask.java:266)
	java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	java.lang.Thread.run(Thread.java:748)
@spring-projects-issues spring-projects-issues added the status: waiting-for-triage An issue we've not yet triaged or decided on label Jun 9, 2020
@Pratikjoy
Copy link

Pratikjoy commented Jun 9, 2020 •
edited

Hello @xuep2006 ,
Can you again run the code with "-Dio.netty.leakDetectionLevel=PARANOID" ?
and let me know if the error still occurs..

@xuep2006
Copy link
Author

Hello @Pratikjoy ,
Thank you for your prompt reply. I use -Dio.netty.leakDetectionLevel=PARANOID to run the code and the error still occurs.

JVM Command line

-Dio.netty.leakDetectionLevel=PARANOID -Dfile.encoding=UTF-8

Log

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::       (v2.1.14.RELEASE)

16:38:04.506|INFO|main|50|o.e.webflux.netty.test.MyApplication:Starting MyApplication on DESKTOP-L2S27F0 with PID 48284 
16:38:04.510|INFO|main|652|o.e.webflux.netty.test.MyApplication:The following profiles are active: dev
16:38:07.655|INFO|main|82|o.s.b.w.e.netty.NettyWebServer:Netty started on port(s): 8090
16:38:07.658|INFO|main|59|o.e.webflux.netty.test.MyApplication:Started MyApplication in 3.709 seconds (JVM running for 4.355)
16:38:45.879|ERROR|reactor-http-nio-1|289|o.s.w.s.a.HttpWebHandlerAdapter:[0ce04e8a] Error [reactor.netty.ReactorNetty$InternalNettyException: java.nio.channels.ClosedChannelException] for HTTP POST "/test", but ServerHttpResponse already committed (200 OK)
16:38:45.882|ERROR|reactor-http-nio-1|289|o.s.w.s.a.HttpWebHandlerAdapter:[1e2ce5d0] Error [reactor.netty.ReactorNetty$InternalNettyException: java.nio.channels.ClosedChannelException] for HTTP POST "/test", but ServerHttpResponse already committed (200 OK)
16:38:50.299|ERROR|pool-3-thread-988|320|io.netty.util.ResourceLeakDetector:LEAK: ByteBuf.release() was not called before it's garbage-collected. See https://netty.io/wiki/reference-counted-objects.html for more information.
Recent access records: 
Created at:
	io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:363)
	io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:187)
	io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:178)
	io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:115)
	org.springframework.core.io.buffer.NettyDataBufferFactory.allocateBuffer(NettyDataBufferFactory.java:71)
	org.springframework.core.io.buffer.NettyDataBufferFactory.allocateBuffer(NettyDataBufferFactory.java:39)
	org.springframework.core.codec.CharSequenceEncoder.lambda$encode$1(CharSequenceEncoder.java:85)
	reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:107)
	reactor.core.publisher.FluxJust$WeakScalarSubscription.request(FluxJust.java:99)
	reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:162)
	reactor.core.publisher.MonoSingle$SingleSubscriber.request(MonoSingle.java:94)
	reactor.core.publisher.Operators$MultiSubscriptionSubscriber.set(Operators.java:1993)
	reactor.core.publisher.Operators$MultiSubscriptionSubscriber.onSubscribe(Operators.java:1867)
	reactor.core.publisher.MonoSingle$SingleSubscriber.onSubscribe(MonoSingle.java:114)
	reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:90)
	reactor.core.publisher.FluxJust.subscribe(FluxJust.java:70)
	reactor.core.publisher.FluxMapFuseable.subscribe(FluxMapFuseable.java:63)
	reactor.core.publisher.MonoSingle.subscribe(MonoSingle.java:58)
	reactor.core.publisher.MonoSwitchIfEmpty.subscribe(MonoSwitchIfEmpty.java:44)
	reactor.core.publisher.MonoFlatMap.subscribe(MonoFlatMap.java:60)
	reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:150)
	reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1623)
	reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:144)
	reactor.core.publisher.MonoPublishOn$PublishOnSubscriber.run(MonoPublishOn.java:178)
	reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:50)
	reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:27)
	java.util.concurrent.FutureTask.run(FutureTask.java:266)
	java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
	java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
	java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	java.lang.Thread.run(Thread.java:748)
2020-06-10T16:38:51.650+0800|ERROR|reactor-http-nio-7|289|o.s.w.s.a.HttpWebHandlerAdapter:[de66a20c] Error [reactor.netty.ReactorNetty$InternalNettyException: java.nio.channels.ClosedChannelException] for HTTP POST "/test", but ServerHttpResponse already committed (200 OK)

@DarrMirr
Copy link

@xuep2006 Hi!

Issue easily reproduced at spring-boot-starter-webflux:2.1.14.RELEASE. But I could not reproduce it at spring-boot-starter-webflux:2.4.2. Error message "LEAK: ByteBuf.release() was not called before it's garbage-collected." is not appeared at log output.

Are you able to update webflux version from 2.1.14.RELEASE to 2.4.2 and try to reproduce issue again?

@rstoyanchev
Copy link
Contributor

Yes this might have been addressed with recent fixes related to #26232.

@rstoyanchev rstoyanchev added the status: waiting-for-feedback We need additional information before we can continue label Feb 2, 2021
@spring-projects-issues
Copy link
Collaborator

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

@spring-projects-issues spring-projects-issues added the status: feedback-reminder We've sent a reminder that we need additional information before we can continue label Feb 9, 2021
@spring-projects-issues
Copy link
Collaborator

Closing due to lack of requested feedback. If you would like us to look at this issue, please provide the requested information and we will re-open the issue.

@spring-projects-issues spring-projects-issues removed status: waiting-for-feedback We need additional information before we can continue status: feedback-reminder We've sent a reminder that we need additional information before we can continue status: waiting-for-triage An issue we've not yet triaged or decided on labels Feb 16, 2021
@rchalicham
Copy link

rchalicham commented May 12, 2021 •
edited

Hi get same error I changed webflux to 2.45
1 actionable task: 1 executed
+--- org.springframework.boot:spring-boot-starter-webflux:2.4.2
| --- org.springframework:spring-webflux:5.3.3 -> 5.1.10.RELEASE
+--- org.springframework.boot:spring-boot-starter-webflux:2.4.2 (n)
+--- org.springframework.boot:spring-boot-starter-webflux:2.4.2
| --- org.springframework:spring-webflux:5.3.3 -> 5.1.10.RELEASE
+--- org.springframework.boot:spring-boot-starter-webflux:2.4.2
| --- org.springframework:spring-webflux:5.3.3 -> 5.1.10.RELEASE
+--- org.springframework.boot:spring-boot-starter-webflux:2.4.2
| --- org.springframework:spring-webflux:5.3.3 -> 5.1.10.RELEASE

the error i get is :

[ntLoopGroup-2-2] io.netty.util.ResourceLeakDetector : LEAK: ByteBuf.release() was not called before it's garbage-collected. See https://netty.io/wiki/reference-counted-objects.html for more information.
Recent access records:
#1:
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:291)
io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1475)
io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1224)
io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1271)
io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:505)
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:444)
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:283)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1421)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:930)
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:697)
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:632)
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:549)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:511)
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:748)
#2:
io.netty.buffer.AdvancedLeakAwareByteBuf.forEachByte(AdvancedLeakAwareByteBuf.java:670)
io.netty.handler.codec.http.HttpObjectDecoder$HeaderParser.parse(HttpObjectDecoder.java:793)
io.netty.handler.codec.http.HttpObjectDecoder.readHeaders(HttpObjectDecoder.java:592)
io.netty.handler.codec.http.HttpObjectDecoder.decode(HttpObjectDecoder.java:218)
io.netty.handler.codec.http.HttpClientCodec$Decoder.decode(HttpClientCodec.java:202)
io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:505)
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:444)
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:283)
io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1475)
io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1224)
io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1271)
io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:505)
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:444)
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:283)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1421)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:930)
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:697)
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:632)
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:549)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:511)
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:748)
#3:
io.netty.buffer.AdvancedLeakAwareByteBuf.forEachByte(AdvancedLeakAwareByteBuf.java:670)
io.netty.handler.codec.http.HttpObjectDecoder$HeaderParser.parse(HttpObjectDecoder.java:793)
io.netty.handler.codec.http.HttpObjectDecoder.readHeaders(HttpObjectDecoder.java:572)
io.netty.handler.codec.http.HttpObjectDecoder.decode(HttpObjectDecoder.java:218)
io.netty.handler.codec.http.HttpClientCodec$Decoder.decode(HttpClientCodec.java:202)
io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:505)
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:444)
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:283)
io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1475)
io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1224)
io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1271)
io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:505)
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:444)
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:283)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1421)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:930)
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:697)
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:632)
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:549)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:511)
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:748)
#4:
io.netty.buffer.AdvancedLeakAwareByteBuf.forEachByte(AdvancedLeakAwareByteBuf.java:670)
io.netty.handler.codec.http.HttpObjectDecoder$HeaderParser.parse(HttpObjectDecoder.java:793)
io.netty.handler.codec.http.HttpObjectDecoder$LineParser.parse(HttpObjectDecoder.java:842)
io.netty.handler.codec.http.HttpObjectDecoder.decode(HttpObjectDecoder.java:199)
io.netty.handler.codec.http.HttpClientCodec$Decoder.decode(HttpClientCodec.java:202)
io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:505)
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:444)
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:283)
io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1475)
io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1224)
io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1271)
io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:505)
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:444)
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:283)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1421)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:930)
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:697)
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:632)
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:549)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:511)
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:748)
#5:
Hint: 'reactor.left.httpCodec' will handle the message from this point.
io.netty.channel.DefaultChannelPipeline.touch(DefaultChannelPipeline.java:116)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:357)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1475)
io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1224)
io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1271)
io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:505)
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:444)
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:283)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1421)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:930)
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:697)
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:632)
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:549)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:511)
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:748)
#6:
io.netty.buffer.AdvancedLeakAwareByteBuf.internalNioBuffer(AdvancedLeakAwareByteBuf.java:736)
io.netty.handler.ssl.SslHandler.toByteBuffer(SslHandler.java:1485)
io.netty.handler.ssl.SslHandler.access$300(SslHandler.java:165)
io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:283)
io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1329)
io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1224)
io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1271)
io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:505)
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:444)
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:283)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1421)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:930)
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:697)
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:632)
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:549)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:511)
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:748)
#7:
io.netty.buffer.AdvancedLeakAwareByteBuf.nioBufferCount(AdvancedLeakAwareByteBuf.java:706)
io.netty.handler.ssl.SslHandler.toByteBuffer(SslHandler.java:1485)
io.netty.handler.ssl.SslHandler.access$300(SslHandler.java:165)
io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:283)
io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1329)
io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1224)
io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1271)
io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:505)
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:444)
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:283)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1421)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:930)
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:697)
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:632)
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:549)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:511)
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:748)
Created at:
io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:349)
io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:187)
io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:178)
io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:115)
io.netty.handler.ssl.SslHandler.allocate(SslHandler.java:2138)
io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1324)
io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1224)
io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1271)
io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:505)
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:444)
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:283)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1421)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:930)
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:697)
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:632)
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:549)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:511)
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:748)
: 12 leak records were discarded because the leak record count is targeted to 4. Use system property io.netty.leakDetection.targetRecords to increase the limit.

'org.springframework.boot' version '2.1.9.RELEASE'
ext {
set('springCloudVersion', 'Greenwich.SR3')
set('springCloudServices', '2.1.4.RELEASE')
}

appreciate your help

@DarrMirr
Copy link

@rchalicham hello!

org.springframework:spring-webflux:5.3.3 -> 5.1.10.RELEASE

According to comment this issue fixed at 5.3.2. It looks like 5.1.10.RELEASE version is used instead of 5.3.3 one. I recommend to manage all spring modules versions only by 'org.springframework.boot' one. And plug any spring-boot starters without setting version.

I think error disappear if you set 'org.springframework.boot' version to 2.4.2 or higher.

@rstoyanchev rstoyanchev mentioned this issue May 17, 2021
Closed
@AtticusLv
Copy link

Faced with the same issue, I use Spring Boot with 2.2.10.RELEASE version.
What's the latest version of 2.2.x version that can be used to fix LEAK issue?
@DarrMirr @rstoyanchev
Thank you very much!

@DarrMirr
Copy link

DarrMirr commented Jan 8, 2022 •
edited

Faced with the same issue, I use Spring Boot with 2.2.10.RELEASE version. What's the latest version of 2.2.x version that can be used to fix LEAK issue? @DarrMirr @rstoyanchev Thank you very much!

@AtticusLv according to comment fix was backported to 5.2.x. on December 2020. Spring boot 2.2.13.RELEASE has released on January 2021. Therefore I think fix should be included to 2.2.13.RELEASE.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@rstoyanchev @xuep2006 @rchalicham @spring-projects-issues @DarrMirr @Pratikjoy @AtticusLv