Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

FlakyTest: MalformedDataAfterHttpMessageTest #2069

Open
bondolo opened this issue Feb 1, 2022 · 11 comments
Open

FlakyTest: MalformedDataAfterHttpMessageTest #2069

bondolo opened this issue Feb 1, 2022 · 11 comments
Labels
flaky tests Unit tests are flaky

Comments

@bondolo
Copy link
Contributor

bondolo commented Feb 1, 2022

https://github.com/apple/servicetalk/pull/2068/checks?check_run_id=5015006266

MalformedDataAfterHttpMessageTest.[2] false

io.servicetalk.transport.netty.internal.StacklessClosedChannelException
	at io.servicetalk.transport.netty.internal.NettyChannelPublisher.channelOnError(Unknown Source)
Caused by: io.servicetalk.http.netty.HttpObjectDecoder$StacklessDecoderException: Invalid preface character before the start-line of the HTTP message
Caused by: io.servicetalk.utils.internal.IllegalCharacterException: '?' (0x00), expected [CR (0x0d), LF (0x0a)]
	at io.servicetalk.http.netty.HttpObjectDecoder.lambda$static$0(HttpObjectDecoder.java:99)
	at io.netty.buffer.AbstractByteBuf.forEachByteAsc0(AbstractByteBuf.java:1301)
	at io.netty.buffer.AbstractByteBuf.forEachByte(AbstractByteBuf.java:1292)
	at io.servicetalk.http.netty.HttpObjectDecoder.skipControlCharacters(HttpObjectDecoder.java:588)
	at io.servicetalk.http.netty.HttpObjectDecoder.decode(HttpObjectDecoder.java:239)
	at io.servicetalk.transport.netty.internal.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:365)
	at io.servicetalk.transport.netty.internal.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:321)
	at io.servicetalk.transport.netty.internal.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:185)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
	at io.servicetalk.transport.netty.internal.CopyByteBufHandlerChannelInitializer$CopyByteBufHandler.channelRead(CopyByteBufHandlerChannelInitializer.java:86)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
	at io.servicetalk.transport.netty.internal.ServiceTalkWireLogger.channelRead(ServiceTalkWireLogger.java:154)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
	at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795)
	at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480)
	at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378)
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.lang.Thread.run(Thread.java:750)
@bondolo bondolo added the flaky tests Unit tests are flaky label Feb 1, 2022
@bondolo
Copy link
Contributor Author

bondolo commented Feb 1, 2022 •
edited

https://github.com/apple/servicetalk/pull/2068/checks?check_run_id=5025494763
https://github.com/apple/servicetalk/runs/5025360620?check_suite_focus=true

MalformedDataAfterHttpMessageTest.[2] false
io.servicetalk.transport.netty.internal.RetryableClosedChannelException

io.servicetalk.transport.netty.internal.RetryableClosedChannelException
Caused by: io.servicetalk.transport.netty.internal.CloseHandler$CloseEventObservedException: CHANNEL_CLOSED_INBOUND(The transport backing this connection has been shutdown (read)) [id: 0xd9f232e5, L:/127.0.0.1:33694 ! R:127.0.0.1/127.0.0.1:41621]
Caused by: io.netty.channel.StacklessClosedChannelException
	at io.netty.channel.AbstractChannel$AbstractUnsafe.write(Object, ChannelPromise)(Unknown Source)
View more details on GitHub Actions

@bondolo
Copy link
Contributor Author

bondolo commented Feb 1, 2022

https://github.com/apple/servicetalk/runs/5029273590?check_suite_focus=true

MalformedDataAfterHttpMessageTest > afterResponseNextClientRequestSucceeds(boolean) > io.servicetalk.http.netty.MalformedDataAfterHttpMessageTest.afterResponseNextClientRequestSucceeds(boolean)[2] FAILED
    io.servicetalk.transport.netty.internal.StacklessClosedChannelException
        at io.servicetalk.transport.netty.internal.NettyChannelPublisher.channelOnError(Unknown Source)

        Caused by:
        io.servicetalk.http.netty.HttpObjectDecoder$StacklessDecoderException: Invalid preface character before the start-line of the HTTP message

            Caused by:
            io.servicetalk.utils.internal.IllegalCharacterException: '' (0x00), expected [CR (0x0d), LF (0x0a)]
                at io.servicetalk.http.netty.HttpObjectDecoder.lambda$static$0(HttpObjectDecoder.java:99)
                at io.netty.buffer.AbstractByteBuf.forEachByteAsc0(AbstractByteBuf.java:1301)
                at io.netty.buffer.AbstractByteBuf.forEachByte(AbstractByteBuf.java:1292)
                at io.servicetalk.http.netty.HttpObjectDecoder.skipControlCharacters(HttpObjectDecoder.java:588)
                at io.servicetalk.http.netty.HttpObjectDecoder.decode(HttpObjectDecoder.java:239)
                at io.servicetalk.transport.netty.internal.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:365)
                at io.servicetalk.transport.netty.internal.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:321)
                at io.servicetalk.transport.netty.internal.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:185)
                at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
                at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
                at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
                at io.servicetalk.transport.netty.internal.CopyByteBufHandlerChannelInitializer$CopyByteBufHandler.channelRead(CopyByteBufHandlerChannelInitializer.java:86)
                at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
                at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
                at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
                at io.servicetalk.transport.netty.internal.ServiceTalkWireLogger.channelRead(ServiceTalkWireLogger.java:154)
                at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
                at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
                at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
                at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
                at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
                at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
                at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
                at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795)
                at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480)
                at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378)
                at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
                at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
                at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
                at java.lang.Thread.run(Thread.java:750)

@bondolo
Copy link
Contributor Author

bondolo commented Feb 2, 2022

https://github.com/apple/servicetalk/pull/2068/checks?check_run_id=5029648741

MalformedDataAfterHttpMessageTest.[2] false
io.servicetalk.transport.netty.internal.StacklessClosedChannelException


io.servicetalk.transport.netty.internal.StacklessClosedChannelException
	at io.servicetalk.transport.netty.internal.NettyChannelPublisher.channelOnError(Unknown Source)
Caused by: io.servicetalk.http.netty.HttpObjectDecoder$StacklessDecoderException: Invalid preface character before the start-line of the HTTP message
Caused by: io.servicetalk.utils.internal.IllegalCharacterException: '?' (0x00), expected [CR (0x0d), LF (0x0a)]
	at io.servicetalk.http.netty.HttpObjectDecoder.lambda$static$0(HttpObjectDecoder.java:99)
	at io.netty.buffer.AbstractByteBuf.forEachByteAsc0(AbstractByteBuf.java:1301)
	at io.netty.buffer.AbstractByteBuf.forEachByte(AbstractByteBuf.java:1292)
	at io.servicetalk.http.netty.HttpObjectDecoder.skipControlCharacters(HttpObjectDecoder.java:588)
	at io.servicetalk.http.netty.HttpObjectDecoder.decode(HttpObjectDecoder.java:239)
	at io.servicetalk.transport.netty.internal.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:365)
	at io.servicetalk.transport.netty.internal.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:321)
	at io.servicetalk.transport.netty.internal.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:185)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
	at io.servicetalk.transport.netty.internal.CopyByteBufHandlerChannelInitializer$CopyByteBufHandler.channelRead(CopyByteBufHandlerChannelInitializer.java:86)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
	at io.servicetalk.transport.netty.internal.ServiceTalkWireLogger.channelRead(ServiceTalkWireLogger.java:154)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
	at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795)
	at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480)
	at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378)
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.lang.Thread.run(Thread.java:750)

@Scottmitch
Copy link
Member

https://github.com/apple/servicetalk/runs/5382461211

MalformedDataAfterHttpMessageTest.[1] true

io.servicetalk.transport.netty.internal.StacklessClosedChannelException
	at io.servicetalk.transport.netty.internal.NettyChannelPublisher.channelOnError(Unknown Source)
Caused by: io.servicetalk.http.netty.HttpObjectDecoder$StacklessDecoderException: Invalid preface character before the start-line of the HTTP message
Caused by: io.servicetalk.utils.internal.IllegalCharacterException: '?' (0x00), expected [CR (0x0d), LF (0x0a)]
	at app//io.servicetalk.http.netty.HttpObjectDecoder.lambda$static$0(HttpObjectDecoder.java:99)
	a

@Scottmitch Scottmitch mentioned this issue Mar 1, 2022
Merged
@Scottmitch
Copy link
Member

https://github.com/apple/servicetalk/runs/6620007764

MalformedDataAfterHttpMessageTest.[1] true

io.servicetalk.transport.netty.internal.StacklessClosedChannelException
	at io.servicetalk.transport.netty.internal.NettyChannelPublisher.channelOnError(Unknown Source)
Caused by: io.servicetalk.http.netty.HttpObjectDecoder$StacklessDecoderException: Invalid preface character before the start-line of the HTTP message
Caused by: io.servicetalk.utils.internal.IllegalCharacterException: '?' (0x00), expected [CR (0x0d), LF (0x0a)]
	at app//io.servicetalk.http.netty.HttpObjectDecoder.lambda$static$0(HttpObjectDecoder.java:99)
	at app//io.netty.buffer.AbstractByteBuf.forEachByteAsc0(AbstractByteBuf.java:1301)
	at app//io.netty.buffer.AbstractByteBuf.forEachByte(AbstractByteBuf.java:1292)
	at app//io.servicetalk.http.netty.HttpObjectDecoder.skipControlCharacters(HttpObjectDecoder.java:619)
	at app//io.servicetalk.http.netty.HttpObjectDecoder.decode(HttpObjectDecoder.java:239)
	at app//io.servicetalk.transport.netty.internal.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:365)
	at app//io.servicetalk.transport.netty.internal.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:321)
	at app//io.servicetalk.transport.netty.internal.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:185)

@bondolo
Copy link
Contributor Author

bondolo commented May 27, 2022

https://github.com/apple/servicetalk/pull/2227/checks?check_run_id=6619966624

MalformedDataAfterHttpMessageTest.[1] true
io.servicetalk.transport.netty.internal.StacklessClosedChannelException
	at io.servicetalk.transport.netty.internal.NettyChannelPublisher.channelOnError(Unknown Source)
Caused by: io.servicetalk.http.netty.HttpObjectDecoder$StacklessDecoderException: Invalid preface character before the start-line of the HTTP message
Caused by: io.servicetalk.utils.internal.IllegalCharacterException: '?' (0x00), expected [CR (0x0d), LF (0x0a)]
	at app//io.servicetalk.http.netty.HttpObjectDecoder.lambda$static$0(HttpObjectDecoder.java:99)
	at app//io.netty.buffer.AbstractByteBuf.forEachByteAsc0(AbstractByteBuf.java:1301)
	at app//io.netty.buffer.AbstractByteBuf.forEachByte(AbstractByteBuf.java:1292)
	at app//io.servicetalk.http.netty.HttpObjectDecoder.skipControlCharacters(HttpObjectDecoder.java:619)
	at app//io.servicetalk.http.netty.HttpObjectDecoder.decode(HttpObjectDecoder.java:239)
	at app//io.servicetalk.transport.netty.internal.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:365)
	at app//io.servicetalk.transport.netty.internal.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:321)
	at app//io.servicetalk.transport.netty.internal.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:185)
	at app//io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	at app//io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
	at app//io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
	at app//io.servicetalk.transport.netty.internal.CopyByteBufHandlerChannelInitializer$CopyByteBufHandler.channelRead(CopyByteBufHandlerChannelInitializer.java:86)
	at app//io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	at app//io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
	at app//io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
	at app//io.servicetalk.transport.netty.internal.ServiceTalkWireLogger.channelRead(ServiceTalkWireLogger.java:154)
	at app//io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	at app//io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
	at app//io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
	at app//io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286)
	at app//io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	at app//io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
	at app//io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
	at app//io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
	at app//io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	at app//io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
	at app//io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
	at app//io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:800)
	at app//io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:487)
	at app//io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:385)
	at app//io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:995)
	at app//io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at app//io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base@11.0.15/java.lang.Thread.run(Thread.java:829)

@bondolo bondolo mentioned this issue May 27, 2022
Merged
@idelpivnitskiy
Copy link
Member

https://github.com/apple/servicetalk/pull/2278/checks?check_run_id=7530726760

MalformedDataAfterHttpMessageTest.[1] true
io.servicetalk.transport.netty.internal.StacklessClosedChannelException
	at io.servicetalk.transport.netty.internal.NettyChannelPublisher.channelOnError(Unknown Source)
Caused by: io.servicetalk.http.netty.HttpObjectDecoder$StacklessDecoderException: Invalid preface character before the start-line of the HTTP message
Caused by: io.servicetalk.utils.internal.IllegalCharacterException: '?' (0x00), expected [CR (0x0d), LF (0x0a)]
	at io.servicetalk.http.netty.HttpObjectDecoder.lambda$static$0(HttpObjectDecoder.java:99)
	at io.netty.buffer.AbstractByteBuf.forEachByteAsc0(AbstractByteBuf.java:1301)
	at io.netty.buffer.AbstractByteBuf.forEachByte(AbstractByteBuf.java:1292)
	at io.servicetalk.http.netty.HttpObjectDecoder.skipControlCharacters(HttpObjectDecoder.java:629)
	at io.servicetalk.http.netty.HttpObjectDecoder.decode(HttpObjectDecoder.java:239)
	at io.servicetalk.transport.netty.internal.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:365)
	at io.servicetalk.transport.netty.internal.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:321)
	at io.servicetalk.transport.netty.internal.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:185)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
	at io.servicetalk.transport.netty.internal.CopyByteBufHandlerChannelInitializer$CopyByteBufHandler.channelRead(CopyByteBufHandlerChannelInitializer.java:86)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
	at io.servicetalk.transport.netty.internal.ServiceTalkWireLogger.channelRead(ServiceTalkWireLogger.java:154)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
	at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
	at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:800)
	at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:487)
	at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:385)
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.lang.Thread.run(Thread.java:750)

@Scottmitch
Copy link
Member

https://github.com/apple/servicetalk/pull/2372/checks?check_run_id=8528212615

MalformedDataAfterHttpMessageTest.[1] true

io.servicetalk.transport.netty.internal.StacklessClosedChannelException
	at io.servicetalk.transport.netty.internal.NettyChannelPublisher.channelOnError(Unknown Source)
Caused by: io.servicetalk.http.netty.HttpObjectDecoder$StacklessDecoderException: Invalid preface character before the start-line of the HTTP message
Caused by: io.servicetalk.utils.internal.IllegalCharacterException: '?' (0x00), expected [CR (0x0d), LF (0x0a)]
	at app//io.servicetalk.http.netty.HttpObjectDecoder.lambda$static$0(HttpObjectDecoder.java:99)
	at app//io.netty.buffer.AbstractByteBuf.forEachByteAsc0(AbstractByteBuf.java:1301)
	at app//io.netty.buffer.AbstractByteBuf.forEachByte(AbstractByteBuf.java:1292)
	at app//io.servicetalk.http.netty.HttpObjectDecoder.skipControlCharacters(HttpObjectDecoder.java:629)
	at app//io.servicetalk.http.netty.HttpObjectDecoder.decode(HttpObjectDecoder.java:239)
	at app//io.servicetalk.transport.netty.internal.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:365)
	at app//io.servicetalk.transport.netty.internal.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:321)
	at app//io.servicetalk.transport.netty.internal.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:185)
	at app//io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	at app//io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
	at app//io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
	at app//io.servicetalk.transport.netty.internal.CopyByteBufHandlerChannelInitializer$CopyByteBufHandler.channelRead(CopyByteBufHandlerChannelInitializer.java:86)
	at app//io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	at app//io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
	at app//io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
	at app//io.servicetalk.transport.netty.internal.ServiceTalkWireLogger.channelRead(ServiceTalkWireLogger.java:154)
	at app//io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	at app//io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
	at app//io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
	at app//io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
	at app//io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	at app//io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
	at app//io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
	at app//io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:800)
	at app//io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499)
	at app//io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397)
	at app//io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
	at app//io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at app//io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)

@Scottmitch Scottmitch mentioned this issue Sep 24, 2022
Merged
@Scottmitch
Copy link
Member

Scottmitch commented Dec 3, 2022 •
edited

https://github.com/apple/servicetalk/pull/2450/checks?check_run_id=9858982000

MalformedDataAfterHttpMessageTest.[1] true

io.servicetalk.transport.netty.internal.StacklessClosedChannelException
	at io.servicetalk.transport.netty.internal.NettyChannelPublisher.channelOnError(Unknown Source)
	Suppressed: io.servicetalk.concurrent.api.internal.BlockingUtils$SuppressedExecutionException: Blocking operation terminated with an error
		at app//io.servicetalk.concurrent.api.SourceToFuture.reportGet(SourceToFuture.java:121)
		at app//io.servicetalk.concurrent.api.SourceToFuture.get(SourceToFuture.java:92)
		at app//io.servicetalk.concurrent.api.internal.BlockingUtils.blockingInvocation(BlockingUtils.java:70)
		at app//io.servicetalk.http.api.BlockingRequestUtils.request(BlockingRequestUtils.java:29)
		at app//io.servicetalk.http.api.StreamingHttpClientToBlockingHttpClient.request(StreamingHttpClientToBlockingHttpClient.java:61)
		at app//io.servicetalk.http.netty.MalformedDataAfterHttpMessageTest.afterResponseNextClientRequestSucceeds(MalformedDataAfterHttpMessageTest.java:146)
		

Caused by: io.servicetalk.http.netty.HttpObjectDecoder$StacklessDecoderException: Invalid preface character before the start-line of the HTTP message
Caused by: io.servicetalk.utils.internal.IllegalCharacterException: '?' (0x00), expected [CR (0x0d), LF (0x0a)]
	at app//io.servicetalk.http.netty.HttpObjectDecoder.lambda$static$0(HttpObjectDecoder.java:99)
	at app//io.netty.buffer.AbstractByteBuf.forEachByteAsc0(AbstractByteBuf.java:1301)
	at app//io.netty.buffer.AbstractByteBuf.forEachByte(AbstractByteBuf.java:1292)
	at app//io.servicetalk.http.netty.HttpObjectDecoder.skipControlCharacters(HttpObjectDecoder.java:629)
	at app//io.servicetalk.http.netty.HttpObjectDecoder.decode(HttpObjectDecoder.java:239)
	at app//io.servicetalk.transport.netty.internal.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:365)
	at app//io.servicetalk.transport.netty.internal.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:321)
	at app//io.servicetalk.transport.netty.internal.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:185)
	at app//io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
	at app//io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at app//io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
	at app//io.servicetalk.transport.netty.internal.CopyByteBufHandlerChannelInitializer$CopyByteBufHandler.channelRead(CopyByteBufHandlerChannelInitializer.java:86)
	at app//io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
	at app//io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at app//io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
	at app//io.servicetalk.transport.netty.internal.ServiceTalkWireLogger.channelRead(ServiceTalkWireLogger.java:154)
	at app//io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
	at app//io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at app//io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
	at app//io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
	at app//io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
	at app//io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at app//io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
	at app//io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:800)
	at app//io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499)
	at app//io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397)
	at app//io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
	at app//io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at app//io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)

@daschl
Copy link
Contributor

daschl commented Dec 12, 2022

#2460

MalformedDataAfterHttpMessageTest > afterResponseNextClientRequestSucceeds(boolean) > io.servicetalk.http.netty.MalformedDataAfterHttpMessageTest.afterResponseNextClientRequestSucceeds(boolean)[1] FAILED
    io.servicetalk.transport.netty.internal.StacklessClosedChannelException
        at io.servicetalk.transport.netty.internal.NettyChannelPublisher.channelOnError(Unknown Source)
        Caused by:
        io.servicetalk.http.netty.HttpObjectDecoder$StacklessDecoderException: Invalid preface character before the start-line of the HTTP message
            Caused by:
            io.servicetalk.utils.internal.IllegalCharacterException: '' (0x00), expected [CR (0x0d), LF (0x0a)]
                at app//io.servicetalk.http.netty.HttpObjectDecoder.lambda$static$0(HttpObjectDecoder.java:99)
                at app//io.netty.buffer.AbstractByteBuf.forEachByteAsc0(AbstractByteBuf.java:1301)
                at app//io.netty.buffer.AbstractByteBuf.forEachByte(AbstractByteBuf.java:1292)
                at app//io.servicetalk.http.netty.HttpObjectDecoder.skipControlCharacters(HttpObjectDecoder.java:629)
                at app//io.servicetalk.http.netty.HttpObjectDecoder.decode(HttpObjectDecoder.java:239)
                at app//io.servicetalk.transport.netty.internal.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:365)
                at app//io.servicetalk.transport.netty.internal.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:321)
                at app//io.servicetalk.transport.netty.internal.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:185)
                at app//io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
                at app//io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
                at app//io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
                at app//io.servicetalk.transport.netty.internal.CopyByteBufHandlerChannelInitializer$CopyByteBufHandler.channelRead(CopyByteBufHandlerChannelInitializer.java:86)

@Scottmitch
Copy link
Member

https://github.com/apple/servicetalk/pull/2649/checks?check_run_id=14987811566

io.servicetalk.transport.netty.internal.StacklessClosedChannelException
	at io.servicetalk.transport.netty.internal.NettyChannelPublisher.channelOnError(Unknown Source)
Caused by: io.servicetalk.http.netty.HttpObjectDecoder$StacklessDecoderException: Invalid preface character before the start-line of the HTTP message
Caused by: io.servicetalk.utils.internal.IllegalCharacterException: '?' (0x00), expected [CR (0x0d), LF (0x0a)]
	at app//io.servicetalk.http.netty.HttpObjectDecoder.lambda$static$0(HttpObjectDecoder.java:99)
	at app//io.netty.buffer.AbstractByteBuf.forEachByteAsc0(AbstractByteBuf.java:1301)
	at app//io.netty.buffer.AbstractByteBuf.forEachByte(AbstractByteBuf.java:1292)
	at app//io.servicetalk.http.netty.HttpObjectDecoder.skipControlCharacters(HttpObjectDecoder.java:628)
	at app//io.servicetalk.http.netty.HttpObjectDecoder.decode(HttpObjectDecoder.java:239)
	at app//io.servicetalk.transport.netty.internal.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:365)
	at app//io.servicetalk.transport.netty.internal.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:321)
	at app//io.servicetalk.transport.netty.internal.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:185)
	at app//io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
	at app//io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at app//io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
	at app//io.servicetalk.transport.netty.internal.CopyByteBufHandlerChannelInitializer$CopyByteBufHandler.channelRead(CopyByteBufHandlerChannelInitializer.java:86)
	at app//io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
	at app//io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at app//io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
	at app//io.servicetalk.transport.netty.internal.ServiceTalkWireLogger.channelRead(ServiceTalkWireLogger.java:154)
	at app//io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
	at app//io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at app//io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
	at app//io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
	at app//io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
	at app//io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at app//io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
	at app//io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:800)
	at app//io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499)
	at app//io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397)
	at app//io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
	at app//io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at app//io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base@17.0.7/java.lang.Thread.run(Thread.java:833)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
flaky tests Unit tests are flaky
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@daschl @bondolo @idelpivnitskiy @Scottmitch