Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exception if Connection between beat and Logstash is closed #16112

Open
jonasLH4 opened this issue Apr 26, 2024 · 0 comments
Open

Exception if Connection between beat and Logstash is closed #16112

jonasLH4 opened this issue Apr 26, 2024 · 0 comments

Comments

@jonasLH4
Copy link

Logstash information:

Please include the following information:

  1. Logstash version (e.g. bin/logstash --version)
    logstash 8.13.2
  2. Logstash installation source (e.g. built from source, with a package manager: DEB/RPM, expanded from tar or zip archive, docker)
    installed via deb package
  3. How is Logstash being run (e.g. as a service/service manager: systemd, upstart, etc. Via command line, docker/kubernetes)
    systemd
    Plugins installed: (bin/logstash-plugin list --verbose)

logstash-codec-avro (3.4.1)
logstash-codec-cef (6.2.7)
logstash-codec-collectd (3.1.0)
logstash-codec-dots (3.0.6)
logstash-codec-edn (3.1.0)
logstash-codec-edn_lines (3.1.0)
logstash-codec-es_bulk (3.1.0)
logstash-codec-fluent (3.4.2)
logstash-codec-graphite (3.0.6)
logstash-codec-json (3.1.1)
logstash-codec-json_lines (3.1.0)
logstash-codec-line (3.1.1)
logstash-codec-msgpack (3.1.0)
logstash-codec-multiline (3.1.1)
logstash-codec-netflow (4.3.2)
logstash-codec-plain (3.1.0)
logstash-codec-rubydebug (3.1.0)
logstash-filter-aggregate (2.10.0)
logstash-filter-anonymize (3.0.7)
logstash-filter-cidr (3.1.3)
logstash-filter-clone (4.2.0)
logstash-filter-csv (3.1.1)
logstash-filter-date (3.1.15)
logstash-filter-de_dot (1.0.4)
logstash-filter-dissect (1.2.5)
logstash-filter-dns (3.2.0)
logstash-filter-drop (3.0.5)
logstash-filter-elastic_integration (0.1.8)
logstash-filter-elasticsearch (3.16.1)
logstash-filter-fingerprint (3.4.4)
logstash-filter-geoip (7.2.13)
logstash-filter-grok (4.4.3)
logstash-filter-http (1.5.1)
logstash-filter-json (3.2.1)
logstash-filter-kv (4.7.0)
logstash-filter-memcached (1.2.0)
logstash-filter-metrics (4.0.7)
logstash-filter-mutate (3.5.8)
logstash-filter-prune (3.0.4)
logstash-filter-ruby (3.1.8)
logstash-filter-sleep (3.0.7)
logstash-filter-split (3.1.8)
logstash-filter-syslog_pri (3.2.1)
logstash-filter-throttle (4.0.4)
logstash-filter-translate (3.4.2)
logstash-filter-truncate (1.0.6)
logstash-filter-urldecode (3.0.6)
logstash-filter-useragent (3.3.5)
logstash-filter-uuid (3.0.5)
logstash-filter-xml (4.2.0)
logstash-input-azure_event_hubs (1.4.5)
logstash-input-beats (6.8.2)
└── logstash-input-elastic_agent (alias)
logstash-input-couchdb_changes (3.1.6)
logstash-input-dead_letter_queue (2.0.0)
logstash-input-elastic_serverless_forwarder (0.1.4)
logstash-input-elasticsearch (4.20.2)
logstash-input-exec (3.6.0)
logstash-input-file (4.4.6)
logstash-input-ganglia (3.1.4)
logstash-input-gelf (3.3.2)
logstash-input-generator (3.1.0)
logstash-input-graphite (3.0.6)
logstash-input-heartbeat (3.1.1)
logstash-input-http (3.8.0)
logstash-input-http_poller (5.5.1)
logstash-input-jms (3.2.2)
logstash-input-pipe (3.1.0)
logstash-input-redis (3.7.0)
logstash-input-snmp (1.3.3)
logstash-input-snmptrap (3.1.0)
logstash-input-stdin (3.4.0)
logstash-input-syslog (3.7.0)
logstash-input-tcp (6.4.1)
logstash-input-twitter (4.1.1)
logstash-input-udp (3.5.0)
logstash-input-unix (3.1.2)
logstash-integration-aws (7.1.6)
├── logstash-codec-cloudfront
├── logstash-codec-cloudtrail
├── logstash-input-cloudwatch
├── logstash-input-s3
├── logstash-input-sqs
├── logstash-output-cloudwatch
├── logstash-output-s3
├── logstash-output-sns
└── logstash-output-sqs
logstash-integration-elastic_enterprise_search (3.0.0)
├── logstash-output-elastic_app_search
└── logstash-output-elastic_workplace_search
logstash-integration-jdbc (5.4.9)
├── logstash-input-jdbc
├── logstash-filter-jdbc_streaming
└── logstash-filter-jdbc_static
logstash-integration-kafka (11.3.4)
├── logstash-input-kafka
└── logstash-output-kafka
logstash-integration-logstash (1.0.2)
├── logstash-input-logstash
└── logstash-output-logstash
logstash-integration-rabbitmq (7.3.3)
├── logstash-input-rabbitmq
└── logstash-output-rabbitmq
logstash-output-csv (3.0.10)
logstash-output-elasticsearch (11.22.5)
logstash-output-email (4.1.3)
logstash-output-file (4.3.0)
logstash-output-graphite (3.1.6)
logstash-output-http (5.6.0)
logstash-output-lumberjack (3.1.9)
logstash-output-nagios (3.0.6)
logstash-output-null (3.0.5)
logstash-output-pipe (3.0.6)
logstash-output-redis (5.0.0)
logstash-output-stdout (3.1.4)
logstash-output-tcp (6.2.0)
logstash-output-udp (3.2.0)
logstash-output-webhdfs (3.1.0)
logstash-patterns-core (4.3.4)

JVM (e.g. java -version):

If the affected version of Logstash is 7.9 (or earlier), or if it is NOT using the bundled JDK or using the 'no-jdk' version in 7.10 (or higher), please provide the following information:

  1. JVM version (java -version)

openjdk 17.0.10 2024-01-16
OpenJDK Runtime Environment Temurin-17.0.10+7 (build 17.0.10+7)
OpenJDK 64-Bit Server VM Temurin-17.0.10+7 (build 17.0.10+7, mixed mode, sharing)

  1. JVM installation source (e.g. from the Operating System's package manager, from source, etc).

bundeled JDK
5. Value of the LS_JAVA_HOME environment variable if set.

OS version (uname -a if on a Unix-like system):
Linux 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Description of the problem including expected versus actual behavior:
Logstash respectively Netty reports an exception if the connection from a beat agent has ended. In this case, the client computer was shutdown.
Steps to reproduce:
End the Connection between Beat and Logstash. Respectively shutdown the beat host machine

Please include a minimal but complete recreation of the problem,
including (e.g.) pipeline definition(s), settings, locale, etc. The easier
you make for us to reproduce it, the more likely that somebody will take the
time to look at it.

Provide logs (if relevant):
r 26 10:53:42 svr-elastic logstash[1048]: [2024-04-26T10:53:42,848][WARN ][io.netty.channel.DefaultChannelPipeline][beat][9a48a55c9bb3112c7f49d105a9848797a353afa217b136ae47bb338df27c510f] An exceptionCaught() event was fired, and it r>Apr 26 10:53:42 svr-elastic logstash[1048]: java.net.SocketException: Connection reset
Apr 26 10:53:42 svr-elastic logstash[1048]: at sun.nio.ch.SocketChannelImpl.throwConnectionReset(SocketChannelImpl.java:394) ~[?:?]
Apr 26 10:53:42 svr-elastic logstash[1048]: at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:426) ~[?:?]
Apr 26 10:53:42 svr-elastic logstash[1048]: at io.netty.buffer.PooledByteBuf.setBytes(PooledByteBuf.java:254) ~[netty-buffer-4.1.100.Final.jar:4.1.100.Final]
Apr 26 10:53:42 svr-elastic logstash[1048]: at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1132) ~[netty-buffer-4.1.100.Final.jar:4.1.100.Final]
Apr 26 10:53:42 svr-elastic logstash[1048]: at io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:357) ~[netty-transport-4.1.100.Final.jar:4.1.100.Final]
Apr 26 10:53:42 svr-elastic logstash[1048]: at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:151) ~[netty-transport-4.1.100.Final.jar:4.1.100.Final]
Apr 26 10:53:42 svr-elastic logstash[1048]: at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) ~[netty-transport-4.1.100.Final.jar:4.1.100.Final]
Apr 26 10:53:42 svr-elastic logstash[1048]: at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) ~[netty-transport-4.1.100.Final.jar:4.1.100.Final]
Apr 26 10:53:42 svr-elastic logstash[1048]: at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) ~[netty-transport-4.1.100.Final.jar:4.1.100.Final]
Apr 26 10:53:42 svr-elastic logstash[1048]: at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) [netty-transport-4.1.100.Final.jar:4.1.100.Final]
Apr 26 10:53:42 svr-elastic logstash[1048]: at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) [netty-common-4.1.100.Final.jar:4.1.100.Final]
Apr 26 10:53:42 svr-elastic logstash[1048]: at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.100.Final.jar:4.1.100.Final]
Apr 26 10:53:42 svr-elastic logstash[1048]: at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-common-4.1.100.Final.jar:4.1.100.Final]
Apr 26 10:53:42 svr-elastic logstash[1048]: at java.lang.Thread.run(Thread.java:840) [?:?]
Apr 26 10:53:42 svr-elastic logstash[1048]: [2024-04-26T10:53:42,848][WARN ][io.netty.channel.DefaultChannelPipeline][beat][9a48a55c9bb3112c7f49d105a9848797a353afa217b136ae47bb338df27c510f] An exceptionCaught() event was fired, and it r>Apr 26 10:53:42 svr-elastic logstash[1048]: java.net.SocketException: Connection reset
Apr 26 10:53:42 svr-elastic logstash[1048]: at sun.nio.ch.SocketChannelImpl.throwConnectionReset(SocketChannelImpl.java:394) ~[?:?]
Apr 26 10:53:42 svr-elastic logstash[1048]: at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:426) ~[?:?]
Apr 26 10:53:42 svr-elastic logstash[1048]: at io.netty.buffer.PooledByteBuf.setBytes(PooledByteBuf.java:254) ~[netty-buffer-4.1.100.Final.jar:4.1.100.Final]
Apr 26 10:53:42 svr-elastic logstash[1048]: at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1132) ~[netty-buffer-4.1.100.Final.jar:4.1.100.Final]
Apr 26 10:53:42 svr-elastic logstash[1048]: at io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:357) ~[netty-transport-4.1.100.Final.jar:4.1.100.Final]
Apr 26 10:53:42 svr-elastic logstash[1048]: at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:151) ~[netty-transport-4.1.100.Final.jar:4.1.100.Final]
Apr 26 10:53:42 svr-elastic logstash[1048]: at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) ~[netty-transport-4.1.100.Final.jar:4.1.100.Final]
Apr 26 10:53:42 svr-elastic logstash[1048]: at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) ~[netty-transport-4.1.100.Final.jar:4.1.100.Final]
Apr 26 10:53:42 svr-elastic logstash[1048]: at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) ~[netty-transport-4.1.100.Final.jar:4.1.100.Final]
Apr 26 10:53:42 svr-elastic logstash[1048]: at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) [netty-transport-4.1.100.Final.jar:4.1.100.Final]
Apr 26 10:53:42 svr-elastic logstash[1048]: at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) [netty-common-4.1.100.Final.jar:4.1.100.Final]
Apr 26 10:53:42 svr-elastic logstash[1048]: at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.100.Final.jar:4.1.100.Final]
Apr 26 10:53:42 svr-elastic logstash[1048]: at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-common-4.1.100.Final.jar:4.1.100.Final]
Apr 26 10:53:42 svr-elastic logstash[1048]: at java.lang.Thread.run(Thread.java:840) [?:?]
Apr 26 10:53:43 svr-elastic logstash[1048]: [2024-04-26T10:53:43,717][INFO ][org.logstash.beats.BeatsHandler][beat][9a48a55c9bb3112c7f49d105a9848797a353afa217b136ae47bb338df27c510f] [local: 172.22.10.169:5044, remote: 172.22.70.23:60067>Apr 26 10:53:43 svr-elastic logstash[1048]: [2024-04-26T10:53:43,717][WARN ][io.netty.channel.DefaultChannelPipeline][beat][9a48a55c9bb3112c7f49d105a9848797a353afa217b136ae47bb338df27c510f] An exceptionCaught() event was fired, and it r>Apr 26 10:53:43 svr-elastic logstash[1048]: java.net.SocketException: Connection reset
Apr 26 10:53:43 svr-elastic logstash[1048]: at sun.nio.ch.SocketChannelImpl.throwConnectionReset(SocketChannelImpl.java:394) ~[?:?]
Apr 26 10:53:43 svr-elastic logstash[1048]: at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:426) ~[?:?]
Apr 26 10:53:43 svr-elastic logstash[1048]: at io.netty.buffer.PooledByteBuf.setBytes(PooledByteBuf.java:254) ~[netty-buffer-4.1.100.Final.jar:4.1.100.Final]
Apr 26 10:53:43 svr-elastic logstash[1048]: at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1132) ~[netty-buffer-4.1.100.Final.jar:4.1.100.Final]
Apr 26 10:53:43 svr-elastic logstash[1048]: at io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:357) ~[netty-transport-4.1.100.Final.jar:4.1.100.Final]
Apr 26 10:53:43 svr-elastic logstash[1048]: at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:151) ~[netty-transport-4.1.100.Final.jar:4.1.100.Final]
Apr 26 10:53:43 svr-elastic logstash[1048]: at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) ~[netty-transport-4.1.100.Final.jar:4.1.100.Final]
Apr 26 10:53:43 svr-elastic logstash[1048]: at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) ~[netty-transport-4.1.100.Final.jar:4.1.100.Final]
Apr 26 10:53:43 svr-elastic logstash[1048]: at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) ~[netty-transport-4.1.100.Final.jar:4.1.100.Final]
Apr 26 10:53:43 svr-elastic logstash[1048]: at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) [netty-transport-4.1.100.Final.jar:4.1.100.Final]
Apr 26 10:53:43 svr-elastic logstash[1048]: at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) [netty-common-4.1.100.Final.jar:4.1.100.Final]
Apr 26 10:53:43 svr-elastic logstash[1048]: at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.100.Final.jar:4.1.100.Final]
Apr 26 10:53:43 svr-elastic logstash[1048]: at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-common-4.1.100.Final.jar:4.1.100.Final]
Apr 26 10:53:43 svr-elastic logstash[1048]: at java.lang.Thread.run(Thread.java:840) [?:?]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant