Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[QUESTION]:ilogtail输出到kafka,默认配置情况下出现消息过大被kafka拒绝 #1483

Open
hailangxing opened this issue May 14, 2024 · 1 comment
Labels
question Further information is requested

Comments

@hailangxing
Copy link

image

image

使用默认的MaxMessageBytes配置,会出现kafka Message was to large,service rejected it to avoid allocation error,从MaxMessageBytes上看是0.96m,没有超过kafka默认消息大小限制1m

@hailangxing hailangxing added the question Further information is requested label May 14, 2024
@shalousun
Copy link
Collaborator

@hailangxing 这种kafka的客户端都是根据服务端状态返回状态的错误了,直接把服务端和客户端的都提升就OK了。本质上一条日志采集上来,发送到kafka最终的message会附带一些信息会比原来的日志大。这个也不属于是iLogtail的问题,发送消息都会碰到。设置留足空余,极限值很容易被拒绝。通常建议是broker端的配置要比producer端高。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants