Java client 5.4.3 frame_max default value on server side

[OlehD]

After upgrade we started having such issue Content headers exceeded max frame size: 262560 > 131072.

Seems like in the new code we started negotiating for the value to be used basing on the frameMax values from client and server sides. On client side in ConnectionFactory, I found that we have a default value set to zero which means unlimited. However, on server we never set a value, and I couldn't find what is default value there.

Does anyone know if there is default value on the server side for this property?

[michaelklishin]

You are leaving out critically important information: what versions have you upgraded from.

The default value is right in the error message: 131072. It does not mean that you cannot send messages of a greater size,
as message payload is framed (split into multiple payload frames) by both sides.

We recommend not touching that value on any side or at least using 131072.

[OlehD]

if i understand the logic correctly then it will be impossible to set unlimited size unless it is set explicitly on both sides client and server.

[michaelklishin]

Why do you need it to be unlimited in the first place? You can publish messages up to 128 MB in size in modern RabbitMQ versions,
and even larger (up to 512 MiB?) with some configuration. No one should do that, however, put those large blobs into a blob store and pass their IDs around.

Every message larger than 128 kiB will be framed by the client. So, a basic.publish frame will be followed by a content (metadata) frame and one or more body frames.

[OlehD]

even if we set client with 128MB the still we need to set server to 0, otherwise with current logic it will pick up min value.

return (clientValue == 0 || serverValue == 0) ?
            Math.max(clientValue, serverValue) :
            Math.min(clientValue, serverValue);

btw, when I set frame max to zero on server side then all worked. The current logic basically limits the client to the value on the server. So, in any case we need to set it on server side.

Yes, agree that this question is more for java client side. Thanks for moving it!

[michaelklishin]

@OlehD we are running in circles. You don't need to override this value. At all. Even if your messages are larger than 128 kiB in size. It's an optional setting, just un set it everywhere and let RabbitMQ and client use the default.

[michaelklishin]

A client cannot be allowed to tell the server what value to use. Our team has learnt this the hard way. Issues that come down to client abuse (deliberate or just sloppy coding or unfortunate configuration values) are not uncommon at all.

[michaelklishin]

Transferred this discussion to the Java client repo as it seems more appropriate here at this point.

[michaelklishin]

@OlehD we are running in circles. You don't need to override this value. At all. Even if your messages are larger than 128 kiB in size. It's an optional setting, just unset it everywhere and let RabbitMQ and client use the default.

If your messages are into 10s of megabytes, and you transfer them using RabbitMQ, then it will technically work but you are doing it wrong. Every part of storage was not designed with large messages in mind, although streams likely will handle them better than any of the queue types. Put them in a blob store and pass their IDs or URIs around.

We won't change the logic to allow the client to override server value. That's a terrible idea and I'm sure you can guess why.

[OlehD]

Do I understand you correctly with this statement - "unset it everywhere and let RabbitMQ and client use the default" - you propose to remove the property frame_max from rabbitmq.conf, and don't set it in ConnectionFactory? If so it was not set in those two places before upgrade.

I believe what happened is that in 3.5.4 it was not working as expected on client side, and in higher versions something was fixed on client side, and it started working as expected since then. However, it requires adjustments in settings when someone does upgrade, and uses big messages in size.... I still try to figure out why upgrade caused it, and if I find something then I will post it here. Also, I agree that it is not the best practice to send big messages, but it is not an easy/quick thing to refactor old legacy code in a big service...

I'm not asking to make changes on client side, but I just wanted to understand why it needed now, so I could make a proper fix in configs for this. Also, maybe it would be good to add something into the specification about how exactly to set up frame max property?

Just to mention that the same issue happened with channel max after upgrade.

[michaelklishin]

Yes, I mean that you don't override the value on either end.

You are the first one to report this. 5.4.3 shipped in
October 2018, more than three years ago. The only relevant change in the last three years that I recall was
the introduction of a hard limit on message size and a lower default (128 MiB) for it.

https://github.com/rabbitmq/rabbitmq-java-client/blame/27203011bd60611af870cc5e092d838dbfe0f3cb/src/main/java/com/rabbitmq/client/impl/AMQConnection.java#L409-L414 demonstrates that the key method in question was last updated 11 years ago.

You can begin using any later 5.x version of the client today, with whatever RabbitMQ version you run. This is not worth
any time spent on code archeology if you ask me.

[michaelklishin]

The only relevant change in RabbitMQ itself that I could find (ignoring frame_max uses in the stream protocol, which this client does not use) is this from 2020: rabbitmq/rabbitmq-server@157808c.

I don't see how it can be relevant since neither the original nor the new value are 0.

[michaelklishin]

Err, I wanted to link to https://github.com/rabbitmq/rabbitmq-java-client/blame/27203011bd60611af870cc5e092d838dbfe0f3cb/src/main/java/com/rabbitmq/client/impl/AMQConnection.java#L633-L636 but the way frame max negotiation method is called is also relevant. Anyhow, this time the changes were 7 and 14 years ago.

So I don't see any evidence of meaningful changes since October 2018. You may be misinterpreting the behavior, what settings
were used at what point in your project's history, and so on.

Just upgrade the client, drop the overrides and move on.

[michaelklishin]

733788e and #640 can also be relevant, they happened after 5.4.3 (in early 2020).

That shipped in 5.8.1.