-
|
After starting multiple producers/consumers which publish with different subjects on the same jetstream I do not see the number of deleted messages in nats-box. As you can see from the following screenshot the stream size is ca. 600k messages, which is ok as expected and also the number of total sent messages is more than 43,6Mil which is also correct. But I would expect too se under deleted messages not 0 but something around 43Mil, which I don't see. When I started another java project with multiple producers/consumers I was able to see the number of deleted messages but this time not any more. Should this parameter in nats-box show the number of deleted messages or am I wrong, because I couldn't find something in this regard?
|
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 6 replies
-
|
Imagine you have a stream with messages 1,2,3,4,5,6,7,8,9,10 the stream removes messages due to limits so you have 5,6,7,8,9,10 this would not be reported as deleted messages. If however you now delete using the delete API message 8, you would have 1 deleted message. If you deleted message 5 you would still have 1 deleted message reported. So its about the definition of deleted messages - they are gaps in the stream if you had 5,6,7,|gap|,10 the gap is 2 big so deleted is 2. |
Beta Was this translation helpful? Give feedback.
-
|
The same pattern would be seen if you say keep 5 messages per subject, if one subject is busy getting messages - so always removing messages - and other subjects dont change, this will cause a lot of gaps and so lots of deleted messages |
Beta Was this translation helpful? Give feedback.
-
|
So just to verify if I understood that correct: I.e., deleted counts gaps on a stream level across subjects and tells you something about "fragmentation" of the stream. Is it a bad thing (performance wise) to have high deleted numbers? |
Beta Was this translation helpful? Give feedback.
-
|
Correct. Deleted msgs are always in the interior, between first and last sequences. 2.10 is much better at handling large interior delete state. |
Beta Was this translation helpful? Give feedback.
-
|
@derekcollison we came around deleted messages actually by accident, when looking at the nats-box. It would be great to have a prom metric for the deleted messages as well, and maybe incorporate it in the nats grafana dashboards. |
Beta Was this translation helpful? Give feedback.
-
|
Looping in @Jarema |
Beta Was this translation helpful? Give feedback.
Imagine you have a stream with messages 1,2,3,4,5,6,7,8,9,10 the stream removes messages due to limits so you have 5,6,7,8,9,10 this would not be reported as deleted messages.
If however you now delete using the delete API message 8, you would have 1 deleted message. If you deleted message 5 you would still have 1 deleted message reported.
So its about the definition of deleted messages - they are gaps in the stream if you had 5,6,7,|gap|,10 the gap is 2 big so deleted is 2.