Min/avg/max aggregation?

[kerlandsson]

The use case is as follows:

When polling values it is easy to miss spikes for things such as CPU utilization or memory usage. The obvious fix when using polling is to poll more frequently. However, as jmxtrans-agent is built now, that also means metrics are sent to the backend more frequently, which results in increased load and storage requirements.

Hence it would be useful to be able to poll values frequently (think every second) but only send a max/min/avg/median aggregate more rarely.

While the functionality would be useful I'm not sure it fits within jmxtrans-agent. Other solutions would be to use statsd (although that introduces another moving part). Another solution would to instead build a separate agent that exposes new MBeans that keep a rolling min/max/avg and query those beans from jmxtrans-agent.

What are your thoughts on this? Should we look into implementing this in jmxtrans-agent or should we try to limit the scope to what it currently does? @cyrille-leclerc do you have any thoughts?

[cyrille-leclerc]

Hi @kerlandsson,

Preliminary note: should we move this discussion to https://groups.google.com/forum/#!forum/jmxtrans ?

I have initiated this kind of aggregation work in the PerMinuteSummarizerOutputWriter (I should have javadoc this class more). The idea of this class is to do the equivalent of Graphite's perSecond(seriesList).

I am also in favor of using statsd but it seems that, in some scenarios, in some organizations, it is too difficult to use statsd for this kind of aggregation.
For this "real life" constraint, I see benefits in aggregating ourselves as Dorpwizard Metrics does it.

I would like to try to implement such aggregation layer as an OutputWriter wrapper.

[kerlandsson]

@cyrille-leclerc I didn't know there was a google group, cool! I'm fine with moving discussions like this to that group, if you think they are better suited there? I guess the chances for more feedback are greater there.

[cyrille-leclerc]

Hi @kerlandsson, sorry I am very busy these days. Yes you could get more feedbacks on Google Groups.