stall-analyser: add output support for flamegraph

[tchaikov]

a typical session is like:

$ ./stall-analyser.py --format trace -e scylla/libexec/scylla \
  reactor_stalled_231d31df_2022_2_twcs_3h.log  > out.folded
$ flamegraph.pl out.folded > /tmp/folded.svg
$ xdg-open /tmp/folded.svg

[bhalevy]

Can you please upload an example of the result?

[tchaikov]

sure. the log file and executable come from scylladb/scylladb#11971

$ sudo dnf install flamegraph # install flamegraph if it's not available
$ ./stall-analyser.py --format trace -e scylla/libexec/scylla \
  reactor_stalled_231d31df_2022_2_twcs_3h.log | \
  flamegraph.pl --title 'Reactor Stalls' --countname "ms" > /tmp/folded.svg
$ xdg-open /tmp/folded.svg

folded.log

[tchaikov]

@bhalevy hi Benny, if the general direction is sane, we could render an SVG on the fly without using flamegraph. this way, we could customize the blocks further. the render will be implemented based on Graph, which has done the heavy lifting. but before that i will try to work on our in-house web service to hook up with this tool.

[bhalevy]

It looks nice visually, but I don't see how it helps understanding the stack traces any better.
Seems distracting to me.

[tchaikov]

@bhalevy hi Benny, thank you for your review. i think it does not provide more information, even worse, it provides less information than the existing plain text output. but the reason i added this output was not just for offering an eye candy.

with a graph we can identify the the hotspot with a quick glance. then we can continue looking at the text output and the logging messages. it serves the same purpose of https://www.brendangregg.com/flamegraphs.html

i am extracting the refactor part out into #2178. would be great to make stall-analyser an importable module in future.