Add some way to clear storage

[LGoossens]

Use case:
On a running program, I want to enable perfmark for a time, and write the trace data to a file.
Then later, I want to restart perfmark again and save a second set of trace data, not containing the trace data from the first time perfmark was activated.

To achieve this there should be some way to clear the Storage, but there is currently no way to this (as far as I could find).
The Storage.resetForTest() method does clear the storage but stops perfmark from properly tracing multiple treads afterwards.
Recording everything and splitting it up later is also not desirable, as this would cause perfmark to take up more and more memory with each use.

[carl-mastrangelo]

Let me think about how to do it. Putting a few thoughts down:

• Each thread assumes it's the only writer so having an external thread try to touch the buffer will likely have some cost.
• The wait-free nature of perfmark may be lost if external writes to a thread local buffer are allowed. Would you be okay with possibly blocking?
• It sounds like you might actually prefer having PerfMark write directly to disk.

[LGoossens]

In my case there would be a button to enable/disable perfmark, where disabling also writes to a file and clears the storage.
In a case like this clearing is not used excessively, so I would think some cost is not likely to be an issue.

[carl-mastrangelo]

In https://github.com/perfmark/perfmark/releases/tag/v0.26.0 PerfMark now has better methods for clearing storage in memory. The methods there are designed to allow existing threads to keep recording, but avoids synchronizing from the clearingThread to the other threads.