Make standalone heap tracing more usefull and easy to read and analyse (IDFGH-12836)

[masterxq]

Is your feature request related to a problem?

tracing down memory leaks currently works like this:

heap_trace_start();
//Wait or try something
heap_trace_stop();
heap_trace_dump();

Now you will see a lot of allocated memory leaks that are false positive. And will be freed soon.

This can be avoided with a some small changes to heap trace api.

Describe the solution you'd like.

As already mentioned many of the false positive will be freed soon and we don't want to see them.

This can be solved if it is possible only to stop tracing malloc and continue tracing free. Then it would be possible to wait longer or restart some services. So many of the false positive leaks will now be freed and new mallocs will not be collected. Now we have a more relevant output of the dump.

heap_trace_start();
//Wait or try something
heap_trace_stop_tracing_malloc(); //Only stop tracing malloc, continue tracing free.
//Wait longer for frees or test some actions that could trigger free
heap_trace_stop();
heap_trace_dump();

Now we should have much less false positives...

The standard workflow would still work, and it should be easily possible without breaking changes.

Describe alternatives you've considered.

Even better would be if this workflow would also be possible, so we better know when or with which action stuff will be freed.

heap_trace_start();
//Wait or try something
heap_trace_stop_tracing_malloc(); //Only stop tracing malloc, continue tracing free.
//Wait longer or test some actions
while(trace_heap)
{
 if(button_pressed)
   heap_trace_dump(); //Don't know if it is currently possible to call dump before stop
}
heap_trace_stop();

Additional context.

I think this would help a lot to make good bug reports about memory leaks without complicated JTAG setup, what really can be hard to setup in products where the problem appears.

After all this could help to fix a lot of memory leaks, by helping users by helping.

I think espressif has a lot of good developer tests, but also a lot of bugs that only appears in production, and there it is hard to analyze for the users, and it is hard to define test cases for the espressif team. So it is important to make the tests that works without special equipment in production as good as possible. This allows users to make tests in the final environment.

Additionally, it would help the user to find leaks in user code. Without have tons of false positives from the framework that will be freed the next few seconds :)

Thanks for reading :)

[SoucheSouche]

Hello @masterxq, thanks for submitting this feature request.

It is perfectly fine to use heap_caps_dump when the tracing is running.
Concerning the feature request, I will investigate it but it seems to make sense to be able to stop recording new allocations while continuing to record free calls.

I will keep you updated about the solution I will choose to implement.

[chipweinberger]

i agree it makes sense

[SoucheSouche]

Hi @masterxq,
I just merged the new feature locally. It will be available with next sync.
I just ended up adding a new function heap_trace_alloc_pause that will internally skip the allocations recording and follow through with the frees.

[masterxq]

Very fast, very cool and very useful!

I will try to honor the work invested and give it a test by using the new feature as soon as possible to create a bug report that is as qualitative as possible for me.

Can it also be merged into 4.4.x?

Please remember to trigger an update of the documentation, so users have the chance to use it, a good espressif with few bugs is good for everybody ^^

Thank you!

[SoucheSouche]

@masterxq,
The documentation has been updated and will now list the new function next to heap_trace_start and heap_trace_stop
Since it is a new feature, it will not be backported.

[masterxq]

Understand, thank you very much!

Best Regards!