memory leaks on windows when using tinyobj_loader_opt.h

[natevm]

Describe the issue

After calling "parseObj", lfpAlloc appears to leak memory. This seems to occur when vectors using lfpAlloc are resized, especially during emplace / push_back. This memory leak can be significantly large when loading hundreds of OBJ files.

By simply removing the "lfpAlloc" from all std::vectors used in tinyobj_loader_opt.h, I no longer leak memory.

To Reproduce

1. repeatedly call parseObj.
2. profile memory usage
3. observe a collection of "lfpAlloc::Node<lfpAlloc::Chunk<<>>"'s that should be free'd once parseOBJ returns, but are not.

Please attach minimal and reproducible files(source codes, .obj/.mtl files, etc)

See @ogrex 's comment on issue #153. Repro can be done by modifying the "viewer.cc" file to repeatedly load a moderately large obj file.

Expected behavior
On return, parseOBJ should free all existing allocations to "lfpAlloc::Node<lfpAlloc::Chunk<<>>".
Alternatively, lfpAlloc should be removed as a dependency of tinyobj_loader_opt.h entirely, as this seems to fix the memory leak. I do not notice a significant impact on performance by removing lfpAlloc, however, more testing might be required to validate this is true.

Environment

• TinyObjLoader version: tinyobj_loader_opt.h on d2cb6d2
• OS: Windows 10
• Compiler: MSVC

[syoyo]

@n8vm Thanks!

Alternatively, lfpAlloc should be removed as a dependency of tinyobj_loader_opt.h entirely

I'd like to go with this proposal. Exposing lfpAlloc to public interface(e.g. struct attrib_t) isn't a good idea.

[syoyo]

@n8vm Could you try opt-refactor branch? https://github.com/tinyobjloader/tinyobjloader/tree/opt-refactor

It removes lfpAlloc dependency from public API and public struct definitions.

I have confirmed at least clang AddressSanitizer on Linux does not report memory leaks.

[natevm]

Hi @syoyo,

Thanks for the quick response!

There still appear to be several vectors still using lfpAlloc, and so I'm still leaking memory in Windows.

If you remove this line, that should point out any remaining lfpAlloc's.

tinyobjloader/experimental/tinyobj_loader_opt.h

Line 1004 in da0b64f

#include "lfpAlloc/Allocator.hpp"

After that point, we could probably remove the "lfpAlloc" folder from the "experimental" folder as well just to clean things up.

[syoyo]

lfpAlloc is mandatory for multithreaded parsing. Without it the performance slowdowns by 2x ~ 3x on modern multicore CPU(e.g. 16 core Threadripper)

You can try other multithread-aware allocators, e.g. mimalloc https://github.com/microsoft/mimalloc (well-maintaned and may have less issues on memory leaks) and how it goes.

[natevm]

Huh, the problem actually might go deeper than lfpAlloc... lfpAlloc definitely leaks the most memory, but there appear to be other leaks as well, so it might not be lfpAlloc that's to blame.

To be honest, I'm not sure where these leaks are coming from. Investigating...

Perhaps it has something to do with memory allocated on the heap by std::threads not being freed once that thread is joined/destructed?

[syoyo]

@n8vm I have added a flag to enable/disable lfpAlloc in opt-refactor branch.

tinyobjloader/experimental/tinyobj_loader_opt.h

Line 1005 in 1369a85

#define TINYOBJ_LOADER_OPT_USE_LFPALLOC

How it goes when disabling lfpAlloc(i.e. using STL default allocator)?

FYI at least ASAN on Linux does not report any memory leak when running objview with multiple parseObj call.

[HakkaTjakka]

Hello.

I have a repo for loading/editing/saving .nbt files.
The repo is from IDidMakeThat on GitHub.
There was a memory leak inside it.
So i added code to find memory leaks.
It uses its own memory allocator, so you can follow exactly what is happening, to find the leak.
Once you found where the leak is, you can use code to exactly find where it happens on the byte.
Maybe its an useful example for some other applications because sometimes its very hard to find.

Just saying.
Have a nice weekend.

https://github.com/HakkaTjakka/MCA-NBT-EDIT