Memory Leak + Reduced Framerate in 0.7

[mbolt35]

This issue appears to be related to the Tilemap plugin, but I did upgrade to bevy 0.8 and this plugin simultaneously, and it's somewhat unclear where this issue might exist.

The code in question uses a G keypress to toggle a "debug grid" tilemap entity over the world tile map.

#[derive(Component)]
pub struct DebugLayer;

const DEBUG_LAYER: f32 = 2.0f32;

/// Handle world actions on keypress. 
pub fn on_keypress(
    mut commands: Commands,
    mut world_debug: ResMut<WorldDebugOptions>,
    game_assets: Res<GameAssets>,
    map_config: Res<MapConfig>,
    keyboard: Res<Input<KeyCode>>,
    map_query: Query<Entity, With<DebugLayer>>)
{
    if keyboard.just_released(KeyCode::G) {
        if world_debug.debug_grid {
            if let Ok(entity) = map_query.get_single() {
                world_debug.debug_grid = false;
                commands.entity(entity).despawn_recursive();
            }
        } else {
            world_debug.debug_grid = true;
            display_debug_grid(&mut commands, &game_assets, &map_config);
        }
    }
}

/// Adds the display of the debug grid, tagging with the `DebugGrid` component.
fn display_debug_grid(
    mut commands: &mut Commands,
    game_assets: &Res<GameAssets>,
    map_config: &Res<MapConfig>)
{
    // Grid Size is the pixel dimensions for each tile
    let tile_dimensions = map_config.tile_dimensions;
    let grid_size = TilemapGridSize{ x: tile_dimensions.x, y: tile_dimensions.y };
    let tile_size = TilemapTileSize{ x: tile_dimensions.x, y: tile_dimensions.y };

    // derive world map size in tiles
    let map_dims = map_config.world_dimensions * map_config.chunk_dimensions;
    let map_size = TilemapSize{ x: map_dims.x, y: map_dims.y };

    // debug grid tile storage
    let debug_texture = game_assets.tiles_for(TileSetType::MainWorld);
    let mut debug_tile_storage = TileStorage::empty(map_size);
    let debug_layer_entity = commands.spawn().id();

    fill_tilemap(
        TileTexture(0),
        map_size,
        TilemapId(debug_layer_entity),
        &mut commands,
        &mut debug_tile_storage);

    // Add Debug Layer
    commands
        .entity(debug_layer_entity)
        .insert_bundle(TilemapBundle {
            grid_size,
            size: map_size,
            storage: debug_tile_storage,
            texture: TilemapTexture(debug_texture),
            tile_size,
            transform: get_centered_transform_2d(
                &map_size,
                &tile_size,
                DEBUG_LAYER,
            ),
            ..Default::default()
        })
        .insert(DebugLayer);
}

The map_size in this case is: 160x160 with a tile_size of 50x50. The tile images are under a non-sharable license, but I can crop the debug tiles if necessary (just let me know).

Here's a video from one of my streams where I can demonstrate the problem occurring. I originally noticed the framerate dropping significantly when constantly toggling, and then discovered that memory tends to increase each time the grid is toggled as well.

bevy-0.8-tilemap-leak.mp4

I also generated a Tracy profile, which just shows that everything in bevy tends to slow down each time the grid is toggled. Here are the trace times towards the end of my test (the last 10s):

Here are the frame times associated with the full run:

I hosted the tracy-profile on google drive (it's 114mb) if you're interested: https://drive.google.com/file/d/1uHzEi_eZWyEYL-GNaQ415cLHIdX58iZM/view?usp=sharing

Thanks again for a wonderful project. I've had an awesome experienced building my game using this plugin. The new API is highly intuitive, loving it so far.

[StarArawn]

I can't say I suggest hiding and showing a grid by spawing and despawing the grid but I still think this is a valid issue. We do a lot of caching on the rendering side and I have a feeling something isn't being cleaned up correctly.

[mbolt35]

@StarArawn Thanks for the response. I agree with the suggestion about despawn/respawn on toggle. I think the reason why this issue concerned me is I am currently working on an "infinite" 2d map using noise, and it would use a chunking design somewhat similar to the examples/chunking.rs where it would despawn the tilemap entities eventually (I was planning on having a "soft removal" where chunks would have to be a specific distance away for some set amount of time before actually despawning to avoid thrashing).

I was able to reproduce this same issue with the examples/chunking.rs example by scrolling around for a few minutes. The problem is much less pronounced with the smaller chunk sizes, but eventually, the FPS will come to a crawl.

Thanks again for the response, I'm not super familiar with the custom render pipeline, but I have some time in the morning that I can poke around. Would love to contribute anyway I can.

[StarArawn]

I've looked into this more closely and this is not an issue with bevy_ecs_tilemap. If you notice reserve_and_flush is rather high in the Tracy profile you posted. I've already submitted an issue about this you can read more here:
bevyengine/bevy#3953
There is also a PR which is suppose to fix this:
bevyengine/bevy#5509

[StarArawn]

One solution as a workaround is to avoid despawning entities and attempt to reuse them instead. For example if you mark a chunk as visible false and then when a new chunk spawns you can pull that chunk in(via a marker component?) change its tiles and transform to the new location.

Hopefully this issue is fixed soon in bevy with the release of 0.8.1 :)

[StarArawn]

The memory leak might indeed be on my side I'm looking for it now.

[mbolt35]

Profiling heap usage isn't exactly straight-forward on windows, so I'm still working on that, but just based on some debug logs I added in prepare and extract, I wasn't able to locate anything that seemed like a leak. Were you able to find anything?

[StarArawn]

Profiling heap usage isn't exactly straight-forward on windows, so I'm still working on that, but just based on some debug logs I added in prepare and extract, I wasn't able to locate anything that seemed like a leak. Were you able to find anything?

There were two hash maps for mapping entities to tile positions but I clear that data now and I still get like 10mb's of leak in a modified chunking example. Still trying to narrow down where.

[StarArawn]

I was able to figure out the issue. Although some memory does still leak somewhere..

In order to use despawn_recursive we need to add each tile as a child of the tilemap entity.

I've opened up a PR with some fixes here:
#219

[mbolt35]

Not sure whether or not this helps, but I was able to get a dhat allocator running against the chunking example:
dhat-heap.zip

You can open the .json in https://nnethercote.github.io/dh_view/dh_view.html

The entries with bevy_ecs_tilemap in the stack:

PP 1.1.1.1/4 {
  │   │   │     Total:     122,445,568 bytes (9.84%, 1,502,088.15/s) in 1,049 blocks (0.38%, 12.87/s), avg size 116,725.99 bytes, avg lifetime 4,798.33 µs (0.01% of program duration)
  │   │   │     Max:       1,835,008 bytes in 1 blocks, avg size 1,835,008 bytes
  │   │   │     At t-gmax: 0 bytes (0%) in 0 blocks (0%), avg size 0 bytes
  │   │   │     At t-end:  0 bytes (0%) in 0 blocks (0%), avg size 0 bytes
  │   │   │     Allocated at {
  │   │   │       ^1: 0x7ff7e009e9d4: dhat::impl$7::alloc (dhat-0.3.0\src\lib.rs:1226:0)
  │   │   │       ^2: 0x7ff7e0081057: core::result::impl$28::from_residual (core\src\result.rs:2105:0)
  │   │   │       ^3: 0x7ff7e0081057: alloc::raw_vec::finish_grow<alloc::alloc::Global> (alloc\src\raw_vec.rs:456:0)
  │   │   │       #4: 0x7ff7e01891e0: alloc::raw_vec::RawVec::grow_amortized (alloc\src\raw_vec.rs:400:0)
  │   │   │       #5: 0x7ff7e01891e0: alloc::raw_vec::RawVec::reserve_for_push<tuple$<bevy_ecs::entity::Entity,bevy_ecs_tilemap::render::extract::ExtractedTileBundle>,alloc::alloc::Global> (alloc\src\raw_vec.rs:298:0)
  │   │   │       #6: 0x7ff7e01a795c: alloc::vec::Vec::push (src\vec\mod.rs:1733:0)
  │   │   │       #7: 0x7ff7e01a795c: bevy_ecs_tilemap::render::extract::extract (src\render\extract.rs:190:0)
  │   │   │       #8: 0x7ff7e01a795c: core::ops::function::FnMut::call_mut (src\ops\function.rs:164:0)
  │   │   │       #9: 0x7ff7e01a795c: core::ops::function::impls::impl$3::call_mut<tuple$<bevy_ecs::system::commands::Commands,bevy_render::extract_param::Extract<bevy_ecs::system::query::Query<tuple$<bevy_ecs::entity::Entity,ref$<bevy_ecs_tilemap::tiles::TilePos>,ref$<bevy_ecs_tilemap::map:: (src\ops\function.rs:290:0)
  │   │   │       #10: 0x7ff7e01a6577: bevy_ecs::system::function_system::impl$26::run::call_inner (src\system\function_system.rs:564:0)
  │   │   │     }
  │   │   │   }

  ├── PP 1.2/13 {
  │     Total:     127,926,272 bytes (10.28%, 1,569,322.11/s) in 488 blocks (0.18%, 5.99/s), avg size 262,144 bytes, avg lifetime 3,167,679.22 µs (3.89% of program duration)
  │     Max:       7,602,176 bytes in 29 blocks, avg size 262,144 bytes
  │     At t-gmax: 3,932,160 bytes (2.35%) in 15 blocks (0.15%), avg size 262,144 bytes
  │     At t-end:  0 bytes (0%) in 0 blocks (0%), avg size 0 bytes
  │     Allocated at {
  │       #1: 0x7ff7e012660f: alloc::alloc::impl$1::allocate (alloc\src\alloc.rs:231:0)
  │       #2: 0x7ff7e012660f: alloc::raw_vec::RawVec::allocate_in (alloc\src\raw_vec.rs:185:0)
  │       #3: 0x7ff7e012660f: alloc::raw_vec::RawVec::with_capacity_in (alloc\src\raw_vec.rs:131:0)
  │       #4: 0x7ff7e012660f: alloc::vec::Vec::with_capacity_in (src\vec\mod.rs:615:0)
  │       #5: 0x7ff7e012660f: alloc::vec::Vec::with_capacity (src\vec\mod.rs:470:0)
  │       #6: 0x7ff7e012660f: bevy_ecs_tilemap::render::chunk::RenderChunk2d::prepare (src\render\chunk.rs:231:0)
  │       #7: 0x7ff7e00b6cca: bevy_ecs_tilemap::render::prepare::prepare (src\render\prepare.rs:166:0)
  │     }
  │   }
  ├── PP 1.3/13 {
  │     Total:     127,926,272 bytes (10.28%, 1,569,322.11/s) in 488 blocks (0.18%, 5.99/s), avg size 262,144 bytes, avg lifetime 3,168,114.56 µs (3.89% of program duration)
  │     Max:       7,602,176 bytes in 29 blocks, avg size 262,144 bytes
  │     At t-gmax: 3,932,160 bytes (2.35%) in 15 blocks (0.15%), avg size 262,144 bytes
  │     At t-end:  0 bytes (0%) in 0 blocks (0%), avg size 0 bytes
  │     Allocated at {
  │       #1: 0x7ff7e01265e0: alloc::alloc::impl$1::allocate (alloc\src\alloc.rs:231:0)
  │       #2: 0x7ff7e01265e0: alloc::raw_vec::RawVec::allocate_in (alloc\src\raw_vec.rs:185:0)
  │       #3: 0x7ff7e01265e0: alloc::raw_vec::RawVec::with_capacity_in (alloc\src\raw_vec.rs:131:0)
  │       #4: 0x7ff7e01265e0: alloc::vec::Vec::with_capacity_in (src\vec\mod.rs:615:0)
  │       #5: 0x7ff7e01265e0: alloc::vec::Vec::with_capacity (src\vec\mod.rs:470:0)
  │       #6: 0x7ff7e01265e0: bevy_ecs_tilemap::render::chunk::RenderChunk2d::prepare (src\render\chunk.rs:230:0)
  │       #7: 0x7ff7e00b6cca: bevy_ecs_tilemap::render::prepare::prepare (src\render\prepare.rs:166:0)
  │     }
  │   }
  ├── PP 1.4/13 {
  │     Total:     127,926,272 bytes (10.28%, 1,569,322.11/s) in 488 blocks (0.18%, 5.99/s), avg size 262,144 bytes, avg lifetime 3,253,004.05 µs (3.99% of program duration)
  │     Max:       7,602,176 bytes in 29 blocks, avg size 262,144 bytes
  │     At t-gmax: 4,980,736 bytes (2.97%) in 19 blocks (0.19%), avg size 262,144 bytes
  │     At t-end:  0 bytes (0%) in 0 blocks (0%), avg size 0 bytes
  │     Allocated at {
  │       #1: 0x7ff7e0124c74: alloc::alloc::impl$1::allocate (alloc\src\alloc.rs:231:0)
  │       #2: 0x7ff7e0124c74: alloc::raw_vec::RawVec::allocate_in (alloc\src\raw_vec.rs:185:0)
  │       #3: 0x7ff7e0124c74: alloc::raw_vec::RawVec::with_capacity_in (alloc\src\raw_vec.rs:131:0)
  │       #4: 0x7ff7e0124c74: alloc::vec::Vec::with_capacity_in (src\vec\mod.rs:615:0)
  │       #5: 0x7ff7e0124c74: alloc::vec::spec_from_elem::impl$0::from_elem (src\vec\spec_from_elem.rs:14:0)
  │       #6: 0x7ff7e0124c74: alloc::vec::from_elem (src\vec\mod.rs:2423:0)
  │       #7: 0x7ff7e0124c74: bevy_ecs_tilemap::render::chunk::RenderChunk2d::new (src\render\chunk.rs:207:0)
  │     }
  │   }
  ├── PP 1.5/13 {
  │     Total:     127,926,272 bytes (10.28%, 1,569,322.11/s) in 488 blocks (0.18%, 5.99/s), avg size 262,144 bytes, avg lifetime 3,167,511.53 µs (3.89% of program duration)
  │     Max:       7,602,176 bytes in 29 blocks, avg size 262,144 bytes
  │     At t-gmax: 3,932,160 bytes (2.35%) in 15 blocks (0.15%), avg size 262,144 bytes
  │     At t-end:  0 bytes (0%) in 0 blocks (0%), avg size 0 bytes
  │     Allocated at {
  │       #1: 0x7ff7e012663e: alloc::alloc::impl$1::allocate (alloc\src\alloc.rs:231:0)
  │       #2: 0x7ff7e012663e: alloc::raw_vec::RawVec::allocate_in (alloc\src\raw_vec.rs:185:0)
  │       #3: 0x7ff7e012663e: alloc::raw_vec::RawVec::with_capacity_in (alloc\src\raw_vec.rs:131:0)
  │       #4: 0x7ff7e012663e: alloc::vec::Vec::with_capacity_in (src\vec\mod.rs:615:0)
  │       #5: 0x7ff7e012663e: alloc::vec::Vec::with_capacity (src\vec\mod.rs:470:0)
  │       #6: 0x7ff7e012663e: bevy_ecs_tilemap::render::chunk::RenderChunk2d::prepare (src\render\chunk.rs:232:0)
  │       #7: 0x7ff7e00b6cca: bevy_ecs_tilemap::render::prepare::prepare (src\render\prepare.rs:166:0)
  │     }
  │   }

PP 1.10/13 {
  │     Total:     47,972,352 bytes (3.86%, 588,495.79/s) in 488 blocks (0.18%, 5.99/s), avg size 98,304 bytes, avg lifetime 3,167,242.03 µs (3.89% of program duration)
  │     Max:       2,850,816 bytes in 29 blocks, avg size 98,304 bytes
  │     At t-gmax: 1,474,560 bytes (0.88%) in 15 blocks (0.15%), avg size 98,304 bytes
  │     At t-end:  0 bytes (0%) in 0 blocks (0%), avg size 0 bytes
  │     Allocated at {
  │       #1: 0x7ff7e01266d9: alloc::alloc::impl$1::allocate (alloc\src\alloc.rs:231:0)
  │       #2: 0x7ff7e01266d9: alloc::raw_vec::RawVec::allocate_in (alloc\src\raw_vec.rs:185:0)
  │       #3: 0x7ff7e01266d9: alloc::raw_vec::RawVec::with_capacity_in (alloc\src\raw_vec.rs:131:0)
  │       #4: 0x7ff7e01266d9: alloc::vec::Vec::with_capacity_in (src\vec\mod.rs:615:0)
  │       #5: 0x7ff7e01266d9: alloc::vec::Vec::with_capacity (src\vec\mod.rs:470:0)
  │       #6: 0x7ff7e01266d9: bevy_ecs_tilemap::render::chunk::RenderChunk2d::prepare (src\render\chunk.rs:234:0)
  │       #7: 0x7ff7e00b6cca: bevy_ecs_tilemap::render::prepare::prepare (src\render\prepare.rs:166:0)

[bzm3r]

Fixed by #219!

[mbolt35]

Might I also add that the "small remaining leak" that seemed to still be present is present in a blank bevy application as long as you leave it running long enough, you'll see memory slowly increase. However, this appears to only exist in debug mode. --release is fine (I don't really have any specific details beyond just the observation).