Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bevy application doesn't terminate upon window close when using AMDVLK Vulkan driver #10260

Closed
robojeb opened this issue Oct 25, 2023 · 8 comments · Fixed by #13236
Closed

Bevy application doesn't terminate upon window close when using AMDVLK Vulkan driver #10260

robojeb opened this issue Oct 25, 2023 · 8 comments · Fixed by #13236
Labels
C-Bug An unexpected or incorrect behavior O-Linux Specific to the Linux desktop operating system S-Needs-Design This issue requires design work to think about how it would best be accomplished

Comments

@robojeb
Copy link
Contributor

robojeb commented Oct 25, 2023

Bevy version

Mainline Bevy

git+https://github.com/bevyengine/bevy.git#66f72dd25bb9e3f3d035f2f14dcbcd25674f968c

[Optional] Relevant system information

  • EndeavourOS

    • Kernel 6.5.8
    • Mesa: 23.2
      • AMDVLK: 2023.Q3.3
      • vulkan-radeon: 23.2.1
  • Rust: 1.75-nightly

  • Broken adapter info:

`AdapterInfo { name: "AMD Radeon RX 7900 XT", vendor: 4098, device: 29772, device_type: DiscreteGpu, driver: "AMD open-source driver", driver_info: "2023.Q3.3 (LLPC)", backend: Vulkan }`

What you did

Ran Bevy with the AMDVLK Vulkan driver, and closed the game window.

What went wrong

The game window closes, but the application continues to run until ctrl+c

2023-10-25T18:25:58.262755Z  INFO bevy_winit::system: Creating new window "App" (0v0)
2023-10-25T18:25:58.263005Z  INFO winit::platform_impl::platform::x11::window: Guessed window scale factor: 1    
2023-10-25T18:25:58.348593Z  INFO bevy_render::renderer: AdapterInfo { name: "AMD Radeon RX 7900 XT", vendor: 4098, device: 29772, device_type: DiscreteGpu, driver: "AMD open-source driver", driver_info: "2023.Q3.3 (LLPC)", backend: Vulkan }
2023-10-25T18:26:01.907275Z  INFO bevy_window::system: No windows are open, exiting
2023-10-25T18:26:01.907772Z  INFO bevy_winit::system: Closing window 0v0
<ctrl+c required here>

Additional information

Forcing the vulkan-radeon driver using environment variables makes the issue go away.

> AMD_VULKAN_ICD=RADV cargo run
2023-10-25T18:24:43.999447Z  INFO bevy_winit::system: Creating new window "App" (0v0)
2023-10-25T18:24:43.999663Z  INFO winit::platform_impl::platform::x11::window: Guessed window scale factor: 1    
2023-10-25T18:24:44.080668Z  INFO bevy_render::renderer: AdapterInfo { name: "AMD Radeon RX 7900 XT (RADV GFX1100)", vendor: 4098, device: 29772, device_type: DiscreteGpu, driver: "radv", driver_info: "Mesa 23.2.1-arch1.2", backend: Vulkan }
2023-10-25T18:24:46.288350Z  INFO bevy_window::system: No windows are open, exiting
2023-10-25T18:24:46.288592Z  INFO bevy_winit::system: Closing window 0v0

So this issue appears to only be related to the AMDVLK driver option.

@robojeb robojeb added C-Bug An unexpected or incorrect behavior S-Needs-Triage This issue needs to be labelled labels Oct 25, 2023
@alice-i-cecile alice-i-cecile added O-Linux Specific to the Linux desktop operating system S-Needs-Investigation This issue requires detective work to figure out what's going wrong and removed S-Needs-Triage This issue needs to be labelled labels Oct 25, 2023
@alice-i-cecile
Copy link
Member

Very interesting: I've seen this issue reported for Linux before, but this is by far the best investigation I've seen. Not entirely sure what should be done to fix this though 🤔

@robojeb
Copy link
Contributor Author

robojeb commented Oct 25, 2023

The issue does reproduce with multi_threaded removed from the project features.

I grabbed a single threaded stack trace from gdb:

#0  0x00007fcd5f7564ae in ?? () from /usr/lib/libc.so.6
#1  0x00007fcd5f761980 in ?? () from /usr/lib/libc.so.6
#2  0x00007fcd5acbe750 in ?? () from /usr/lib/amdvlk64.so
#3  0x00007fcd5abb201e in ?? () from /usr/lib/amdvlk64.so
#4  0x00007fcd5a9856ab in ?? () from /usr/lib/amdvlk64.so
#5  0x00007fcd4de797fe in DispatchDestroySwapchainKHR (device=device@entry=0x564b0f065d60, swapchain=<optimized out>, swapchain@entry=0xfef35a00000000a0, 
    pAllocator=pAllocator@entry=0x0)
    at /usr/src/debug/vulkan-validation-layers/Vulkan-ValidationLayers-sdk-1.3.261.1/layers/layer_chassis_dispatch_manual.cpp:393
#6  0x00007fcd4dd840ef in vulkan_layer_chassis::DestroySwapchainKHR (device=0x564b0f065d60, swapchain=0xfef35a00000000a0, pAllocator=0x0)
    at /usr/src/debug/vulkan-validation-layers/Vulkan-ValidationLayers-sdk-1.3.261.1/layers/vulkan/generated/chassis.cpp:5689
#7  0x00007fcd61b5eeb7 in ash::extensions::khr::swapchain::Swapchain::destroy_swapchain ()
    at /home/jeb/.local/share/cargo/registry/src/index.crates.io-6f17d22bba15001f/ash-0.37.3+1.3.251/src/extensions/khr/swapchain.rs:70
#8  wgpu_hal::vulkan::instance::{impl#5}::unconfigure (self=<optimized out>, device=<optimized out>) at src/vulkan/instance.rs:808
#9  0x00007fcd619c2c26 in wgpu_core::hub::Hub<wgpu_hal::vulkan::Api, wgpu_core::identity::IdentityManagerFactory>::surface_unconfigure<wgpu_hal::vulkan::Api, wgpu_core::identity::IdentityManagerFactory> (self=<optimized out>, device_id=..., surface=0x7fff77597ec0)
    at /home/jeb/.local/share/cargo/registry/src/index.crates.io-6f17d22bba15001f/wgpu-core-0.17.1/src/hub.rs:659
#10 0x00007fcd61a48803 in wgpu_core::instance::{impl#6}::surface_drop::unconfigure<wgpu_core::identity::IdentityManagerFactory, wgpu_hal::vulkan::Api> (
    global=<optimized out>, surface=<optimized out>, present=<optimized out>)
    at /home/jeb/.local/share/cargo/registry/src/index.crates.io-6f17d22bba15001f/wgpu-core-0.17.1/src/instance.rs:653
#11 wgpu_core::global::Global<wgpu_core::identity::IdentityManagerFactory>::surface_drop<wgpu_core::identity::IdentityManagerFactory> (self=0x564b0f0045f0, 
    id=...) at /home/jeb/.local/share/cargo/registry/src/index.crates.io-6f17d22bba15001f/wgpu-core-0.17.1/src/instance.rs:659
#12 0x00007fcd614e9db1 in core::ptr::drop_in_place<wgpu::Surface> () at /rustc/49691b1f70d71dd7b8349c332b7f277ee527bf08/library/core/src/ptr/mod.rs:498
#13 0x00007fcd61504062 in core::ptr::drop_in_place<bevy_render::view::window::SurfaceData> ()
    at /rustc/49691b1f70d71dd7b8349c332b7f277ee527bf08/library/core/src/ptr/mod.rs:498
#14 core::ptr::drop_in_place<core::option::Option<bevy_render::view::window::SurfaceData>> ()
    at /rustc/49691b1f70d71dd7b8349c332b7f277ee527bf08/library/core/src/ptr/mod.rs:498
#15 bevy_render::view::window::WindowSurfaces::remove (self=0x564b0f171570, window=<optimized out>) at src/view/window/mod.rs:209
#16 0x00007fcd614d579b in bevy_render::view::window::extract_windows (extracted_windows=..., screenshot_manager=..., closed=..., windows=..., removed=..., 
    window_surfaces=...) at src/view/window/mod.rs:173
#17 core::ops::function::FnMut::call_mut<fn(bevy_ecs::change_detection::ResMut<bevy_render::view::window::ExtractedWindows>, bevy_render::extract_param::Extract<bevy_ecs::change_detection::Res<bevy_render::view::window::screenshot::ScreenshotManager>>, bevy_render::extract_param::Extract<bevy_ecs::event::EventReader<bevy_window::event::WindowClosed>>, bevy_render::extract_param::Extract<bevy_ecs::system::query::Query<(bevy_ecs::entity::Entity, &bevy_window::window::Window, &bevy_window::raw_handle::RawHandleWrapper, core::option::Option<&bevy_window::window::PrimaryWindow>), ()>>, bevy_render::extract_param::Extract<bevy_ecs::removal_detection::RemovedComponents<bevy_window::raw_handle::RawHandleWrapper>>, bevy_ecs::change_detection::ResMut<bevy_render::view::window::WindowSurfaces>), (bevy_ecs::change_detection::ResMut<bevy_render::view::window::ExtractedWindows>, bevy_render::extract_param::Extract<bevy_ecs::change_detection::Res<bevy_render::view::window::screenshot::ScreenshotManager>>, bevy_render::extract_param::Extract<bevy_ecs::event::EventReader<bevy_window::event::WindowClosed>>, bevy_render::extract_param::Extract<bevy_ecs::system::query::Query<(bevy_ecs::entity::Entity, &bevy_window::window::Window, &bevy_window::raw_handle::RawHandleWrapper, core::option::Option<&bevy_window::window::PrimaryWindow>), ()>>, bevy_render::extract_param::Extract<bevy_ecs::removal_detection::RemovedComponents<bevy_window::raw_handle::RawHandleWrapper>>, bevy_ecs::change_detection::ResMut<bevy_render::view::window::WindowSurfaces>)> ()
    at /rustc/49691b1f70d71dd7b8349c332b7f277ee527bf08/library/core/src/ops/function.rs:166
#18 core::ops::function::impls::{impl#3}::call_mut<(bevy_ecs::change_detection::ResMut<bevy_render::view::window::ExtractedWindows>, bevy_render::extract_param::Extract<bevy_ecs::change_detection::Res<bevy_render::view::window::screenshot::ScreenshotManager>>, bevy_render::extract_param::Extract<bevy_ecs::event::EventReader<bevy_window::event::WindowClosed>>, bevy_render::extract_param::Extract<bevy_ecs::system::query::Query<(bevy_ecs::entity::Entity, &bevy_window::window::Window, &bevy_window::raw_handle::RawHandleWrapper, core::option::Option<&bevy_window::window::PrimaryWindow>), ()>>, bevy_render::extract_param::Extract<bevy_ecs::removal_detection::RemovedComponents<bevy_window::raw_handle::RawHandleWrapper>>, bevy_ecs::change_detection::ResMut<bevy_render::view::window::WindowSurfaces>), fn(bevy_ecs::change_detection::ResMut<bevy_render::view::window::ExtractedWindows>, bevy_render::extract_param::Extract<bevy_ecs::change_detection::Res<bevy_render::view::window::screenshot::ScreenshotManager>>, bevy_render::extract_param::Extract<bevy_ecs::event::EventReader<bevy_window::event::WindowClosed>>, bevy_render::extract_param::Extract<bevy_ecs::system::query::Query<(bevy_ecs::entity::Entity, &bevy_window::window::Window, &bevy_window::raw_handle::RawHandleWrapper, core::option::Option<&bevy_window::window::PrimaryWindow>), ()>>, bevy_render::extract_param::Extract<bevy_ecs::removal_detection::RemovedComponents<bevy_window::raw_handle::RawHandleWrapper>>, bevy_ecs::change_detection::ResMut<bevy_render::view::window::WindowSurfaces>)> (
    self=<optimized out>, args=...) at /rustc/49691b1f70d71dd7b8349c332b7f277ee527bf08/library/core/src/ops/function.rs:294
#19 bevy_ecs::system::function_system::{impl#21}::run::call_inner<(), bevy_ecs::change_detection::ResMut<bevy_render::view::window::ExtractedWindows>, bevy_render::extract_param::Extract<bevy_ecs::change_detection::Res<bevy_render::view::window::screenshot::ScreenshotManager>>, bevy_render::extract_param::Extract<bevy_ecs::event::EventReader<bevy_window::event::WindowClosed>>, bevy_render::extract_param::Extract<bevy_ecs::system::query::Query<(bevy_ecs::entity::Entity, &bevy_window::window::Window, &bevy_window::raw_handle::RawHandleWrapper, core::option::Option<&bevy_window::window::PrimaryWindow>), ()>>, bevy_render::extract_param::Extract<bevy_ecs::removal_detection::RemovedComponents<bevy_window::raw_handle::RawHandleWrapper>>, bevy_ecs::change_detection::ResMut<bevy_render::view::window::WindowSurfaces>, &mut fn(bevy_ecs::change_detection::ResMut<bevy_render::view::window::ExtractedWindows>, bevy_render::extract_param::Extract<bevy_ecs::change_detection::Res<bevy_render::view::window::screenshot::ScreenshotManager>>, bevy_render::extract_param::Extract<bevy_ecs::event::EventReader<bevy_window::event::WindowClosed>>, bevy_render::extract_param::Extract<bevy_ecs::system::query::Query<(bevy_ecs::entity::Entity, &bevy_window::window::Window, &bevy_window::raw_handle::RawHandleWrapper, core::option::Option<&bevy_window::window::PrimaryWindow>), ()>>, bevy_render::extract_param::Extract<bevy_ecs::removal_detection::RemovedComponents<bevy_window::raw_handle::RawHandleWrapper>>, bevy_ecs::change_detection::ResMut<bevy_render::view::window::WindowSurfaces>)> (f=<optimized out>, F0=..., F1=..., F2=..., F3=..., F4=..., F5=...)
    at /home/jeb/.local/share/cargo/git/checkouts/bevy-f7ffde730c324c74/66f72dd/crates/bevy_ecs/src/system/function_system.rs:641
#20 bevy_ecs::system::function_system::{impl#21}::run<(), fn(bevy_ecs::change_detection::ResMut<bevy_render::view::window::ExtractedWindows>, bevy_render::extract_param::Extract<bevy_ecs::change_detection::Res<bevy_render::view::window::screenshot::ScreenshotManager>>, bevy_render::extract_param::Extract<bevy_ecs::event::EventReader<bevy_window::event::WindowClosed>>, bevy_render::extract_param::Extract<bevy_ecs::system::query::Query<(bevy_ecs::entity::Entity, &bevy_window::window::Window, &bevy_window::raw_handle::RawHandleWrapper, core::option::Option<&bevy_window::window::PrimaryWindow>), ()>>, bevy_render::extract_param::Extract<bevy_ecs::removal_detection::RemovedComponents<bevy_window::raw_handle::RawHandleWrapper>>, bevy_ecs::change_detection::ResMut<bevy_render::view::window::WindowSurfaces>), bevy_ecs::change_detection::ResMut<bevy_render::view::window::ExtractedWindows>, bevy_render::extract_param::Extract<bevy_ecs::change_detection::Res<bevy_render::view::window::screenshot::ScreenshotManager>>, bevy_render::extract_param::Extract<bevy_ecs::event::EventReader<bevy_window::event::WindowClosed>>, bevy_render::extract_param::Extract<bevy_ecs::system::query::Query<(bevy_ecs::entity::Entity, &bevy_window::window::Window, &bevy_window::raw_handle::RawHandleWrapper, core::option::Option<&bevy_window::window::PrimaryWindow>), ()>>, bevy_render::extract_param::Extract<bevy_ecs::removal_detection::--Type <RET> for more, q to quit, c to continue without paging-- 
RemovedComponents<bevy_window::raw_handle::RawHandleWrapper>>, bevy_ecs::change_detection::ResMut<bevy_render::view::window::WindowSurfaces>> (
    self=<optimized out>, param_value=...)
    at /home/jeb/.local/share/cargo/git/checkouts/bevy-f7ffde730c324c74/66f72dd/crates/bevy_ecs/src/system/function_system.rs:644
#21 bevy_ecs::system::function_system::{impl#6}::run_unsafe<fn(bevy_ecs::change_detection::ResMut<bevy_render::view::window::ExtractedWindows>, bevy_render::extract_param::Extract<bevy_ecs::change_detection::Res<bevy_render::view::window::screenshot::ScreenshotManager>>, bevy_render::extract_param::Extract<bevy_ecs::event::EventReader<bevy_window::event::WindowClosed>>, bevy_render::extract_param::Extract<bevy_ecs::system::query::Query<(bevy_ecs::entity::Entity, &bevy_window::window::Window, &bevy_window::raw_handle::RawHandleWrapper, core::option::Option<&bevy_window::window::PrimaryWindow>), ()>>, bevy_render::extract_param::Extract<bevy_ecs::removal_detection::RemovedComponents<bevy_window::raw_handle::RawHandleWrapper>>, bevy_ecs::change_detection::ResMut<bevy_render::view::window::WindowSurfaces>), fn(bevy_ecs::change_detection::ResMut<bevy_render::view::window::ExtractedWindows>, bevy_render::extract_param::Extract<bevy_ecs::change_detection::Res<bevy_render::view::window::screenshot::ScreenshotManager>>, bevy_render::extract_param::Extract<bevy_ecs::event::EventReader<bevy_window::event::WindowClosed>>, bevy_render::extract_param::Extract<bevy_ecs::system::query::Query<(bevy_ecs::entity::Entity, &bevy_window::window::Window, &bevy_window::raw_handle::RawHandleWrapper, core::option::Option<&bevy_window::window::PrimaryWindow>), ()>>, bevy_render::extract_param::Extract<bevy_ecs::removal_detection::RemovedComponents<bevy_window::raw_handle::RawHandleWrapper>>, bevy_ecs::change_detection::ResMut<bevy_render::view::window::WindowSurfaces>)> (
    self=0x564b0f3369c0, input=<optimized out>, world=...)
    at /home/jeb/.local/share/cargo/git/checkouts/bevy-f7ffde730c324c74/66f72dd/crates/bevy_ecs/src/system/function_system.rs:484
#22 0x00007fcd6230cfe0 in bevy_ecs::schedule::executor::single_threaded::{impl#0}::run::{closure#0} () at src/schedule/executor/single_threaded.rs:92
#23 core::ops::function::FnOnce::call_once<bevy_ecs::schedule::executor::single_threaded::{impl#0}::run::{closure_env#0}, ()> ()
    at /rustc/49691b1f70d71dd7b8349c332b7f277ee527bf08/library/core/src/ops/function.rs:250
#24 core::panic::unwind_safe::{impl#23}::call_once<(), bevy_ecs::schedule::executor::single_threaded::{impl#0}::run::{closure_env#0}> (self=...)
    at /rustc/49691b1f70d71dd7b8349c332b7f277ee527bf08/library/core/src/panic/unwind_safe.rs:271
#25 std::panicking::try::do_call<core::panic::unwind_safe::AssertUnwindSafe<bevy_ecs::schedule::executor::single_threaded::{impl#0}::run::{closure_env#0}>, ()> (data=<optimized out>) at /rustc/49691b1f70d71dd7b8349c332b7f277ee527bf08/library/std/src/panicking.rs:504
#26 std::panicking::try<(), core::panic::unwind_safe::AssertUnwindSafe<bevy_ecs::schedule::executor::single_threaded::{impl#0}::run::{closure_env#0}>> (
    f=...) at /rustc/49691b1f70d71dd7b8349c332b7f277ee527bf08/library/std/src/panicking.rs:468
#27 std::panic::catch_unwind<core::panic::unwind_safe::AssertUnwindSafe<bevy_ecs::schedule::executor::single_threaded::{impl#0}::run::{closure_env#0}>, ()> (
    f=...) at /rustc/49691b1f70d71dd7b8349c332b7f277ee527bf08/library/std/src/panic.rs:142
#28 bevy_ecs::schedule::executor::single_threaded::{impl#0}::run (self=0x564b0f04d910, schedule=0x7fff775991a0, world=0x564b0f222350)
    at src/schedule/executor/single_threaded.rs:91
#29 0x00007fcd6172766e in bevy_ecs::world::{impl#3}::run_schedule::{closure#0}<bevy_render::ExtractSchedule> (world=0x564b0f222350, sched=0x7fff77599180)
    at /home/jeb/.local/share/cargo/git/checkouts/bevy-f7ffde730c324c74/66f72dd/crates/bevy_ecs/src/world/mod.rs:2088
#30 bevy_ecs::world::World::try_schedule_scope<(), bevy_render::ExtractSchedule, bevy_ecs::world::{impl#3}::run_schedule::{closure_env#0}<bevy_render::ExtractSchedule>> (self=0x564b0f222350, label=..., f=...)
    at /home/jeb/.local/share/cargo/git/checkouts/bevy-f7ffde730c324c74/66f72dd/crates/bevy_ecs/src/world/mod.rs:2007
#31 bevy_ecs::world::World::schedule_scope<(), bevy_render::ExtractSchedule, bevy_ecs::world::{impl#3}::run_schedule::{closure_env#0}<bevy_render::ExtractSchedule>> (self=0x564b0f222350, label=..., f=...)
    at /home/jeb/.local/share/cargo/git/checkouts/bevy-f7ffde730c324c74/66f72dd/crates/bevy_ecs/src/world/mod.rs:2059
#32 0x00007fcd615724d3 in bevy_ecs::world::World::run_schedule<bevy_render::ExtractSchedule> (self=<optimized out>)
    at /home/jeb/.local/share/cargo/git/checkouts/bevy-f7ffde730c324c74/66f72dd/crates/bevy_ecs/src/world/mod.rs:2088
#33 bevy_render::extract (main_world=0x7fff7759b718, render_app=0x564b0f222350) at src/lib.rs:396
#34 0x00007fcd622a443b in alloc::boxed::{impl#49}::call<(&mut bevy_ecs::world::World, &mut bevy_app::app::App), (dyn core::ops::function::Fn<(&mut bevy_ecs::world::World, &mut bevy_app::app::App), Output=()> + core::marker::Send), alloc::alloc::Global> (self=0x564b0f222340, args=...)
    at /rustc/49691b1f70d71dd7b8349c332b7f277ee527bf08/library/alloc/src/boxed.rs:2021
#35 bevy_app::app::SubApp::extract (self=0x564b0f222340, main_world=0x7fff7759b718) at src/app.rs:166
#36 bevy_app::app::App::update (self=0x7fff7759b718) at src/app.rs:264
#37 0x00007fcd60592720 in bevy_winit::winit_runner::{closure#0} (event=..., event_loop=0x564b0d97cea0, control_flow=0x7fff7759a740) at src/lib.rs:818
#38 0x00007fcd605c887e in winit::platform_impl::platform::sticky_exit_callback<(), bevy_winit::winit_runner::{closure_env#0}> (evt=..., target=0x0, 
    control_flow=0x7fff7759a740, callback=0x7fff7759afd0)
    at /home/jeb/.local/share/cargo/registry/src/index.crates.io-6f17d22bba15001f/winit-0.28.7/src/platform_impl/linux/mod.rs:884
#39 winit::platform_impl::platform::x11::{impl#2}::run_return::single_iteration<(), bevy_winit::winit_runner::{closure_env#0}> (this=0x7fff7759ab80, 
    control_flow=0x7fff7759a740, cause=0x7fff7759a7f0, callback=0x7fff7759afd0)
    at /home/jeb/.local/share/cargo/registry/src/index.crates.io-6f17d22bba15001f/winit-0.28.7/src/platform_impl/linux/x11/mod.rs:358
#40 0x00007fcd605c8ea8 in winit::platform_impl::platform::x11::EventLoop<()>::run_return<(), bevy_winit::winit_runner::{closure_env#0}> (
    self=0x7fff7759ab80, callback=...)
    at /home/jeb/.local/share/cargo/registry/src/index.crates.io-6f17d22bba15001f/winit-0.28.7/src/platform_impl/linux/x11/mod.rs:483
#41 0x00007fcd605ca98d in winit::platform_impl::platform::x11::EventLoop<()>::run<(), bevy_winit::winit_runner::{closure_env#0}> (self=..., 
    callback=<error reading variable: Cannot access memory at address 0x1a1>)
    at /home/jeb/.local/share/cargo/registry/src/index.crates.io-6f17d22bba15001f/winit-0.28.7/src/platform_impl/linux/x11/mod.rs:498
#42 0x00007fcd60584368 in winit::platform_impl::platform::EventLoop<()>::run<(), bevy_winit::winit_runner::{closure_env#0}> (
    self=<error reading variable: Cannot access memory at address 0x0>, callback=<error reading variable: Cannot access memory at address 0x18>)
    at /home/jeb/.local/share/cargo/registry/src/index.crates.io-6f17d22bba15001f/winit-0.28.7/src/platform_impl/linux/mod.rs:792
#43 winit::event_loop::EventLoop<()>::run<(), bevy_winit::winit_runner::{closure_env#0}> (
    event_handler=<error reading variable: Cannot access memory at address 0x18>)
    at /home/jeb/.local/share/cargo/registry/src/index.crates.io-6f17d22bba15001f/winit-0.28.7/src/event_loop.rs:305
#44 bevy_winit::run<bevy_winit::winit_runner::{closure_env#0}, ()> (event_handler=<error reading variable: Cannot access memory at address 0x18>)
    at src/lib.rs:235
#45 bevy_winit::winit_runner (app=...) at src/lib.rs:902
#46 0x00007fcd60575902 in core::ops::function::FnOnce::call_once<fn(bevy_app::app::App), (bevy_app::app::App)> ()
    at /rustc/49691b1f70d71dd7b8349c332b7f277ee527bf08/library/core/src/ops/function.rs:250
#47 core::ops::function::FnOnce::call_once<fn(bevy_app::app::App), (bevy_app::app::App)> ()
    at /rustc/49691b1f70d71dd7b8349c332b7f277ee527bf08/library/core/src/ops/function.rs:250
#48 0x00007fcd622a45f9 in alloc::boxed::{impl#47}::call_once<(bevy_app::app::App), (dyn core::ops::function::FnOnce<(bevy_app::app::App), Output=()> + core::marker::Send), alloc::alloc::Global> (self=..., args=...) at /rustc/49691b1f70d71dd7b8349c332b7f277ee527bf08/library/alloc/src/boxed.rs:2007
--Type <RET> for more, q to quit, c to continue without paging--
#49 bevy_app::app::App::run (self=<optimized out>) at src/app.rs:315
#50 0x0000564b0be29272 in general_logistics::main () at src/bin/general_logistics.rs:5

Its getting stuck deep in the AMDVLK code, so it might be their issue? Unless wgpu is doing something "non conforming" when destroying the swapchain, that other drivers handle better.

For now I will probably just go back to using radeon-vulkan, I only installed AMDVLK for some raytracing tests and I don't need to keep it installed.

@robojeb
Copy link
Contributor Author

robojeb commented Oct 26, 2023

I created a minimal example that can control the issue by changing the window PresentMode

use bevy::{prelude::*, window::PresentMode};

fn main() {
    App::new()
        .add_plugins(DefaultPlugins.set(WindowPlugin {
            primary_window: Some(Window {
                present_mode: PresentMode::AutoNoVsync,  /* Works */
                ..default()
            }),
            ..default()
        }))
        .run();
}

The issue only reproduces with PresentMode::AutoVsync and PresentMode::Fifo (which I think is the default in AutoVsync mode).
The issue also doesn't appear to reproduce if the WinitSettings::desktop_app() resource is provided.

Interestingly, the issue does not reproduce running wgpu boids or cube examples, even though they should be using PresentMode::Fifo as well.
I checked on both wgpu 0.17.1 and trunk.

I haven't been able to narrow down what Bevy is doing that causes FIFO mode to apparently deadlock when the swap-chain is destroyed.

@robojeb
Copy link
Contributor Author

robojeb commented Oct 26, 2023

@alice-i-cecile I think I have narrowed the issue.

In PresentMode::Fifo when the winit::window::Window structure is dropped before wgpu::Surface, then the AMDVLK driver will hang while trying to destroy the SwapChain.
If I skip dropping (or leak) the winit::window::Window, the hang does not occur.

I think that Bevy might be violating the WGPU surface docs on this, from here

raw_window_handle must remain valid until after the returned Surface is dropped.

But Bevy will always destroy the winit Window before destroying the Surface.

It isn't clear why other drivers handle this more gracefully.

@nicopap
Copy link
Contributor

nicopap commented Oct 27, 2023

Known issue. I've tried several times to solve it, but stopped, since for me, since the pipelined rendering change, it stopped happening altogether. See the already opened issues:

If it is really what you describe, I'm hyped. All I can say.

@robojeb
Copy link
Contributor Author

robojeb commented Oct 27, 2023

Thanks for bringing together all the prior issues.

It heartens me that #839 and #1908 have stack-traces which show the stall occurring during swap-chain destruction.
It makes it more likely we are actually looking at the same issue across all these machines. (It looks like it may even affect the intel driver as well).
#5524 doesn't specifically mention the swap-chain but does occur while the surface is being unconfigured.

@robojeb
Copy link
Contributor Author

robojeb commented Oct 28, 2023

I built and installed the amdvlk driver with debug symbols to see if I could dig further.

The issue is most definitely caused by closing the winit Window prior to shutting down the wgpu Surface in the renderer.

Internally the amdvlk driver spawns a background thread to handle presenting images to whatever the display back-end happens to be.
In FIFO mode, after every frame presented this background thread will wait for the X-server to send an XCB_PRESENT_COMPLETE_NOTIFY by calling xcb_wait_for_special_event().
Because the window was closed, this never happens (see similar issue here).

Because the background thread never advances in its loop, it fails to process a Notify event that the Bevy render thread sends while tearing down the Swap-chain which sets a timeout for UINT32_MAX seconds (so technically Bevy will eventually close... in 136 years).

So the root cause is:

Bevy --[blocks on]--> amdvlk worker --[waiting for event from]--> Xorg

While it could be argued that the amdvlk driver is doing something not very resilient, I do think that Bevy should fix the Window-Surface close order. This will make Bevy more resilient, and will comply with the wgpu documentation.

I put together a prototype which stores an Atomic token in the main app World to back-channel information from the Render World about the state of the wgpu Surface.

This is the most minimal change that I could think of, but is somewhat of a hack.
If someone knows a better way to back-channel information from the Render World, or can move ownership of the Surface to the Main App World that might be a better solution, but I couldn't think of a way that didn't involve introducing heavy new dependencies into one of the crates.

Edit: I think this issue should be tagged with S-Needs-Design

@alice-i-cecile alice-i-cecile added S-Needs-Design This issue requires design work to think about how it would best be accomplished and removed S-Needs-Investigation This issue requires detective work to figure out what's going wrong labels Oct 28, 2023
@Friz64
Copy link
Contributor

Friz64 commented May 9, 2024

#13236 fixes this.

github-merge-queue bot pushed a commit that referenced this issue May 12, 2024
# Objective

Fixes two issues related to #13208.

First, we ensure render resources for a window are always dropped first
to ensure that the `winit::Window` always drops on the main thread when
it is removed from `WinitWindows`. Previously, changes in #12978 caused
the window to drop in the render world, causing issues.

We accomplish this by delaying despawning the window by a frame by
inserting a marker component `ClosingWindow` that indicates the window
has been requested to close and is in the process of closing. The render
world now responds to the equivalent `WindowClosing` event rather than
`WindowCloseed` which now fires after the render resources are
guarunteed to be cleaned up.

Secondly, fixing the above caused (revealed?) that additional events
were being delivered to the the event loop handler after exit had
already been requested: in my testing `RedrawRequested` and
`LoopExiting`. This caused errors to be reported try to send an exit
event on the close channel. There are two options here:
- Guard the handler so no additional events are delivered once the app
is exiting. I ~considered this but worried it might be confusing or bug
prone if in the future someone wants to handle `LoopExiting` or some
other event to clean-up while exiting.~ We are now taking this approach.
- Only send an exit signal if we are not already exiting. ~It doesn't
appear to cause any problems to handle the extra events so this seems
safer.~
 
Fixing this also appears to have fixed #13231.

Fixes #10260.

## Testing

Tested on mac only.

---

## Changelog

### Added
- A `WindowClosing` event has been added that indicates the window will
be despawned on the next frame.

### Changed
- Windows now close a frame after their exit has been requested.

## Migration Guide
- Ensure custom exit logic does not rely on the app exiting the same
frame as a window is closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-Bug An unexpected or incorrect behavior O-Linux Specific to the Linux desktop operating system S-Needs-Design This issue requires design work to think about how it would best be accomplished
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants