Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stack overflow after upgrade to 7.0.2 #1506

Open
negezor opened this issue Mar 30, 2024 · 9 comments
Open

Stack overflow after upgrade to 7.0.2 #1506

negezor opened this issue Mar 30, 2024 · 9 comments
Labels
bug Something isn't working

Comments

@negezor
Copy link
Contributor

negezor commented Mar 30, 2024

I've been experiencing stack overflow in runtime, I didn't immediately notice this problem as I hadn't run tests for a long time during local development.

After hours of comparing changes, I found out that it happened after an upgrade to 7.0.2. After spending some more time trying to figure out what broke between 7.0.1 and 7.0.2 I found the cause.

The problem turned out to be the following PR #1468.

Expected Behavior

Working code as on 7.0.1

Actual Behavior

When calling GraphQL mutation and then executing a database transaction, application crashes

thread 'actix-rt|system:0|arbiter:7' has overflowed its stack
fatal runtime error: stack overflow
fish: Job 1, 'cargo run' terminated by signal SIGABRT (Abort)

Using backtrace-on-stack-overflow I got the following stacktrace.

Stacktrace
Stack Overflow:
   0: backtrace_on_stack_overflow::handle_sigsegv
   1: <unknown>
   2: <(A,B,C,D,E,F,G,H,I,J,K,L,M,N,O,P,Q,R,S,T,U) as nom::branch::Alt<Input,Output,Error>>::choice
   3: nom::branch::alt::{{closure}}
   4: <F as nom::internal::Parser<I,O,E>>::parse
   5: <(A,B,C,D,E,F,G,H,I,J,K,L,M,N,O,P,Q,R,S,T,U) as nom::branch::Alt<Input,Output,Error>>::choice
   6: nom::branch::alt::{{closure}}
   7: <F as nom::internal::Parser<I,O,E>>::parse
   8: <(A,B,C,D,E,F,G,H,I,J,K,L,M,N,O,P,Q,R,S,T,U) as nom::branch::Alt<Input,Output,Error>>::choice
   9: nom::branch::alt::{{closure}}
  10: <F as nom::internal::Parser<I,O,E>>::parse
  11: <(A,B,C,D,E,F,G,H,I,J,K,L,M,N,O,P,Q,R,S,T,U) as nom::branch::Alt<Input,Output,Error>>::choice
  12: nom::branch::alt::{{closure}}
  13: <F as nom::internal::Parser<I,O,E>>::parse
  14: <(A,B,C,D,E,F,G,H,I,J,K,L,M,N,O,P,Q,R,S,T) as nom::branch::Alt<Input,Output,Error>>::choice
  15: nom::branch::alt::{{closure}}
  16: <F as nom::internal::Parser<I,O,E>>::parse
  17: <(A,B,C,D,E,F,G,H,I,J,K,L,M,N,O,P,Q,R,S,T,U) as nom::branch::Alt<Input,Output,Error>>::choice
  18: nom::branch::alt::{{closure}}
  19: <F as nom::internal::Parser<I,O,E>>::parse
  20: <(A,B,C,D,E,F,G,H,I,J,K,L,M,N,O,P,Q,R,S,T,U) as nom::branch::Alt<Input,Output,Error>>::choice
  21: nom::branch::alt::{{closure}}
  22: <F as nom::internal::Parser<I,O,E>>::parse
  23: <(A,B,C,D,E,F,G,H,I,J,K,L,M,N,O,P,Q,R,S,T) as nom::branch::Alt<Input,Output,Error>>::choice
  24: nom::branch::alt::{{closure}}
  25: sqlformat::tokenizer::get_plain_reserved_token
  26: core::ops::function::FnMut::call_mut
  27: <F as nom::internal::Parser<I,O,E>>::parse
  28: <(A,B,C,D) as nom::branch::Alt<Input,Output,Error>>::choice
  29: nom::branch::alt::{{closure}}
  30: sqlformat::tokenizer::get_reserved_word_token
  31: sqlformat::tokenizer::get_next_token::{{closure}}
  32: core::result::Result<T,E>::or_else
  33: sqlformat::tokenizer::get_next_token
  34: sqlformat::tokenizer::tokenize
  35: sqlformat::format
  36: sqlx_core::logger::QueryLogger::finish
  37: <sqlx_core::logger::QueryLogger as core::ops::drop::Drop>::drop
  38: core::ptr::drop_in_place<sqlx_core::logger::QueryLogger>
  39: sqlx_postgres::connection::executor::<impl sqlx_postgres::connection::PgConnection>::run::{{closure}}::{{closure}}::{{closure}}
  40: <core::pin::Pin<P> as core::future::future::Future>::poll
  41: <futures_util::future::future::fuse::Fuse<Fut> as core::future::future::Future>::poll
  42: futures_util::future::future::FutureExt::poll_unpin
  43: <sqlx_core::ext::async_stream::TryAsyncStream<T> as futures_core::stream::Stream>::poll_next
  44: <core::pin::Pin<P> as futures_core::stream::Stream>::poll_next
  45: <S as futures_core::stream::TryStream>::try_poll_next
  46: futures_util::stream::try_stream::TryStreamExt::try_poll_next_unpin
  47: <futures_util::stream::try_stream::try_next::TryNext<St> as core::future::future::Future>::poll
  48: sqlx_postgres::connection::executor::<impl sqlx_core::executor::Executor for &mut sqlx_postgres::connection::PgConnection>::fetch_optional::{{closure}}
  49: <core::pin::Pin<P> as core::future::future::Future>::poll
  50: <F as futures_core::future::TryFuture>::try_poll
  51: <futures_util::future::try_future::into_future::IntoFuture<Fut> as core::future::future::Future>::poll
  52: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll
  53: <futures_util::future::future::Map<Fut,F> as core::future::future::Future>::poll
  54: <futures_util::future::try_future::MapOk<Fut,F> as core::future::future::Future>::poll
  55: <F as futures_core::future::TryFuture>::try_poll
  56: <futures_util::future::try_future::try_flatten::TryFlatten<Fut,<Fut as futures_core::future::TryFuture>::Ok> as core::future::future::Future>::poll
  57: <futures_util::future::try_future::TryFlatten<Fut1,Fut2> as core::future::future::Future>::poll
  58: <futures_util::future::try_future::AndThen<Fut1,Fut2,F> as core::future::future::Future>::poll
  59: <core::pin::Pin<P> as core::future::future::Future>::poll
  60: sqlx_core::query::Query<DB,A>::fetch_one::{{closure}}
  61: <sea_orm::database::transaction::DatabaseTransaction as sea_orm::database::connection::ConnectionTrait>::query_one::{{closure}}::{{closure}}
  62: <sea_orm::database::transaction::DatabaseTransaction as sea_orm::database::connection::ConnectionTrait>::query_one::{{closure}}
  63: <core::pin::Pin<P> as core::future::future::Future>::poll
  64: sea_orm::executor::select::SelectorRaw<S>::one::{{closure}}
  65: sea_orm::executor::insert::exec_insert_with_returning::{{closure}}
  66: sea_orm::entity::active_model::ActiveModelTrait::insert::{{closure}}
  67: <core::pin::Pin<P> as core::future::future::Future>::poll
  68: siro_api::graphql::schema::teams::pending_team_project::PendingTeamProjectsMutation::accept_create_team_project::{{closure}}::{{closure}}::{{closure}}
  69: <core::pin::Pin<P> as core::future::future::Future>::poll
  70: sea_orm::database::transaction::DatabaseTransaction::run::{{closure}}::{{closure}}
  71: sea_orm::database::transaction::DatabaseTransaction::run::{{closure}}
  72: sea_orm::driver::sqlx_postgres::SqlxPostgresPoolConnection::transaction::{{closure}}::{{closure}}
  73: sea_orm::driver::sqlx_postgres::SqlxPostgresPoolConnection::transaction::{{closure}}
  74: <sea_orm::database::db_connection::DatabaseConnection as sea_orm::database::connection::TransactionTrait>::transaction::{{closure}}::{{closure}}
  75: <sea_orm::database::db_connection::DatabaseConnection as sea_orm::database::connection::TransactionTrait>::transaction::{{closure}}
  76: <core::pin::Pin<P> as core::future::future::Future>::poll
  77: siro_api::graphql::schema::teams::pending_team_project::PendingTeamProjectsMutation::accept_create_team_project::{{closure}}
  78: siro_api::graphql::schema::teams::pending_team_project::_::<impl siro_api::graphql::schema::teams::pending_team_project::PendingTeamProjectsMutation>::__accept_create_team_project_resolver::{{closure}}::{{closure}}
  79: siro_api::graphql::schema::teams::pending_team_project::_::<impl siro_api::graphql::schema::teams::pending_team_project::PendingTeamProjectsMutation>::__accept_create_team_project_resolver::{{closure}}
  80: siro_api::graphql::schema::teams::pending_team_project::_::<impl async_graphql::resolver_utils::container::ContainerType for siro_api::graphql::schema::teams::pending_team_project::PendingTeamProjectsMutation>::resolve_field::{{closure}}
  81: <&T as async_graphql::resolver_utils::container::ContainerType>::resolve_field::{{closure}}
  82: <async_graphql::types::merged_object::MergedObject<A,B> as async_graphql::resolver_utils::container::ContainerType>::resolve_field::{{closure}}
  83: <async_graphql::types::merged_object::MergedObject<A,B> as async_graphql::resolver_utils::container::ContainerType>::resolve_field::{{closure}}
  84: <async_graphql::types::merged_object::MergedObject<A,B> as async_graphql::resolver_utils::container::ContainerType>::resolve_field::{{closure}}
  85: <async_graphql::types::merged_object::MergedObject<A,B> as async_graphql::resolver_utils::container::ContainerType>::resolve_field::{{closure}}
  86: <async_graphql::types::merged_object::MergedObject<A,B> as async_graphql::resolver_utils::container::ContainerType>::resolve_field::{{closure}}
  87: <async_graphql::types::merged_object::MergedObject<A,B> as async_graphql::resolver_utils::container::ContainerType>::resolve_field::{{closure}}
  88: <async_graphql::types::merged_object::MergedObject<A,B> as async_graphql::resolver_utils::container::ContainerType>::resolve_field::{{closure}}
  89: <async_graphql::types::merged_object::MergedObject<A,B> as async_graphql::resolver_utils::container::ContainerType>::resolve_field::{{closure}}
  90: <async_graphql::types::merged_object::MergedObject<A,B> as async_graphql::resolver_utils::container::ContainerType>::resolve_field::{{closure}}
  91: <async_graphql::types::merged_object::MergedObject<A,B> as async_graphql::resolver_utils::container::ContainerType>::resolve_field::{{closure}}
  92: <async_graphql::types::merged_object::MergedObject<A,B> as async_graphql::resolver_utils::container::ContainerType>::resolve_field::{{closure}}
  93: <async_graphql::types::merged_object::MergedObject<A,B> as async_graphql::resolver_utils::container::ContainerType>::resolve_field::{{closure}}
  94: <async_graphql::types::merged_object::MergedObject<A,B> as async_graphql::resolver_utils::container::ContainerType>::resolve_field::{{closure}}
  95: <async_graphql::types::merged_object::MergedObject<A,B> as async_graphql::resolver_utils::container::ContainerType>::resolve_field::{{closure}}
  96: <async_graphql::types::merged_object::MergedObject<A,B> as async_graphql::resolver_utils::container::ContainerType>::resolve_field::{{closure}}
  97: <async_graphql::types::merged_object::MergedObject<A,B> as async_graphql::resolver_utils::container::ContainerType>::resolve_field::{{closure}}
  98: <async_graphql::types::merged_object::MergedObject<A,B> as async_graphql::resolver_utils::container::ContainerType>::resolve_field::{{closure}}
  99: <async_graphql::types::merged_object::MergedObject<A,B> as async_graphql::resolver_utils::container::ContainerType>::resolve_field::{{closure}}
 100: <async_graphql::types::merged_object::MergedObject<A,B> as async_graphql::resolver_utils::container::ContainerType>::resolve_field::{{closure}}
 101: <async_graphql::types::merged_object::MergedObject<A,B> as async_graphql::resolver_utils::container::ContainerType>::resolve_field::{{closure}}
 102: <async_graphql::types::merged_object::MergedObject<A,B> as async_graphql::resolver_utils::container::ContainerType>::resolve_field::{{closure}}
 103: <async_graphql::types::merged_object::MergedObject<A,B> as async_graphql::resolver_utils::container::ContainerType>::resolve_field::{{closure}}
 104: <async_graphql::types::merged_object::MergedObject<A,B> as async_graphql::resolver_utils::container::ContainerType>::resolve_field::{{closure}}
 105: <async_graphql::types::merged_object::MergedObject<A,B> as async_graphql::resolver_utils::container::ContainerType>::resolve_field::{{closure}}
 106: <async_graphql::types::merged_object::MergedObject<A,B> as async_graphql::resolver_utils::container::ContainerType>::resolve_field::{{closure}}
 107: <async_graphql::types::merged_object::MergedObject<A,B> as async_graphql::resolver_utils::container::ContainerType>::resolve_field::{{closure}}
 108: <async_graphql::types::merged_object::MergedObject<A,B> as async_graphql::resolver_utils::container::ContainerType>::resolve_field::{{closure}}
 109: <async_graphql::types::merged_object::MergedObject<A,B> as async_graphql::resolver_utils::container::ContainerType>::resolve_field::{{closure}}
 110: <async_graphql::types::merged_object::MergedObject<A,B> as async_graphql::resolver_utils::container::ContainerType>::resolve_field::{{closure}}
 111: <async_graphql::types::merged_object::MergedObject<A,B> as async_graphql::resolver_utils::container::ContainerType>::resolve_field::{{closure}}
 112: <async_graphql::types::merged_object::MergedObject<A,B> as async_graphql::resolver_utils::container::ContainerType>::resolve_field::{{closure}}
 113: <async_graphql::types::merged_object::MergedObject<A,B> as async_graphql::resolver_utils::container::ContainerType>::resolve_field::{{closure}}
 114: <async_graphql::types::merged_object::MergedObject<A,B> as async_graphql::resolver_utils::container::ContainerType>::resolve_field::{{closure}}
 115: <async_graphql::types::merged_object::MergedObject<A,B> as async_graphql::resolver_utils::container::ContainerType>::resolve_field::{{closure}}
 116: <async_graphql::types::merged_object::MergedObject<A,B> as async_graphql::resolver_utils::container::ContainerType>::resolve_field::{{closure}}
 117: <siro_api::graphql::schema::MutationRoot as async_graphql::resolver_utils::container::ContainerType>::resolve_field::{{closure}}
 118: <core::pin::Pin<P> as core::future::future::Future>::poll
 119: <&mut F as core::future::future::Future>::poll
 120: async_graphql::extensions::NextResolve::run::{{closure}}
 121: <async_graphql::extensions::apollo_tracing::ApolloTracingExtension as async_graphql::extensions::Extension>::resolve::{{closure}}
 122: <core::pin::Pin<P> as core::future::future::Future>::poll
 123: async_graphql::extensions::NextResolve::run::{{closure}}
 124: async_graphql::extensions::Extension::resolve::{{closure}}
 125: <core::pin::Pin<P> as core::future::future::Future>::poll
 126: async_graphql::extensions::NextResolve::run::{{closure}}
 127: async_graphql::extensions::Extension::resolve::{{closure}}
 128: <core::pin::Pin<P> as core::future::future::Future>::poll
 129: async_graphql::extensions::NextResolve::run::{{closure}}
 130: async_graphql::extensions::Extensions::resolve::{{closure}}
 131: async_graphql::resolver_utils::container::Fields::add_set::{{closure}}
 132: <core::pin::Pin<P> as core::future::future::Future>::poll
 133: async_graphql::resolver_utils::container::resolve_container_inner::{{closure}}
 134: async_graphql::resolver_utils::container::resolve_container_serial::{{closure}}
 135: async_graphql::schema::Schema<Query,Mutation,Subscription>::execute_once::{{closure}}
 136: async_graphql::schema::Schema<Query,Mutation,Subscription>::execute::{{closure}}::{{closure}}::{{closure}}::{{closure}}
 137: <core::pin::Pin<P> as core::future::future::Future>::poll
 138: async_graphql::extensions::NextExecute::internal_run::{{closure}}
 139: async_graphql::extensions::NextExecute::run::{{closure}}
 140: <async_graphql::extensions::apollo_tracing::ApolloTracingExtension as async_graphql::extensions::Extension>::execute::{{closure}}
 141: <core::pin::Pin<P> as core::future::future::Future>::poll
 142: async_graphql::extensions::NextExecute::internal_run::{{closure}}
 143: async_graphql::extensions::NextExecute::run::{{closure}}
 144: async_graphql::extensions::Extension::execute::{{closure}}
 145: <core::pin::Pin<P> as core::future::future::Future>::poll
 146: async_graphql::extensions::NextExecute::internal_run::{{closure}}
 147: async_graphql::extensions::NextExecute::run::{{closure}}
 148: async_graphql::extensions::Extension::execute::{{closure}}
 149: <core::pin::Pin<P> as core::future::future::Future>::poll
 150: async_graphql::extensions::NextExecute::internal_run::{{closure}}
 151: async_graphql::extensions::NextExecute::run::{{closure}}
 152: async_graphql::extensions::Extensions::execute::{{closure}}
 153: async_graphql::schema::Schema<Query,Mutation,Subscription>::execute::{{closure}}::{{closure}}
 154: <core::pin::Pin<P> as core::future::future::Future>::poll
 155: <&mut F as core::future::future::Future>::poll
 156: async_graphql::extensions::NextRequest::run::{{closure}}
 157: async_graphql::extensions::Extension::request::{{closure}}
 158: <core::pin::Pin<P> as core::future::future::Future>::poll
 159: async_graphql::extensions::NextRequest::run::{{closure}}
 160: <async_graphql::extensions::analyzer::AnalyzerExtension as async_graphql::extensions::Extension>::request::{{closure}}
 161: <core::pin::Pin<P> as core::future::future::Future>::poll
 162: async_graphql::extensions::NextRequest::run::{{closure}}
 163: async_graphql::extensions::Extension::request::{{closure}}
 164: <core::pin::Pin<P> as core::future::future::Future>::poll
 165: async_graphql::extensions::NextRequest::run::{{closure}}
 166: async_graphql::extensions::Extensions::request::{{closure}}
 167: async_graphql::schema::Schema<Query,Mutation,Subscription>::execute::{{closure}}
 168: siro_api::server::graphql_index::{{closure}}
 169: actix_web::handler::handler_service::{{closure}}::{{closure}}
 170: <core::pin::Pin<P> as core::future::future::Future>::poll
 171: <actix_web::resource::Resource<T> as actix_web::service::HttpServiceFactory>::register::{{closure}}::{{closure}}
 172: <core::pin::Pin<P> as core::future::future::Future>::poll
 173: <actix_cors::middleware::CorsMiddleware<S> as actix_service::Service<actix_web::service::ServiceRequest>>::call::{{closure}}
 174: <core::pin::Pin<P> as core::future::future::Future>::poll
 175: <opentelemetry::trace::context::WithContext<T> as core::future::future::Future>::poll
 176: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll
 177: <futures_util::future::future::Map<Fut,F> as core::future::future::Future>::poll
 178: <core::pin::Pin<P> as core::future::future::Future>::poll
 179: <actix_service::map_err::MapErrFuture<A,Req,F,E> as core::future::future::Future>::poll
 180: actix_http::h1::dispatcher::InnerDispatcher<T,S,B,X,U>::poll_response
 181: <actix_http::h1::dispatcher::Dispatcher<T,S,B,X,U> as core::future::future::Future>::poll
 182: <actix_http::service::HttpServiceHandlerResponse<T,S,B,X,U> as core::future::future::Future>::poll
 183: <actix_service::and_then::AndThenServiceResponse<A,B,Req> as core::future::future::Future>::poll
 184: <actix_server::service::StreamService<S,I> as actix_service::Service<(actix_server::worker::WorkerCounterGuard,actix_server::socket::MioStream)>>::call::{{closure}}
 185: tokio::runtime::task::core::Core<T,S>::poll::{{closure}}
 186: tokio::runtime::task::core::Core<T,S>::poll
 187: tokio::runtime::task::harness::poll_future::{{closure}}
 188: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
 189: std::panicking::try::do_call
 190: __rust_try
 191: std::panicking::try
 192: std::panic::catch_unwind
 193: tokio::runtime::task::harness::poll_future
 194: tokio::runtime::task::harness::Harness<T,S>::poll_inner
 195: tokio::runtime::task::harness::Harness<T,S>::poll
 196: tokio::runtime::task::raw::poll
 197: tokio::runtime::task::raw::RawTask::poll
 198: tokio::runtime::task::LocalNotified<S>::run
 199: tokio::task::local::LocalSet::tick::{{closure}}
 200: tokio::task::local::LocalSet::tick
 201: <tokio::task::local::RunUntil<T> as core::future::future::Future>::poll::{{closure}}
 202: tokio::task::local::LocalSet::with::{{closure}}
 203: std::thread::local::LocalKey<T>::try_with
 204: std::thread::local::LocalKey<T>::with
 205: tokio::task::local::LocalSet::with
 206: <tokio::task::local::RunUntil<T> as core::future::future::Future>::poll
 207: tokio::task::local::LocalSet::run_until::{{closure}}
 208: <core::pin::Pin<P> as core::future::future::Future>::poll
 209: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}::{{closure}}
 210: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}
 211: tokio::runtime::scheduler::current_thread::Context::enter
 212: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}
 213: tokio::runtime::scheduler::current_thread::CoreGuard::enter::{{closure}}
 214: tokio::runtime::context::scoped::Scoped<T>::set
 215: tokio::runtime::context::set_scheduler::{{closure}}
 216: std::thread::local::LocalKey<T>::try_with
 217: std::thread::local::LocalKey<T>::with
 218: tokio::runtime::context::set_scheduler
 219: tokio::runtime::scheduler::current_thread::CoreGuard::enter
 220: tokio::runtime::scheduler::current_thread::CoreGuard::block_on
 221: tokio::runtime::scheduler::current_thread::CurrentThread::block_on::{{closure}}
 222: tokio::runtime::context::runtime::enter_runtime
 223: tokio::runtime::scheduler::current_thread::CurrentThread::block_on
 224: tokio::runtime::runtime::Runtime::block_on
 225: tokio::task::local::LocalSet::block_on
 226: actix_rt::runtime::Runtime::block_on
 227: actix_rt::arbiter::Arbiter::with_tokio_rt::{{closure}}
 228: std::sys_common::backtrace::__rust_begin_short_backtrace
 229: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}
 230: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
 231: std::panicking::try::do_call
 232: __rust_try
 233: std::panicking::try
 234: std::thread::Builder::spawn_unchecked_::{{closure}}
 235: core::ops::function::FnOnce::call_once{{vtable.shim}}
 236: std::sys::pal::unix::thread::Thread::new::thread_start
 237: <unknown>
 238: <unknown>

//

I don't encounter stack overflow in --release, but that should not be the final solution.

Steps to Reproduce the Problem

I'm not quite sure exactly how to reproduce this.

Specifications

  • Version: 7.0.2
  • Platform: WSL 2 Arch Linux
  • Subsystem: N/A
  • Rust: 1.77
  • Features enabled: url, chrono, decimal, dataloader, playground, opentelemetry, apollo_tracing, email-validator, apollo_persisted_queries,
@negezor negezor added the bug Something isn't working label Mar 30, 2024
@ckruse
Copy link

ckruse commented Apr 3, 2024

I have a similar problem. I managed to get some queries to work with boxing a lot of the structs, but when the query gets more complex I still get the stack overflow (besides boxing the objects queried). So just boxing everything is not a solution...

@xamgore
Copy link
Contributor

xamgore commented Apr 4, 2024

  1. Try switching off the ApolloTracingExtension, it has a lot of downsides on the performance and response size.
  2. Also, I have concerns about the MergedObject, as it represents a list of schemas, which in turn could be optimized on the library level. As I see, you have a MutationRoot with 36 entities inside. Can you try splitting them into a tree of some kind? Like Merged Root: { L: [18], R: [18] } or even steeper. Does the stack size decreases?
  3. It would be nice to understand each stack frame's size. Check your own Extensions and field resolvers. Are there any long local arrays or fat enums clippy is warning about?
    Pin the future on the heap in tests to prevent stack overflows tokio-rs/tokio#5136 (comment)

@ckruse
Copy link

ckruse commented Apr 5, 2024

For people who have the same problem: I tried to solve this by trying to box things and had some success doing so. Simple queries and mutations work. But more complex queries still lead to a stack overflow, often not even in my code.

In release builds (cargo build --release) everything works for me, even for expensive queries.

For development and tests I just gave up and increased the stack size. Instead of using #[tokio::main] I now use the builder:

    #[cfg(not(debug_assertions))]
    let runtime = tokio::runtime::Builder::new_multi_thread()
        .enable_all()
        .build()
        .unwrap();

    #[cfg(debug_assertions)]
    let runtime = tokio::runtime::Builder::new_multi_thread()
        .enable_all()
        .thread_stack_size(3 * 1024 * 1024)
        .build()
        .unwrap();

    runtime.block_on(async {
        let listen_interface = std::env::var("LISTEN").unwrap_or_else(|_| "127.0.0.1:8000".to_owned());
        print!("\n🚀 GraphiQL IDE: http://{}/graphql\n\n", listen_interface);

        let listener = TcpListener::bind(listen_interface)
            .await
            .expect("could not create listener");

        axum::serve(
            listener,
            make_app(None, None, true)
                .await
                .into_make_service_with_connect_info::<SocketAddr>(),
        )
        .await
        .unwrap();
    });

Integration tests I run with RUST_MIN_STACK=3145728 cargo test --all.

This works for me.

Edit: as far as I understand it, this happens because async_trait moved the future via a Pin<Box<Future<...>>> to the heap and native async traits don't do that. And due to debug code containing a lot of unnecessary allocations this leads to a stack overflow in debug builds and works nonetheless in release builds.

@Sytten
Copy link

Sytten commented Apr 28, 2024

Related issue #1516

@Sytten
Copy link

Sytten commented Apr 29, 2024

I am going to work on a feature flag to revert the removal of async_trait since this was clearly a bad move for a lot of people (myself included).

@McleanWorkshield
Copy link

@Sytten

We are hitting the same issue. We also have a large merged object with pretty extensive input objects (with generics).

If it helps:

  1. I can reproduce the issue on everything down to async-graphql v7.0.0
  2. Using v6.0.11 works without issue.
  3. We can also get v7.0.3 to run by increasing the stack limit to 10mb. RUST_MIN_STACK=10485760 cargo run - it does not run at 9mb. Prob not ideal.

Keep us up to date if you attempt to work on this and I'd be happy to help!

@Sytten
Copy link

Sytten commented May 6, 2024

I started to work on it, but the issue really is that the crate trait-variant which should be now the default instead of async-trait is just not up to the task which makes it hard to support both usecases

@McleanWorkshield
Copy link

I started to work on it, but the issue really is that the crate trait-variant which should be now the default instead of async-trait is just not up to the task which makes it hard to support both usecases

Interesting. Thanks for the update here. Maybe an unintelligent question - would moving to something like Dynamic Schemas still produce the same issue?

@Sytten
Copy link

Sytten commented May 7, 2024

Yes since it uses the same traits

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

5 participants