Sentry gets deadlocked during offline startup

[molind]

Platform

iOS

Environment

Production

Installed

Swift Package Manager

Version

8.25.0

Did it work on previous versions?

8.21.0

Steps to Reproduce

1. Switch device to the flight mode.
2. SentrySDK.start inside application(_: UIApplication, didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]? = nil)

It's not guaranteed to happen. But in crashlog we can follow to the cause.

Main thread inside the dispatch_once tries to access [SentryCrashWrapper systemInfo] and at the same time second thread locks [SentrySDK currentHub], calls [SentryCrashWrapper systemInfo] and waits for main thread to finish. But main thread can't finish because inside of that dispatch_once it's also tries to access [SentrySDK options].

-------------------------------------
Translated Report (Full Report Below)
-------------------------------------

Incident Identifier: 82C9C585-8649-42DA-A0F8-62C3D085EF43
CrashReporter Key:   3c8d5e561b90c78666cc1a6eb487bbdce220b9e7
Hardware Model:      iPhone14,2
Process:             Guru [3540]
Path:                /private/var/containers/Bundle/Application/D308A662-48B8-45D6-8F68-AD0817F04B3C/Guru.app/Guru
Identifier:          com.bodunov.galileo
Version:             5.5.6 (9247)
AppStoreTools:       15E204
AppVariant:          1:iPhone14,2:15
Code Type:           ARM-64 (Native)
Role:                Foreground
Parent Process:      launchd [1]
Coalition:           com.bodunov.galileo [1445]

Date/Time:           2024-05-08 13:39:25.2179 +0200
Launch Time:         2024-05-08 13:39:05.0456 +0200
OS Version:          iPhone OS 17.4.1 (21E236)
Release Type:        User
Baseband Version:    3.50.04
Report Version:      104

Exception Type:  EXC_CRASH (SIGKILL)
Exception Codes: 0x0000000000000000, 0x0000000000000000
Termination Reason: FRONTBOARD 2343432205 
<RBSTerminateContext| domain:10 code:0x8BADF00D explanation:scene-create watchdog transgression: app<com.bodunov.galileo(1455943A-E1D8-4E68-9A07-72D45E6103AB)>:3540 exhausted real (wall clock) time allowance of 19.89 seconds
ProcessVisibility: Foreground
ProcessState: Running
WatchdogEvent: scene-create
WatchdogVisibility: Foreground
WatchdogCPUStatistics: (
"Elapsed total CPU time (seconds): 13.360 (user 9.770, system 3.590), 11% CPU",
"Elapsed application CPU time (seconds): 0.029, 0% CPU"
) reportType:CrashLog maxTerminationResistance:Interactive>

Triggered by Thread:  0

Thread 0 name:  9247 Dispatch queue: com.apple.main-thread
Thread 0 Crashed:
0   libsystem_kernel.dylib        	       0x1de3aa93c __ulock_wait2 + 8
1   libsystem_platform.dylib      	       0x1f1d70ea8 _os_unfair_lock_lock_slow + 187
2   libobjc.A.dylib               	       0x18dd7b8b8 objc_sync_enter + 19
3   Guru                          	       0x102925c5c +[SentrySDK options] (in Guru) + 24 + 4086876
4   Guru                          	       0x102936268 -[SentryDependencyContainer crashReporter] (in Guru) + 88 + 4153960
5   Guru                          	       0x1028db0f8 __32-[SentryCrashWrapper systemInfo]_block_invoke (in Guru) + 40 + 3780856
6   libdispatch.dylib             	       0x19dd3edd4 _dispatch_client_callout + 19
7   libdispatch.dylib             	       0x19dd40654 _dispatch_once_callout + 31
8   Guru                          	       0x1028db0cc -[SentryCrashWrapper systemInfo] (in Guru) + 60 + 3780812
9   Guru                          	       0x1028db244 -[SentryCrashWrapper enrichScope:] (in Guru) + 164 + 3781188
10  Guru                          	       0x1028f67e4 -[SentryHub initWithClient:andScope:] (in Guru) + 732 + 3893220
11  Guru                          	       0x1029261a0 +[SentrySDK startWithOptions:] (in Guru) + 640 + 4088224
12  Guru                          	       0x102926614 +[SentrySDK startWithConfigureOptions:] (in Guru) + 88 + 4089364
13  Guru                          	       0x10259e71c specialized AppDelegate.application(_:didFinishLaunchingWithOptions:) (in Guru) (<compiler-generated>:0) + 386844
14  Guru                          	       0x10259ab1c @objc AppDelegate.application(_:didFinishLaunchingWithOptions:) (in Guru) (<compiler-generated>:123) + 371484
15  UIKitCore                     	       0x1982cdc70 -[UIApplication _handleDelegateCallbacksWithOptions:isSuspended:restoreState:] + 319
16  UIKitCore                     	       0x1982ccdd8 -[UIApplication _callInitializationDelegatesWithActions:forCanvas:payload:fromOriginatingProcess:] + 2855
17  UIKitCore                     	       0x1982cbdbc -[UIApplication _runWithMainScene:transitionContext:completion:] + 855
18  UIKitCore                     	       0x1982cba34 -[_UISceneLifecycleMultiplexer completeApplicationLaunchWithFBSScene:transitionContext:] + 175
19  UIKitCore                     	       0x1982cb964 -[UIApplication _compellApplicationLaunchToCompleteUnconditionally] + 47
20  UIKitCore                     	       0x1982e8eb8 -[UIApplication _run] + 851
21  UIKitCore                     	       0x1982e8518 UIApplicationMain + 339
22  Guru                          	       0x10259cb28 main (in Guru) (AppDelegate.swift:16) + 379688
23  dyld                          	       0x1b9396d84 start + 2239

Thread 1 name:   Dispatch queue: com.apple.NSURLSession-delegate
Thread 1:
0   libsystem_kernel.dylib        	       0x1de39e578 __ulock_wait + 8
1   libdispatch.dylib             	       0x19dd3f7c4 _dlock_wait + 55
2   libdispatch.dylib             	       0x19dd3f6f4 _dispatch_once_wait + 111
3   Guru                          	       0x1028db0cc -[SentryCrashWrapper systemInfo] (in Guru) + 60 + 3780812
4   Guru                          	       0x1028db244 -[SentryCrashWrapper enrichScope:] (in Guru) + 164 + 3781188
5   Guru                          	       0x1028f67e4 -[SentryHub initWithClient:andScope:] (in Guru) + 732 + 3893220
6   Guru                          	       0x102925bf4 +[SentrySDK currentHub] (in Guru) + 60 + 4086772
7   Guru                          	       0x10290ffc8 __68-[SentryRequestOperation initWithSession:request:completionHandler:]_block_invoke (in Guru) + 316 + 3997640
8   CFNetwork                     	       0x196f89248 0x196f89065 + 483
9   CFNetwork                     	       0x196fa7210 0x196fa7171 + 159
10  libdispatch.dylib             	       0x19dd3d13c _dispatch_call_block_and_release + 31
11  libdispatch.dylib             	       0x19dd3edd4 _dispatch_client_callout + 19
12  libdispatch.dylib             	       0x19dd46400 _dispatch_lane_serial_drain + 747
13  libdispatch.dylib             	       0x19dd46f64 _dispatch_lane_invoke + 431
14  libdispatch.dylib             	       0x19dd51cb4 _dispatch_root_queue_drain_deferred_wlh + 287
15  libdispatch.dylib             	       0x19dd51528 _dispatch_workloop_worker_thread + 403
16  libsystem_pthread.dylib       	       0x1f1e24f20 _pthread_wqthread + 287
17  libsystem_pthread.dylib       	       0x1f1e24fc0 start_wqthread + 7

Thread 2:
0   libsystem_pthread.dylib       	       0x1f1e24fb8 start_wqthread + 0

Thread 3 name:  com.apple.uikit.eventfetch-thread
Thread 3:
0   libsystem_kernel.dylib        	       0x1de39daf8 mach_msg2_trap + 8
1   libsystem_kernel.dylib        	       0x1de39d890 mach_msg2_internal + 79
2   libsystem_kernel.dylib        	       0x1de39d7a8 mach_msg_overwrite + 435
3   libsystem_kernel.dylib        	       0x1de39d5e8 mach_msg + 23
4   CoreFoundation                	       0x195e7801c __CFRunLoopServiceMachPort + 159
5   CoreFoundation                	       0x195e75f04 __CFRunLoopRun + 1207
6   CoreFoundation                	       0x195e75968 CFRunLoopRunSpecific + 607
7   Foundation                    	       0x194d044a8 -[NSRunLoop(NSRunLoop) runMode:beforeDate:] + 211
8   Foundation                    	       0x194d2e4e8 -[NSRunLoop(NSRunLoop) runUntilDate:] + 63
9   UIKitCore                     	       0x19824bac8 -[UIEventFetcher threadMain] + 419
10  Foundation                    	       0x194d75a9c __NSThread__start__ + 731
11  libsystem_pthread.dylib       	       0x1f1e25a90 _pthread_start + 135
12  libsystem_pthread.dylib       	       0x1f1e24fcc thread_start + 7


Thread 0 crashed with ARM Thread State (64-bit):
    x0: 0xfffffffffffffffc   x1: 0x0000000000000000   x2: 0x0000000000001902   x3: 0x0000000000000000
    x4: 0x0000000000000000   x5: 0x0000000000000900   x6: 0x0000000000000011   x7: 0x00000001f1c2a08c
    x8: 0x0000000000001902   x9: 0x0000000000001903  x10: 0x00000003038ac0a0  x11: 0x0000000000000001
   x12: 0x0000000300e8f990  x13: 0x0000000000000000  x14: 0x0000000000000000  x15: 0xffffffffffffffff
   x16: 0x0000000000000220  x17: 0x8000000301bc2124  x18: 0x0000000000000000  x19: 0x0000000000000103
   x20: 0x0000000000000000  x21: 0x0000000000000000  x22: 0x0000000106004998  x23: 0x0000000001000002
   x24: 0x0000000000001902  x25: 0x0000000000000000  x26: 0x00000000ffffffff  x27: 0x0000000000000103
   x28: 0x0000000102acd000   fp: 0x000000016d8bed00   lr: 0x00000001f1d70ea8
    sp: 0x000000016d8becb0   pc: 0x00000001de3aa93c cpsr: 0x40001000
   far: 0x0000000000000000  esr: 0x56000080  Address size fault

Expected Result

It should start fine

Actual Result

It's locked in deadlock, then app is killed by watchdog.

Are you willing to submit a PR?

I'll try. :)

[molind]

To reproduce it you can add sleep into the systemInfo. And there should be a cached envelope in client.

- (NSDictionary *)systemInfo
{
    static NSDictionary *sharedInfo = nil;
    static dispatch_once_t onceToken;
    dispatch_once(&onceToken, ^{
        sleep(5);
        sharedInfo = SentryDependencyContainer.sharedInstance.crashReporter.systemInfo;
    });
    return sharedInfo;
}

[brustolin]

Thanks for reaching out @molind. And thanks for the help with the PR.
I believe we can continue the rest of the conversation in the PR.

[philipphofmann]

This is related to #3899.