fatal error: concurrent map read and map write

[yz89122]

What happened?

When I run pulumi up I get the following error:

(The origin output was too large, GitHub doesn't allow me to post that. So I paste only the running goroutines.)

Previewing update (*****):
Downloading plugin gcp v6.39.0: 44.87 MiB / 44.87 MiB [=============] 100.00% 3s
     Type                 Name                         Plan
     pulumi:pulumi:Stack  *****  running
fatal error: concurrent map read and map write
fatal error: concurrent map read and map write

goroutine 303 [running]:
github.com/pulumi/pulumi/pkg/v3/resource/deploy.(*resmon).RegisterResource(0xc002c5c000, {0xc001cf9858?, 0x1003da7f5?}, 0xc0013f5500)
	/private/tmp/pulumi-20221025-4619-1yxrsgo/pkg/resource/deploy/source_eval.go:972 +0x305
github.com/pulumi/pulumi/sdk/v3/proto/go._ResourceMonitor_RegisterResource_Handler.func1({0x10247b5f8, 0xc0018cd740}, {0x101e72a20?, 0xc0013f5500})
	/private/tmp/pulumi-20221025-4619-1yxrsgo/sdk/proto/go/resource.pb.go:1702 +0x78
github.com/grpc-ecosystem/grpc-opentracing/go/otgrpc.OpenTracingServerInterceptor.func1({0x10247b5f8, 0xc0018cd620}, {0x101e72a20, 0xc0013f5500}, 0xc0019d2f80, 0xc00048be00)
	/Users/brew/Library/Caches/Homebrew/go_mod_cache/pkg/mod/github.com/grpc-ecosystem/grpc-opentracing@v0.0.0-20180507213350-8e809c8a8645/go/otgrpc/server.go:57 +0x3f9
github.com/pulumi/pulumi/sdk/v3/proto/go._ResourceMonitor_RegisterResource_Handler({0x101da57c0?, 0xc002c5c000}, {0x10247b5f8, 0xc0018cd620}, 0xc0003111f0, 0xc000ad4260)
	/private/tmp/pulumi-20221025-4619-1yxrsgo/sdk/proto/go/resource.pb.go:1704 +0x138
google.golang.org/grpc.(*Server).processUnaryRPC(0xc002c5e000, {0x1024859a0, 0xc002016680}, 0xc001897e60, 0xc001143500, 0x103487440, 0x0)
	/Users/brew/Library/Caches/Homebrew/go_mod_cache/pkg/mod/google.golang.org/grpc@v1.49.0/server.go:1301 +0xb2b
google.golang.org/grpc.(*Server).handleStream(0xc002c5e000, {0x1024859a0, 0xc002016680}, 0xc001897e60, 0x0)
	/Users/brew/Library/Caches/Homebrew/go_mod_cache/pkg/mod/google.golang.org/grpc@v1.49.0/server.go:1642 +0xa2f
google.golang.org/grpc.(*Server).serveStreams.func1.2()
	/Users/brew/Library/Caches/Homebrew/go_mod_cache/pkg/mod/google.golang.org/grpc@v1.49.0/server.go:938 +0x98
created by google.golang.org/grpc.(*Server).serveStreams.func1
	/Users/brew/Library/Caches/Homebrew/go_mod_cache/pkg/mod/google.golang.org/grpc@v1.49.0/server.go:936 +0x28a

<truncated>

goroutine 328 [running]:
	goroutine running on other thread; stack unavailable
created by google.golang.org/grpc.(*Server).serveStreams.func1
	/Users/brew/Library/Caches/Homebrew/go_mod_cache/pkg/mod/google.golang.org/grpc@v1.49.0/server.go:936 +0x28a

<truncated>

goroutine 328 [running]:
github.com/pulumi/pulumi/pkg/v3/resource/deploy.(*resmon).RegisterResource(0xc002c5c000, {0xc001c73858?, 0x1003da7f5?}, 0xc000754c00)
	/private/tmp/pulumi-20221025-4619-1yxrsgo/pkg/resource/deploy/source_eval.go:972 +0x305
github.com/pulumi/pulumi/sdk/v3/proto/go._ResourceMonitor_RegisterResource_Handler.func1({0x10247b5f8, 0xc00189b950}, {0x101e72a20?, 0xc000754c00})
	/private/tmp/pulumi-20221025-4619-1yxrsgo/sdk/proto/go/resource.pb.go:1702 +0x78
github.com/grpc-ecosystem/grpc-opentracing/go/otgrpc.OpenTracingServerInterceptor.func1({0x10247b5f8, 0xc00189b830}, {0x101e72a20, 0xc000754c00}, 0xc001da0420, 0xc00187c4f8)
	/Users/brew/Library/Caches/Homebrew/go_mod_cache/pkg/mod/github.com/grpc-ecosystem/grpc-opentracing@v0.0.0-20180507213350-8e809c8a8645/go/otgrpc/server.go:57 +0x3f9
github.com/pulumi/pulumi/sdk/v3/proto/go._ResourceMonitor_RegisterResource_Handler({0x101da57c0?, 0xc002c5c000}, {0x10247b5f8, 0xc00189b830}, 0xc0004bc7e0, 0xc000ad4260)
	/private/tmp/pulumi-20221025-4619-1yxrsgo/sdk/proto/go/resource.pb.go:1704 +0x138
google.golang.org/grpc.(*Server).processUnaryRPC(0xc002c5e000, {0x1024859a0, 0xc002016680}, 0xc0018a0b40, 0xc001143500, 0x103487440, 0x0)
	/Users/brew/Library/Caches/Homebrew/go_mod_cache/pkg/mod/google.golang.org/grpc@v1.49.0/server.go:1301 +0xb2b
google.golang.org/grpc.(*Server).handleStream(0xc002c5e000, {0x1024859a0, 0xc002016680}, 0xc0018a0b40, 0x0)
	/Users/brew/Library/Caches/Homebrew/go_mod_cache/pkg/mod/google.golang.org/grpc@v1.49.0/server.go:1642 +0xa2f
google.golang.org/grpc.(*Server).serveStreams.func1.2()
	/Users/brew/Library/Caches/Homebrew/go_mod_cache/pkg/mod/google.golang.org/grpc@v1.49.0/server.go:938 +0x98
created by google.golang.org/grpc.(*Server).serveStreams.func1
	/Users/brew/Library/Caches/Homebrew/go_mod_cache/pkg/mod/google.golang.org/grpc@v1.49.0/server.go:936 +0x28a

another shot

fatal error: concurrent map writes

goroutine 165 [running]:
github.com/pulumi/pulumi/pkg/v3/resource/deploy.(*resmon).RegisterResource(0xc001ca5680, {0xc001e07858?, 0x1003da7f5?}, 0xc00120e180)
	/private/tmp/pulumi-20221025-4619-1yxrsgo/pkg/resource/deploy/source_eval.go:1204 +0x2b07
github.com/pulumi/pulumi/sdk/v3/proto/go._ResourceMonitor_RegisterResource_Handler.func1({0x10247b5f8, 0xc000f12960}, {0x101e72a20?, 0xc00120e180})
	/private/tmp/pulumi-20221025-4619-1yxrsgo/sdk/proto/go/resource.pb.go:1702 +0x78
github.com/grpc-ecosystem/grpc-opentracing/go/otgrpc.OpenTracingServerInterceptor.func1({0x10247b5f8, 0xc000f12840}, {0x101e72a20, 0xc00120e180}, 0xc000718b00, 0xc000203248)
	/Users/brew/Library/Caches/Homebrew/go_mod_cache/pkg/mod/github.com/grpc-ecosystem/grpc-opentracing@v0.0.0-20180507213350-8e809c8a8645/go/otgrpc/server.go:57 +0x3f9
github.com/pulumi/pulumi/sdk/v3/proto/go._ResourceMonitor_RegisterResource_Handler({0x101da57c0?, 0xc001ca5680}, {0x10247b5f8, 0xc000f12840}, 0xc000504a80, 0xc0002edca0)
	/private/tmp/pulumi-20221025-4619-1yxrsgo/sdk/proto/go/resource.pb.go:1704 +0x138
google.golang.org/grpc.(*Server).processUnaryRPC(0xc001ba65a0, {0x1024859a0, 0xc00106c340}, 0xc001c10240, 0xc001088660, 0x103487440, 0x0)
	/Users/brew/Library/Caches/Homebrew/go_mod_cache/pkg/mod/google.golang.org/grpc@v1.49.0/server.go:1301 +0xb2b
google.golang.org/grpc.(*Server).handleStream(0xc001ba65a0, {0x1024859a0, 0xc00106c340}, 0xc001c10240, 0x0)
	/Users/brew/Library/Caches/Homebrew/go_mod_cache/pkg/mod/google.golang.org/grpc@v1.49.0/server.go:1642 +0xa2f
google.golang.org/grpc.(*Server).serveStreams.func1.2()
	/Users/brew/Library/Caches/Homebrew/go_mod_cache/pkg/mod/google.golang.org/grpc@v1.49.0/server.go:938 +0x98
created by google.golang.org/grpc.(*Server).serveStreams.func1
	/Users/brew/Library/Caches/Homebrew/go_mod_cache/pkg/mod/google.golang.org/grpc@v1.49.0/server.go:936 +0x28a

Steps to reproduce

1. pulumi login xxx
2. pulumi --stack "yyy" up

Expected Behavior

No crash

Actual Behavior

Crash

Output of pulumi about

CLI          
Version      3.44.1
Go Version   go1.19.2
Go Compiler  gc

Plugins
NAME    VERSION
nodejs  unknown

Host     
OS       darwin
Version  12.6
Arch     x86_64

This project is written in nodejs: executable='/Users/*****/.nvm/versions/node/v17.1.0/bin/node' version='v17.1.0'

Current Stack: ******

TYPE                                                               URN
pulumi:pulumi:Stack                                                urn:pulumi:******::******::pulumi:pulumi:Stack::******
<about 2000 lines truncated>


Found no pending operations associated with *****

Backend        
Name           ******
URL            file://~
User           ******
Organizations  

Pulumi locates its logs in /var/folders/pl/n_7kfrtd67n85qq7q_t286k80000gn/T/ by default

Additional context

v3.43.1 is fine, this issue starts from v3.44.0

I've tried another run, combine those outputs, "I guess" the problem might occurred between these two lines.

pulumi/pkg/resource/deploy/source_eval.go

Line 972 in 4ca146e

if parentsProviders, parentIsComponent := rm.componentResourceProviders[parent]; !custom &&

pulumi/pkg/resource/deploy/source_eval.go

Line 1204 in 4ca146e

rm.componentResourceProviders[result.State.URN] = req.GetProviders()

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[Frassle]

Fixed by #11151, we'll be doing a patch release for 3.44.2 shortly with that included.