Backport 1.9: Address a data race issue within identity_store_util::processLocalAlias (#13093) #13094
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…as (#13093) - When loading an existing alias within processLocalAlias we aren't cloning the object from the memory store. There seems to be a data race within the function when calling entity.UpsertAlias and a concurrent invalidation routine. ================== WARNING: DATA RACE Read at 0x00c00bd03d08 by goroutine 94: google.golang.org/protobuf/internal/impl.pointer.Elem() /go/pkg/mod/google.golang.org/protobuf@v1.27.1/internal/impl/pointer_unsafe.go:118 +0x2b3 google.golang.org/protobuf/internal/impl.(*MessageInfo).sizePointerSlow() /go/pkg/mod/google.golang.org/protobuf@v1.27.1/internal/impl/encode.go:76 +0x265 google.golang.org/protobuf/internal/impl.(*MessageInfo).sizePointer() /go/pkg/mod/google.golang.org/protobuf@v1.27.1/internal/impl/encode.go:56 +0x12a google.golang.org/protobuf/internal/impl.(*MessageInfo).size() /go/pkg/mod/google.golang.org/protobuf@v1.27.1/internal/impl/encode.go:40 +0x95 google.golang.org/protobuf/internal/impl.(*MessageInfo).size-fm() /go/pkg/mod/google.golang.org/protobuf@v1.27.1/internal/impl/encode.go:33 +0x6c google.golang.org/protobuf/proto.MarshalOptions.marshal() /go/pkg/mod/google.golang.org/protobuf@v1.27.1/proto/encode.go:153 +0x1f3 google.golang.org/protobuf/proto.MarshalOptions.MarshalAppend() /go/pkg/mod/google.golang.org/protobuf@v1.27.1/proto/encode.go:122 +0xa5 github.com/golang/protobuf/proto.marshalAppend() /go/pkg/mod/github.com/golang/protobuf@v1.5.2/proto/wire.go:40 +0xe4 github.com/golang/protobuf/proto.Marshal() /go/pkg/mod/github.com/golang/protobuf@v1.5.2/proto/wire.go:23 +0x64 github.com/hashicorp/vault/helper/identity.(*Entity).Clone() /go/src/github.com/hashicorp/vault/helper/identity/identity.go:34 +0x150 github.com/hashicorp/vault/vault.(*IdentityStore).MemDBEntitiesByBucketKeyInTxn() /go/src/github.com/hashicorp/vault/vault/identity_store_util.go:1214 +0x306 github.com/hashicorp/vault/vault.(*IdentityStore).Invalidate() /go/src/github.com/hashicorp/vault/vault/identity_store.go:216 +0xd6c github.com/hashicorp/vault/vault.(*IdentityStore).Invalidate-fm() /go/src/github.com/hashicorp/vault/vault/identity_store.go:160 +0x6d github.com/hashicorp/vault/sdk/framework.(*Backend).InvalidateKey() /go/src/github.com/hashicorp/vault/sdk/framework/backend.go:347 +0x8a github.com/hashicorp/vault/vault.(*IdentityStore).InvalidateKey() <autogenerated>:1 +0x7d github.com/hashicorp/vault/vault.(*Core).asyncInvalidateKey() /go/src/github.com/hashicorp/vault/vault/replication_invalidation_ent.go:58 +0x390 github.com/hashicorp/vault/vault.(*Core).asyncInvalidateHandler() /go/src/github.com/hashicorp/vault/vault/replication_invalidation_ent.go:71 +0x9b github.com/hashicorp/vault/vault.startReplicationEnt·dwrap·453() /go/src/github.com/hashicorp/vault/vault/replication_util_ent.go:331 +0x71 Previous write at 0x00c00bd03d08 by goroutine 52: github.com/hashicorp/vault/helper/identity.(*Entity).UpsertAlias() /go/src/github.com/hashicorp/vault/helper/identity/identity.go:55 +0x271 github.com/hashicorp/vault/vault.(*IdentityStore).processLocalAlias() /go/src/github.com/hashicorp/vault/vault/identity_store_util.go:720 +0x672 github.com/hashicorp/vault/vault.possiblyForwardEntityCreation() /go/src/github.com/hashicorp/vault/vault/request_handling_util_ent.go:230 +0x286 github.com/hashicorp/vault/vault.(*Core).handleLoginRequest() /go/src/github.com/hashicorp/vault/vault/request_handling.go:1345 +0x234a github.com/hashicorp/vault/vault.(*Core).handleCancelableRequest() /go/src/github.com/hashicorp/vault/vault/request_handling.go:607 +0x1a11 github.com/hashicorp/vault/vault.(*Core).switchedLockHandleRequest() /go/src/github.com/hashicorp/vault/vault/request_handling.go:442 +0x5b5 github.com/hashicorp/vault/vault.(*Core).HandleRequest() /go/src/github.com/hashicorp/vault/vault/request_handling.go:408 +0xf2 github.com/hashicorp/vault/http.request() /go/src/github.com/hashicorp/vault/http/handler.go:953 +0xb1 github.com/hashicorp/vault/http.handleLogicalInternal.func1() /go/src/github.com/hashicorp/vault/http/logical.go:341 +0xca net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d github.com/hashicorp/vault/http.handleRequestForwarding.func1() /go/src/github.com/hashicorp/vault/http/handler.go:887 +0x4eb net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d net/http.(*ServeMux).ServeHTTP() /usr/local/go/src/net/http/server.go:2424 +0xc5 github.com/hashicorp/vault/http.wrapHelpHandler.func1() /go/src/github.com/hashicorp/vault/http/help.go:23 +0x281 net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d github.com/hashicorp/vault/http.wrapCORSHandler.func1() /go/src/github.com/hashicorp/vault/http/cors.go:29 +0xb0e net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d github.com/hashicorp/vault/http.rateLimitQuotaWrapping.func1() /go/src/github.com/hashicorp/vault/http/util.go:97 +0xf28 net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d github.com/hashicorp/vault/http.wrapDRSecondaryHandler.func1() /go/src/github.com/hashicorp/vault/http/util_ent.go:81 +0x7e3 net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d github.com/hashicorp/vault/http.wrapGenericHandler.func1() /go/src/github.com/hashicorp/vault/http/handler.go:465 +0x1843 net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d github.com/hashicorp/go-cleanhttp.PrintablePathCheckHandler.func1() /go/pkg/mod/github.com/hashicorp/go-cleanhttp@v0.5.2/handlers.go:42 +0xc1 net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d net/http.serverHandler.ServeHTTP() /usr/local/go/src/net/http/server.go:2878 +0x89a net/http.initALPNRequest.ServeHTTP() /usr/local/go/src/net/http/server.go:3479 +0x34d net/http.(*initALPNRequest).ServeHTTP() <autogenerated>:1 +0x8f net/http.Handler.ServeHTTP-fm() /usr/local/go/src/net/http/server.go:87 +0x75 net/http.(*http2serverConn).runHandler() /usr/local/go/src/net/http/h2_bundle.go:5832 +0xdd net/http.(*http2serverConn).processHeaders·dwrap·31() /usr/local/go/src/net/http/h2_bundle.go:5562 +0x64
ncabatoff
approved these changes
Nov 9, 2021
vishalnayak
approved these changes
Nov 9, 2021
mlevitt
pushed a commit
to mlevitt/vault
that referenced
this pull request
Dec 23, 2021
…rocessLocalAlias (hashicorp#13093) (hashicorp#13094) * Address a data race issue within identity_store_util::processLocalAlias (hashicorp#13093) - When loading an existing alias within processLocalAlias we aren't cloning the object from the memory store. There seems to be a data race within the function when calling entity.UpsertAlias and a concurrent invalidation routine. ================== WARNING: DATA RACE Read at 0x00c00bd03d08 by goroutine 94: google.golang.org/protobuf/internal/impl.pointer.Elem() /go/pkg/mod/google.golang.org/protobuf@v1.27.1/internal/impl/pointer_unsafe.go:118 +0x2b3 google.golang.org/protobuf/internal/impl.(*MessageInfo).sizePointerSlow() /go/pkg/mod/google.golang.org/protobuf@v1.27.1/internal/impl/encode.go:76 +0x265 google.golang.org/protobuf/internal/impl.(*MessageInfo).sizePointer() /go/pkg/mod/google.golang.org/protobuf@v1.27.1/internal/impl/encode.go:56 +0x12a google.golang.org/protobuf/internal/impl.(*MessageInfo).size() /go/pkg/mod/google.golang.org/protobuf@v1.27.1/internal/impl/encode.go:40 +0x95 google.golang.org/protobuf/internal/impl.(*MessageInfo).size-fm() /go/pkg/mod/google.golang.org/protobuf@v1.27.1/internal/impl/encode.go:33 +0x6c google.golang.org/protobuf/proto.MarshalOptions.marshal() /go/pkg/mod/google.golang.org/protobuf@v1.27.1/proto/encode.go:153 +0x1f3 google.golang.org/protobuf/proto.MarshalOptions.MarshalAppend() /go/pkg/mod/google.golang.org/protobuf@v1.27.1/proto/encode.go:122 +0xa5 github.com/golang/protobuf/proto.marshalAppend() /go/pkg/mod/github.com/golang/protobuf@v1.5.2/proto/wire.go:40 +0xe4 github.com/golang/protobuf/proto.Marshal() /go/pkg/mod/github.com/golang/protobuf@v1.5.2/proto/wire.go:23 +0x64 github.com/hashicorp/vault/helper/identity.(*Entity).Clone() /go/src/github.com/hashicorp/vault/helper/identity/identity.go:34 +0x150 github.com/hashicorp/vault/vault.(*IdentityStore).MemDBEntitiesByBucketKeyInTxn() /go/src/github.com/hashicorp/vault/vault/identity_store_util.go:1214 +0x306 github.com/hashicorp/vault/vault.(*IdentityStore).Invalidate() /go/src/github.com/hashicorp/vault/vault/identity_store.go:216 +0xd6c github.com/hashicorp/vault/vault.(*IdentityStore).Invalidate-fm() /go/src/github.com/hashicorp/vault/vault/identity_store.go:160 +0x6d github.com/hashicorp/vault/sdk/framework.(*Backend).InvalidateKey() /go/src/github.com/hashicorp/vault/sdk/framework/backend.go:347 +0x8a github.com/hashicorp/vault/vault.(*IdentityStore).InvalidateKey() <autogenerated>:1 +0x7d github.com/hashicorp/vault/vault.(*Core).asyncInvalidateKey() /go/src/github.com/hashicorp/vault/vault/replication_invalidation_ent.go:58 +0x390 github.com/hashicorp/vault/vault.(*Core).asyncInvalidateHandler() /go/src/github.com/hashicorp/vault/vault/replication_invalidation_ent.go:71 +0x9b github.com/hashicorp/vault/vault.startReplicationEnt·dwrap·453() /go/src/github.com/hashicorp/vault/vault/replication_util_ent.go:331 +0x71 Previous write at 0x00c00bd03d08 by goroutine 52: github.com/hashicorp/vault/helper/identity.(*Entity).UpsertAlias() /go/src/github.com/hashicorp/vault/helper/identity/identity.go:55 +0x271 github.com/hashicorp/vault/vault.(*IdentityStore).processLocalAlias() /go/src/github.com/hashicorp/vault/vault/identity_store_util.go:720 +0x672 github.com/hashicorp/vault/vault.possiblyForwardEntityCreation() /go/src/github.com/hashicorp/vault/vault/request_handling_util_ent.go:230 +0x286 github.com/hashicorp/vault/vault.(*Core).handleLoginRequest() /go/src/github.com/hashicorp/vault/vault/request_handling.go:1345 +0x234a github.com/hashicorp/vault/vault.(*Core).handleCancelableRequest() /go/src/github.com/hashicorp/vault/vault/request_handling.go:607 +0x1a11 github.com/hashicorp/vault/vault.(*Core).switchedLockHandleRequest() /go/src/github.com/hashicorp/vault/vault/request_handling.go:442 +0x5b5 github.com/hashicorp/vault/vault.(*Core).HandleRequest() /go/src/github.com/hashicorp/vault/vault/request_handling.go:408 +0xf2 github.com/hashicorp/vault/http.request() /go/src/github.com/hashicorp/vault/http/handler.go:953 +0xb1 github.com/hashicorp/vault/http.handleLogicalInternal.func1() /go/src/github.com/hashicorp/vault/http/logical.go:341 +0xca net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d github.com/hashicorp/vault/http.handleRequestForwarding.func1() /go/src/github.com/hashicorp/vault/http/handler.go:887 +0x4eb net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d net/http.(*ServeMux).ServeHTTP() /usr/local/go/src/net/http/server.go:2424 +0xc5 github.com/hashicorp/vault/http.wrapHelpHandler.func1() /go/src/github.com/hashicorp/vault/http/help.go:23 +0x281 net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d github.com/hashicorp/vault/http.wrapCORSHandler.func1() /go/src/github.com/hashicorp/vault/http/cors.go:29 +0xb0e net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d github.com/hashicorp/vault/http.rateLimitQuotaWrapping.func1() /go/src/github.com/hashicorp/vault/http/util.go:97 +0xf28 net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d github.com/hashicorp/vault/http.wrapDRSecondaryHandler.func1() /go/src/github.com/hashicorp/vault/http/util_ent.go:81 +0x7e3 net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d github.com/hashicorp/vault/http.wrapGenericHandler.func1() /go/src/github.com/hashicorp/vault/http/handler.go:465 +0x1843 net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d github.com/hashicorp/go-cleanhttp.PrintablePathCheckHandler.func1() /go/pkg/mod/github.com/hashicorp/go-cleanhttp@v0.5.2/handlers.go:42 +0xc1 net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d net/http.serverHandler.ServeHTTP() /usr/local/go/src/net/http/server.go:2878 +0x89a net/http.initALPNRequest.ServeHTTP() /usr/local/go/src/net/http/server.go:3479 +0x34d net/http.(*initALPNRequest).ServeHTTP() <autogenerated>:1 +0x8f net/http.Handler.ServeHTTP-fm() /usr/local/go/src/net/http/server.go:87 +0x75 net/http.(*http2serverConn).runHandler() /usr/local/go/src/net/http/h2_bundle.go:5832 +0xdd net/http.(*http2serverConn).processHeaders·dwrap·31() /usr/local/go/src/net/http/h2_bundle.go:5562 +0x64 * Add missing changelog for pr hashicorp#13093 (hashicorp#13095)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport PR of #13093
cloning the object from the memory store. There seems to be a data
race within the function when calling entity.UpsertAlias and
a concurrent invalidation routine.