Crash in grpc c++ lib when trying to query grpc::SecureAuthContext::FindPropertyValues( "XYZ" )

[mudhireddy]

What version of gRPC and what language are you using?

grpc 1.60.0

What operating system (Linux, Windows,...) and version?

x86_64 GNU/Linux

What runtime / compiler are you using (e.g. python version or version of gcc)

gcc version 11.3.1 20221121 (Red Hat 11.3.1-4) (GCC)

What did you do?

Please provide either 1) A unit test for reproducing the bug or 2) Specific steps for us to follow to reproduce the bug. If there’s not enough information to debug the problem, gRPC team may close the issue at their discretion. You’re welcome to re-open the issue once you have a reproduction.

grpc/include/grpcpp/security/auth_metadata_processor.h

Line 66 in 4e735be

grpc::AuthContext* context,

As part of handling the "Process" callback referenced above, following call to find a property value in the context is crashing on rare occasions. It works most of the time and interestingly crash is seen right after the system restart when a grpc client is trying to connect.

std::vectorgrpc::string_ref property = context->FindPropertyValues( "xyz" )

basically this is the assert that is getting hit

grpc/src/core/lib/security/context/security_context.cc

Line 196 in da04df7

GPR_ASSERT(prop->name != nullptr);

5668 security_context.cc:196] ASSERTION FAILED: prop->name != nullptr

What did you expect to see?

expect to not crash

What did you see instead?

A crash with the following backtrace

#0 0x00007fa6c91ca1d6 in __pthread_kill_implementation () from /lib64/libc.so.6
#1 0x00007fa6c917b061 in raise () from /lib64/libc.so.6
#2 0x00007fa6c914e83b in abort () from /lib64/libc.so.6
#3 0x00007fa6a3eb5a3b in grpc_core::Crash(std::basic_string_view<char, std::char_traits >, grpc_core::SourceLocation) () from /lib64/libgpr.so.37
#4 0x00007fa6a3eb35c7 in gpr_assertion_failed () from /lib64/libgpr.so.37
#5 0x00007fa6a44e8068 in grpc_auth_property_iterator_next () from /lib64/libgrpc.so.37
#6 0x00007fa6a4fb4fdd in grpc::SecureAuthContext::FindPropertyValues(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&) const () from /lib64/libgrpc++.so.1.60
#7 0x00007fa6a51a01ff in GrpcServer::CustomAuthenticationProcessor::Process (this=0x386d, auth_metadata=<error reading variable: Cannot access memory at address 0x2e>, context=0x7fa67dffa4c0)

Make sure you include information that can help us debug (full error message, exception listing, stack trace, logs).

See TROUBLESHOOTING.md for how to diagnose problems better.

Anything else we should know about your project / environment?