Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When grpc client is deployed in Kubernetes, what is the usecase of HealthIndicator (actuator/health) that reports the service as OUT_OF_SERVICE if any client has the TRANSIENT_FAILURE status? #1022

Open
matevskial opened this issue Dec 30, 2023 · 0 comments
Labels
question A question about this library or its usage

Comments

@matevskial
Copy link

matevskial commented Dec 30, 2023

The context

I have deployed microservice on Kubernetes that has configured liveness probe that uses the spring actuator's endpoint to check the health of microservice. If the livenesss probe returns status code greater than 400, then kubernetes will trigger a pod restart.

The question

What is the usecase of HealthIndicator (actuator/health) that reports the service as OUT_OF_SERVICE if any client has the TRANSIENT_FAILURE status in this context?

Let's say we want to execute a gRPC call with our client to some gRPC server and lets say the gRPC server is temporarily unavailable. In the current situation, the client won't be able to reach the server, and grpc client's internal Channel will report a TRANSIENT_FAILURE status to the used GrpcChannelFactory and thus make the client microservice restart when Kubernetes tests the liveness probe.

The solution of course is to disable health integration altogether either by:

  • not using actuator
  • configuring
management:
  health:
    grpcChannel:
      enabled: false
  • or excluding autoconfiguration for health interation\
@EnableAutoConfiguration(exclude = {GrpcClientHealthAutoConfiguration.class})

Then the question remains, what is the point of this? I just lost my whole day investigating why the microservice was constantly restarting when the microservice tries to call temporarily unavailable grpc server???
Stacktraces and logs

No stactraces and logs are needed.

The application's environment

Which versions do you use?

  • Spring (boot): 3.2.0
  • grpc-java: 1.58.0
  • grpc-spring-boot-starter: 2.15.0.RELEASE
  • java: 19 64bit
  • Other relevant libraries...

Additional information

No additional information needed.

@matevskial matevskial added the question A question about this library or its usage label Dec 30, 2023
@matevskial matevskial changed the title When grpc client is deployed in Kubernetes, what is the usecase of HealthIndicator (actuator/health) that reports the service as OUT_OF_SERVICE if any client has the TRANSIENT_FAILURE status When grpc client is deployed in Kubernetes, what is the usecase of HealthIndicator (actuator/health) that reports the service as OUT_OF_SERVICE if any client has the TRANSIENT_FAILURE status? Dec 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question A question about this library or its usage
Projects
None yet
Development

No branches or pull requests

1 participant