ConcurrentModificationException when querying /v3/api-docs/{group} concurrently for different groups.

[laech]

Describe the bug

After application startup, when querying /v3/api-docs/{group} concurrently for different groups, often an error is returned caused by:

java.util.ConcurrentModificationException: null
        at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1631)
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
        at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
        at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:693)
        at org.springdoc.core.GenericResponseService.getGenericMapResponse(GenericResponseService.java:636)
        at org.springdoc.core.GenericResponseService.build(GenericResponseService.java:144)
        at org.springdoc.api.AbstractOpenApiResource.calculatePath(AbstractOpenApiResource.java:460)
        at org.springdoc.api.AbstractOpenApiResource.calculatePath(AbstractOpenApiResource.java:614)
        at org.springdoc.webmvc.api.OpenApiResource.lambda$calculatePath$10(OpenApiResource.java:225)
        at java.base/java.util.Optional.ifPresent(Optional.java:178)
        at org.springdoc.webmvc.api.OpenApiResource.calculatePath(OpenApiResource.java:207)
        at org.springdoc.webmvc.api.OpenApiResource.lambda$getPaths$2(OpenApiResource.java:178)
        at java.base/java.util.Optional.ifPresent(Optional.java:178)
        at org.springdoc.webmvc.api.OpenApiResource.getPaths(OpenApiResource.java:157)
        at org.springdoc.api.AbstractOpenApiResource.getOpenApi(AbstractOpenApiResource.java:323)
        at org.springdoc.webmvc.api.OpenApiResource.openapiJson(OpenApiResource.java:132)
        at org.springdoc.webmvc.api.OpenApiWebMvcResource.openapiJson(OpenApiWebMvcResource.java:111)
        at org.springdoc.webmvc.api.MultipleOpenApiWebMvcResource.openapiJson(MultipleOpenApiWebMvcResource.java:86)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:568)
        at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
        at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:150)
        at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:117)
        at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:895)
        at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:808)
        at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
        at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1067)
        at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:963)
        at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
        at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:655)

After some digging, it seems the problem lies within GenericResponseService:

public class GenericResponseService {

  private List<ControllerAdviceInfo> controllerAdviceInfos = new ArrayList<>();

GenericResponseService is a singleton service, and controllerAdviceInfos is an ArrayList which is not thread safe, and it's being mutated at runtime by the service, therefore when concurrent requests come in, ConcurrentModificationException could occur.

The workaround that I currently have is to override buildGenericResponse and build (the two methods that read/write controllerAdviceInfos) to mark them as synchronized to avoid concurrent access, which is possibly not the optimal solution:

  @Bean
  GenericResponseService responseBuilder(
      OperationService operationService,
      List<ReturnTypeParser> returnTypeParsers,
      SpringDocConfigProperties springDocConfigProperties,
      PropertyResolverUtils propertyResolverUtils) {
    return new GenericResponseService(
        operationService, returnTypeParsers, springDocConfigProperties, propertyResolverUtils) {
      @Override
      public synchronized void buildGenericResponse(
          Components components, Map<String, Object> findControllerAdvice, Locale locale) {
        super.buildGenericResponse(components, findControllerAdvice, locale);
      }

      @Override
      public synchronized ApiResponses build(
          Components components,
          HandlerMethod handlerMethod,
          Operation operation,
          MethodAttributes methodAttributes) {
        return super.build(components, handlerMethod, operation, methodAttributes);
      }
    };
  }

To Reproduce
Steps to reproduce the behavior:

• What version of spring-boot you are using?
  2.6.7
• What modules and versions of springdoc-openapi are you using?
  spring-doc-openapi-ui 1.6.8
• What is the actual and the expected result using OpenAPI Description (yml or json)?
  When query /v3/api-docs/{group} concurrently for different groups, some requests are failing due to the above error.
• Provide with a sample code (HelloController) or Test that reproduces the problem
  Sorry have not had the time to do this, hopefully the description provides enough info.

Expected behavior

Able to query /v3/api-docs/{group} concurrently for different groups without error.

[bnasslahsen]

@laech,

Thank you for reporting this issue and your deeper analysis of the root cause.
I can confirm it's reproducible.
Feel free to test with the latest SNAPSHOT and provide your feedback.

[laech]

Thanks for such a quick turnaround, I've tried the snapshot and it has indeed fixed the issue.

[bianjp]

Meet the same problem. When will a new release version be available?

[bianjp]

Updated to 1.6.9 but the problem persisted:

Exception in thread "pool-3-thread-1" java.util.ConcurrentModificationException
  at java.util.ArrayList$ArrayListSpliterator.tryAdvance(ArrayList.java:1363)
  at java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:126)
  at java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:499)
  at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:486)
  at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
  at java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:230)
  at java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:196)
  at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
  at java.util.stream.ReferencePipeline.anyMatch(ReferencePipeline.java:516)
  at org.springdoc.api.AbstractOpenApiResource.isHiddenRestControllers(AbstractOpenApiResource.java:822)
  at org.springdoc.api.AbstractOpenApiResource.lambda$getOpenApi$3(AbstractOpenApiResource.java:309)
  at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:174)
  at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
  at java.util.HashMap$EntrySpliterator.forEachRemaining(HashMap.java:1723)
  at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
  at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
  at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
  at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
  at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:566)
  at org.springdoc.api.AbstractOpenApiResource.getOpenApi(AbstractOpenApiResource.java:310)
  at org.springdoc.api.AbstractOpenApiResource.getOpenApi(AbstractOpenApiResource.java:256)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  at java.lang.Thread.run(Thread.java:750)

It seems that Collections#synchronizedList isn't enough for thread-safety on iteration: https://stackoverflow.com/questions/25113987/why-is-there-a-concurrentmodificationexception-even-when-list-is-synchronized

[bianjp]

@bnasslahsen Can you please implement another fix for the problem?

I thought synchronizing on the list might be a possible solution. eg:

synchronized (HIDDEN_REST_CONTROLLERS) {
    return HIDDEN_REST_CONTROLLERS.stream().anyMatch(clazz -> clazz.isAssignableFrom(rawClass));
}

[bnasslahsen]

@bianjp,

Please feel free to propose a PR.