Consuming from multiple queues using different listeners

[alogvinov-forter]

Hi folks,

Our Spring Boot application needs to consume from two different queues, and the messages are different. It was working fine with one method, but when I added the second one, I got a weird error:

<#db707541> j.l.IllegalStateException: executor not accepting a task
...
	at i.n.u.c.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470)
	at i.n.c.n.NioEventLoop.run(NioEventLoop.java:569)
	at i.n.u.c.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
	at i.n.u.i.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	... 1 common frames omitted
Wrapped by: <#1e440384> s.a.a.c.e.SdkClientException: Unable to execute HTTP request: executor not accepting a task
	at s.a.a.c.e.SdkClientException$BuilderImpl.build(SdkClientException.java:111)
	at s.a.a.c.e.SdkClientException.create(SdkClientException.java:47)
	at s.a.a.c.i.h.p.s.u.RetryableStageHelper.setLastException(RetryableStageHelper.java:223)
	at s.a.a.c.i.h.p.s.u.RetryableStageHelper.setLastException(RetryableStage...

Unfortunately, I can't easily track it to the root cause as part of the exception was cut off by filebeat.

Both listeners are in the same class, separate methods annotated with @SqsListener. Is there any configuration I need to supply to make it work?

Thanks in advance 🙏

[maciejwalkowiak]

It does look strange indeed. Are you able to prepare sample repository that reproduces this issue?

[alogvinov-forter]

I tried to reproduce it locally, but everything seems to be working fine - I'm able to consume from both queues without any problems. Will check this again next week and get back to you with more details, thanks!

[tomazfernandes]

Wrapped by: <#1e440384> s.a.a.c.e.SdkClientException: Unable to execute HTTP request: executor not accepting a task

From this part of the stack trace it seems the Executor being used for the SqsAsyncClient is being closed while there are still messages being processed.

Spring Cloud AWS does not interfere with the client lifecycle other than declaring it as a Spring Bean, which ties it to the Spring Context lifecycle.

Here are some questions to help your troubleshooting:

• Are you declaring a custom Executor for you client?
• Are you declaring a custom client rather than the auto configured one?
• Might your application be shutting down for some reason?

Let me know if that makes sense to you.

[alogvinov-forter]

Thanks for your reply! No custom executor or async client, everything is auto-configured. I'll attempt to run it again next week and try to get more logs.

[alogvinov-forter]

I've tested this setup locally and wasn't able to reproduce the issue - messages were consumed from both queues, as expected.

After deploying the change one more time, I figured out the problem was that the service was lacking AWS permissions for one of the queues.