accept_tcp_from_lwip error: queue overrun; shouldn't happen with lwIP listen backlog #1998
-
|
under what other conditions, beside high cpu load/usage/throttle can this happen |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 3 replies
-
|
do you have a fuller trace or sample code we can look at? |
Beta Was this translation helpful? Give feedback.
-
|
I'll try to reproduce it, but atm don't have more info other than that log from production systems. I'm just trying to narrow it down and understand so I can focus on that area. |
Beta Was this translation helpful? Give feedback.
-
|
Did you get that error message from an instance with multiple vCPUs? I can see in the code that this could happen when there are multiple concurrent connection requests that are processed in parallel by different vCPUs; it's a flaw in the code that we should be able to fix by reworking the logic that protects a listening socket from concurrent access to its struct fields. |
Beta Was this translation helpful? Give feedback.
-
|
In this case there is no smp involved. The service is deployed on gcp f1-micro and single cpu is available/exposed from nanos perspective. What happened there was a high cpu usage that lasted for a long time and being a shared cpu with 0.2 cpu thread available (burstable to 0.4 for few seconds) things start to fall apart. I did expect troubles under such conditions, but nanos had no other major issues handling that. |
Beta Was this translation helpful? Give feedback.
Did you get that error message from an instance with multiple vCPUs? I can see in the code that this could happen when there are multiple concurrent connection requests that are processed in parallel by different vCPUs; it's a flaw in the code that we should be able to fix by reworking the logic that protects a listening socket from concurrent access to its struct fields.
Assuming you are using multi-vCPU instances, basically this could happen when your server has a TCP listen backlog close to the maximum supported value (255) and there is a high rate of client connection requests.