KNative Queue-Proxy memory limits.

[msgurikar]

Hi,
Do Queue-Proxy memory requests and limits needs to be increased in service yaml, if my application containers accepts large paylod(300MB).?

annotations:
queue.sidecar.serving.knative.dev/cpu-resource-request: "1"
queue.sidecar.serving.knative.dev/cpu-resource-limit: "2"
queue.sidecar.serving.knative.dev/memory-resource-request: "1Gi"
queue.sidecar.serving.knative.dev/memory-resource-limit: "2Gi"

[ReToCode]

There is no general answer to scaling, so you need to test with your setup. Also it is highly dependant on your backend startup times and latencies.

But based on that really large request size, you probably want to increase these resources and also take a good look at the resource limits + scaling of the activator and ingress-gateway (kourier, contour or istio) components.

[msgurikar]

Thank you for your reply.
You are suggesting to increase resource limits of queue-proxy, activator and ingress-gateway?

Also, when i send large number of requests around 80 with this each payload size of 300MB and my maxScale is 80, Activator does not increase the count to 80, sometimes it manages with 50. How do i make all 80 reqeusts reach to activator to scale pods to 80.

Thank you.

[ReToCode]

You are suggesting to increase resource limits of queue-proxy, activator and ingress-gateway?

Yes probably. But as said, you need to test with your expected workload to find the best settings.

Also, when i send large number of requests around 80 with this each payload size of 300MB and my maxScale is 80, Activator does not increase the count to 80, sometimes it manages with 50. How do i make all 80 reqeusts reach to activator to scale pods to 80.

It might be that 50 pods can handle you sending 80 requests while only sending one request at a time to a backend. It depends a bit on how fast your backends respond. Also scaling takes some time. So make sure you run your tests continuoulsy.

[msgurikar]

Ok. Right. I have been running for several days, i see that when we have small paylod request(<50MB) with 80 requests going out, if network speed is high, they reach KNative quickly and activator takes little time to send scale up requests, but it works once they are up.

But when i have large paylod(>200Mb), network is slow to send requests to Knative and most of the time i recieve timeout error or OS error
"logger":"activator","caller":"network/error_handler.go:33","message":"error reverse proxying request; sockstat: sockets: used 89\nTCP: inuse 80 orphan 0 tw 18 alloc 305 mem 236\nUDP: inuse 0 mem 1\nUDPLITE: inuse 0\nRAW: inuse 0\nFRAG: inuse 0 memory 0\n","commit":"f60eb32","knative.dev/controller":"activator","knative.dev/pod":"activator-887865954-7w8r9","knative.dev/key":"testsvc/test-service-latest","error":"context canceled","stacktrace":"knative.dev/pkg/network.ErrorHandler.func1\n\tknative.dev/pkg@v0.0.0-20230718152110-aef227e72ead/network/error_handler.go:33\nknative.dev/serving/pkg/activator/handler.(*activationHandler).proxyRequest.func1\n\tknative.dev/serving/pkg/activator/handler/handler.go:140\nnet/http/httputil.(*ReverseProxy).ServeHTTP\n\tnet/http/httputil/reverseproxy.go:475\nknative.dev/serving/pkg/activator/handler.(*activationHandler).proxyRequest\n\tknative.dev/serving/pkg/activator/handler/handler.go:143\nknative.dev/serving/pkg/activator/handler.(*activationHandler).ServeHTTP.func1\n\tknative.dev/serving/pkg/activator/handler/handler.go:96\nknative.dev/serving/pkg/activator/net.(*revisionThrottler).try.func1\n\tknative.dev/serving/pkg/activator/net/throttler.go:236\nknative.dev/serving/pkg/queue.(*Breaker).Maybe\n\tknative.dev/serving/pkg/queue/breaker.go:155\nknative.dev/serving/pkg/activator/net.(*revisionThrottler).try\n\tknative.dev/serving/pkg/activator/net/throttler.go:226\nknative.dev/serving/pkg/activator/net.(*Throttler).Try\n\tknative.dev/serving/pkg/activator/net/throttler.go:521\nknative.dev/serving/pkg/activator/handler.(*activationHandler).ServeHTTP\n\tknative.dev/serving/pkg/activator/handler/handler.go:89\nknative.dev/serving/pkg/http/handler.(*timeoutHandler).ServeHTTP.func4\n\tknative.dev/serving/pkg/http/handler/timeout.go:118"}

Not sure how do i fix this.

I have already increased queue-proxy resources, it didnt help. Does increasing cpu/memory resource of activator and ingress-gateway wiill help?

It might be that 50 pods can handle you sending 80 requests while only sending one request at a time to a backend. It depends a bit on how fast your backends respond. Also scaling takes some time. So make sure you run your tests continuoulsy.
=> I totally agree on this, this is happening clearly in my case. I am running tests continuously to understand it better. For now, i feel like network speed is bottleneck, also its not consistent, sometimes its faster and slower.