RPS is not increasing as the number of users

[tengerye]

Prerequisites

• I am using the latest version of Locust
• I am reporting a bug, not asking a question

Description

I am using Locust for load test of gRPC with web UI. I set the "number of users (peak concurrency)" to be 100 in the hope that the request per second could reach 100. However, not matter what the number of users I set, the RPS seems never change.

Command line

locust -f src/content_moderation_ocr/locustfile.py

Locustfile contents

gRPC client file: src/util/locust_load_test.py

from locust import User
from locust.exception import LocustError

import time
from typing import Any, Callable

import grpc
import grpc.experimental.gevent as grpc_gevent
from grpc_interceptor import ClientInterceptor

# patch grpc so that it uses gevent instead of asyncio
grpc_gevent.init_gevent()


class LocustInterceptor(ClientInterceptor):
    def __init__(self, environment, *args, **kwargs):
        super().__init__(*args, **kwargs)

        self.env = environment

    def intercept(
        self,
        method: Callable,
        request_or_iterator: Any,
        call_details: grpc.ClientCallDetails,
    ):
        response = None
        exception = None
        start_perf_counter = time.perf_counter()
        response_length = 0
        try:
            response = method(request_or_iterator, call_details)
            response_length = response.result().ByteSize()
        except grpc.RpcError as e:
            exception = e

        self.env.events.request.fire(
            request_type="grpc",
            name=call_details.method,
            response_time=(time.perf_counter() - start_perf_counter) * 1000,
            response_length=response_length,
            response=response,
            context=None,
            exception=exception,
        )
        return response


class GrpcUser(User):
    abstract = True
    stub_class = None

    def __init__(self, environment):
        super().__init__(environment)
        for attr_value, attr_name in ((self.host, "host"), (self.stub_class, "stub_class")):
            if attr_value is None:
                raise LocustError(f"You must specify the {attr_name}.")

        self._channel = grpc.insecure_channel(self.host)
        interceptor = LocustInterceptor(environment=environment)
        self._channel = grpc.intercept_channel(self._channel, interceptor)

        self.stub = self.stub_class(self._channel)

LocustFile: src/cm_ocr/locustfile.py

from locust import events, task
from bson import ObjectId # pylint: disable=import-error
import gevent
from src.util.locust_load_test import GrpcUser
from v3alpha.moderation_pb2 import (
    OCRModRequest, 
    OCRModResponse,
    Image,
    Reason,
    Status,
    StatusCode                                                         
)
from v3alpha.moderation_pb2_grpc import (
    ImageOCRService,
    add_ImageOCRServiceServicer_to_server,
    ImageOCRServiceStub
)




class OCRGrpcUser(GrpcUser):
    host = "content-moderation-ocr-dev.service.consul:8081"
    stub_class = ImageOCRServiceStub

    @task
    def OCRMod(self):
        self.stub.OCRMod(
            OCRModRequest(
            trace_id=str(ObjectId()),
            image=Image(image_url="https://website.com/3358a31964107656322c60bcf969fcd2.jpg"),
        ),
        timeout=20
        )

### Python version

2.24.0

### Locust version

2.24.0

### Operating system

ubuntu 20.04

[cyberw]

Response times are high, you have reached the limit of the system you are testing. Try a slower ramp up to validate if response times are increasing as new users are added.

https://github.com/locustio/locust/wiki/FAQ#increase-my-request-raterps

[tengerye]

Hi @cyberw thank you for your kind reply. I have done three experiments to validate if the limit of gRPC server has reached the limit.

For the first experiment, on the server, I return the response immediately receiving the request. The following image displays configuration and result.

For the second experiment, on the server, I return the response after waiting for one second after receiving the request. The following image displays the configuration and result.

For the third one, on the server, I return the response after waiting for two seconds after receiving the request. The following image displays the configuration and result.

My server has 800 threads listening. My expectation is around 400 RPS for 1 second and 200 RPS for 2 seconds delay. If the threads are sufficient, the latency of the server should be independent from the throughput of that, but I suppose they are coupled in Locust, therefore, I think there is something wrong with it in my humble opinion.

Or is there a way that I can set strict RPS without users please?

[cyberw]

It is really hard for me to know what the error might be.

Did you try running locust distributed to see if there is any difference?

Try running with a slower ramp up to see if that shows when the limit is reached more clearly.

Or is there a way that I can set strict RPS without users please?

You can use a high number of users and combine it with constant_pacing/constant_ips wait_time to constrain the RPS, but there probably wont be a difference, because something is limiting your requests. Adding more users is likely to just make the queue longer or eventually give errors because your system/whatever is limiting you cant keep up.

[cyberw]

But most importantly, check your server logs to confirm what it is doing. Can you get it to log response times as well? With no more info to go on, I still think it is likely that the limitation is not in locust.

[tengerye]

@cyberw Yes, I can get the log response time. May I ask is that enough?

[cyberw]

Well, you'll want to see if the logged response times match then ones in locust. If they do, then you problem is most certainly on your server side and we should close this ticket, because its not a locust issue :)

[github-actions]

This issue was closed because it has been stalled for 10 days with no activity. This does not necessarily mean that the issue is bad, but it most likely means that nobody is willing to take the time to fix it. If you have found Locust useful, then consider contributing a fix yourself!