Incorrect RPS when decreasing the number of users

[mpindado]

Describe the bug

In the UI and HTML report, the RPS chart shows the latest RPS computed when users>0. It could be this stat is only tracked on new messages. I came to this using custom shapes but this also happens when editing the test.

Expected behavior

The RPS should reflect the actual RPS on the given period.

Actual behavior

The RPS has the last value computed when there were users, even if there are 0 users.

Steps to reproduce

• Create the most simple user, wait time = 1 second:

class User(HttpUser):
    host = "https://example.com"
    wait_time = constant(1)

    @task
    def main(self):
        self.client.get("/")

• Launch locust interactive.
• New test, 2 users, spawn rate 2.
• Wait some time to estabilize RPS to 2.
• Edit test, 0 users, spawn rate 2.
• RPS is always 2.
  (Actually will happen also with just 1 user).

Environment

• OS: Windows / Linux
• Python version: 3.9
• Locust version: 2.16.1
• Locust command line that you ran: locust
• Locust file contents (anonymized if necessary): NA

[cyberw]

Can you maybe try this using the modern ui? --modern-ui

[mpindado]

Hi, first of all, thanks for your reply. Please also notice that this is not a very important bug, but one can get into this using custom shapes.

Currently, it seems there is a major difference on behaviour between old and modern UI, when the number of users reach 0. In old UI, when the number of users drops to 0, the UI charts still are being updated rolling to the right with time. In the modern UI, when users are 0, the charts "freeze". This invalidates the value of the time axis when the number of users is 0. You could stop the test or define a shape with 0 users during some time, in the modern UI this will not be correctly reflected in time axis.

For example, consider this shape:

class StagesShape(LoadTestShape):
    stages = [
        {"duration": 60, "users": 2, "spawn_rate": 1},
        {"duration": 300, "users": 0, "spawn_rate": 1},
        {"duration": 360, "users": 2, "spawn_rate": 10}
    ]

    def tick(self):
        run_time = self.get_run_time()

        for stage in self.stages:
            if run_time < stage["duration"]:
                tick_data = (stage["users"], stage["spawn_rate"])
                return tick_data

        return None

We'd expect a square graph for both the users and the RPS. This is what we see:

• Default UI:

• Modern UI:

As you can see, in modern UI, the 5 minutes stage with 0 users is incorrectly reflected in the graph.

For both UIs, we do not see the square wave in RPS but a simple glitch, we'd expect the RPS to behave similarly (delayed) as the User graph. This will happen if the users drop to 0 quickly, as the RPS seems to not update in this case and retain the last RPS value calculated.

For modern-UI, I would say freezing is a little misleading, unless some sign was shown on the graph to reflect that time axis is stopped.

Modern-ui also does not take into account custom shapes. Default UI does not let you modify the number of users or spawn rate. This could be misleading in modern ui.

[cyberw]

@andrewbaldwin44 Maybe something you can take a look at?

I'm fine with leaving the old UI broken, my plan is that next minor release should switch the default to Modern UI, and this is not a critical issue.

[andrewbaldwin44]

Yeah I see there is a bit of a false assumption that is being made in the new UI which causes this "freezing" of the charts. When refetching the stats, we rely on the fact that with each refetch the stats data should change and thus the hook will re-run. I can open a PR to fix this assumption shortly.

As for the response time, I believe it is showing correctly. The RPS monitor in the top right shows the average response time from the aggregated data. Although it seems a bit weird that it still shows "1" when there are no users (or when the tests are stopped), the number is correct

As for the charts, I believe that once we fix the false assumption, the data is being displayed correctly

• The topmost chart is once again using the average response time from the aggregated data. We can see that the data is taken from the last stat entry, and the last stat entry always corresponds to the aggregated row report["total_rps"] = stats[len(stats) - 1]["current_rps"]
• The response times takes some time to update due to cache, however once the cache is invalidated we see that both current_response_time_percentile_1 and current_response_time_percentile_2 are None and thus no data is displayed in the chart (@cyberw we could consider invalidating the cache right away, when the swarm is updated)

[cyberw]

@mpindado can you retest with latest prerelease (pip install -U --pre locust). It should be available in a few minutes.

[mpindado]

Tested with Successfully installed locust-2.18.4.dev3.
The UI does not freeze anymore, the user graph displays correctly.
The RPS still gets value from latest point, as mentioned, when users are 0, the latest RPS is shown on the graph. This only happens when users = 0. I think that this only could be fixed in the runner updating the rps to 0 when users go down to 0. As aforementioned, it is not big deal as this is a corner case.

Note: Still different behaviour with custom shapes, not very important.