Document how to benchmark Dragonfly #101

chakaz · 2023-06-15T07:03:36Z

Fixes #60

vercel · 2023-06-15T07:03:39Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
documentation	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jun 15, 2023 9:17am

chakaz · 2023-06-15T07:04:19Z

Hi @romange, please let me know if this is roughly what you had in mind, and what should be added / changed if not.

romange · 2023-06-15T07:20:23Z

docs/development/benchmarking.md

+
+```shell
+sudo apt install linux-tools-common linux-tools-generic
+sudo cpupower frequency-set --governor performance


Are you sure you need it on cloud instances?
i have never done it, actually.

sorry I missed the sentence before :(
I would frame the whole page around the cloud or on-prem datacenter server. I do not advise benchmarking locally.

Sure, I removed this whole paragraph with the command line example

romange · 2023-06-15T07:28:20Z

docs/development/benchmarking.md

+
+## Choosing an Environment
+
+A benchmark is done to assess the performance aspects of a system. In the case of Dragonfly, a


Was this paragraph generated by ChatGPT? :)

I thought about providing specific requirements of how to say reach 1M qps on m5 family instance.

Specifiying Linux os version, even distribution, mentioning that Dragonfly runs best with iouring,

talking about what's the minimal instance size that allows reaching 1M qps on m5.

Talking about running the load test on a separate machine. machine size for the load test .

Specifying that it is better to choose a bigger size for load test machine to avoid bottlenecks on the client side.

Provide rule of thumb for --threads should not be higher than number of vcpus on that machine.

Provide suggested configuration for memtier to reach 1M qps on Dragonfly.

Provide suggested configuration + instance type to reach 2M qps on GCP instance etc

i.e. keep everything very technical and specific.

or any other instance family and interesting target goal.

It was not generated by ChatGPT, but I've been called worse :)

re/ 1: done
re/ 2: what is indeed the minimal instance?
re/ 3: I already talk about it below
re/ 4: done
re/ 5: done (but I think you meant --proactor_threads?)
re/ 6+7: do you have these? or would you like me to run them until I figure it out?

Sorry 😞 , I was joking.

re/5 - I meant -t on memtier side. Dragonfly actually spans all the cpus automatically. memtier always uses 4 by default.

romange · 2023-06-15T07:30:19Z

docs/development/benchmarking.md

+traffic, you should definitely use them. That would be the closest estimation to what a real
+production deployment with a backing Dragonfly would look like.
+
+If, like many others, you do not (yet) have such a tool, you could either write your own tool to


Lets be opinionated and say - we usually use memtier - and this is how we do it.
I would not suggest folks to write their own loadtest tool :)

We can also mention redis-benchmark - from my experience (have not been using it for the last two years) it's less efficient than memtier but it has more predefined loadtest options specific to redis.
but each such statement must be checked verified and run personally by you.

I would actually suggest that users write their on load tests, as an end-to-end kind of thing, but that's really beside the point here :)

romange · 2023-06-15T07:31:56Z

docs/development/benchmarking.md

+    --requests=<requests per client>
+```
+
+## Having Troubles? Anything Unclear?


btw, every docs page has "edit page" button at the end...

Shall I remove this then?

romange · 2023-06-15T07:32:48Z

docs/development/benchmarking.md

+Then, when you're done with the benchmark you could reboot your machine or run the following:
+
+```shell
+sudo cpupower frequency-set --governor powersave


I would not suggest it unless you saw difference in Dragonfly's performance with and without this option on the cloud.

Removed as per discussed

romange · 2023-06-15T07:37:00Z

docs/development/benchmarking.md

+consider running the benchmark in a similar way.
+
+In practice, it means that any other systems in your setup (like other services & databases) should
+run in other machines. Importantly, also the software that sends the traffic should run in another


i would even make the last sentence bold or move it to the start because it's the most relevant sentence here for practical purposes.

romange · 2023-06-15T13:49:34Z

docs/development/benchmarking.md

+
+Dragonfly supports both `epoll` and [`io_uring`](https://en.wikipedia.org/wiki/Io_uring) Linux APIs.
+`io_uring` is a newer API, which is faster. Dragonfly runs best with `io_uring`, but it is only
+available with Linux kernels >= 5.1.


Dragonfly requires 5.10 or later for iouring. Before that iouring API was partial and not reliable.

romange · 2023-06-15T13:50:54Z

docs/development/benchmarking.md

+`io_uring` is available in Debian versions Bullseye (11) or later, Ubuntu 21.04 or later, Red Hat
+Enterprise Linux 9.3 or later, Fedora 37 or later.
+
+To find if your machine has `io_uring` support you could run the following:


since we require iouring starting from specific version the most straightforward approach would be to use uname -a in this case.

worldsoup · 2023-07-05T17:28:44Z

@chakaz is this ready to merge?

chakaz · 2023-07-06T06:30:50Z

I'm afraid not. It's going to take a while, as I'm actively learning how to do some more advanced benchmarking from Roman..

romange · 2023-10-10T13:21:19Z

@chakaz should we close this PR?

chakaz · 2023-10-10T13:50:10Z

Yeah, let's close it. I'll create another PR at a later time.

Document how to benchmark Dragonfly

1bf2e95

Fixes #60

chakaz requested a review from romange June 15, 2023 07:04

vercel bot deployed to Preview June 15, 2023 07:04 View deployment

romange requested changes Jun 15, 2023

View reviewed changes

Feedback

e233ff5

vercel bot deployed to Preview June 15, 2023 09:17 View deployment

romange reviewed Jun 15, 2023

View reviewed changes

chakaz closed this Oct 10, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Document how to benchmark Dragonfly #101

Document how to benchmark Dragonfly #101

chakaz commented Jun 15, 2023

vercel bot commented Jun 15, 2023 •

edited

chakaz commented Jun 15, 2023

romange Jun 15, 2023

romange Jun 15, 2023 •

edited

chakaz Jun 15, 2023

romange Jun 15, 2023

romange Jun 15, 2023

chakaz Jun 15, 2023

romange Jun 15, 2023

romange Jun 15, 2023

romange Jun 15, 2023

chakaz Jun 15, 2023

romange Jun 15, 2023

chakaz Jun 15, 2023

romange Jun 15, 2023

chakaz Jun 15, 2023

romange Jun 15, 2023

romange Jun 15, 2023

romange Jun 15, 2023

worldsoup commented Jul 5, 2023

chakaz commented Jul 6, 2023

romange commented Oct 10, 2023

chakaz commented Oct 10, 2023


		## Choosing an Environment

		A benchmark is done to assess the performance aspects of a system. In the case of Dragonfly, a

Document how to benchmark Dragonfly #101

Document how to benchmark Dragonfly #101

Conversation

chakaz commented Jun 15, 2023

vercel bot commented Jun 15, 2023 • edited

chakaz commented Jun 15, 2023

Choose a reason for hiding this comment

romange Jun 15, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

worldsoup commented Jul 5, 2023

chakaz commented Jul 6, 2023

romange commented Oct 10, 2023

chakaz commented Oct 10, 2023

vercel bot commented Jun 15, 2023 •

edited

romange Jun 15, 2023 •

edited