Skip to content

Latest commit

 

History

History
141 lines (115 loc) · 7.29 KB

README.md

File metadata and controls

141 lines (115 loc) · 7.29 KB
Raw

Performance Tests

This folder has scripts and resources that can execute performance tests to examine the performance of Istio Ambient and compare it to Istio sidecar and no-mesh setups.

For HTTP tests with nighthawk, the script will execute a nighthawk client that will send requests to the server service which is also a nighthawk running in a server mode.

For HTTP tests with fortio, the script will execute a fortio client that will send requests to a httpbin service.

For TCP tests, the script will execute an iperf3 client that will send requests to an iperf3 server for a given number of connections, defaulting to 1000.

Dependencies

  • yq and jq are required

  • The performance test script in this project are executing redirection scripts and Istio installation from the Istio Sidecarless repo

  • OPTIONAL: Prometheus, Grafana and Node-Exporter can be installed to collect metrics and view the provided analysis dashboard:

    cat <<\EOF > ./values.yaml
alertmanager:
  enabled: false
kubeStateMetrics:
  enabled: false
nodeExporter:
  enabled: true 
prometheus:
  prometheusSpec:
    retention: 60d
EOF

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update

helm install kube-prometheus-stack \
prometheus-community/kube-prometheus-stack \
--version 30.0.1 \
--namespace monitoring \
--create-namespace \
-f values.yaml

Running with a config

To allow chaining multiple clusters with the same config, a number of changes have been made.

You can now use config.yaml to configure your test parameters.

Valid test types:

  • http
  • tcp
  • tcp-throughput

For http test types, there are two possible performance clients:

  • nighthawk
  • fortio

An example:

service_port_name: "tcp"
params: '--concurrency 2 --output-format json --prefetch-connections --open-loop --experimental-h1-connection-reuse-strategy lru --connections 1 --rps 1000 --duration 60 --request-header "x-nighthawk-test-server-config: {response_body_size:1024}" --request-body-size 1024'
final_result: "/home/daniel/results/test.txt"
test_type: "http"
perf_client: "nighthawk"
istioctl_path: "/home/daniel/dev/ambient/istioctl"
clusters:
- context: "gke_solo-test-236622_us-west1-c_daniel-ambient"
  result_file: "/home/daniel/results/http/1.csv"
  params: '--concurrency 2 --output-format json --prefetch-connections --open-loop --experimental-h1-connection-reuse-strategy lru --connections 1 --rps 1000 --duration 60 --request-header "x-nighthawk-test-server-config: {response_body_size:1024}" --request-body-size 1024'

Then just run:

./run_tests.sh

Config options

Some options can be defined globally, and/or in a cluster. Globally will define defaults, with cluster overriding the default (if a cluster-level config option).

Option Can be defined in cluster Default Description
context yes N/A The kubeconfig context to use.
continue_on_fail no yes If yes, will continue to next cluster if a test fails.
count yes 1000 How many times to run the test. Only used for tcp test_type.
final_result no /tmp/results.txt The file to write the final results (ASCII table) to.
hub no null The hub to use for Istio.
params yes N/A The parameters to pass to the performance client.
perf_client yes nighthawk The performance client to use. Only used for http test_type. Valid values are: nighthawk, fortio
result_file yes /tmp/results.csv The file to write test results to, only a cluster level config item.
server_scale yes 1 How many replicas to deploy, only used for http test types with the fortio performance client.
tag no null The tag to use for Istio.
test_type yes http The type of test to run. Valid values are: http, tcp, tcp-throughput
test_wait yes 1 The amount of time, in seconds, to wait after a test completes before starting the next.

Example Configs

service_port_name: "tcp"
params: '-n 100M --parallel 1 --bidir'
final_result: "/home/daniel/results/testt.txt"
hub: "us-docker.pkg.dev/solo-io-ambient/istio"
tag: "1.16-dev"
test_type: "tcp-throughput"
count: 1000
clusters:
- context: "gke_solo-test-236622_us-west1-c_daniel-ambient"
  result_file: "/home/daniel/results/test-throughput.csv"
  params: '-n 100M'
- context: "gke_solo-test-236622_us-west1-c_daniel-ambient"
  test_type: "tcp"
  result_file: "/home/daniel/results/test-latency.csv"
  params: '-n 1K --parallel 1 --bidir'
params: '--concurrency 1 --simple-warmup --output-format json --rps 200 --duration 60'
final_result: "/home/daniel/results/overall-gke.txt"
hub: "us-docker.pkg.dev/solo-io-ambient/istio"
tag: "1.16-dev"
test_type: "http"
context: "gke_solo-test-236622_us-west1-c_daniel-ambient"
clusters:
- result_file: "/home/daniel/results/http/1.csv"
  params: '--concurrency 2 --output-format json --prefetch-connections --open-loop --experimental-h1-connection-reuse-strategy lru --connections 1 --rps 1000 --duration 60 --request-header "x-nighthawk-test-server-config: {response_body_size:1024}" --request-body-size 1024'
- result_file: "/home/daniel/results/http/2.csv"
  params: '--concurrency 2 --output-format json --prefetch-connections --open-loop --experimental-h1-connection-reuse-strategy lru --connections 2 --rps 1000 --duration 60 --request-header "x-nighthawk-test-server-config: {response_body_size:1024}" --request-body-size 1024'
- result_file: "/home/daniel/results/http/8.csv"
  params: '--concurrency 2 --output-format json --prefetch-connections --open-loop --experimental-h1-connection-reuse-strategy lru --connections 8 --rps 1000 --duration 60 --request-header "x-nighthawk-test-server-config: {response_body_size:1024}" --request-body-size 1024'
- result_file: "/home/daniel/results/http/16.csv"
  params: '--concurrency 2 --output-format json --prefetch-connections --open-loop --experimental-h1-connection-reuse-strategy lru --connections 16 --rps 1000 --duration 60 --request-header "x-nighthawk-test-server-config: {response_body_size:1024}" --request-body-size 1024'
- result_file: "/home/daniel/results/http/32.csv"
  params: '--concurrency 2 --output-format json --prefetch-connections --open-loop --experimental-h1-connection-reuse-strategy lru --connections 32 --rps 1000 --duration 60 --request-header "x-nighthawk-test-server-config: {response_body_size:1024}" --request-body-size 1024'
- result_file: "/home/daniel/results/http/64.csv"
  params: '--concurrency 2 --output-format json --prefetch-connections --open-loop --experimental-h1-connection-reuse-strategy lru --connections 64 --rps 1000 --duration 60 --request-header "x-nighthawk-test-server-config: {response_body_size:1024}" --request-body-size 1024'
- result_file: "/home/daniel/results/http/128.csv"
  params: '--concurrency 2 --output-format json --prefetch-connections --open-loop --experimental-h1-connection-reuse-strategy lru --connections 128 --rps 1000 --duration 60 --request-header "x-nighthawk-test-server-config: {response_body_size:1024}" --request-body-size 1024'