The Rook Test Framework is used to run end to end and integration tests on Rook. The framework depends on a running instance of Kubernetes. The framework also provides scripts for starting Kubernetes so users can quickly spin up a Kubernetes cluster. The Test framework is designed to install Rook, run tests, and uninstall Rook.
You can choose any Kubernetes install of your choice. The test framework only depends on kubectl
being configured. To install kubectl
, please see the official guide.
To install Minikube follow the official guide. It is recommended to use the kvm2 driver when running on a Linux machine and the hyperkit driver when running on a MacOS. Both allow to create and attach additional disks to the virtual machine. This is required for the Ceph OSD to consume one drive. We don't recommend any other drivers for Rook. You will need a Minikube version 1.23 or higher.
Starting the cluster on Minikube is as simple as running:
# On Linux
minikube start --disk-size=40g --extra-disks=1 --driver kvm2
# On MacOS
minikube start --disk-size=40g --extra-disks=1 --driver hyperkit
It is recommended to install a Docker client on your host system too. Depending on your operating system follow the official guide.
Stopping the cluster and destroying the Minikube virtual machine can be done with:
minikube delete
Use helm.sh to install Helm and set up Rook charts defined under _output/charts
(generated by build):
- To install and set up Helm charts for Rook run
tests/scripts/helm.sh up
. - To clean up
tests/scripts/helm.sh clean
.
NOTE: These helper scripts depend on some artifacts under the _output/
directory generated during build time.
These scripts should be run from the project root.
NOTE: If Helm is not available in your PATH
, Helm will be downloaded to a temporary directory (/tmp/rook-tests-scripts-helm
) and used from that directory.
Now that the Kubernetes cluster is running we need to populate the Docker registry environment variables:
eval $(minikube docker-env -p minikube)
The build process will build and push the images to the Docker registry inside the Minikube virtual machine.
make build
Now tag the newly built images to rook/ceph:local-build
for running tests, or rook/ceph:master
if creating sample manifests::
docker tag $(docker images|awk '/build-/ {print $1}') rook/ceph:local-build
docker tag rook/ceph:local-build rook/ceph:master
Some settings are available to run the tests under different environments. The settings are all configured with environment variables. See environment.go for the available environment variables. At least you should set the following variables.
export TEST_HELM_PATH=/tmp/rook-tests-scripts-helm/linux-amd64/helm
export TEST_BASE_DIR=WORKING_DIR
export TEST_SCRATCH_DEVICE=/dev/vdb
Please note that the integration tests erase the contents of TEST_SCRATCH_DEVICE.
To run all integration tests:
go test -v -timeout 7200s github.com/rook/rook/tests/integration
After running tests, you can get test logs under "tests/integration/_output".
To run a specific suite (uses regex):
go test -v -timeout 1800s -run CephSmokeSuite github.com/rook/rook/tests/integration
To run specific tests inside a suite:
go test -v -timeout 1800s -run CephSmokeSuite github.com/rook/rook/tests/integration -testify.m TestARookClusterInstallation_SmokeTest
- Setup OpenShift environment and export KUBECONFIG before executing the tests.
- Make sure
oc
executable file is in the PATH. - Only
CephSmokeSuite
is currently supported on OpenShift. - Set few environment variables:
export TEST_ENV_NAME=openshift
export TEST_STORAGE_CLASS=gp2
export TEST_BASE_DIR=/tmp
export RETRY_MAX=40
To run the CephSmokeSuite
:
go test -v -timeout 1800s -run CephSmokeSuite github.com/rook/rook/tests/integration