docs: add details about HPA via KEDA

By default HPA can use details about memory or CPU consumption for
autoscaling, but also it can use custom metrics as well. There are alot
provides supports HPA via customer and one of them is KEDA project. Here
it is done with help of Prometheus Scaler

Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>

Documentation/ceph-monitoring.md

@@ -210,3 +210,30 @@ labels:
     prometheus: k8s
 [...]
 ```
+
+### Horizontal Pod Scaling using Kubernetes Event-driven Autoscaling (KEDA)
+
+Using metrics exported from the Prometheus service, the horizontal pod scaling can use the custom metrics other than CPU and memory consumption. It can be done with help of Prometheus Scaler provided by the [KEDA](https://keda.sh/docs/2.4/scalers/prometheus/). See the [KEDA deployment guide](https://keda.sh/docs/2.4/deploy/) for details.
+
+Following is an example to autoscale RGW:
+```YAML
+apiVersion: keda.k8s.io/v1alpha1
+kind: ScaledObject
+metadata:
+ name: rgw-scale
+ namespace: rook-ceph
+spec:
+ scaleTargetRef:
+   kind: Deployment
+   deploymentName: rook-ceph-rgw-my-store-a # deployment for the autoscaling
+ minReplicaCount: 1
+ maxReplicaCount: 5
+ triggers:
+ - type: prometheus
+   metadata:
+     serverAddress: http://rook-prometheus.rook-ceph.svc:9090
+     metricName: collecting_ceph_rgw_put
+     query: |
+       sum(rate(ceph_rgw_put[2m])) # promethues query used for autoscaling
+     threshold: "90"
+```

cluster/examples/kubernetes/ceph/keda-sample-rgw.yaml

@@ -0,0 +1,19 @@
+apiVersion: keda.k8s.io/v1alpha1
+kind: ScaledObject
+metadata:
+  name: rgw-scale
+  namespace: rook-ceph
+spec:
+  scaleTargetRef:
+    kind: Deployment
+    deploymentName: rook-ceph-rgw-my-store-a
+  minReplicaCount: 1
+  maxReplicaCount: 5
+  triggers:
+    - type: prometheus
+      metadata:
+      serverAddress: http://rook-prometheus.rook-ceph.svc:9090
+      metricName: ceph_rgw_put_collector
+      query: |
+        sum(rate(ceph_rgw_put[2m]))
+      threshold: "90"