[receiver/elasticsearch] Add metric definitions (#6892)

* Add metrics metadata Also move to experimental mdatagen Move go generate directive to separate codegen.go * Update some descriptions in metadata. * Update readme to be more clear about the purpose of this scraper. * small tweaks to wording in readme * Some PR feedback * Make every description a full sentence * Ensure punctuation at the end of each description * Fuse node.fs.disk.usage and node.fs.disk.free * Minor tweaks to wording * Rename thread_pool tasks metrics * Add newline to end of file * remove cluster health metric for now * rerun go generate * Use value override for attributes that are obvious in context * rename some attributes to make more sense. * update some metric names * depluralize units * make sure metrics units are plural * add enabled so that mdatagen works again * Add changelog entry * node.fs.io.operations: -> node.fs.operations: * remove year from copyright * fix clunky sentence in README * shard_type -> shard_state * Minor tweaks to metric descriptions * go generate * Fix JVM metrics * remove elasticsearch.node.fs.operations metric * Add in missing JVM metrics * add cluster health status metric * run go generate
open-telemetry · Jan 12, 2022 · 0bdb885 · 0bdb885
1 parent 2089c54
commit 0bdb885
Show file tree

Hide file tree

Showing 8 changed files with 2,683 additions and 73 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -6,6 +6,7 @@
 
 - `cloudfoundryreceiver`: Enable Cloud Foundry client (#7060)
 - `elasticsearchexporter`: add elasticsearchexporter to the components exporter list (#6002)
+- `elasticsearchreceiver`: Add metric metadata (#6892)
 - `datadogexporter`: Add http.status_code tag to trace stats (#6889)
 - `tanzuobservabilityexporter`: Support delta histograms (#6897)
 - `mysqlreceiver`: Add the receiver to available components (#7078)

diff --git a/receiver/elasticsearchreceiver/README.md b/receiver/elasticsearchreceiver/README.md
@@ -1,6 +1,6 @@
 # Elasticsearch Receiver
 
-This receiver queries the Elasticsearch [node stats](https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-nodes-stats.html) and [cluster health](https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-health.html) endpoints.
+This receiver queries the Elasticsearch [node stats](https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-nodes-stats.html) and [cluster health](https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-health.html) endpoints in order to scrape metrics from a running elasticsearch cluster.
 
 Supported pipeline types: `metrics`
 

diff --git a/receiver/elasticsearchreceiver/codegen.go b/receiver/elasticsearchreceiver/codegen.go
@@ -0,0 +1,20 @@
+// Copyright The OpenTelemetry Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//      http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//go:build !windows
+// +build !windows
+
+//go:generate mdatagen --experimental-gen metadata.yaml
+
+package elasticsearchreceiver // import "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/elasticsearchreceiver"
diff --git a/receiver/elasticsearchreceiver/documentation.md b/receiver/elasticsearchreceiver/documentation.md
@@ -8,8 +8,55 @@ These are the metrics available for this scraper.
 
 | Name | Description | Unit | Type | Attributes |
 | ---- | ----------- | ---- | ---- | ---------- |
+| elasticsearch.cluster.data_nodes | The number of data nodes in the cluster. | {nodes} | Sum(Int) | <ul> </ul> |
+| elasticsearch.cluster.nodes | The total number of nodes in the cluster. | {nodes} | Sum(Int) | <ul> </ul> |
+| elasticsearch.cluster.shards | The number of shards in the cluster. | {shards} | Sum(Int) | <ul> <li>shard_state</li> </ul> |
+| elasticsearch.node.cache.evictions | The number of evictions from the cache. | {evictions} | Sum(Int) | <ul> <li>cache_name</li> </ul> |
+| elasticsearch.node.cache.memory.usage | The size in bytes of the cache. | By | Sum(Int) | <ul> <li>cache_name</li> </ul> |
+| elasticsearch.node.cluster.connections | The number of open tcp connections for internal cluster communication. | {connections} | Sum(Int) | <ul> </ul> |
+| elasticsearch.node.cluster.io | The number of bytes sent and received on the network for internal cluster communication. | By | Sum(Int) | <ul> <li>direction</li> </ul> |
+| elasticsearch.node.documents | The number of documents on the node. | {documents} | Sum(Int) | <ul> <li>document_state</li> </ul> |
+| elasticsearch.node.fs.disk.available | The amount of disk space available across all file stores for this node. | By | Sum(Int) | <ul> </ul> |
+| elasticsearch.node.http.connections | The number of HTTP connections to the node. | {connections} | Sum(Int) | <ul> </ul> |
+| elasticsearch.node.jvm.gc.classes.loaded | The number of loaded classes. | {classes} | Sum(Int) | <ul> </ul> |
+| elasticsearch.node.jvm.gc.collections.count | The number of garbage collections performed by the JVM. | {collections} | Sum(Int) | <ul> <li>generation</li> </ul> |
+| elasticsearch.node.jvm.gc.collections.time | The total time spent by the JVM running the garbage collector. | ms | Sum(Int) | <ul> <li>generation</li> </ul> |
+| elasticsearch.node.jvm.memory.committed | The amount of committed memory. | By | Sum(Int) | <ul> <li>segment</li> </ul> |
+| elasticsearch.node.jvm.memory.init | The initial size of memory. | By | Sum(Int) | <ul> <li>segment</li> </ul> |
+| elasticsearch.node.jvm.memory.limit | The maximum memory used by the JVM. | By | Gauge(Int) | <ul> <li>segment</li> </ul> |
+| elasticsearch.node.jvm.memory.pool.commit | The memory committed by the memory pool. | By | Sum(Int) | <ul> <li>memory_pool_name</li> </ul> |
+| elasticsearch.node.jvm.memory.pool.init | The initial size of the memory pool. | By | Sum(Int) | <ul> <li>memory_pool_name</li> </ul> |
+| elasticsearch.node.jvm.memory.pool.limit | The max memory in use by the memory pool. | By | Gauge(Int) | <ul> <li>memory_pool_name</li> </ul> |
+| elasticsearch.node.jvm.memory.pool.usage | The memory in use by the memory pool. | By | Sum(Int) | <ul> <li>memory_pool_name</li> </ul> |
+| elasticsearch.node.jvm.memory.usage | The memory in use by the JVM. | By | Sum(Int) | <ul> <li>segment</li> </ul> |
+| elasticsearch.node.jvm.threads.count | The number of running threads in the node's JVM process. | {threads} | Sum(Int) | <ul> </ul> |
+| elasticsearch.node.jvm.threads.peak | The highest number of concurrently running threads in the current lifetime of the node's JVM process. | {threads} | Gauge(Int) | <ul> </ul> |
+| elasticsearch.node.open_files | The number of open file descriptors held by the node. | {files} | Sum(Int) | <ul> </ul> |
+| elasticsearch.node.operations.completed | The number of operations completed. | {operations} | Sum(Int) | <ul> <li>operation</li> </ul> |
+| elasticsearch.node.operations.time | Time spent on operations. | ms | Sum(Int) | <ul> <li>operation</li> </ul> |
+| elasticsearch.node.shards.size | The size of the shards assigned to this node. | By | Sum(Int) | <ul> </ul> |
+| elasticsearch.node.thread_pool.tasks.finished | The number of tasks finished by the thread pool. | {tasks} | Sum(Int) | <ul> <li>thread_pool_name</li> <li>task_state</li> </ul> |
+| elasticsearch.node.thread_pool.tasks.queued | The number of queued tasks in the thread pool. | {tasks} | Sum(Int) | <ul> <li>thread_pool_name</li> </ul> |
+| elasticsearch.node.thread_pool.threads | The number of threads in the thread pool. | {threads} | Sum(Int) | <ul> <li>thread_state</li> </ul> |
+| elasticserach.cluster.health | The health status of the cluster. Health status is based on the state of its primary and replica shards. Green indicates all shards are assigned. Yellow indicates that one or more replica shards are unassigned. Red indicates that one or more primary shards are unassigned, making some data unavailable.  | {status} | Sum(Int) | <ul> <li>health_status</li> </ul> |
 
 ## Attributes
 
 | Name | Description |
 | ---- | ----------- |
+| cache_name | The name of cache. |
+| direction | The direction of network data. |
+| disk_usage_state | The state of a section of space on disk. |
+| document_state | The state of the document. |
+| elasticsearch.cluster.name | The name of the elasticsearch cluster. |
+| elasticsearch.node.name | The name of the elasticsearch node. |
+| fs_direction | The direction of filesystem IO. |
+| generation | The generation on which garbage collection was performed. |
+| health_status | The health status of the cluster. |
+| memory_pool_name | The name of the JVM memory pool |
+| operation | The type of operation. |
+| segment | The segment of JVM memory. |
+| shard_state | The state of the shard. |
+| task_state | The state of the task. |
+| thread_pool_name | The name of the thread pool. |
+| thread_state | The state of the thread. |
diff --git a/receiver/elasticsearchreceiver/factory.go b/receiver/elasticsearchreceiver/factory.go
@@ -14,8 +14,6 @@
 
 package elasticsearchreceiver // import "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/elasticsearchreceiver"
 
-//go:generate mdatagen metadata.yaml
-
 import (
 	"context"
 	"errors"

diff --git a/receiver/elasticsearchreceiver/internal/metadata/generated_metrics.go b/receiver/elasticsearchreceiver/internal/metadata/generated_metrics.go