wasmCloud · ericgregory · Mar 19, 2024 · Mar 19, 2024
@@ -6,19 +6,23 @@ sidebar_position: 2
 
 In addition to host-level logging, wasmCloud supports log exports using the [OpenTelemetry (OTEL)](https://opentelemetry.io/) specification.
 
-For more information on wasmCloud's implementation of OpenTelemetry, including advanced configuration options, see [Observability with OpenTelemetry](/docs/deployment/observability/observability-with-opentelemetry.mdx).
+For more information on wasmCloud's implementation of OpenTelemetry, including advanced configuration options, see [Observability with OpenTelemetry](/docs/1.0/deployment/observability/observability-with-opentelemetry).
 
 ### Configuration
 
 By default, wasmCloud hosts write `info` level logs to `stderr` in plaintext. 
 
-Enable the wasmCloud host to emit logs as an OTEL signal with the `OTEL_TRACES_EXPORTER` environment variable:
+To enable all OTEL signals, including logs, use the `WASMCLOUD_OBSERVABILITY_ENABLED` environment variable:
 
 ```bash
-OTEL_TRACES_EXPORTER=otlp
+WASMCLOUD_OBSERVABILITY_ENABLED="true"
 ```
 
-In v0.82, this enables all supported OTEL signals&mdash;not just traces. (A more accurately descriptive environment variable and command line argument is to come in v1.0.) 
+To enable only logs, use the `WASMCLOUD_LOGS_ENABLED` environment variable:
+
+```bash
+WASMCLOUD_LOGS_ENABLED="true"
+```
 
 Enabling OTEL signals automatically structures logs for OTEL export. (In this case, local `stderr` logs remain plaintext.)
 
@@ -30,7 +34,7 @@ The endpoint for exports may be overwritten with the `OTEL_EXPORTER_OTLP_ENDPOIN
 OTEL_EXPORTER_OTLP_ENDPOINT=http://my-custom-host:4318
 ```
 
-These configurations can also be set with the `--otel-traces-exporter` and `--otel-exporter-otlp-endpoint` [host config](/docs/reference/host-config) options.
+See the [host config](/docs/1.0/reference/host-config) options for equivalent host configuration CLI flags. 
 
 ### Log levels
 
@@ -41,7 +45,7 @@ The wasmCloud host supports the following log levels:
 - `debug`
 - `trace`
 
-To override the log level, use the `--log-level` [host config](/docs/reference/host-config) argument.
+To override the log level, use the `--log-level` [host config](/docs/1.0/reference/host-config) argument.
 
 :::warning
 
@@ -53,14 +57,14 @@ Take care when enabling `trace` or `debug` level logs in a production environmen
 
 In a production environment, it's more common to emit logs in a structured format and aggregate logs from all hosts in a centralized location. Enabling OTEL signals automatically structures JSON logs for OTEL export, while local `stderr` logs remain plaintext.
 
-To enable structured log output *without* enabling OTEL signals, use the `--enable-structured-logging` [host config](/docs/reference/host-config) argument. This will cause the host to emit logs as JSON, which are suitable for forwarding to a log aggregator.
+To enable structured log output *without* enabling OTEL signals, use the `--enable-structured-logging` [host config](/docs/1.0/reference/host-config) argument. This will cause the host to emit logs as JSON, which are suitable for forwarding to a log aggregator.
 
 ```json
 {"timestamp":"2024-01-09T17:54:59.113161Z","level":"INFO","fields":{"message":"wasmCloud host started","host_id":"NDBYMWNPI3GSXBNM6H4WG6JTVB7K42KQWPOPICMIBVUXQL36ISJZHK4D"},"target":"wasmcloud_host::wasmbus"}
 ```
 
 ### Viewing logs with Grafana and Loki
 
-A set of sample configurations and Docker Compose files are available to help launch a local observability environment. After following the instructions to [**launch a wasmCloud observability ecosystem**](/docs/deployment/observability/observability-with-opentelemetry#launch-a-wasmcloud-observability-ecosystem), you can navigate to [http://localhost:5050/explore](http://localhost:5050/explore), click the dropdown in the upper left, and select **"Loki"** to view logs.
+A set of sample configurations and Docker Compose files are available to help launch a local observability environment. After following the instructions to [**launch a wasmCloud observability ecosystem**](/docs/1.0/deployment/observability/observability-with-opentelemetry#launch-a-wasmcloud-observability-ecosystem), you can navigate to [http://localhost:5050/explore](http://localhost:5050/explore), click the dropdown in the upper left, and select **"Loki"** to view logs.
 
-Refer to the `OTEL_` environment variables in the [Docker Compose file](https://github.com/wasmCloud/wasmCloud/blob/main/examples/docker/docker-compose-otel.yml) for a complete example configuration. The example uses Grafana and Loki, but wasmCloud works with any OTEL-compatible backend.
+Refer to the environment variables in the [Docker Compose file](https://github.com/wasmCloud/wasmCloud/blob/main/examples/docker/docker-compose-otel.yml) for a complete example configuration. The example uses Grafana and Loki, but wasmCloud works with any OTEL-compatible backend.
@@ -4,10 +4,62 @@ description: "How to get aggregate data on the availability and performance of a
 sidebar_position: 4
 ---
 
-:::info
+The wasmCloud host can export metrics using the [OpenTelemetry](https://opentelemetry.io/) (OTEL) specification. 
 
-🚧 Under construction 🚧
+For more information on wasmCloud's implementation of OpenTelemetry, including advanced configuration options, see [Observability with OpenTelemetry](/docs/deployment/observability/observability-with-opentelemetry.mdx).
 
-Design for exporting [metrics via OpenTelemetry](https://opentelemetry.io/docs/specs/otel/metrics/) is underway. See the [RFC](https://github.com/wasmCloud/wasmCloud/issues/664) for more information.
+### Configuration
 
-:::
+By default, metrics are disabled in the wasmCloud host. The host can be configured using [environment variables or host config flags](https://wasmcloud.com/docs/reference/host-config). 
+
+To enable all OTEL signals, including metrics, use the `WASMCLOUD_OBSERVABILITY_ENABLED` environment variable:
+
+```bash
+WASMCLOUD_OBSERVABILITY_ENABLED="true"
+```
+
+To enable only metrics, use the `WASMCLOUD_METRICS_ENABLED` environment variable:
+
+```bash
+WASMCLOUD_METRICS_ENABLED="true"
+```
+
+By default, wasmCloud hosts export signals via HTTP/OpenTelemetry Protocol (`otlp`) on port 4318.
+
+The endpoint for metrics exports may be overwritten with the `OTEL_EXPORTER_OTLP_METRICS_ENDPOINT` environment variable:
+
+```bash
+OTEL_EXPORTER_OTLP_METRICS_ENDPOINT=http://my-custom-host:4318
+```
+
+See the [host config](/docs/1.0/reference/host-config) options for equivalent host configuration CLI flags. 
+
+# Format
+
+Metrics are exported in [OpenTelemetry format](/docs/1.0/deployment/observability/observability-with-opentelemetry). The [OTEL collector](https://opentelemetry.io/docs/collector/) converts OTEL-formatted metrics to [Prometheus format](https://prometheus.io/docs/concepts/data_model/) for use with Prometheus systems. 
+
+### Metric options
+
+The following specific metrics are included with wasmCloud:
+
+- **Actors**:
+    * Time to handle invocation in nanoseconds (`wasmcloud_host.handle_rpc_message.duration`)
+	* Count of the number of invocations (`wasmcloud_host.actor.invocations`)
+    * Count of errors (`wasmcloud_host.actor.errors`)
+
+User feedback and proposals for specific metrics is encouraged&mdash;[join the Slack](https://slack.wasmcloud.com/) or [open an issue](https://github.com/wasmCloud/wasmCloud/issues) to discuss your ideas.
+
+### Viewing metrics with Grafana and Prometheus
+
+A set of sample configurations and Docker Compose files are available to help launch a local observability environment. After following the instructions to [**launch a wasmCloud observability ecosystem**](/docs/1.0/deployment/observability/observability-with-opentelemetry#launch-a-wasmcloud-observability-ecosystem), there are two different ways to query metrics emitted by the wasmCloud host:
+
+* **Prometheus' built-in query interface** at [http://localhost:9090/graph](http://localhost:9090/graph).
+* **Grafana's "Explore" interface** at [http://localhost:5050/explore](http://localhost:5050/explore). Once you are in the "View," select "Prometheus" from the dropdown on the top-left corner and under the "Metric" field select the metric you would like to explore.
+
+Depending on what you are trying to accomplish with the metrics, one or the other interface may be a better fit for you.
+
+* If you are in the process of emitting new metrics, or you are interested in exploring the metrics in a Prometheus-native interface, the Prometheus built-in query interface is probably the better fit.
+
+* If you are interested in seeing how the metrics would look in an interface that you might use in production, or you are looking to develop a metrics dashboard, the Grafana interface is the better fit.
+
+Refer to the environment variables in the [Docker Compose file](https://github.com/wasmCloud/wasmCloud/blob/main/examples/docker/docker-compose-otel.yml) for a complete example configuration. The example uses Grafana and Prometheus, but wasmCloud works with any OTEL-compatible backend.
@@ -4,7 +4,7 @@ description: "Using OpenTelemetry for wasmCloud observability"
 sidebar_position: 1
 ---
 
-wasmCloud is designed to support cloud native observability standards using [OpenTelemetry (OTEL)](https://opentelemetry.io/). OpenTelemetry is an open source project defining common APIs and standards for telemetry in cloud native systems. As of v0.82.0, wasmCloud emits **traces** and **logs** as OTEL [**signals**](https://opentelemetry.io/docs/concepts/signals/), with **metrics** support slated for v1.0.
+wasmCloud is designed to support cloud native observability standards using [OpenTelemetry (OTEL)](https://opentelemetry.io/). OpenTelemetry is an open source project defining common APIs and standards for telemetry in cloud native systems. As of v1.0, wasmCloud emits **traces**, **logs**, and **metrics** as OTEL [**signals**](https://opentelemetry.io/docs/concepts/signals/).
 
 ### Data flow
 
@@ -14,28 +14,28 @@ A wasmCloud observability ecosystem must include an [**OTEL collector**](https:/
 
 The OTEL documentation provides detailed instructions on how to [install and run the OTEL collector](https://opentelemetry.io/docs/collector/quick-start/). Additionally, you can find a [sample configuration for an OTEL collector](https://github.com/wasmCloud/wasmCloud/blob/main/examples/docker/otel-collector.yaml) in the wasmCloud observability ecosystem demo on GitHub.
 
-While the OTEL collector is the recommended point of configuration for complex signal routing, there are a set of core configurations available at the level of a [wasmCloud host](https://wasmcloud.com/docs/concepts/hosts).
+While the OTEL collector is the recommended point of configuration for complex signal routing, there are a set of core configurations available at the level of a [wasmCloud host](https://wasmcloud.com/docs/concepts/hosts). The host can be configured using [environment variables or host config flags](/docs/1.0/reference/host-config).
 
 ### Configuration
 
-Enable the wasmCloud host to emit OTEL signals with the `OTEL_TRACES_EXPORTER` environment variable:
+Enable the wasmCloud host to emit all OTEL signals with the `WASMCLOUD_OBSERVABILITY_ENABLED` environment variable:
 
 ```bash
-OTEL_TRACES_EXPORTER=otlp
+WASMCLOUD_OBSERVABILITY_ENABLED="true"
 ```
 
-In v0.82, this enables all supported OTEL signals at once&mdash;not just traces. (A more accurately descriptive environment variable and command line argument is to come in v1.0.)  
-
-By default, OTEL signals are exported on port 4318, the standard **HTTP/OpenTelemetry Protocol (`otlp`)** port. wasmCloud only supports the `otlp` format. The endpoint for `otlp` exports may be overwritten with the `OTEL_EXPORTER_OTLP_ENDPOINT` environment variable:
+By default, OTEL signals are exported on port 4318, the standard **HTTP/OpenTelemetry Protocol (`otlp`)** port. wasmCloud only supports the `otlp` format. The endpoint for all `otlp` exports may be overwritten with the `OTEL_EXPORTER_OTLP_ENDPOINT` environment variable:
 
 ```bash
 OTEL_EXPORTER_OTLP_ENDPOINT=http://my-custom-host:4318
 ```
 
-While these options are typically passed as part of [host configuration](https://wasmcloud.com/docs/reference/host-config), you can launch a wasmCloud observability ecosystem locally by passing environment variables to `wash up`:
+It is also possible to enable and overwrite endpoints for only specific signals&mdash;see the pages for [traces](/docs/1.0/deployment/observability/tracing), [logs](/docs/1.0/deployment/observability/logs), and [metrics](/docs/1.0/deployment/observability/metrics) for more details. 
+
+While these options are typically passed as part of [host configuration](/docs/1.0/reference/host-config), you can launch a wasmCloud observability ecosystem locally by passing environment variables to `wash up`:
 
 ```bash
-OTEL_TRACES_EXPORTER=otlp wash up
+WASMCLOUD_OBSERVABILITY_ENABLED="true" wash up
 ```
 
 ### Launch a wasmCloud observability ecosystem
@@ -67,7 +67,7 @@ docker compose -f docker-compose-otel.yml up
 Make sure a wasmCloud host is running locally with OTEL signals enabled:
 
 ```bash
-OTEL_TRACES_EXPORTER=otlp wash up
+WASMCLOUD_OBSERVABILITY_ENABLED="true" wash up
 ```
 
 Now you can access Grafana's "Explore" interface at [http://localhost:5050/explore](http://localhost:5050/explore).
@@ -76,16 +76,14 @@ Now you can access Grafana's "Explore" interface at [http://localhost:5050/explo
 
 Because wasmCloud's implementation of OpenTelemetry observability uses upstream OTEL libraries, a variety of advanced configuration options are available via environment variables defined in the upstream [OTEL SDK](https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/) and [OTLP Exporter SDK](https://opentelemetry.io/docs/languages/sdk-configuration/otlp-exporter/). 
 
-For example, users have several options for [trace sampling](https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/#general-sdk-configuration), [log batch management](https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/#batch-logrecord-processor), and setting [per signal overrides for endpoints](https://opentelemetry.io/docs/languages/sdk-configuration/otlp-exporter/#otel_exporter_otlp_traces_endpoint). 
-
-Not all upstream options are supported&mdash;for example, the wasmCloud host does not support changing format to gRPC.
+For example, users have several options for [trace sampling](https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/#general-sdk-configuration) and [log batch management](https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/#batch-logrecord-processor). Not all upstream options are supported&mdash;for example, the wasmCloud host does not support changing format to gRPC.
 
 :::info[Note]
 In cases where the system receives conflicting configurations for the same option via environment variable and command line argument, upstream OpenTelemetry libraries prioritize configurations set via environment variable. Refer to the [OpenTelemetry documentation](https://opentelemetry.io/docs/languages/sdk-configuration/) for more details.
 :::
 
 ### Further reading
 
-Signal-specific options on the wasmCloud host are detailed on the [traces](/docs/deployment/observability/tracing.mdx), [logs](/docs/deployment/observability/logs.mdx), and [metrics](/docs/deployment/observability/metrics.mdx) pages. For more information on OpenTelemetry, refer to the [project documentation](https://opentelemetry.io/docs/).
+Signal-specific options on the wasmCloud host are detailed on the [traces](/docs/1.0/deployment/observability/tracing), [logs](/docs/1.0/deployment/observability/logs), and [metrics](/docs/1.0/deployment/observability/metrics) pages. For more information on OpenTelemetry, refer to the [project documentation](https://opentelemetry.io/docs/).
 
 
@@ -8,29 +8,37 @@ Traces are essential for understanding the flow of messages through a distribute
 
 The wasmCloud host can record and export distributed traces using the [OpenTelemetry](https://opentelemetry.io/) (OTEL) specification. If you haven't worked with distributed traces before, [this primer](https://opentelemetry.io/docs/concepts/observability-primer/#distributed-traces) is a great place to start. 
 
-For more information on wasmCloud's implementation of OpenTelemetry, including advanced configuration options, see [Observability with OpenTelemetry](/docs/deployment/observability/observability-with-opentelemetry.mdx).
+For more information on wasmCloud's implementation of OpenTelemetry, including advanced configuration options, see [Observability with OpenTelemetry](/docs/1.0/deployment/observability/observability-with-opentelemetry).
 
 ### Configuration
 
-By default, tracing is disabled in the wasmCloud host. To enable traces, use the `OTEL_TRACES_EXPORTER` environment variable:
+By default, tracing is disabled in the wasmCloud host. The host can be configured using [environment variables or host config flags](/docs/1.0/reference/host-config). 
+
+To enable all OTEL signals, including traces, use the `WASMCLOUD_OBSERVABILITY_ENABLED` environment variable:
+
+```bash
+WASMCLOUD_OBSERVABILITY_ENABLED="true"
+```
+
+To enable only traces, use the `WASMCLOUD_TRACES_ENABLED` environment variable:
 
 ```bash
-OTEL_TRACES_EXPORTER=otlp
+WASMCLOUD_TRACES_ENABLED="true"
 ```
 
 By default, wasmCloud hosts export traces via HTTP/OpenTelemetry Protocol (`otlp`) on port 4318.
 
-The endpoint for trace exports may be overwritten with the `OTEL_EXPORTER_OTLP_ENDPOINT` environment variable:
+The endpoint for trace exports may be overwritten with the `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT` environment variable:
 
 ```bash
-OTEL_EXPORTER_OTLP_ENDPOINT=http://my-custom-host:4318
+OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=http://my-custom-host:4318
 ```
 
-These configurations can also be set with the `--otel-traces-exporter` and `--otel-exporter-otlp-endpoint` [host config](/docs/reference/host-config) options.
+See the [host config](/docs/1.0/reference/host-config) options for equivalent host configuration CLI flags. 
 
 ### Viewing traces with Grafana and Tempo
 
-A set of sample configurations and Docker Compose files are available to help launch a local observability environment. After following the instructions to [**launch a wasmCloud observability ecosystem**](/docs/deployment/observability/observability-with-opentelemetry#launch-a-wasmcloud-observability-ecosystem), you can navigate to [http://localhost:5050/explore](http://localhost:5050/explore), click the dropdown in the upper left, and select **"Tempo"** to view traces.
+A set of sample configurations and Docker Compose files are available to help launch a local observability environment. After following the instructions to [**launch a wasmCloud observability ecosystem**](/docs/1.0/deployment/observability/observability-with-opentelemetry#launch-a-wasmcloud-observability-ecosystem), you can navigate to [http://localhost:5050/explore](http://localhost:5050/explore), click the dropdown in the upper left, and select **"Tempo"** to view traces.
 
 There are several ways to query traces in Tempo. To see all traces from the host, change the "Query type" tab to "Search," then click the "Service Name" dropdown and select "wasmcloud-host." You can also increase the "Limit" field to something more than the default (20). To search, press Shift-Enter. You can click on any of the Trace IDs to view all the spans associated with the trace.