Instrument identity, registration conflicts, multi-callback registration, unregister support

specification/metrics/api.md

@@ -129,19 +129,24 @@ This API MUST accept the following parameters:
 * [since 1.4.0] `schema_url` (optional): Specifies the Schema URL that should be
   recorded in the emitted telemetry.
 
-It is unspecified whether or under which conditions the same or different
-`Meter` instances are returned from this functions.
-
-Implementations MUST NOT require users to repeatedly obtain a `Meter` again with
-the same name+version+schema_url to pick up configuration changes. This can be
+Meters are identified by all of these fields.  When more than one
+Meter of the same `name`, `version`, and `schema_url` is created, it
+is unspecified whether or under which conditions the same or different
+`Meter` instances are returned.  The term _identical_ applied to
+Meters describes instances where all identifying fields are equal.
+The term _distinct_ applied to Meters describes instances where at
+least one identifying field has a different value.
+
+Implementations MUST NOT require users to repeatedly obtain a `Meter` with
+the same identity to pick up configuration changes. This can be
 achieved either by allowing to work with an outdated configuration or by
 ensuring that new configuration applies also to previously returned `Meter`s.
 
 Note: This could, for example, be implemented by storing any mutable
 configuration in the `MeterProvider` and having `Meter` implementation objects
 have a reference to the `MeterProvider` from which they were obtained. If
 configuration must be stored per-meter (such as disabling a certain meter), the
-meter could, for example, do a look-up with its name+version+schema_url in a map
+meter could, for example, do a look-up with its identity in a map
 in the `MeterProvider`, or the `MeterProvider` could maintain a registry of all
 returned `Meter`s and actively update their configuration if it changes.
 
@@ -172,22 +177,56 @@ Also see the respective sections below for more information on instrument creati
 ## Instrument
 
 Instruments are used to report [Measurements](#measurement). Each Instrument
-will have the following information:
+will have the following fields:
 
 * The `name` of the Instrument
-* The `kind` of the Instrument - whether it is a [Counter](#counter) or other
-  instruments, whether it is synchronous or asynchronous
+* The `kind` of the Instrument - whether it is a [Counter](#counter) or
+  one of the other kinds, whether it is synchronous or asynchronous
 * An optional `unit` of measure
 * An optional `description`
 
-Instruments are associated with the Meter during creation, and are identified by
-the name:
+Instruments are associated with the Meter during creation. Instruments
+are identified by all of these fields.
+
+Language-level features such as the distinction between integer and
+floating point numbers SHOULD be considered as identifying.
+
+<a name="instrument-type-conflict-detection"></a>
+
+When more than one Instrument of the same `name` is created for
+identical Meters, denoted _duplicate instrument registration_, the
+implementation MUST create a valid Instrument in every case.  Here,
+"valid" means an instrument that is functional and can expected to
+export data, despite potentially creating a [semantic error in the
+data
+model](datamodel.md#opentelemetry-protocol-data-model-producer-recommendations).
+
+It is unspecified whether or under which conditions the same or
+different Instrument instance will be returned as a result of
+duplicate instrument registration.  The term _identical_ applied to
+Instruments describes instances where all identifying fields are
+equal.  The term _distinct_ applied to Instruments describes instances
+where at least one field value is different.
+
+The implementation MUST aggregate data from identical Instruments
+together in its export pipeline.
+
+When more than one distinct Instrument is registered with the same
+`name` for identical Meters, the implementation SHOULD emit a warning
+to the user informing them of duplicate registration conflict(s).
+
+__Note the warning about duplicate Instrument registration conflicts
+is meant to help avoid the semantic error state described in the
+[OpenTelemetry Metrics data
+model](datamodel.md#opentelemetry-protocol-data-model-producer-recommendations)
+when more than one `Metric` is written for a given instrument `name`
+and Meter identity by the same MeterProvider.
+
+<a name="instrument-namespace"></a>
 
-* Meter implementations MUST return an error when multiple Instruments are
-  registered under the same Meter instance using the same name.
-* Different Meters MUST be treated as separate namespaces. The names of the
-  Instruments under one Meter SHOULD NOT interfere with Instruments under
-  another Meter.
+Distinct Meters MUST be treated as separate namespaces for the
+purposes of detecting [duplicate instrument registration
+conflicts](#instrument-type-conflict-detection).
 
 <a name="instrument-naming-rule"></a>
 
@@ -210,7 +249,7 @@ DIGIT = %x30-39 ; 0-9
 
 <a name="instrument-unit"></a>
 
-The `unit` is an optional string provided by the author of the instrument. It
+The `unit` is an optional string provided by the author of the Instrument. It
 SHOULD be treated as an opaque string from the API and SDK (e.g. the SDK is not
 expected to validate the unit of measurement, or perform the unit conversion).
 
@@ -239,7 +278,7 @@ instrument. It MUST be treated as an opaque string from the API and SDK.
 * It MUST support at least 1023 characters. [OpenTelemetry
   API](../overview.md#api) authors MAY decide if they want to support more.
 
-Instruments can be categorized based on whether they are synchronous or
+Instruments are categorized on whether they are synchronous or
 asynchronous:
 
 <a name="synchronous-instrument"></a>
@@ -264,6 +303,27 @@ Please note that the term *synchronous* and *asynchronous* have nothing to do
 with the [asynchronous
 pattern](https://en.wikipedia.org/wiki/Asynchronous_method_invocation).
 
+The API MUST support creation of asynchronous instruments by passing a
+single callback function to be registered to the newly created
+instrument.
+
+The API SHOULD support creation of asynchronous instruments
+by passing a variable number of callback functions, greater than or
+equal to zero, to be registered to the newly created instrument.
+
+The API SHOULD support registration of callbacks bound to
+one or more instruments outside of instrument construcotrs, provided
+the API declaratively states which instrument(s) will be be used.
+
+The API SHOULD support unregistration of callbacks.
+
+Callback functions SHOULD NOT take an indefinite amount of time.
+
+Callback functions SHOULD NOT make duplicate observations from asynchronous
+instrument callbacks.  The resulting behavior when a callback observes
+multiple values for identical instrument and attributes is explicitly
+not specified.
+
 ### Counter
 
 `Counter` is a [synchronous Instrument](#synchronous-instrument) which supports
@@ -396,9 +456,9 @@ The API MUST accept the following parameters:
   rule](#instrument-unit).
 * An optional `description`, following the [instrument description
   rule](#instrument-description).
-* A `callback` function.
+* Zero or more `callback` functions.  [See the general requirements](#asynchronous-instrument).
 
-The `callback` function is responsible for reporting the
+The `callback` function is responsible for reporting
 [Measurement](#measurement)s. It will only be called when the Meter is being
 observed. [OpenTelemetry API](../overview.md#api) authors SHOULD define whether
 this callback function needs to be reentrant safe / thread safe or not.
@@ -408,10 +468,6 @@ callback function reports the absolute value of the counter. To determine the
 reported rate the counter is changing, the difference between successive
 measurements is used.
 
-The callback function SHOULD NOT take indefinite amount of time. If multiple
-independent SDKs coexist in a running process, they MUST invoke the callback
-function(s) independently.
-
 [OpenTelemetry API](../overview.md#api) authors MAY decide what is the idiomatic
 approach. Here are some examples:
 
@@ -613,17 +669,13 @@ The API MUST accept the following parameters:
   rule](#instrument-unit).
 * An optional `description`, following the [instrument description
   rule](#instrument-description).
-* A `callback` function.
+* Zero or more `callback` functions.  [See the general requirements](#asynchronous-instrument).
 
-The `callback` function is responsible for reporting the
+The `callback` function is responsible for reporting
 [Measurement](#measurement)s. It will only be called when the Meter is being
 observed. [OpenTelemetry API](../overview.md#api) authors SHOULD define whether
 this callback function needs to be reentrant safe / thread safe or not.
 
-The callback function SHOULD NOT take indefinite amount of time. If multiple
-independent SDKs coexist in a running process, they MUST invoke the callback
-function(s) independently.
-
 [OpenTelemetry API](../overview.md#api) authors MAY decide what is the idiomatic
 approach. Here are some examples:
 
@@ -885,9 +937,9 @@ The API MUST accept the following parameters:
   rule](#instrument-unit).
 * An optional `description`, following the [instrument description
   rule](#instrument-description).
-* A `callback` function.
+* Zero or more `callback` functions.  [See the general requirements](#asynchronous-instrument).
 
-The `callback` function is responsible for reporting the
+The `callback` function is responsible for reporting
 [Measurement](#measurement)s. It will only be called when the Meter is being
 observed. [OpenTelemetry API](../overview.md#api) authors SHOULD define whether
 this callback function needs to be reentrant safe / thread safe or not.
@@ -897,10 +949,6 @@ the callback function reports the absolute value of the Asynchronous
 UpDownCounter. To determine the reported rate the Asynchronous UpDownCounter is
 changing, the difference between successive measurements is used.
 
-The callback function SHOULD NOT take indefinite amount of time. If multiple
-independent SDKs coexist in a running process, they MUST invoke the callback
-function(s) independently.
-
 [OpenTelemetry API](../overview.md#api) authors MAY decide what is the idiomatic
 approach. Here are some examples:
 

specification/metrics/datamodel.md

@@ -15,6 +15,9 @@
   * [Event Model](#event-model)
   * [Timeseries Model](#timeseries-model)
   * [OpenTelemetry Protocol data model](#opentelemetry-protocol-data-model)
+    + [OpenTelemetry Protocol data model: Producer recommendations](#opentelemetry-protocol-data-model-producer-recommendations)
+    + [OpenTelemetry Protocol data model: Consumer recommendations](#opentelemetry-protocol-data-model-consumer-recommendations)
+    + [Point kinds](#point-kinds)
 - [Metric points](#metric-points)
   * [Sums](#sums)
   * [Gauge](#gauge)
@@ -28,8 +31,8 @@
       - [Negative Scale: Extract and Shift the Exponent](#negative-scale-extract-and-shift-the-exponent)
       - [All Scales: Use the Logarithm Function](#all-scales-use-the-logarithm-function)
       - [Positive Scale: Use a Lookup Table](#positive-scale-use-a-lookup-table)
-      - [Producer Recommendations](#producer-recommendations)
-    + [Consumer Expectations](#consumer-expectations)
+    + [ExponentialHistogram: Producer Recommendations](#exponentialhistogram-producer-recommendations)
+    + [ExponentialHistogram: Consumer Recommendations](#exponentialhistogram-consumer-recommendations)
   * [Summary (Legacy)](#summary-legacy)
 - [Exemplars](#exemplars)
 - [Single-Writer](#single-writer)
@@ -233,7 +236,7 @@ consisting of several metadata properties:
 
 - Metric name
 - Attributes (dimensions)
-- Kind of point (integer, floating point, etc)
+- Value type of the point (integer, floating point, etc)
 - Unit of measurement
 
 The primary data of each timeseries are ordered (timestamp, value) points, with
@@ -257,22 +260,89 @@ to map into, but is used as a reference throughout this document.
 
 ### OpenTelemetry Protocol data model
 
-The OpenTelemetry protocol data model is composed of Metric data streams. These
-streams are in turn composed of metric data points. Metric data streams
-can be converted directly into Timeseries, and share the same identity
-characteristics for a Timeseries. A metric stream is identified by:
-
-- The originating `Resource`
-- The metric stream's `name`.
-- The attached `Attribute`s
-- The metric stream's point kind.
-
-It is possible (and likely) that more than one metric stream is created per
-`Instrument` in the event model.
-
-**Note: The same `Resource`, `name` and `Attribute`s but differing point kind
-coming out of an OpenTelemetry SDK is considered an "error state" that SHOULD
-be handled by an SDK.**
+The OpenTelemetry protocol (OTLP) data model is composed of Metric data
+streams.  These streams are in turn composed of metric data points.
+Metric data streams can be converted directly into Timeseries.
+
+Metric streams are grouped into individual `Metric` objects,
+identified by:
+
+- The originating `Resource` attributes
+- The instrumentation `Scope` (e.g., instrumentation library name, version)
+- The metric stream's `name`
+
+Including `name`, the `Metric` object is defined by the following
+properties:
+
+- The data point type (e.g. `Sum`, `Gauge`, `Histogram` `ExponentialHistogram`, `Summary`)
+- The metric stream's `unit`
+- The metric stream's `description`
+- Intrinsic data point properties, where applicable: `AggregationTemporality`, `Monotonic`
+
+The data point type, `unit`, and intrinsic properties are considered
+identifying, whereas the `description` field is explicitly not
+identifying in nature.
+
+Extrinsic properties of specific points are not considered
+identifying; these include but are not limited to:
+
+- Bucket boundaries of a `Histogram` data point
+- Scale or bucket count of a `ExponentialHistogram` data point.
+
+The `Metric` object contains individual streams, identified by the set
+of `Attributes`.  Within the individual streams, points are identified
+by one or two timestamps, details vary by data point type.
+
+Within certain data point types (e.g., `Sum` and `Gauge`) there is
+variation permitted in the numeric point value; in this case, the
+associated variation (i.e., floating-point vs. integer) is not
+considered identifying.
+
+#### OpenTelemetry Protocol data model: Producer recommendations
+
+Producers SHOULD prevent the presence of multiple `Metric` identities
+for a given `name` with the same `Resource` and `Scope` attributes.
+Producers are expected to aggregate data for identical `Metric`
+objects as a basic feature, so the appearance of multiple `Metric`,
+considered a "semantic error", generally requires duplicate
+conflicting instrument registration to have occurred somewhere.
+
+Producers MAY be able to remediate the problem, depending on whether
+they are an SDK or a downstream processor:
+
+1. If the potential conflict involves a non-identifying property (i.e.,
+   `description`), the producer SHOULD choose the longer string.
+2. If the potential conflict involves similar but disagreeing units
+   (e.g., "ms" and "s"), an implementation MAY convert units to avoid
+   semantic errors; otherwise an implementation SHOULD inform the user
+   of a semantic error and pass through conflicting data.
+3. If the potential conflict involves an `AggregationTemporality`
+   property, an implementation MAY convert temporality using a
+   Cumulative-to-Delta or a Delta-to-Cumulative transformation;
+   otherwise, an implementation SHOULD inform the user of a semantic
+   error and pass through conflicting data.
+4. Generally, for potential conflicts involving an identifying
+   property (i.e., all properties except `description`), the producer
+   SHOULD inform the user of a semantic error and pass through
+   conflicting data.
+
+When semantic errors such as these occur inside an implementation of
+the OpenTelemetry API, there is an presumption of a fixed `Resource`
+value.  Consequently, SDKs implementing the OpenTelemetry API have
+complete information about the origin of duplicate instrument
+registration conflicts and are sometimes able to help users avoid
+semantic errors.  See the SDK specification for specific details.
+
+#### OpenTelemetry Protocol data model: Consumer recommendations
+
+Consumers MAY reject OpenTelemetry Metrics data containing semantic
+errors (i.e., more than one `Metric` identity for a given `name`,
+`Resource`, and `Scope`).
+
+OpenTelemetry does not specify any means for conveying such an outcome
+to the end user, although this subject deserves attention.
+
+#### Point kinds
 
 A metric stream can use one of these basic point kinds, all of
 which satisfy the requirements above, meaning they define a decomposable
@@ -653,7 +723,7 @@ For positive scales, lookup table methods have been demonstrated
 that are able to exactly compute the index in constant time from a
 lookup table with `O(2**scale)` entries.
 
-##### Producer Recommendations
+#### ExponentialHistogram: Producer Recommendations
 
 At the lowest or highest end of the 64 bit IEEE floating point, a
 bucket's range may only be partially representable by the floating
@@ -674,7 +744,7 @@ perform an exact computation.  As a result, ExponentialHistogram
 exemplars could map into buckets with zero count.  We expect to find
 such values counted in the adjacent buckets.
 
-#### Consumer Expectations
+#### ExponentialHistogram: Consumer Recommendations
 
 ExponentialHistogram bucket indices are expected to map into buckets
 where both the upper and lower boundaries can be represented
@@ -749,11 +819,11 @@ All metric data streams within OTLP MUST have one logical writer.  This means,
 conceptually, that any Timeseries created from the Protocol MUST have one
 originating source of truth.  In practical terms, this implies the following:
 
-- All metric data streams produced by OTel SDKs MUST be globally uniquely
-  produced and free from duplicates.   All metric data streams can be uniquely
-  identified in some way.
+- All metric data streams produced by OTel SDKs SHOULD have globally
+  unique identity at any given point in time. [`Metric` identity is defined
+  above.](#opentelemetry-protocol-data-model-producer-recommendations)
 - Aggregations of metric streams MUST only be written from a single logical
-  source.
+  source at any given point time.
   **Note: This implies aggregated metric streams must reach one destination**.
 
 In systems, there is the possibility of multiple writers sending data for the
@@ -779,6 +849,13 @@ scenarios and take corrective actions.  Additionally, it ensures that
 well-behaved systems can perform metric stream manipulation without undesired
 degradation or loss of visibility.
 
+Note that violations of the Single-Writer principle are not semantic
+errors, generally they result from misconfiguration.  Whereas semantic
+errors can sometimes be corrected by configuring Views, violations of
+the Single-Writer principle can be corrected by differentiating the
+`Resource` used or by ensuring that streams for a given `Resource` and
+`Attribute` set do not overlap in time.
+
 ## Temporality
 
 **Status**: [Stable](../document-status.md)