New benchmark derived from otel-launcher-go

[jmacd]

This is a near-identical copy of the benchmark used in developing the Lighstep metrics SDK.

This measures end-to-end performance of the synchronous instrument implementation using Counters. This should help settle the question in #3199.

The current numbers:

goos: darwin
goarch: arm64
pkg: go.opentelemetry.io/otel/sdk/metric
BenchmarkCounterAddNoAttrs-10                 	51228051	        22.61 ns/op	       0 B/op	       0 allocs/op
BenchmarkCounterAddOneAttr-10                 	 7272368	       164.0 ns/op	     152 B/op	       3 allocs/op
BenchmarkCounterAddOneInvalidAttr-10          	 4732094	       254.3 ns/op	     280 B/op	       3 allocs/op
BenchmarkCounterAddManyAttrs-10               	 2170496	       638.3 ns/op	     301 B/op	       3 allocs/op
BenchmarkCounterAddManyInvalidAttrs-10        	 1676473	       658.9 ns/op	     378 B/op	       3 allocs/op
BenchmarkCounterAddManyFilteredAttrs-10       	 1610043	       676.1 ns/op	     381 B/op	       3 allocs/op
BenchmarkCounterCollectOneAttr-10             	 2484734	       475.9 ns/op	     456 B/op	       8 allocs/op
BenchmarkCounterCollectTenAttrs-10            	  507835	      2352 ns/op	    2512 B/op	      35 allocs/op
BenchmarkCounterCollectTenAttrsTenTimes-10    	   51271	     23737 ns/op	   25120 B/op	     350 allocs/op

[codecov]

Codecov Report

Merging #3395 (fea2e28) into main (c8a13d6) will decrease coverage by 0.2%.
The diff coverage is 61.8%.

Additional details and impacted files

@@           Coverage Diff           @@
##            main   #3395     +/-   ##
=======================================
- Coverage   77.9%   77.7%   -0.3%     
=======================================
  Files        164     164             
  Lines      11361   11457     +96     
=======================================
+ Hits        8861    8909     +48     
- Misses      2301    2346     +45     
- Partials     199     202      +3     

Impacted Files	Coverage Δ
exporters/otlp/otlpmetric/otlpmetricgrpc/config.go	41.8% <0.0%> (-3.3%)	⬇️
exporters/otlp/otlpmetric/otlpmetrichttp/config.go	85.7% <0.0%> (-14.3%)	⬇️
sdk/metric/aggregation/aggregation.go	77.4% <ø> (ø)
sdk/metric/reader.go	100.0% <ø> (+24.4%)	⬆️
exporters/otlp/otlpmetric/exporter.go	67.2% <25.0%> (-17.8%)	⬇️
exporters/otlp/otlpmetric/otlpmetricgrpc/client.go	84.9% <42.8%> (-3.0%)	⬇️
exporters/otlp/otlpmetric/otlpmetrichttp/client.go	77.5% <42.8%> (-1.6%)	⬇️
sdk/metric/manual_reader.go	78.6% <65.2%> (-6.0%)	⬇️
...xporters/otlp/otlpmetric/internal/oconf/options.go	79.8% <72.4%> (-1.7%)	⬇️
exporters/stdout/stdoutmetric/config.go	72.9% <72.4%> (-0.8%)	⬇️
... and 8 more

[jmacd]

Although these benchmarks allow comparing the two SDKs, that wasn't my goal. Here is the corresponding output for otel-launcher-go:

goos: darwin
goarch: arm64
pkg: github.com/lightstep/otel-launcher-go/lightstep/sdk/metric
BenchmarkCounterAddNoAttrs-10                 	30648045	        37.15 ns/op	       0 B/op	       0 allocs/op
BenchmarkCounterAddOneAttr-10                 	12716595	        93.52 ns/op	      64 B/op	       1 allocs/op
BenchmarkCounterAddOneInvalidAttr-10          	 8578038	       140.8 ns/op	     128 B/op	       1 allocs/op
BenchmarkCounterAddManyAttrs-10               	 1000000	      1157 ns/op	     569 B/op	       6 allocs/op
BenchmarkCounterAddManyInvalidAttrs-10        	  743876	      1799 ns/op	    1092 B/op	      10 allocs/op
BenchmarkCounterAddManyFilteredAttrs-10       	 1000000	      1426 ns/op	     954 B/op	       8 allocs/op
BenchmarkCounterCollectOneAttrNoReuse-10      	 2412259	       493.3 ns/op	     400 B/op	       7 allocs/op
BenchmarkCounterCollectOneAttrWithReuse-10    	 3470206	       345.5 ns/op	     136 B/op	       3 allocs/op
BenchmarkCounterCollectTenAttrs-10            	  663994	      1762 ns/op	     712 B/op	      12 allocs/op
BenchmarkCounterCollectTenAttrsTenTimes-10    	   68974	     17650 ns/op	    7129 B/op	     120 allocs/op

See it uses fewer allocations and is somewhat faster when the attribute set is re-used (due to "fingerprinting"), otherwise is somewhat slower due to extra complexity associated with memory management.

[jmacd]

Also note @bogdandrutu I could not identify a performance impact of the following diff

diff --git a/sdk/metric/view/view.go b/sdk/metric/view/view.go
index c6f20018..963c7240 100644
--- a/sdk/metric/view/view.go
+++ b/sdk/metric/view/view.go
@@ -71,7 +71,7 @@ func New(opts ...Option) (View, error) {
 
 // TransformInstrument will check if an instrument matches this view
 // and will convert it if it does.
-func (v View) TransformInstrument(inst Instrument) (transformed Instrument, match bool) {
+func (v *View) TransformInstrument(inst Instrument) (transformed Instrument, match bool) {
 	if !v.match(inst) {
 		return Instrument{}, false
 	}
@@ -89,7 +89,7 @@ func (v View) TransformInstrument(inst Instrument) (transformed Instrument, matc
 
 // AttributeFilter returns a function that returns only attributes specified by
 // WithFilterAttributes. If no filter was provided nil is returned.
-func (v View) AttributeFilter() func(attribute.Set) attribute.Set {
+func (v *View) AttributeFilter() func(attribute.Set) attribute.Set {
 	if v.filter == nil {
 		return nil
 	}
@@ -99,27 +99,27 @@ func (v View) AttributeFilter() func(attribute.Set) attribute.Set {
 	}
 }
 
-func (v View) matchName(name string) bool {
+func (v *View) matchName(name string) bool {
 	return v.instrumentName == nil || v.instrumentName.MatchString(name)
 }
 
-func (v View) matchScopeName(name string) bool {
+func (v *View) matchScopeName(name string) bool {
 	return v.scope.Name == "" || name == v.scope.Name
 }
 
-func (v View) matchScopeVersion(version string) bool {
+func (v *View) matchScopeVersion(version string) bool {
 	return v.scope.Version == "" || version == v.scope.Version
 }
 
-func (v View) matchScopeSchemaURL(schemaURL string) bool {
+func (v *View) matchScopeSchemaURL(schemaURL string) bool {
 	return v.scope.SchemaURL == "" || schemaURL == v.scope.SchemaURL
 }
 
-func (v View) matchInstrumentKind(kind InstrumentKind) bool {
+func (v *View) matchInstrumentKind(kind InstrumentKind) bool {
 	return v.instrumentKind == undefinedInstrument || kind == v.instrumentKind
 }
 
-func (v View) match(i Instrument) bool {
+func (v *View) match(i Instrument) bool {
 	return v.matchName(i.Name) &&
 		v.matchScopeName(i.Scope.Name) &&
 		v.matchScopeSchemaURL(i.Scope.SchemaURL) &&

I believe this settles the question in #3199: it's no big deal to use structs for Views. If there's a statistical difference here, it's slight.

[hanyuancheung]

👍