v0.13.4 - Improved gRPC and external probes, HTTP latency breakdown and more

[manugarg]

New Features / Enhancements

• Latency Breakdown for HTTP Probes
  Provide a way to report latency breakdown for HTTP probes ([probes.http] Provide a way to report latency breakdown. #699).

• gRPC Probe Enhancements

  • Wait for successful connection before sending the request ([probes.grpc] Wait for successful connection #726, [probes.grpc] Use grpcurl's blocking dial #729). By default gRPC's DialContext returns immediately even for the non-retryable errors. This mechanism doesn't work very well for probing -- there is no error message to show problems with the connection and connection is in the TRANSIENT_FAILURE state. We now use grpcurl's BlockingDial which takes care of these issues.
  • Use client TLS by default for encryption. ([probes.grpc] Use client TLS by default for encryption #727)
  • Capture request error messages better. ([probes.grpc] Use response's status to determine if gRPC call failed #731)

• Streaming Metrics for external probes
  Parse and export external probes' metrics as soon as they are available. ([probes.external] Parse stdout as soon as it is available. #708, [probes.external] Fix stdout streaming. #712, [probes.external] Make test more reliable #713, [probes.external] Catch and log any scanner errors #715, [probes.external] Test: Kill all started processes before exiting. #716, [probes.external] Increase max scanner token size to 256kB (from 64kB). #722)
  This feature enables use cases where external probe runs less frequently (say 1 every 60s) but runs many tasks (for performance measurement, e.g.) and export results many times (say every 5s) within that interval. See discussion in Streaming of stdout for metric emission in external probe #691 and All custom metric emitted on stdout are converted into metric together for external probe #689 for more background.

• Bulk writes in postgres surfacer
  Batch postgres surfacer writes to improve peformance ([surfacer.postgres] Postgresql surfacer bulk inserts #717).

• Allow DNS Overrides
  Allow overriding DNS server (Provide DNSOverride for targets #707).

Other Changes

• Add a command line flag to control prometheus metrics prefix: --prometheus_metrics_prefix ([surfacers.prometheus] Allow configuring metrics prefix through a flag. #732).
• Fix ostgres surfacer metric filter (Postgres surfacer metric filter bugfix #711).
• [prober.saveconfig] Write probe configs to disk in order ([prober.saveconfig] Write probe configs to disk in order. #721).
• [tls] Reload server certificates as well ([tls] Reload server certificates as well. #719).
• [build.tools] Fix where python proto stub is copied to ([build.tools] Fix where python proto stub is copied to. #700).
• [build] Update net dependency for sec alert ([build] Update net dependency for sec alert #728).
• [build] Update protobuf package to fix security alert ([build] Update protobuf package to fix security alert #714).

Contributors

• @manugarg
• @markoposavec made their first contribution in Postgres surfacer metric filter bugfix #711
• @sfc-gh-raram made their first contribution in Provide DNSOverride for targets #707

Full Changelog: v0.13.3...v0.13.4

---

This discussion was created from the release v0.13.4 - Improved gRPC and external probes, HTTP latency breakdown and more.