Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sanitize metric type prefixes #319

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

sysedwinistrator
Copy link
Contributor

@sysedwinistrator sysedwinistrator commented Mar 19, 2024

When more than one prefix matches the same metric descriptor, this will throw the error collected metric xxx was collected before with the same name and label values.

For example, using the metric type prefixes
foo.googleapis.com/bar (a prefix)
and
foo.googleapis.com/bar/baz (a metric)
will result in an error because both match the metric
foo.googleapis.com/bar/baz.

Further, using the metric type prefixes
foo.googleapis.com/bar/baz (a metric)
and
foo.googleapis.com/bar/baz_count (a metric)
will result in an error because both match the metric
foo.googleapis.com/bar/baz_count.

While the first pitfall could be expected by the user, the latter will come as a complete surprise to anyone who is not aware that stackdriver-exporter internally uses an MQL query in the form of
metric.type = starts_with("<prefix>")
to filter the metrics.

Avoid this by sanitizing the provided metric type prefixes in the following way:

  • Drop any duplicate prefixes
  • Sort the prefixes (required by the next step)
  • Drop any prefixes that start with another prefix present in the input

Fixes #103

When more than one prefix matches the same metric descriptor, this will
throw the error "collected metric xxx was collected before with the
same name and label values".

For example, using the metric type prefixes
  foo.googleapis.com/bar (a prefix)
and
  foo.googleapis.com/bar/baz (a metric)
will result in an error because both match the metric
  foo.googleapis.com/bar/baz.

Further, using the metric type prefixes
  foo.googleapis.com/bar/baz (a metric)
and
  foo.googleapis.com/bar/baz_count (a metric)
will result in an error because both match the metric
  foo.googleapis.com/bar/baz_count.

While the first pitfall could be expected by the user, the latter will
come as a complete surprise to anyone who is not aware that
stackdriver-exporter internally uses an MQL query in the form of
  metric.type = starts_with("<prefix>")
to filter the metrics.

Avoid this by sanitizing the provided metric type prefixes in the
following way:
- Drop any duplicate prefixes
- Sort the prefixes (required by the next step)
- Drop any prefixes that start with another prefix present in the input

Signed-off-by: Edwin Mackenzie-Owen <edwin.mowen@gmail.com>
@sysedwinistrator sysedwinistrator marked this pull request as draft March 20, 2024 13:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

How to workaround duplicate metrics from Stackdriver
1 participant