-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Reduce cardinality in Dapr metrics and add more information to API logs #6919
Merged
Commits on Oct 20, 2023
-
Reduce cardinality in Dapr metrics and add more information to API logs
Fixes dapr#6723 Includes: 1. Drastically reduce cardinality of metrics, especially those emitted by the HTTP server: - Also removes the possibility of PII being included in the metrics endpoint. 1. Add more information to API logs for both HTTP and gRPC, including things that are not going to be included in metrics by default anymore: - Response status code - Response size (for HTTP only & if possible) - Latency 3. When `obfuscateURLs` is enabled for API logging, in API logs we now include a more descriptive name for the method rather than the path that was matched in the router. The descriptive names map to the names of the methods in gRPC, for example `SaveState` in place of `POST /state/{storeName}/{name}`. Since gRPC API logs are always, only "obfuscated" (because the params are in the body and not in the "URL"), this makes HTTP and gRPC API logs more consistent too 4. Refactors how tracing and API logging (and the API token auth middleware) get the data from the handler/router: - The new approach is a lot more declarative, and less based on heuristics (such as parsing the path from the URL again) - The new approach also reduces the number of maps that are allocated and used in each request, which generally contained duplicate information generated in multiple parts of the code 5. For both HTTP and gRPC, the way metadata is added to a tracing span has changed and it's now more declarative: - When looking at how tracing spans were composed, the metadata was added in `pkg/diagnostics`, separately from the endpoints. Only a VERY SMALL number of APIs had proper tracing configured (as a matter of fact, see the number of "TODO"'s in this PR), in large part due to the fact that this was in a separate package. The approach also relied a lot on heuristics. - For HTTP, now the `Endpoint` struct contains a property to add a function that adds properties to a span for tracing purposes. This lives right next to the handler. - For gRPC, messages defined in the protos whose name ends in "Request" now must define an `AppendSpanAttribute` method (this is enforced by a unit test) 6. Update API Allowlisting/Denylisting for HTTP to: - Make sure that the same constants can be used for both HTTP and gRPC, especially versions. Right now, versions are in the format "v1.0-alpha1" for HTTP, and "v1alpha1" for gRPC. This PR changes the HTTP versions to be in the format "v1alpha1" too ("v1.0-alpha1" is preserved for bacwkards-compatibility) - Improved perf of the HTTP Allowlist/Denylist, especially when using the "new" format (e.g. versions with "v1alpha1"): checking the allowlist is now a simple map lookup rather than an iteration over the entire allowlist. Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Configuration menu - View commit details
-
Copy full SHA for 2012ffd - Browse repository at this point
Copy the full SHA 2012ffdView commit details -
Added missing workflow beta1 APIs
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Configuration menu - View commit details
-
Copy full SHA for fbb3624 - Browse repository at this point
Copy the full SHA fbb3624View commit details -
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Configuration menu - View commit details
-
Copy full SHA for e95974e - Browse repository at this point
Copy the full SHA e95974eView commit details -
Update tests/integration/framework/binary/binary.go
Signed-off-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com>
Configuration menu - View commit details
-
Copy full SHA for 80ab29a - Browse repository at this point
Copy the full SHA 80ab29aView commit details -
Configuration menu - View commit details
-
Copy full SHA for 7da569d - Browse repository at this point
Copy the full SHA 7da569dView commit details
Commits on Oct 21, 2023
-
Configuration menu - View commit details
-
Copy full SHA for d8761dd - Browse repository at this point
Copy the full SHA d8761ddView commit details -
Configuration menu - View commit details
-
Copy full SHA for 3fb6b3d - Browse repository at this point
Copy the full SHA 3fb6b3dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 4538ed2 - Browse repository at this point
Copy the full SHA 4538ed2View commit details -
Configuration menu - View commit details
-
Copy full SHA for 07d74da - Browse repository at this point
Copy the full SHA 07d74daView commit details -
Configuration menu - View commit details
-
Copy full SHA for 30bfca0 - Browse repository at this point
Copy the full SHA 30bfca0View commit details
Commits on Oct 23, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 62899a1 - Browse repository at this point
Copy the full SHA 62899a1View commit details -
Configuration menu - View commit details
-
Copy full SHA for 50d6074 - Browse repository at this point
Copy the full SHA 50d6074View commit details
Commits on Oct 24, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 2803b4b - Browse repository at this point
Copy the full SHA 2803b4bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 4d31a47 - Browse repository at this point
Copy the full SHA 4d31a47View commit details
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.