forked from dapr/dapr
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Reduce cardinality in Dapr metrics and add more information to API logs
Fixes dapr#6723 Includes: 1. Drastically reduce cardinality of metrics, especially those emitted by the HTTP server: - Also removes the possibility of PII being included in the metrics endpoint. 1. Add more information to API logs for both HTTP and gRPC, including things that are not going to be included in metrics by default anymore: - Response status code - Response size (for HTTP only & if possible) - Latency 3. When `obfuscateURLs` is enabled for API logging, in API logs we now include a more descriptive name for the method rather than the path that was matched in the router. The descriptive names map to the names of the methods in gRPC, for example `SaveState` in place of `POST /state/{storeName}/{name}`. Since gRPC API logs are always, only "obfuscated" (because the params are in the body and not in the "URL"), this makes HTTP and gRPC API logs more consistent too 4. Refactors how tracing and API logging (and the API token auth middleware) get the data from the handler/router: - The new approach is a lot more declarative, and less based on heuristics (such as parsing the path from the URL again) - The new approach also reduces the number of maps that are allocated and used in each request, which generally contained duplicate information generated in multiple parts of the code 5. For both HTTP and gRPC, the way metadata is added to a tracing span has changed and it's now more declarative: - When looking at how tracing spans were composed, the metadata was added in `pkg/diagnostics`, separately from the endpoints. Only a VERY SMALL number of APIs had proper tracing configured (as a matter of fact, see the number of "TODO"'s in this PR), in large part due to the fact that this was in a separate package. The approach also relied a lot on heuristics. - For HTTP, now the `Endpoint` struct contains a property to add a function that adds properties to a span for tracing purposes. This lives right next to the handler. - For gRPC, messages defined in the protos whose name ends in "Request" now must define an `AppendSpanAttribute` method (this is enforced by a unit test) 6. Update API Allowlisting/Denylisting for HTTP to: - Make sure that the same constants can be used for both HTTP and gRPC, especially versions. Right now, versions are in the format "v1.0-alpha1" for HTTP, and "v1alpha1" for gRPC. This PR changes the HTTP versions to be in the format "v1alpha1" too ("v1.0-alpha1" is preserved for bacwkards-compatibility) - Improved perf of the HTTP Allowlist/Denylist, especially when using the "new" format (e.g. versions with "v1alpha1"): checking the allowlist is now a simple map lookup rather than an iteration over the entire allowlist. Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
- Loading branch information
1 parent
4e5c51b
commit 7e3a423
Showing
62 changed files
with
3,293 additions
and
2,547 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,68 @@ | ||
/* | ||
Copyright 2023 The Dapr Authors | ||
Licensed under the Apache License, Version 2.0 (the "License"); | ||
you may not use this file except in compliance with the License. | ||
You may obtain a copy of the License at | ||
http://www.apache.org/licenses/LICENSE-2.0 | ||
Unless required by applicable law or agreed to in writing, software | ||
distributed under the License is distributed on an "AS IS" BASIS, | ||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
See the License for the specific language governing permissions and | ||
limitations under the License. | ||
*/ | ||
|
||
package consts | ||
|
||
import ( | ||
semconv "go.opentelemetry.io/otel/semconv/v1.10.0" | ||
) | ||
|
||
const ( | ||
// DaprInternalSpanAttrPrefix is the internal span attribution prefix. | ||
// Middleware will not populate it if the span key starts with this prefix. | ||
DaprInternalSpanAttrPrefix = "__dapr." | ||
// DaprAPISpanNameInternal is the internal attribution, but not populated to span attribution. | ||
DaprAPISpanNameInternal = DaprInternalSpanAttrPrefix + "spanname" | ||
|
||
// Span attribute keys | ||
// Reference trace semantics https://github.com/open-telemetry/opentelemetry-specification/tree/master/specification/trace/semantic_conventions | ||
DBSystemSpanAttributeKey = string(semconv.DBSystemKey) | ||
DBNameSpanAttributeKey = string(semconv.DBNameKey) | ||
DBStatementSpanAttributeKey = string(semconv.DBStatementKey) | ||
DBConnectionStringSpanAttributeKey = string(semconv.DBConnectionStringKey) | ||
MessagingSystemSpanAttributeKey = string(semconv.MessagingSystemKey) | ||
MessagingDestinationSpanAttributeKey = string(semconv.MessagingDestinationKey) | ||
MessagingDestinationKindSpanAttributeKey = string(semconv.MessagingDestinationKindKey) | ||
GrpcServiceSpanAttributeKey = string(semconv.RPCServiceKey) | ||
NetPeerNameSpanAttributeKey = string(semconv.NetPeerNameKey) | ||
|
||
DaprAPISpanAttributeKey = "dapr.api" | ||
DaprAPIStatusCodeSpanAttributeKey = "dapr.status_code" | ||
DaprAPIProtocolSpanAttributeKey = "dapr.protocol" | ||
DaprAPIInvokeMethod = "dapr.invoke_method" | ||
DaprAPIActorTypeID = "dapr.actor" | ||
|
||
DaprAPIHTTPSpanAttrValue = "http" | ||
DaprAPIGRPCSpanAttrValue = "grpc" | ||
|
||
StateBuildingBlockType = "state" | ||
SecretBuildingBlockType = "secrets" | ||
BindingBuildingBlockType = "bindings" | ||
PubsubBuildingBlockType = "pubsub" | ||
|
||
DaprGRPCServiceInvocationService = "ServiceInvocation" | ||
DaprGRPCDaprService = "Dapr" | ||
|
||
// Keys used in the context's metadata for streaming calls | ||
// Note: these keys must always be all-lowercase | ||
DaprCallLocalStreamMethodKey = "__dapr_calllocalstream_method" | ||
) | ||
|
||
// MessagingDestinationTopicKind is effectively const, but isn't a const from upstream. | ||
var MessagingDestinationTopicKind = semconv.MessagingDestinationKindTopic.Value.AsString() | ||
|
||
// GrpcAppendSpanAttributesFn is the interface that applies to gRPC requests that add span attributes. | ||
type GrpcAppendSpanAttributesFn interface { | ||
// AppendSpanAttributes appends attributes to the map used for the span in tracing for the gRPC method. | ||
AppendSpanAttributes(rpcMethod string, m map[string]string) | ||
} |
Oops, something went wrong.