Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add MDC support to logging #1077

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Add MDC support to logging #1077

wants to merge 1 commit into from

Conversation

blzsaa
Copy link

@blzsaa blzsaa commented Jun 16, 2023

Hello,

With this pull request Airlift will have an MDC like logging functionality.

Adding the following code to a class:

MDC.put("foo", "bar");
LOGGER.info("message");

would create the following JSON log:

{
  "timestamp": "2023-06-16T15:01:23.048080448Z",
  "level": "INFO",
  "thread": "THREAD123",
  "logger": "com.example.Main",
  "message": "message",
  "mdc": {
    "foo": "bar"
  }
}

`

I plan to submit another pull request (PR) but this time in the Trino repository after a successful merge and Trino is updated to the new Airlift version,
The second PR is needed to ensure that Trino's PluginClassLoader doesn't load the MDC class twice. In that case, the key-value pairs would be added to a different MDC than the one used for logging. To address this issue, it is necessary to include the io.airlift.log.mdc package under PluginManager.SPI_PACKAGES.

@electrum
Copy link
Member

Can you explain more about the use case for this in Trino? We recently added support for OpenTelemetry, which provides context in tracing spans. If we could reuse that for logging, we wouldn't need to duplicate context.

@blzsaa
Copy link
Author

blzsaa commented Jun 16, 2023

  • with MDC we can add any custom key-value pair to the log for a thread (e.g username, retry counter) to make the log more readable
  • with OpenTelemetry the focus should be on observability in distributed systems trace, span...
  • with the current logging of OpenTelemetry context we can only log out traceId, spanId and traceFlags fields and

E.g let's say we have a custom plugin that uses an incoming JwtToken token, that contains a username as well, to communicate with another server and writes lots of logs in that case we can add the username to all logs for easier log following.


A typical use case for using MDC (Mapped Diagnostic Context) is when you want to enhance the logs generated by your application with additional contextual information. MDC provides a way to associate key-value pairs with log statements, typically on a per-thread basis. Here are some common use cases where MDC can be beneficial:

  • User Context: When handling user requests in a multi-threaded application, you can use MDC to store user-related information such as user IDs, usernames, or session IDs. This allows you to track and correlate logs specific to each user, making it easier to analyze their activities or troubleshoot user-specific
  • Request Context: In web applications, MDC can be used to store request-specific information like request IDs, HTTP headers, or other metadata. This helps in tracking the flow of a request across various components and correlating logs related to the same request, even when multiple threads or services are
  • Security Context: MDC can be utilized to store security-related information, such as authentication details, access levels, or security tokens. This can be helpful for auditing and tracking security-related events or diagnosing security-related
  • Transaction Context: When dealing with long-running transactions or multi-step processes, MDC can be used to store transaction-specific information. This allows you to correlate logs generated during the different stages of a transaction, aiding in troubleshooting and understanding the behavior of complex
  • Performance Analysis: MDC can be used to track performance-related information, such as timestamps, execution times, or performance metrics. This can assist in identifying bottlenecks, analyzing response times, or tracing performance-related issues within the application.

I

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants