Custom Logger And Extra Input

[bcolley]

When using the logger I can add extra information like the following:
logger.info('Additional Metrics', { countOfA: 0, countOfB: 0, countOfC: 0 })

That will add three new fields to my log statement with corresponding names:

{
    "message": "Additional Metrics",
    "service": "MyService",
    "awsRegion": "us-east-1",
    "correlationIds": {
        "awsRequestId": "ID",
        "xRayTraceId": "ID"
    },
    "lambdaFunction": {
        "name": "functionName",
        "arn": "functionArn",
        "memoryLimitInMB": 256,
        "version": "$LATEST",
        "coldStart": true
    },
    "logLevel": "INFO",
    "timestamp": "2023-08-01T21:45:11.289Z",
    "logger": {},
    "countOfA": 500,
    "countOfB": 0,
    "countOfC": 500
}

However when logging with Context the message and additional fields, the data I most want at a glance, is pushed off of the screen and I must expand the log to see it.

{"message":"Additional Metrics","service":"MyService","awsRegion":"us-east-1","correlationIds":{"awsRequestId"...

To combat this I tried creating a custom logger (just used the example one)

import { LogFormatter } from '@aws-lambda-powertools/logger';
import { LogAttributes, UnformattedAttributes } from '@aws-lambda-powertools/logger/lib/types';

type CustomLog = LogAttributes;

export class CustomLogFormatter extends LogFormatter {
  public formatAttributes(attributes: UnformattedAttributes): CustomLog {
    return {
      message: attributes.message,
      service: attributes.serviceName,
      environment: attributes.environment,
      awsRegion: attributes.awsRegion,
      correlationIds: {
        awsRequestId: attributes.lambdaContext?.awsRequestId,
        xRayTraceId: attributes.xRayTraceId,
      },
      lambdaFunction: {
        name: attributes.lambdaContext?.functionName,
        arn: attributes.lambdaContext?.invokedFunctionArn,
        memoryLimitInMB: attributes.lambdaContext?.memoryLimitInMB,
        version: attributes.lambdaContext?.functionVersion,
        coldStart: attributes.lambdaContext?.coldStart,
      },
      logLevel: attributes.logLevel,
      timestamp: this.formatTimestamp(attributes.timestamp),
      logger: {
        sampleRateValue: attributes.sampleRateValue,
      },
    };
  }
}

And that makes the message the first field, but I can't find any way to add my custom fields so that the message and additional fields are printed first in the log, followed by everything else.

I CAN just do:

logger.info(`Additional Metrics ${JSON.stringify({ countOfA: 0, countOfB: 0, countOfC: 0 })`)

But that gets me

"message": "{\"Additional Metrics\":{\"countOfA\":500,\"countOfB\":0,\"countOfC\":500}}"

Which isn't as useful for log insights.

The main concern I have here is I want to see MY data first in the log statement so that I don't have to expand it unless I need more information. Does anyone have any suggestions for how to approach this?

[dreamorosi]

Hi @bcolley, thank you for opening this discussion and for considering using Powertools!

Your findings are correct. At the moment the LogFormatter.formatAttributes() method only receives the standard attributes (aka the ones managed by Logger) but doesn't receive the ones you add afterwards. For reference, here you can see which fields are included in the UnformattedAttributes type.

This is a limitation of the current implementation and we have an issue (#1261) tracking it. As discussed in the issue giving access to these attributes will require a breaking change, which we are going to release in our next major release.

As part of another work stream that also involves the Logger utility, we have completed an RFC #1500 for a feature that will allow full customizability of the format, including the attributes you specify.

The feature was merged into the v2 branch in #1511 and we plan to release it between Q3 and early Q4 this year, according to our public roadmap.

[bcolley]

Thanks for the heads up! I've managed to deal with it for now by adding a JSON.stringify-ed version of the object in the message, and also including the raw object as additional input.