Error in Parsing Lambda Result Logs

[RomanOttGmx]

I get an error during executing state machine in the executor lambda.

Logs from Execute Lambda:
{
"errorType": "SyntaxError",
"errorMessage": "Unexpected end of JSON input",
"stack": [
"SyntaxError: Unexpected end of JSON input",
" at JSON.parse ()",
" at /var/task/utils.js:552:54",
" at Array.map ()",
" at module.exports.extractDurationFromJSON (/var/task/utils.js:552:35)",
" at module.exports.extractDuration (/var/task/utils.js:529:22)",
" at /var/task/utils.js:473:22",
" at Array.map ()",
" at module.exports.parseLogAndExtractDurations (/var/task/utils.js:470:17)",
" at computeStatistics (/var/task/executor.js:180:29)",
" at module.exports.handler (/var/task/executor.js:71:12)"
]
}

Because the result inherits empty lines.

In utils.js there is a module function: module.exports.extractDurationFromJSON = (log) => {

const lines = log.split('\n').filter((line) => line.startsWith('{')).map((line) => {

Replacing it with:
const lines = log.split('\n').filter(line => line.trim() !== '').map((line) => JSON.parse(line));

it works. Best Regards

[patmeiler]

Hi @RomanOttGmx , thanks for raising this. I'll look into it. Could you clarify a bit further how this leads to an empty lines in the JSON? I'd like to check if there are any other points where this could happen.

[Marcel-Brouwer]

Having the same issue. We are seeing a trailing newline in the logs when logging the incoming event, using the injectLambdaContext middleware from Powertools Logger.

https://docs.powertools.aws.dev/lambda/typescript/latest/core/logger/#log-incoming-event

This leads to an empty line.

[alexcasalboni]

hey @Marcel-Brouwer @RomanOttGmx thanks for sharing!

The current implementation (here):

const lines = log.split('\n').filter((line) => line.startsWith('{')).map((line) => {...});

should already manage empty lines quite well (as they don't start with {).

Could you please look into the logs that generate the Unexpected end of JSON input error and share here? It's true that JSON.parse("") throws such an error, but we aren't parsing empty lines. Even if it was a JSON line with some weird syntax (like a missing } at the end), the current implementation handles that and should print out:

console.error(`Detected invalid JSON line: ${line}`);

I'd like to add a regression test to catch this case, if possible :)

[alexcasalboni]

maybe the Powertools Logger is logging out prettyfied logs? (with newlines in between fields for readability)

something like:

{
  "cold_start": true,
  "function_arn": "arn:aws:lambda:eu-west-1:123456789012:function:LogEventFn",
  "function_memory_size": "128",
  "function_name": "LogEventFn",
  "function_request_id": "0a9df60d-e2de-447d-ba3e-45f149eae6c9",
  "level": "INFO",
  "message": "Lambda invocation event",
  "sampling_rate": 0,
  "service": "service_undefined",
  "timestamp": "2024-08-14T10:08:06.199Z",
  "xray_trace_id": "1-66bc8205-21f8b5190da519d22b2b0533",
  "event": {
    "foo": "bar"
  }
}

instead of

{ "cold_start": true, "function_arn": "arn:aws:lambda:eu-west-1:123456789012:function:LogEventFn", "function_memory_size": "128", "function_name": "LogEventFn", "function_request_id": "0a9df60d-e2de-447d-ba3e-45f149eae6c9", "level": "INFO", "message": "Lambda invocation event", "sampling_rate": 0, "service": "service_undefined", "timestamp": "2024-08-14T10:08:06.199Z", "xray_trace_id": "1-66bc8205-21f8b5190da519d22b2b0533", "event": {   "foo": "bar" }}

In that case, our implementation would break since we expect every log to be one line.

[tavosansal]

I ran into this when my lambda had Log Format set to JSON.
Once I changed it back to Text format the power tuner ran fine.

[shawnxue]

Same as @tavosansal , this issue was gone after I changed Lambda log format to text. The funny thing for me is that: We have the same lambda functions deployed in AWS AMAP regions, EMEA regions and China. Only China region has this issue.

[alexcasalboni]

@shawnxue that's interesting! I imagine you're using the SAR app to deploy Lambda Power Tuning in China as well?