[BUG] ApproximateCreationDateTime from DynamoDBStreamRecord in milliseconds when originating from Kinesis, not seconds.

[seanlane]

This is essentially the same issue as aws/aws-lambda-dotnet#839, but without crashing deserialization, which I'm guessing is due to the use of float64 that avoids overflowing with the larger value to deserialize. The relevant points of discussion are:

• DynamoDB events have milliseconds, not seconds, in their timestamps which crashes deserialization aws-lambda-dotnet#839 (comment)
• DynamoDB events have milliseconds, not seconds, in their timestamps which crashes deserialization aws-lambda-dotnet#839 (comment)

It seems that the value of ApproximateCreationDateTime will be in seconds when coming from a DynamoDB Stream, but in milliseconds when coming from a Kinesis Stream:

• DynamoDB Streams - https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_streams_StreamRecord.html

The approximate date and time when the stream record was created, in UNIX epoch time format and rounded down to the closest second

• Kinesis - https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/kds.html

ApproximateCreationDateTime indicates the time of the modification in milliseconds.

There appears to be an internal ticket that's being tracked, so I wanted to open an issue here as well for AWS to monitor and hopefully resolve in the near future. Thanks!

[seanlane]

It may not crash, but Unmarshaling and Marshaling a value will cause an overflow for most Unix timestamps due to calling time.UnixNano, toy example:

package main

import (
	"encoding/json"
	"fmt"
	"time"
)

// SecondsEpochTime serializes a time.Time in JSON as a UNIX epoch time in seconds
type SecondsEpochTime struct {
	time.Time
}

const secondsToNanoSecondsFactor = 1000000000
const milliSecondsToNanoSecondsFactor = 1000000

func TestUnmarshalJSON(epoch float64) time.Time {
	epochSec := int64(epoch)
	epochNano := int64((epoch - float64(epochSec)) * float64(secondsToNanoSecondsFactor))
	return time.Unix(epochSec, epochNano)
}

func TestMarshalJSON(t time.Time) ([]byte, error) {
	// UnixNano() returns the epoch in nanoseconds
	unixTime := float64(t.UnixNano()) / float64(secondsToNanoSecondsFactor)
	return json.Marshal(unixTime)
}

func testVal(test float64) time.Time {
	convertedTime := TestUnmarshalJSON(test)
	convertBack, _ := TestMarshalJSON(convertedTime)
	fmt.Printf("%f\t%s\t%s\n", test, convertedTime.String(), convertBack)
	return convertedTime
}

func main() {
	testVal(1669739327580.0) // Millisecond
	testVal(1669739327.0)    // Second
}

1669739327580.000000	54881-12-06 15:53:00 +0000 UTC	-8914383127.569197
1669739327.000000	2022-11-29 16:28:47 +0000 UTC	1669739327

[bmoffatt]

aws/aws-lambda-dotnet#839 (comment) claims that this only occurs when the dynamo event is first passed through kinesis. Do I understand that correctly? If so, I'm not sure if this is something that's supportable, and the function should operate on the Kinesis event rather than the Dynamo event