[WIP]beats @timestamp support nanoseconds #9818

pytimer · 2018-12-28T09:41:18Z

This PR adds nanoseconds support to beats. Because this pr update libbeat dtfmt module, so it maybe influence all beats.

This PR relate to #7576.

elasticmachine · 2018-12-28T09:41:19Z

Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually?

libbeat/outputs/codec/common.go

ruflin · 2018-12-28T10:00:11Z

@urso Any chance you could have a look at this?

@pytimer Thanks for the PR. Could you make houndci happy? Please also add a changelog entry.

pytimer · 2018-12-29T04:54:14Z

@ruflin I squash commits. houndci is OK.

But travis-ci failed, and i see details logs, i found a question that update @timestamp support nanoseconds and remove old milliseconds function, many testing case about @timestamp failed. Should i update these testing or use other ways to make it support nanoseconds?

I don't know which method should i do, if you know, please tell me. Thanks.

pytimer · 2018-12-29T05:39:58Z

libbeat/outputs/elasticsearch/enc.go

@@ -89,8 +89,8 @@ func (b *jsonEncoder) resetState() {

 	b.folder, err = gotype.NewIterator(visitor,
 		gotype.Folders(
-			codec.MakeTimestampEncoder(),


@ruflin @urso This is old milliseconds encoder, now i remove it. many testing failed. So i am not sure that is it this change correct?

urso · 2019-01-02T15:56:18Z

The change of TsLayout broke the ParseTime function. See libbeat test output which fails in datatime_test.go. ParseTime is used to parse user provided timestamps (e.g. via JSON). The behavior of this function should not break. Make sure it supports inputs with millis and nanoseconds + add more tests :)

pytimer · 2019-01-03T00:55:26Z

@urso Thanks for you reply.

Now ParseTime supports inputs with milliseconds and nanoseconds, but output always nanoseconds layout, for example, inputs: 2015-01-24T14:06:05.071Z, outputs: 2015-01-24T14:06:05.071000000Z. Due to this reason, many testing failed, such as metricbeat, filebeat, etc. because these testing expected output is milliseconds layout.

Should i update these testing expected output? Or make ParseTime support output with millis and nanoseconds?

In addition, i will add more tests in datatime_test.go.

urso · 2019-01-04T04:01:36Z

ParseTime returns a timestamp of type time.Time. The method (Time).Format has a nice implementation of handling nanoseconds and milliseconds:

A decimal point followed by one or more nines represents a fractional second, printed to the given number of decimal places, with trailing zeros removed.

As the encoding formatter is more similar to traditional date-time formatting packages as can be found in other languages (like Java) I think this use case can be better handled by extending the 'S' formatter.

E.g. checking javas DateTimeFormat docs, it says 'n' is nano (similar to your implementation), but 'S' is a fraction. Playing with the java.time package I found S prints the exact number of digits and actually behaves like n as you have implemented it. If n is used in Java, then it always prints the 9-digit nanosecond value, no matter how many n are given and what the current nano value in the timestamp is. Using the DataTimeFormatterBuilder one has better control over fractions. When adding a fraction of min length 3 and max length 9 it only prints between 3 and 9 digits, ommitting any trailing zeros. That is, millisecond accurate timestamps will not have any trailing zeros.

How about changing the behavior of n and S like this (which means we remove the millis handler in favor of nanos always):

n always prints the nanosecond with 9 chars.
S up to 3 behaves as n right now.
S between 3 and 9 behaves like n, but ommits any trailing zeros.

If a timestamp parsed with ParseTime includes only milliseconds, then we will print the timestamp with millisecond accuracy when changing the default pattern for the formatter to use SSSSSSSSS instead of nnnnnnnnn.

Following the trail of fractions, we could also introduce f for fraction. If f is 3 we print millis, without omitting zeros. If f is >3 and <=6 we print up to microseconds, omitting zeros only if the last 3 digits are zero. If f is >3, but <9 we use nanoseconds, but remove trailing zeros if the count of trailing zeros is 3 or 6. This way we can have nanosecond precision if the timestamps parse/acquired have nanosecond precision, but do not change the output if the parsed timestamps have only millisecond or microsecond precision.

Dealing with time can be tricky at times :(

As the switch to nanoseconds is kind of a breaking change we will have to check if the tests fail for a good reason or not and adapt them.

pytimer · 2019-01-07T05:39:40Z

@urso Let me confirm that i understanding is correct. If wrong, hope that you can point out.

Two ways that implement timestamp support millis and nanoseconds, or microseconds:

extend the S formatter.
add new formatter f.

If use SSSSSSSSS instead of nnnnnnnnn, is there no need to add f formatter, SSSSSSSSS can implement f feature?

If we want to auto omitting the trailing zeros, it means that beats store data to elasticsearch timestamp maybe different. Is it will cause other unknown question?

urso · 2019-01-07T21:30:49Z

From playing with java.time I found:

n behaves exactly like SSSSSSSSS. No matter how many n's are given.
Your implementation of n matches the implementation of S in java.I'd propose to remove millisOfSeconds and use nanosOfSeconds always when S is used. This removes some duplicate code for handling millis or nanos (basically the same, only div factor changes).

Neither n nor S have any logic of dealing with trailing zeros in hava. n always prints 9 digits and S prints as many digits as S-characters are configured.

I think we can make the semantics of S and n equivalent to the java implementation.

I was thinking to also add support to S to remove trailing zeros, but I don't think that is necessary. Difference between S and f in my original think was:

S always removes trailing zeros, no matter the amount of trailing zeros (e.g. 1 or 4)
f only removes pairs of 3 consecutive trailing zeros.

e.g. a timestamp with nanos: 123400000 would produce:

1234 if SSSSSSSSS is used
123400000 if n is used
123400 if fffffffff is used.

Rethinking it (again) we can just make S behave like your current implementation of n, change n to always print 9 digits. Support for f would be nice.

pytimer · 2019-01-08T01:17:54Z

We update parsePatternTo function deal with S and n, add f formatter?

Like code below:

case 'S': // fraction of second
    b.nanoOfSecond(tokLen)
case 'n': // nano second
   if len(tokLen) < 9 {
       tokLen = 9
    }
    b.nanoOfSecond(tokLen)
case 'f':  // fraction of second
     b.fractionOfSecond(tokLen)

I have a question, that we use default timestamp formatter in MakeNanoTimestampEncoder now. Should we change this formatter use SSSSSSSSS or fffffffff instead of nnnnnnnnn?
If we use one of formatter above, is it other formatter no effect when output data? Because i see formatter is only used in filebeat modules and outputs. If other code use, can you tell me? Thanks.

libbeat/common/dtfmt/elems.go

libbeat/common/dtfmt/fmt.go

urso · 2019-01-08T18:51:48Z

Oh I see you didn't push new code. Can you also push changes to the PR, so I have a chance to comment on the change? If unsure you can create a PR in your own repo against your development branch so we can discuss over there.

We update parsePatternTo function deal with S and n, add f formatter?
Like code below:

LGTM. maybe change len(tokLen) < 9 to len(tokLen) != 9 for n.

I have a question, that we use default timestamp formatter in MakeNanoTimestampEncoder now. Should we change this formatter use SSSSSSSSS or fffffffff instead of nnnnnnnnn?

I think I'd prefer fffffffff. Assuming tests publishing to Elasticsearch don't fail, then we will have to use n. Using fffffffff we will print timestamps with millisecond accuracy if we did parse timestamps with millisecond accuracy only (output matches original input).

Because i see formatter is only used in filebeat modules and outputs. If other code use, can you tell me? Thanks.

I think there is no other code depending on the date format changes.

The formatter is only used for printing, but I'd love to have parsing support as well, so we could allow users to configure custom timestamp formats.

pytimer · 2019-01-09T01:47:19Z

@urso I will push new code according to the discussion above soon.

The formatter is only used for printing, but I'd love to have parsing support as well, so we could allow users to configure custom timestamp formats.

I think this change maybe send a new PR to do it.

urso · 2019-01-09T16:21:16Z

Cool :)

pytimer · 2019-01-14T11:07:05Z

Hi, @urso

I have a question when i implement S. Now nanoOfSecond use appendExtDecimal, this div is 100000000,but there have a problem that paddedNumber.compile() convert int to byte, however, div is too large, so byte convert error.
I have two ways to implement S:

update paddedNumber to support big int convert byte.
add paddedBigNumber and nanoOfSecond use this element.

Now i use the second method to implement. If you have other method or idea, i hope you can tell me. Thanks.

pytimer · 2019-01-14T12:14:43Z

libbeat/common/dtfmt/elems.go

@@ -50,6 +50,14 @@ type paddedNumber struct {
 	signed               bool
 }

+type paddedBigNumber struct {


@urso, i add a new elements. It works, what do you think?

I don't think we need this structure if we introduce (or extend) the opcodes to also stores a div up to 32bit. paddedNumber would need to choose the right opcode based on the size of div. The we wouldn't need to cast to float or use math.Pow.

If we wants to stores a div up to 32bit, i think we should change prog structure, is it?

I don't think we need to change the structure. For example see opCopyLong (it uses a 16bit length counter). Putting a 32bit value into prog can easily be done via binary.LittleEndian.PutUint32.

Checking all usages of paddedNumber we could actually change all uses of paddedNumber to report the exponent for 10^exp (like you did for divLen).

type paddedNumber struct { ft fieldType divExp int // <- replace div with exponent for computing 10^exp minDigits, maxDigits int signed bool } func (n paddedNumber) compile() (prog, error) { if n.divExp == 0 { ... } return makeProg(opExtNumPadded, byte(n.ft), byte(n.divExp), byte(n.maxDigits)) }

And use a table instead of math.Pow.
This changes smeantics of opExtNumPadded, but I didn't find any place where we need the actual divisor, so that's fine:

var pow10Table [10]int func init() { x := 1 for i := range pow10Table { pow10Table[i] = x x *= 10 } } ... case opExtNumPadded: ft, divExp, digits := fieldType(p.p[i]), int(p.p[i+1]), int(p.p[i+2]) div := pow10Table[divExp] i += 3 v, err := getIntField(ft, ctx, t) if err != nil { return bytes, err } bytes = appendPadded(bytes, v/div, digits)

What's nice about this is that we don't need to add any new cases to prog. We can also remove opNumPadded now, because a divExp value of 0 will divide by 1 anyways.

I think this later approach is more close to the implementation you provided, but also unifies all fraction handling in the package to use the pow10 based approach of yours (minus the float operations).

If we remove div and add divExp , the paddednumber only support 10^exp, and i'm not sure whether it affect the appendDecimalValue function, so i add a paddedBigNumber structure before.

I'm thinking change the compile() function like below when div too large, it use binary.PutVarint save the int to byte and binary.Varint to convert byte to int now. But i'm not sure this way whether correct.

My way maybe not good, what do you think?

func (n paddedNumber) compile() (prog, error) { if n.div == 0 { return makeProg(opNumPadded, byte(n.ft), byte(n.maxDigits)) } buf := make([]byte, 4) bvar := binary.PutVarint(buf, int64(div)) bufLen := len(buf[:n]) newBuf := make([]byte, 0, bufLen) newBuf = append(newBuf, opExtNumPadded, byte(n.ft), byte(bufLen))) newBuf = append(newBuf, bvar ...) newBuf = append(newBuf, byte(n.maxDigits)) return makeProg(newBuf...) }

If we remove div and add divExp , the paddednumber only support 10^exp, and i'm not sure whether it affect the appendDecimalValue function, so i add a paddedBigNumber structure before.

The appendDecimalValue method always sets div to 0. The only call that sets the div is appendExtDecimal. All uses of appendExtDecimal have a divisor that is dividable by 10. This is why I did propose this approach.

Your approach with PutVarint will also work. Using Varint will add some decoding overhead whenever we want to print the data. this is why I opted for using a 32bit value by using binary.LittleEndian.PutUint32.

boernd · 2019-10-02T13:00:53Z

What's the status of this PR? Would be great to have this feature.

TomGudman · 2020-01-07T07:16:05Z

@urso : are you aware of any other attempts to get nanoseconds in filebeat?

because it sounds like this second attempt to get nanoseconds to (file)beats is lost in limbo again :) I was so excited to upgrade our ELK stack to 7.5.1 to then discover that filebeat does not support nanoseconds... (e.g: type:container aka docker logs)

TomGudman · 2020-01-07T07:19:55Z

@ruflin: are you aware of any other attempts to get nanoseconds in filebeat?

ruflin · 2020-01-07T12:14:20Z

@TomGudman Thanks for the ping. AFAIK there wasn't more progress on this but @urso is the one that will know here.

urso · 2020-01-07T16:08:03Z

libbeat/common/dtfmt/elems.go

-		c.enableClock()
-		c.enableMillis()
+	case ftNanoOfSecond:
+		c.enableNano()


ftMillisX should be removed from the enum if they are not used anymore.

urso · 2020-01-08T16:47:38Z

CHANGELOG.next.asciidoc

+- Rename beat.timezone to event.timezone. {pull}9458[9458]
+- Use _doc as document type. {pull}9056[9056]{pull}9573[9573]
+- Update to Golang 1.11.3. {pull}9560[9560]
+- Add @timestamp support nanoseconds. {pull}9818[9818]


Looks like the git 3-way-merge did mess with the Changelog :)

I did skim code for other potential merge failures, but it looks all good.

pytimer · 2020-01-10T04:40:48Z

I squash commits and push it. I remove the code related about f format, so this commit only add nanoseconds timestamp and it not implement f. If i change the TsLayout in libbeat/common/datetime.go, testing case should change when use common.MustParseTime(), like https://github.com/elastic/beats/blob/master/libbeat/reader/readjson/json_test.go#L261, i did not change these testing case now.

@urso You can review this pr, and if something missing or wrong, hope you point out.

urso · 2020-01-10T12:06:40Z

Jenkins, test this.

urso · 2020-01-10T12:08:39Z

The CI failures are definitely related to this change. There are some failing unit tests in libbeat + filebeat json parsing (includes a timestamp) also breaks:

======================================================================

FAIL: Should be able to make use of a `@timestamp` field if it exists in the

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/Users/travis/gopath/src/github.com/elastic/beats/filebeat/tests/system/test_json.py", line 181, in test_timestamp_in_message

    assert output[0]["@timestamp"] == "2016-04-05T18:47:18.444Z"

AssertionError

libbeat/common/dtfmt/builder.go

urso · 2020-01-10T12:33:51Z

Code for encoding looks good.

Some outstanding tasks:

The timestamp parsing must support milliseconds and nanoseconds as input (I guess filebeat json parsing test fails because it presume the timestamp to be written in millis format).
Fix other failing tests in libbeat, metricbeat, packetbeat (tests presumes timestamp uses millis)
update libbeat/common/dtfmt/doc.go
update '@timestamp' to use date_nanos in fields.yml

pytimer · 2020-01-10T14:48:31Z

The timestamp parsing must support milliseconds and nanoseconds as input (I guess filebeat json parsing test fails because it presume the timestamp to be written in millis format).

Timestamp parse use const time layout, https://github.com/elastic/beats/blob/master/libbeat/common/datetime.go#L59 . The tests use time.Parse(), should we support milliseconds or nanoseconds input when use Parse function?

urso · 2020-01-10T14:59:49Z

I'd say both formats. time.Parse is unfortunately quite strict and we want to support millis and nanos. For example: https://play.golang.org/p/R4Hc_RAi3fd

pytimer · 2020-01-11T02:00:07Z

I want to change common.MustParseTimefunction, it will try nanoseconds layout first, then try milliseconds layout, if both fail, return error.
For example: https://play.golang.org/p/ezEpbiiEEsu

pytimer · 2020-01-13T08:23:26Z

@urso I fix some fail tests, send a new commit about parse milliseconds and nanoseconds format.

urso · 2020-01-13T13:35:51Z

libbeat/common/datetime.go

+
+	for i := 0; i < 3; i++ {
+		trailZero += "000"
+		tsLayout = fmt.Sprintf("%s.%sZ", TsLayout, trailZero)


fmt.Sprintf is a rather expensive operation on the critical path. Given that we have only 3 patterns better create a table with supported layouts. This allows devs to even add more layout types in the future for probing.

urso · 2020-01-13T14:16:55Z

I did restart some tests on travis, but there are still a many that are failing due to the timestamp changes. Even some unit tests in libbeat it seems.

pytimer · 2020-01-13T23:25:43Z

I use make test in my environment, tests pass except kibana tests. Can you tell me which tests fail?

urso · 2020-01-14T12:01:27Z

Right now your tests are run on travis (not sure you can see the list of checks at the bottom of this page). Following the travis link the last failed CI run is here: https://travis-ci.org/elastic/beats/builds/636201972?utm_medium=notification&utm_source=github_status

In order to run a more complete set of tests use TEST_ENVIRONMENT=true SYSTEM_TESTS=true make testsuite (you will need to have docker installed).

Filebeat system tests error:

======================================================================

FAIL: Should be able to make use of a `@timestamp` field if it exists in the

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/go/src/github.com/elastic/beats/filebeat/tests/system/test_json.py", line 181, in test_timestamp_in_message

    assert output[0]["@timestamp"] == "2016-04-05T18:47:18.444Z"

AssertionError

libbeat logstash output integration test errors:

--- FAIL: TestLogstashElasticOutputPluginBulkCompatibleMessageTCP (1.05s)

    logstash_integration_test.go:543: 

        	Error Trace:	logstash_integration_test.go:543

        	            				logstash_integration_test.go:533

        	            				logstash_integration_test.go:478

        	Error:      	Not equal: 

        	            	expected: "2020-01-13T06:38:29.386Z"

        	            	actual  : "2020-01-13T06:38:29.386366722Z"

        	            	

        	            	Diff:

        	            	--- Expected

        	            	+++ Actual

        	            	@@ -1 +1 @@

        	            	-2020-01-13T06:38:29.386Z

        	            	+2020-01-13T06:38:29.386366722Z

        	Test:       	TestLogstashElasticOutputPluginBulkCompatibleMessageTCP

and

--- FAIL: TestLogstashElasticOutputPluginBulkCompatibleMessageTLS (1.38s)

    logstash_integration_test.go:543: 

        	Error Trace:	logstash_integration_test.go:543

        	            				logstash_integration_test.go:533

        	            				logstash_integration_test.go:482

        	Error:      	Not equal: 

        	            	expected: "2020-01-13T06:38:30.707Z"

        	            	actual  : "2020-01-13T06:38:30.707490642Z"

        	            	

        	            	Diff:

        	            	--- Expected

        	            	+++ Actual

        	            	@@ -1 +1 @@

        	            	-2020-01-13T06:38:30.707Z

        	            	+2020-01-13T06:38:30.707490642Z

        	Test:       	TestLogstashElasticOutputPluginBulkCompatibleMessageTLS

Metricbeat unit test failure:

--- FAIL: ExampleWrapper (0.05s)

got:

{

  "@metadata": {

    "beat": "noindex",

    "type": "_doc",

    "version": "1.2.3"

  },

  "@timestamp": "2016-05-10T23:27:58.485000000Z",

  "event": {

    "dataset": "fake.eventfetcher",

    "duration": 111,

    "module": "fake"

  },

  "fake": {

    "eventfetcher": {

      "metric": 1

    }

  },

  "metricset": {

    "name": "eventfetcher",

    "period": 10000

  },

  "service": {

    "type": "fake"

  }

}

want:

{

  "@metadata": {

    "beat": "noindex",

    "type": "_doc",

    "version": "1.2.3"

  },

  "@timestamp": "2016-05-10T23:27:58.485Z",

  "event": {

    "dataset": "fake.eventfetcher",

    "duration": 111,

    "module": "fake"

  },

  "fake": {

    "eventfetcher": {

      "metric": 1

    }

  },

  "metricset": {

    "name": "eventfetcher",

    "period": 10000

  },

  "service": {

    "type": "fake"

  }

}

FAIL

coverage: 54.6% of statements

FAIL	github.com/elastic/beats/metricbeat/mb/module	0.076s

FAIL

And packetbeat system test failures:

======================================================================

FAIL: test_2_pings (test_0050_icmp.Test)

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/home/travis/gopath/src/github.com/elastic/beats/packetbeat/tests/system/test_0050_icmp.py", line 13, in test_2_pings

    assert objs[0]["@timestamp"] == "2015-10-19T21:47:49.900Z"

AssertionError

======================================================================

FAIL: test_icmp4_ping (test_0050_icmp.Test)

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/home/travis/gopath/src/github.com/elastic/beats/packetbeat/tests/system/test_0050_icmp.py", line 28, in test_icmp4_ping

    assert objs[0]["@timestamp"] == "2015-10-19T20:49:23.817Z"

AssertionError

======================================================================

FAIL: test_icmp4_ping_over_vlan (test_0050_icmp.Test)

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/home/travis/gopath/src/github.com/elastic/beats/packetbeat/tests/system/test_0050_icmp.py", line 40, in test_icmp4_ping_over_vlan

    assert objs[0]["@timestamp"] == "2015-10-19T20:49:23.849Z"

AssertionError

======================================================================

FAIL: test_icmp6_ping (test_0050_icmp.Test)

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/home/travis/gopath/src/github.com/elastic/beats/packetbeat/tests/system/test_0050_icmp.py", line 52, in test_icmp6_ping

    assert objs[0]["@timestamp"] == "2015-10-19T20:49:23.872Z"

AssertionError

======================================================================

FAIL: test_icmp6_ping_over_vlan (test_0050_icmp.Test)

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/home/travis/gopath/src/github.com/elastic/beats/packetbeat/tests/system/test_0050_icmp.py", line 64, in test_icmp6_ping_over_vlan

    assert objs[0]["@timestamp"] == "2015-10-19T20:49:23.901Z"

AssertionError

======================================================================

FAIL: Should correctly create a keyspace in Cassandra

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/home/travis/gopath/src/github.com/elastic/beats/packetbeat/tests/system/test_0062_cassandra.py", line 29, in test_create_keyspace

    assert o["event.end"] == "2016-06-28T09:03:53.502Z"

AssertionError

======================================================================

FAIL: Should correctly interpret an A query to google.com

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/home/travis/gopath/src/github.com/elastic/beats/packetbeat/tests/system/test_0032_dns.py", line 29, in test_A

    assert o["event.start"] == "2015-08-27T08:00:55.638Z"

AssertionError

----------------------------------------------------------------------

pytimer · 2020-01-14T12:15:52Z

OK, i will retry in my pc use TEST_ENVIRONMENT=true SYSTEM_TESTS=true make testsuite to testing.

TomGudman · 2020-01-17T02:21:02Z

@pytimer almost there! Good progress!

pytimer · 2020-01-21T06:42:21Z

@urso Sorry, i'm on the holiday until next month, i commit code to github will be interrupted due to bad network, so this work maybe i can't continue during this time, can you continue to do it? Thanks.

urso · 2020-01-21T12:31:17Z

Will do. Thanks for the contribution.

urso · 2020-01-27T18:01:06Z

@pytimer fiy; I merged this PR into a feature branch (feature-timestamp-nano), so we can follow up with PRs from other authors (e.g. me).
The in progress PR finally merging the effort into master is #15872

I think you have finished the hard part, now we have to adapt the tests. Thanks for your efforts.

来自这个pr:elastic#9818 但未被合并进master

skldfm · 2020-09-21T20:16:16Z

Is this issue related to that: https://discuss.elastic.co/t/filebeat-timestamp-processsor-doesnt-parse-microseconds-part/248591 ?

@timestamp

* beats: add @timestamp support nanoseconds * datetime support milli and nano seconds format

pytimer requested a review from a team as a code owner December 28, 2018 09:41

houndci-bot reviewed Dec 28, 2018

View reviewed changes

libbeat/outputs/codec/common.go Outdated Show resolved Hide resolved

libbeat/outputs/codec/common.go Outdated Show resolved Hide resolved

pytimer mentioned this pull request Dec 28, 2018

filebeat @timestamp support nanoseconds #7576

Closed

ruflin requested a review from urso December 28, 2018 09:59

ruflin added review libbeat labels Dec 28, 2018

pytimer commented Dec 29, 2018

View reviewed changes

urso reviewed Jan 8, 2019

View reviewed changes

libbeat/common/dtfmt/elems.go Outdated Show resolved Hide resolved

urso reviewed Jan 8, 2019

View reviewed changes

libbeat/common/dtfmt/fmt.go Outdated Show resolved Hide resolved

pytimer changed the title ~~beats @timestamp support nanoseconds~~ [WIP]beats @timestamp support nanoseconds Jan 14, 2019

pytimer requested a review from a team as a code owner January 14, 2019 10:57

pytimer commented Jan 14, 2019

View reviewed changes

urso reviewed Jan 7, 2020

View reviewed changes

urso reviewed Jan 8, 2020

View reviewed changes

TomGudman mentioned this pull request Jan 10, 2020

Feature request: support for nanosecond timestamps elastic/logstash#10822

Closed

urso reviewed Jan 10, 2020

View reviewed changes

libbeat/common/dtfmt/builder.go Show resolved Hide resolved

pytimer added 2 commits January 13, 2020 14:10

beats: add @timestamp support nanoseconds

9658c5f

datetime support milli and nano seconds format

316a696

urso reviewed Jan 13, 2020

View reviewed changes

urso mentioned this pull request Jan 27, 2020

Add support for nanosecond timestamps #15871

Closed

21 tasks

urso changed the base branch from master to feature-timestamp-nano January 27, 2020 17:46

Merge branch 'feature-timestamp-nano' into master

7dde936

urso merged commit 1454462 into elastic:feature-timestamp-nano Jan 27, 2020

andresrc added the Team:Integrations Label for the Integrations team label Mar 6, 2020

silenceper added a commit to silenceper/beats that referenced this pull request Apr 10, 2020

timestamp支持nanoseconds

95bc812

来自这个pr:elastic#9818 但未被合并进master

leweafan pushed a commit to leweafan/beats that referenced this pull request Apr 28, 2023

[WIP]beats @timestamp support nanoseconds (elastic#9818)

d9e6884

* beats: add @timestamp support nanoseconds * datetime support milli and nano seconds format

[WIP]beats @timestamp support nanoseconds #9818

[WIP]beats @timestamp support nanoseconds #9818

Conversation

pytimer commented Dec 28, 2018

elasticmachine commented Dec 28, 2018

ruflin commented Dec 28, 2018

pytimer commented Dec 29, 2018

Choose a reason for hiding this comment

urso commented Jan 2, 2019

pytimer commented Jan 3, 2019

urso commented Jan 4, 2019

pytimer commented Jan 7, 2019

urso commented Jan 7, 2019

pytimer commented Jan 8, 2019 • edited Loading

urso commented Jan 8, 2019 • edited Loading

pytimer commented Jan 9, 2019

urso commented Jan 9, 2019

pytimer commented Jan 14, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pytimer Jan 9, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

boernd commented Oct 2, 2019

TomGudman commented Jan 7, 2020

TomGudman commented Jan 7, 2020

ruflin commented Jan 7, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pytimer commented Jan 10, 2020

urso commented Jan 10, 2020

urso commented Jan 10, 2020

urso commented Jan 10, 2020

pytimer commented Jan 10, 2020

urso commented Jan 10, 2020

pytimer commented Jan 11, 2020

pytimer commented Jan 13, 2020

Choose a reason for hiding this comment

urso commented Jan 13, 2020

pytimer commented Jan 13, 2020

urso commented Jan 14, 2020

pytimer commented Jan 14, 2020

TomGudman commented Jan 17, 2020

pytimer commented Jan 21, 2020 • edited Loading

urso commented Jan 21, 2020

urso commented Jan 27, 2020

skldfm commented Sep 21, 2020

pytimer commented Jan 8, 2019 •

edited

Loading

urso commented Jan 8, 2019 •

edited

Loading

pytimer commented Jan 14, 2019 •

edited

Loading

pytimer Jan 9, 2020 •

edited

Loading

pytimer commented Jan 21, 2020 •

edited

Loading