-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add eTLD+1 to the DNS documents published by Packetbeat #1107
Conversation
@andrewkroh Nice one! Is ready for review? |
It can be reviewed, but not merged. There's a dependency on the SOCKS proxy PR which adds a common dependency. I'll add the review label as soon as it's mergeable. |
231c4fa
to
9c4265f
Compare
9c4265f
to
4820582
Compare
This is now ready to go. I rebased it to pull in the |
Add eTLD+1 to the DNS documents published by Packetbeat
@@ -6,7 +6,7 @@ This file is generated! See etc/fields.yml and scripts/generate_field_docs.py | |||
[[exported-fields]] | |||
== Exported Fields | |||
|
|||
This document describes the fields that are exported by Packetbeat. They are | |||
This document describes the fields that are exported by Docs/Fields.Asciidoc. They are |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't understand why you changed Packetbeat to Docs/Fields.Asciidoc here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Me either. 😕 I used make update
re-build the Packetbeat docs because I added a field. I'll open a new PR to undo this change.
The effective top level domain plus one more label is really useful for clustering DNS requests (hostnames). For example, the eTLD+1 for "foo.bar.golang.org." is "golang.org.".
This was the basis for the aggregations used in Detecting DNS Tunnels with Packetbeat and Watcher.
I used a Groovy script to do this in the blog post, but this has several downsides.
www.google.co.uk
because it would returnco.uk
resulting in a huge cluster of domains. This uses the data from http://publicsuffix.org so it will correctly returngoogle.co.uk
.Unrelated changes
This change depends on the SOCKS5 PR which adds the
golang.org/x/net/publicsuffix
dep.