[ECS] Upgrade modules to ECS 1.10

[P1llus]

Updating all modules to ECS 1.10 after release

New fields:
Datastream: https://github.com/elastic/ecs/blob/master/rfcs/text/0009-data_stream-fields.md

Beta fields:
Orchestrator: https://github.com/elastic/ecs/blob/master/rfcs/text/0012-orchestrator-field-set.md

Experimental:
Threat fields: https://github.com/elastic/ecs/blob/master/rfcs/text/0018-extend-threat-group-software.md

[elasticmachine]

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

[andrewkroh]

We should consider adding an alias where data_stream.dataset -> event.dataset. It may help ease the transition (way) later when apps and rules remove support for querying both event.dataset and data_stream.dataset (currently we are making apps/rules work with data that has either field, but the long term plan is to only use the data_stream.dataset). Then older data you had indexed with Beats can continue to be queried by apps and rules.

[P1llus]

Related modules and fixes:

Processors (planed for 7.15 to adopt orchestrator fields)

• Kubernetes Autodiscover
• Kubernetes Metadata Processor
• Nomad Autodiscover
• Nomad Metadata Processor
• add_cloud_metadata
• add_cloudfoundry_metadata
• add_docker_metadata
• add_observer_metadata

Global ECS file update
Change updating Libbeat and updating each beat: #25931
#26121

• Libbeat
• Journalbeat
• Heartbeat
• OSquerybeat
• Winlogbeat
• Metricbeat
• Auditbeat
• Packetbeat
• Filebeat

Journalbeat

• Update ECS version [Journalbeat] updating journalbeat to ecs 1.10.0 #25933

Heartbeat

• Update ECS version [Heartbeat] updating heartbeat to ecs 1.10.0 #25934

OSquerybeat

• Update ECS version Osquerybeat: Align with the rest of the beats, set the ECS version #26324

Winlogbeat

• Update ECS version [Winlogbeat] updating winlogbeat to ecs 1.10.0 #25932

Metricbeat

• Update ECS version

Auditbeat:

• Update ECS version [Auditbeat] updating auditbeat to ecs 1.10.0 #25930

Filebeat:
OSS:

• Apache2 [Filebeat] updating apache to ecs 1.10.0 #25935
• auditd [Filebeat] updating auditd to ecs 1.10.0 #25874
• elasticsearch [Filebeat] updating elasticsearch to ecs 1.10.0 #25957
• haproxy [Filebeat] updating haproxy to ecs 1.10.0 #25936
• icinga [Filebeat] updating icinga to ecs 1.10.0 #25937
• iis [Filebeat] updating iis to ecs 1.10.0 #25938
• kafka [Filebeat] updating kafka to ecs 1.10.0 #25939
• kibana [Filebeat] updating kibana to ecs 1.10.0 #25944
• logstash [Filebeat] updating logstash to ecs 1.10.0 #25945
• mongodb [Filebeat] updating mongodb to ecs 1.10.0 #25946
• mysql [Filebeat] updating mysql to ecs 1.10.0 #25949
• nats [Filebeat] updating nats to ecs 1.10.0 #25950
• nginx [Filebeat] updating nginx to ecs 1.10.0 #25951
• osquery [Filebeat] updating osquery to ecs 1.10.0 #25918
• pensando [Filebeat] updating pensando to ecs 1.10.0 #25875
• postgresql [Filebeat] updating postgresql to ecs 1.10.0 #25952
• redis [Filebeat] updating redis to ecs 1.10.0 #25953
• santa [Filebeat] updating santa to ecs 1.10.0 #25919
• system [Filebeat] updating system to ecs 1.10.0 #25920
• traefik [Filebeat] updating traefik to ecs 1.10.0 #25921

X-Pack:

• activemq [Filebeat] updating activemq to ecs 1.10.0 #25959
• aws (cloudtrail) [Filebeat] updating aws cloudtrail to ecs 1.10.0 #25876
• aws (other)
• awsfargate
• azure
• barracuda [Filebeat] updating barracuda to ecs 1.10.0 #25913
• bluecoat [Filebeat] updating bluecoat to ecs 1.10.0 #25914
• cef [Filebeat] updating cef to ecs 1.10.0 #25915
• checkpoint [Filebeat] updating checkpoint to ecs 1.10.0 #25916
• cisco [Filebeat] updating cisco to ecs 1.10.0 #25877
• coredns [Filebeat] updating coredns to ecs 1.10.0 #25879
• crowdstrike [Filebeat] updating crowdstrike to ecs 1.10.0 #25878
• cyberark [Filebeat] updating cyberark to ecs 1.10.0 #25880
• cyberarkpas [Filebeat] updating cyberarkpas to ecs 1.10.0 #25881
• cylance [Filebeat] updating cylance to ecs 1.10.0 #25882
• envoyproxy [Envoyproxy] Update envoyproxy ECS version #26277
• f5 [Filebeat] updating f5 to ecs 1.10.0 #25883
• fortinet [Filebeat] updating fortinet to ecs 1.10.0 #25884
• gcp [FileBeat] GCP module enhancement - Populate orchestrator.* fields for K8S logs #25368
• google_workspace [Filebeat] updating google_workspace to ecs 1.10.0 #25887
• googlecloud [Filebeat] updating google_workspace to ecs 1.10.0 #25887
• gsuite [Filebeat] updating gsuite to ecs 1.10.0 #25917
• ibmmq [Filebeat] updating ibmmq to ecs 1.10.0 #25960
• imperva [Filebeat] updating imperva to ecs 1.10.0 #25888
• infoblox [Filebeat] updating infoblox to ecs 1.10.0 #25889
• iptables [Filebeat] updating iptables to ecs 1.10.0 #25890
• juniper [Filebeat] updating juniper to ecs 1.10.0 #25891
• microsoft [Filebeat] updating microsoft to ecs 1.10.0 #25892
• misp [Filebeat] updating misp to ecs 1.10.0 #25893
• mssql [Filebeat] updating mssql to ecs 1.10.0 #25961
• mysqlenterprise [Filebeat] updating mysqlenterprise to ecs 1.10.0 #25894
• netflow [Filebeat] updating netflow to ecs 1.10.0 #25895
• netscout [Filebeat] updating netscout to ecs 1.10.0 #25896
• o365 [Filebeat] updating o365 to ecs 1.10.0 #25897
• okta [Filebeat] updating okta to ecs 1.10.0 #25898
• oracle [Filebeat] updating oracle to ecs 1.10.0 #25962
• panw [Filebeat] updating panw to ecs 1.10.0 #25899
• proofpoint [Filebeat] updating proofpoint to ecs 1.10.0 #25900
• rabbitmq [Filebeat] updating rabbitmq to ecs 1.10.0 #25963
• radware [Filebeat] updating radware to ecs 1.10.0 #25901
• snort [Filebeat] updating snort to ecs 1.10.0 #25902
• snyk [Filebeat] updating snyk to ecs 1.10.0 #25903
• sonicwall [Filebeat] updating sonicwall to ecs 1.10.0 #25904
• sophos [Filebeat] updating sophos to ecs 1.10.0 #25905
• squid [Filebeat] updating squid to ecs 1.10.0 #25906
• suricata [Filebeat] updating suricata to ecs 1.10.0 #25907
• threatintel: [Threatintel] update threatintel ECS version #26274
• tomcat [Filebeat] updating tomcat to ecs 1.10.0 #25908
• zeek [Filebeat] updating zeek to ecs 1.10.0 #25911
• zookeeper [Filebeat] updating zookeeper to ecs 1.10.0 #25964
• zoom [Filebeat] updating zoom to ecs 1.10.0 #25909
• zscaler [Filebeat] updating zscaler to ecs 1.10.0 #25912

[andrewstucki]

Heartbeat, Winlogbeat and OSQuerybeat should likely also be updated. Heads up that you'll also need to update the default ECS version in each beat, in case a module doesn't explicitly set it (such as here), the various test fixtures, and then also regenerate the embedded field yml files and docs for each beat after you globally update the shared ECS field definitions.

[P1llus]

Will update the issue accordingly, thanks @andrewstucki !