Use new json type for labels and annotations

[jsoriano]

Uses new type for queryable object fields for docker and kubernetes labels and annotations, so any label name is allowed without modifications. This should fix mapping issues we are having with some combinations of labels.

I haven't removed the dedotting options in case some user wants to continue using it, but probably we could remove it also for 7.0.

It depends on this feature to be merged in ES and on its support for terms aggregations

[elasticmachine]

Pinging @elastic/infrastructure

[jsoriano]

Added issue in ECS to discuss the introduction of this type also there elastic/ecs#198

[ruflin]

What happens if we would put it still in the same format?

[jsoriano]

Do you mean with Put? Put creates the intermediate maps, something we don't want here. In this case we want to use directly the labels as keys, no matter if they have dots or not.

[ruflin]

What was more or less my question. So if the tree is created inside a json type the keys can't be used directly? This is mainly for my understanding.

[jsoriano]

Yes, they can be used too, but this can be problematic, for example for these labels:

docker.container.labels.foo=something
docker.container.labels.foo.response=42

Using Put there can be two different possible results depending on what label is "put" first, this happens with the json type or without it, because this is caused by how we build the event:

{
  "docker": {
    "container": {
      "labels": {
        "foo": "something",
      }
    }
  }
}

Or

{
  "docker": {
    "container": {
      "labels": {
        "foo": {
          "response": "42"
        }
      }
    }
  }
}

With safemapstr this was resolved storing this like:

{
  "docker": {
    "container": {
      "labels": {
        "foo": {
          "response": "42",
          "value": "something"
        }
      }
    }
  }
}

Assigning directly (without Put) and using the new json type, the labels keep their format in the stored event:

{
  "docker": {
    "container": {
      "labels": {
        "foo.response": "42",
        "foo": "something"
      }
    }
  }
}

In all cases the keys can be used directly, but they can be different, and only with the last option they are kept as they are.

[jsoriano]

Closing this by now as this wouldn't support grouping based on terms aggregations on initial implementations, something we want to maintain.