label based slicer to select nested object

[mattijn]

It would be nice to make a label based slice to select a nested object. I can use the existing project transform as such:

{
  "$schema": "https://vega.github.io/schema/vega/v3.0.json",
  "data": [
    {
      "name": "source",
      "values": [
        {"foo": {"a": 5, "b": "string_0"}, "bah": 0},
        {"foo": {"a": 6, "b": "string_1"}, "bah": 1},
        {"foo": {"a": 7, "b": "string_2"}, "bah": 2}
      ],
      "transform": [{"type": "project", "fields": ["foo"]}]
    }
  ]
}

but this still returns a nested object:

[{foo: {a: 5, b: "string_0"}}, 
 {foo: {a: 6, b: "string_1"}}, 
 {foo: {a: 7, b: "string_2"}}]

Where I was hoping for:

[{a: 5, b: "string_0"},
 {a: 6, b: "string_1"},
 {a: 7, b: "string_2"}]

Maybe in the project transform the as can accept null value to make this possible?
"transform": [{"type": "project", "fields": ["foo"], "as": null}]

Or, completely different, as a real slicer under the data format:
"format": {"type": "json", "property": "[:,'foo']"}

[silvestrelosada]

+1 I think this feature is really needed, mosto of systems return data using nested objects (ie elasticsearch aggregations) And you cannot select nested objects to visualize them. This is new feature request that will be very usefull

[jheer]

I certainly understand the desire for convenience here, but also want to point out that you can already accomplish un-nesting of semi-structured data with the existing Vega transforms.

For example, in the example provided above, you can simply provide explicit nested field names to the project transform:

"transform": [{"type": "project", "fields": ["foo.a", "foo.b"], "as": ["a", "b"}]

While the above example is slightly more verbose, it doesn't add a disproportional costs: if you are going to project them, I assume you already have to reference these fields by name elsewhere in your specification in order to visualize them. I've updated the project transform documentation to include an example of this form.

Also, in addition to project, the flatten transform can be used to help deal with arrays of objects contained within nested JSON input.

Nevertheless, the idea of a slicer syntax -- either under the data format property or as a new transform -- is interesting. While not currently on our high-priority list of features, feel free to share suggestions for the design of an appropriate query syntax.

[silvestrelosada]

I have another example that come from elasticsearch
[
{"foo": [{"a": 5, "b": "string_0"},{"a": 5, "b": "string_2"},{"a": 5, "b": "string_3"}], "bah": 0},
{"foo": [{"a": 6, "b": "string_1"}{"a": 6, "b": "string_2"},{"a": 6, "b": "string_3"}], "bah": 1},
{"foo": [{"a": 7, "b": "string_1"}{"a": 7, "b": "string_2"},{"a": 7, "b": "string_3"}], "bah": 2}
]

The ouput should be

[{"a": 5, "b": "string_0"},
{"a": 5, "b": "string_2"},
{"a": 5, "b": "string_3"},
{"a": 6, "b": "string_1"},
{"a": 6, "b": "string_2"},
{"a": 6, "b": "string_3"},
{"a": 7, "b": "string_1"},
{"a": 7, "b": "string_2"},
{"a": 7, "b": "string_3"}]

if ne nested value is an array or an object containing other objects I think project wont work.

Best

[jheer]

As I noted in my previous reply, you can use the flatten transform to handle array-valued data. Here is a complete example -- based on your data above -- that uses flatten and project together to map semi-structured data to a flat table structure.

{
  "$schema": "https://vega.github.io/schema/vega/v4.json",
  "data": [
    {
      "name": "data",
      "values": [
        {"foo": [{"a": 5, "b": "string_0"},{"a": 5, "b": "string_2"},{"a": 5, "b": "string_2"}], "bah": 0},
        {"foo": [{"a": 6, "b": "string_1"},{"a": 6, "b": "string_2"},{"a": 6, "b": "string_2"}], "bah": 1},
        {"foo": [{"a": 7, "b": "string_1"},{"a": 7, "b": "string_2"},{"a": 7, "b": "string_2"}], "bah": 2}
      ],
      "transform": [
        {
          "type": "flatten",
          "fields": ["foo"],
          "as": ["foo"]
        },
        {
          "type": "project",
          "fields": ["bah", "foo.a", "foo.b"],
          "as": ["bah", "a", "b"]
        }
      ]
    }
  ]
}

[silvestrelosada]

Thanks!!

[jheer]

I'm closing out this issue for now, as the needed functionality is already provided. If someone would like to propose a more complete design for alternative un-nesting procedures, please feel free to share them in a new feature request issue.