Add basic support for field aliases in index mappings. #31287

jtibshirani · 2018-06-13T02:14:38Z

Functionality covered in this PR:

Almost all parts of the search request support field aliases. In particular, aliases can be used in queries, aggregations, and sort fields, as well as when requesting docvalue_fields, suggestions, and highlights.
Field aliases can be used in the capabilities API.
If field names are provided as wildcards, field aliases will be matched in addition to concrete fields.
A concrete field can be added in one mappings update, and an alias added in a separate update.
If a concrete field type is updated, any aliases it has will always refer to the latest version.
Attempts to write to a field alias will cause document parsing to fail.

This PR has become quite large -- I tried to keep each commit in the PR well-scoped, but let me know if it would help to split this into smaller PRs.

elasticmachine · 2018-06-13T14:03:06Z

Pinging @elastic/es-search-aggs

jpountz

👍 I like the way it is done.

jpountz · 2018-06-18T07:50:10Z

server/src/main/java/org/elasticsearch/index/mapper/DocumentParser.java

+        } else if (mapper instanceof FieldAliasMapper) {
+            throw new IllegalArgumentException("Cannot write to a field alias [" + mapper.name() + "].");
+        } else {
+            throw new IllegalStateException("The provided mapper [" + mapper.name() + "] has an unrecognized type.");


should we add the class of the field to make debugging easier?

jpountz · 2018-06-18T08:03:54Z

server/src/main/java/org/elasticsearch/index/mapper/FieldAliasMapper.java

+import java.util.Iterator;
+import java.util.Map;
+
+public class FieldAliasMapper extends Mapper {


let's make final and add some javadocs?

jpountz · 2018-06-18T08:04:42Z

server/src/main/java/org/elasticsearch/index/mapper/FieldAliasMapper.java

+            Object pathField = node.remove(Names.PATH);
+            String path = XContentMapValues.nodeStringValue(pathField, null);
+            if (path == null) {
+                throw new MapperParsingException("The [path] property must be specified.");


let's add the field name in the error message

jpountz · 2018-06-18T08:08:39Z

server/src/main/java/org/elasticsearch/index/mapper/FieldTypeLookup.java

+        String resolvedField = aliasToConcreteName.get(field);
+        return resolvedField == null
+            ? fullNameToFieldType.get(field)
+            : fullNameToFieldType.get(resolvedField);


things would be a bit easier to read for me this way:

String resolvedField = aliasToConcreteName.getOrDefault(field, field); return fullNameToFieldType.get(resolvedField);

Makes sense, will change.

jpountz · 2018-06-18T08:09:50Z

server/src/main/java/org/elasticsearch/index/mapper/MapperUtils.java

        if (mapper instanceof RootObjectMapper) {
            // root mapper isn't really an object mapper
        } else if (mapper instanceof ObjectMapper) {
            objectMappers.add((ObjectMapper)mapper);
        } else if (mapper instanceof FieldMapper) {
            fieldMappers.add((FieldMapper)mapper);
+        } else if (mapper instanceof FieldAliasMapper) {
+            fieldAliasMappers.add((FieldAliasMapper) mapper);
        }


maybe we should add an else block that throws an exception like you added in other places?

jtibshirani · 2018-06-18T19:33:41Z

@elasticmachine run sample packaging tests

* Add basic support for field aliases through a new top-level mapper type. * Add tests for queries, aggregations, sorting, and fetching doc values. * Make sure we properly handle wildcard fields in query string queries. * Allow for aliases when requesting suggestions. * Allow for aliases when requesting highlights. * Add a test for field capabilities.

* Add basic support for field aliases in index mappings. (#31287) * Allow for aliases when fetching stored fields. (#31411) * Add tests around accessing field aliases in scripts. (#31417) * Add documentation around field aliases. (#31538) * Add validation for field alias mappings. (#31518) * Return both concrete fields and aliases in DocumentFieldMappers#getMapper. (#31671) * Make sure that field-level security is enforced when using field aliases. (#31807) * Add more comprehensive tests for field aliases in queries + aggregations. (#31565) * Remove the deprecated method DocumentFieldMappers#getFieldMapper. (#32148)

* Add basic support for field aliases in index mappings. (#31287) * Allow for aliases when fetching stored fields. (#31411) * Add tests around accessing field aliases in scripts. (#31417) * Return both concrete fields and aliases in DocumentFieldMappers#getMapper. (#31671) * Add documentation around field aliases. (#31538) * Add validation for field alias mappings. (#31518) * Make sure that field-level security is enforced when using field aliases. (#31807) * Add more comprehensive tests for field aliases in queries + aggregations. (#31565) * Remove the deprecated method DocumentFieldMappers#getFieldMapper. (#32148) * Ensure that field aliases cannot be used in multi-fields. (#32219) * Make sure that field aliases count towards the total fields limit. (#32222) * Fix a test bug around nested aggregations and field aliases. (#32287) * Make sure the _uid field is correctly loaded in scripts. * Fix the failing test case FieldLevelSecurityTests#testParentChild_parentField. * Enforce that field aliases can only be specified on indexes with a single type.

jtibshirani added the WIP label Jun 13, 2018

jtibshirani force-pushed the feature/field-aliases branch from afd01b8 to f7b5b3d Compare June 13, 2018 03:43

colings86 added the :Search Foundations/Mapping Index mappings, including merging and defining field types label Jun 13, 2018

jtibshirani force-pushed the feature/field-aliases branch 3 times, most recently from 5279839 to b08f5a4 Compare June 13, 2018 20:34

jtibshirani mentioned this pull request Jun 13, 2018

New field type=alias including support for querying, aggs, suggestions + more read ops #30230

Closed

jtibshirani force-pushed the feature/field-aliases branch 2 times, most recently from 4ca2f04 to c3cb62b Compare June 15, 2018 18:01

jtibshirani changed the base branch from master to field-aliases June 15, 2018 20:12

jtibshirani force-pushed the feature/field-aliases branch from c3cb62b to 8fbfb6d Compare June 15, 2018 20:14

jtibshirani mentioned this pull request Jun 15, 2018

Field aliases implementation tracking #31372

Closed

9 tasks

jtibshirani changed the title ~~Add support for field aliases in index mappings.~~ Add basic support for field aliases in index mappings. Jun 15, 2018

jtibshirani force-pushed the feature/field-aliases branch 2 times, most recently from 9aed225 to 115f1c5 Compare June 15, 2018 22:25

jtibshirani removed the WIP label Jun 16, 2018

jtibshirani requested a review from jpountz June 16, 2018 01:11

jpountz approved these changes Jun 18, 2018

View reviewed changes

jtibshirani force-pushed the feature/field-aliases branch 2 times, most recently from 67c17ad to 71a7820 Compare June 18, 2018 18:37

jtibshirani added 7 commits June 18, 2018 12:03

Add basic support for field aliases through a new top-level mapper type.

5924fe0

Add tests for queries, aggregations, sorting, and fetching doc values.

89b7937

Make sure we properly handle wildcard fields in query string queries.

a921c37

Add a test for field capabilities.

870c54d

Allow for aliases when requesting suggestions.

b00a4f1

Allow for aliases when requesting highlights.

66741fc

Address code review feedback.

0f9c196

jtibshirani force-pushed the feature/field-aliases branch from 71a7820 to 0f9c196 Compare June 18, 2018 19:03

jtibshirani merged commit 12c45fa into elastic:field-aliases Jun 18, 2018

jtibshirani deleted the feature/field-aliases branch June 18, 2018 21:15

jtibshirani added the >feature label Jun 22, 2018

jtibshirani added a commit to jtibshirani/elasticsearch that referenced this pull request Jul 24, 2018

Add basic support for field aliases in index mappings. (elastic#31287)

3fd380c

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add basic support for field aliases in index mappings. #31287

Add basic support for field aliases in index mappings. #31287

jtibshirani commented Jun 13, 2018 •

edited

Loading

elasticmachine commented Jun 13, 2018

jpountz left a comment

jpountz Jun 18, 2018

jtibshirani Jun 18, 2018

jpountz Jun 18, 2018

jtibshirani Jun 18, 2018

jpountz Jun 18, 2018

jtibshirani Jun 18, 2018

jpountz Jun 18, 2018

jtibshirani Jun 18, 2018

jpountz Jun 18, 2018

jtibshirani Jun 18, 2018

jtibshirani commented Jun 18, 2018

Add basic support for field aliases in index mappings. #31287

Add basic support for field aliases in index mappings. #31287

Conversation

jtibshirani commented Jun 13, 2018 • edited Loading

elasticmachine commented Jun 13, 2018

jpountz left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jtibshirani commented Jun 18, 2018

jtibshirani commented Jun 13, 2018 •

edited

Loading