Discrepancy between multisearch and multisearch template response formats.

[swallez]

Elasticsearch version (bin/elasticsearch --version): 7.16, 8.0, 8.1.

Description of the problem including expected versus actual behavior:

The format of the multisearch response and the multisearch template response are different. The former adds a status field in every response item while the latter does not. I guess this is a discrepancy that was introduced when one was updated and not the other.

Reported by a user of the Java API Client: elastic/elasticsearch-java#110, workaround added to the API specification: elastic/elasticsearch-specification#1320

MultiSearchResponse:

elasticsearch/server/src/main/java/org/elasticsearch/action/search/MultiSearchResponse.java

Lines 153 to 171 in 533f6e0

	public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
	builder.startObject();
	builder.field("took", tookInMillis);
	builder.startArray(Fields.RESPONSES);
	for (Item item : items) {
	builder.startObject();
	if (item.isFailure()) {
	ElasticsearchException.generateFailureXContent(builder, params, item.getFailure(), true);
	builder.field(Fields.STATUS, ExceptionsHelper.status(item.getFailure()).getStatus());
	} else {
	item.getResponse().innerToXContent(builder, params);
	builder.field(Fields.STATUS, item.getResponse().status().getStatus());
	}
	builder.endObject();
	}
	builder.endArray();
	builder.endObject();
	return builder;
	}

MultiSearchTemplateResponse:

elasticsearch/modules/lang-mustache/src/main/java/org/elasticsearch/script/mustache/MultiSearchTemplateResponse.java

Lines 144 to 160 in 533f6e0

	public XContentBuilder toXContent(XContentBuilder builder, ToXContent.Params params) throws IOException {
	builder.startObject();
	builder.field("took", tookInMillis);
	builder.startArray(Fields.RESPONSES);
	for (Item item : items) {
	if (item.isFailure()) {
	builder.startObject();
	ElasticsearchException.generateFailureXContent(builder, params, item.getFailure(), true);
	builder.endObject();
	} else {
	item.getResponse().toXContent(builder, params);
	}
	}
	builder.endArray();
	builder.endObject();
	return builder;
	}

[elasticmachine]

Pinging @elastic/es-search (Team:Search)

[javanna]

Good catch, status was added to multi search responses with #18586 and should have been added to msearch template too. Would be nice to share some code between the two if possible.

[AbeleMM]

@javanna Hey! Sorry for resurrecting the issue, but I wondered if help is still wanted for it. If so, I think I have a fix, so I would be happy to submit a PR and have it checked out.

[javanna]

Help is still wanted, please go ahead @AbeleMM !

[cbuescher]

@swallez we are going to add the "status" field by default in 8.3 with #85496, just pinging you in case there are any follow ups or actions for the clients team in that case.

[swallez]

Thanks @cbuescher! I've opened elastic/elasticsearch-specification#1721 so that we update the API specification for 8.3