Implement update by query #1644

ffaoudi · 2021-01-13T15:17:41Z

You have read the Spring Data contribution guidelines.
There is a ticket in the bug tracker for the project in our issue tracker.
You use the code formatters provided here and have them applied to your changes. Don’t submit any formatting related changes.
You submit test cases (unit or integration tests) that back your changes.
You added yourself as author in the headers of the classes you touched. Amend the date range in the Apache license header if needed. For new types, add the license header (copy from another file and set the current year only).

sothawo

#1446 is about adding the update by query feature to the library. This means that both ReactiveDocumentOperations and DocumentOperations and their implementations (both on-reactive and reactive) must have either their update methods enhanced to support this case.

Your PR only adds code to the reactive client and not the the operations/templates.

ffaoudi · 2021-01-13T21:53:57Z

Hi @sothawo, for DocumentOperations and ReactiveDocumentOperations, i see that input and output are spring data type not elasticsearch data type, this mean that for the update by query i need to create a new type and transforme it to the UpdateByQueryRequest elasticsearch type?

sothawo · 2021-01-14T19:32:25Z

Yes, we do not want to leak Elasticsearch types into the Spring Data Elasticsearch API. Although there are a few places where this happens, returning Aggregations for example. These places are either old code that has not been fixed or where we did not have the resources yet to change that.

For new additions we should not do this.

About how a solution could be done, check my comment in the issue and let me know what you think about it.

sothawo

This looks pretty good, thanks a lot for your work. I made some remarks.

Can you please rebase your branch onto master and force push it? I had a PR merged that deals with custom routing and I want to see how this fits together with the routing for the updates

src/main/java/org/springframework/data/elasticsearch/core/query/BulkByScrollResponse.java

src/main/java/org/springframework/data/elasticsearch/core/ScriptType.java

src/test/java/org/springframework/data/elasticsearch/core/ElasticsearchRestTemplateTests.java

sothawo · 2021-01-20T18:57:20Z

src/main/java/org/springframework/data/elasticsearch/core/RequestFactory.java

+		return updateByQueryRequest;
+	}
+
+	public UpdateByQueryRequestBuilder updateByQueryRequestBuilderFor(Client client, UpdateQuery query, IndexCoordinates index) {


Suggested change

public UpdateByQueryRequestBuilder updateByQueryRequestBuilderFor(Client client, UpdateQuery query, IndexCoordinates index) {

public UpdateByQueryRequestBuilder updateByQueryRequestBuilder(Client client, UpdateQuery query, IndexCoordinates index) {

please remove these For suffixes. Don't know why I named this function that way.

sothawo · 2021-01-20T18:59:53Z

src/main/java/org/springframework/data/elasticsearch/core/query/BulkByScrollResponse.java

+		return new BulkByScrollResponseBuilder();
+	}
+
+	public static BulkByScrollResponse translateFrom(org.elasticsearch.index.reindex.BulkByScrollResponse bulkByScrollResponse) {


Suggested change

public static BulkByScrollResponse translateFrom(org.elasticsearch.index.reindex.BulkByScrollResponse bulkByScrollResponse) {

public static BulkByScrollResponse from(org.elasticsearch.index.reindex.BulkByScrollResponse bulkByScrollResponse) {

We name such factory methods normally just from(...) or of(...)

done for of(...)

sothawo · 2021-01-20T19:02:32Z

src/main/java/org/springframework/data/elasticsearch/client/util/RequestConverters.java

+		Params params = new Params(request)
+				.withRouting(updateByQueryRequest.getRouting())
+				.withPipeline(updateByQueryRequest.getPipeline())
+				.withRefresh(updateByQueryRequest.isRefresh())
 				.withTimeout(updateByQueryRequest.getTimeout())
 				.withWaitForActiveShards(updateByQueryRequest.getWaitForActiveShards())
 				.withRequestsPerSecond(updateByQueryRequest.getRequestsPerSecond())


If you want to have this formatting, you need to add // to the end of every line. Otherwise the formatter settings will set it back to the longer lines.

I would prefer to change the formatter, but that's the Spring Data style.

sothawo

two potential places with wrong code, otherwise minor stuff

sothawo · 2021-01-21T19:06:33Z

src/main/java/org/springframework/data/elasticsearch/core/RequestFactory.java

@@ -1404,7 +1398,7 @@ public UpdateRequest updateRequest(UpdateQuery query, IndexCoordinates index) {
 			if (params == null) {
 				params = new HashMap<>();
 			}
-			Script script = new Script(ScriptType.INLINE, query.getLang(), query.getScript(), params);
+			Script script = new Script(getScriptType(ScriptType.INLINE), query.getLang(), query.getScript(), params);


shouldn't that be

Suggested change

Script script = new Script(getScriptType(ScriptType.INLINE), query.getLang(), query.getScript(), params);

Script script = new Script(getScriptType(query.getScriptType()), query.getLang(), query.getScript(), params);

sothawo · 2021-01-21T19:07:04Z

src/main/java/org/springframework/data/elasticsearch/core/RequestFactory.java

@@ -1478,7 +1472,7 @@ public UpdateRequestBuilder updateRequestBuilderFor(Client client, UpdateQuery q
 			if (params == null) {
 				params = new HashMap<>();
 			}
-			Script script = new Script(ScriptType.INLINE, query.getLang(), query.getScript(), params);
+			Script script = new Script(getScriptType(ScriptType.INLINE), query.getLang(), query.getScript(), params);


Suggested change

Script script = new Script(getScriptType(ScriptType.INLINE), query.getLang(), query.getScript(), params);

Script script = new Script(getScriptType(query.getScriptType(), query.getLang(), query.getScript(), params);

sothawo · 2021-01-21T19:13:01Z

src/main/java/org/springframework/data/elasticsearch/core/query/UpdateByQueryResponse.java

+
+	private UpdateByQueryResponse(long took, boolean timedOut, long total, long updated, long deleted, int batches,
+								  long versionConflicts, long noops, long bulkRetries, long searchRetries,
+								  String reasonCancelled, List<Failure> failures) {


Suggested change

String reasonCancelled, List<Failure> failures) {

@Nullable String reasonCancelled, List<Failure> failures) {

sothawo · 2021-01-21T19:13:41Z

src/main/java/org/springframework/data/elasticsearch/core/query/UpdateByQueryResponse.java

+		 * @param failure {@link BulkItemResponse.Failure} to translate
+		 * @return a new {@link Failure}
+		 */
+		public static Failure translateFrom(BulkItemResponse.Failure failure) {


Suggested change

public static Failure translateFrom(BulkItemResponse.Failure failure) {

public static Failure of(BulkItemResponse.Failure failure) {

spring-projects-issues added the status: waiting-for-triage An issue we've not yet triaged label Jan 13, 2021

sothawo requested changes Jan 13, 2021

View reviewed changes

sothawo removed the status: waiting-for-triage An issue we've not yet triaged label Jan 14, 2021

spring-projects-issues added the status: waiting-for-triage An issue we've not yet triaged label Jan 14, 2021

ffaoudi force-pushed the add-UpdateByQueryRequest branch from 1a65f9f to 30ad335 Compare January 19, 2021 22:49

sothawo requested changes Jan 20, 2021

View reviewed changes

sothawo removed the status: waiting-for-triage An issue we've not yet triaged label Jan 20, 2021

spring-projects-issues added the status: waiting-for-triage An issue we've not yet triaged label Jan 20, 2021

ffaoudi force-pushed the add-UpdateByQueryRequest branch from 30ad335 to d04bdc1 Compare January 20, 2021 23:51

sothawo requested changes Jan 21, 2021

View reviewed changes

ffaoudi force-pushed the add-UpdateByQueryRequest branch from d04bdc1 to 75f5a70 Compare January 22, 2021 08:13

Implement update by query

9904ddd

ffaoudi force-pushed the add-UpdateByQueryRequest branch from 75f5a70 to 9904ddd Compare January 22, 2021 08:13

sothawo approved these changes Jan 23, 2021

View reviewed changes

sothawo merged commit 0ee0164 into spring-projects:master Jan 23, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement update by query #1644

Implement update by query #1644

ffaoudi commented Jan 13, 2021 •

edited

Loading

sothawo left a comment

ffaoudi commented Jan 13, 2021

sothawo commented Jan 14, 2021

sothawo left a comment

sothawo Jan 20, 2021

ffaoudi Jan 20, 2021

sothawo Jan 20, 2021

ffaoudi Jan 20, 2021

sothawo Jan 20, 2021

ffaoudi Jan 20, 2021

sothawo left a comment

sothawo Jan 21, 2021

ffaoudi Jan 22, 2021

sothawo Jan 21, 2021

ffaoudi Jan 22, 2021

sothawo Jan 21, 2021

ffaoudi Jan 22, 2021

sothawo Jan 21, 2021

ffaoudi Jan 22, 2021

	public UpdateByQueryRequestBuilder updateByQueryRequestBuilderFor(Client client, UpdateQuery query, IndexCoordinates index) {
	public UpdateByQueryRequestBuilder updateByQueryRequestBuilder(Client client, UpdateQuery query, IndexCoordinates index) {

	public static BulkByScrollResponse translateFrom(org.elasticsearch.index.reindex.BulkByScrollResponse bulkByScrollResponse) {
	public static BulkByScrollResponse from(org.elasticsearch.index.reindex.BulkByScrollResponse bulkByScrollResponse) {

	Script script = new Script(getScriptType(ScriptType.INLINE), query.getLang(), query.getScript(), params);
	Script script = new Script(getScriptType(query.getScriptType()), query.getLang(), query.getScript(), params);

	String reasonCancelled, List<Failure> failures) {
	@Nullable String reasonCancelled, List<Failure> failures) {

	public static Failure translateFrom(BulkItemResponse.Failure failure) {
	public static Failure of(BulkItemResponse.Failure failure) {

Implement update by query #1644

Implement update by query #1644

Conversation

ffaoudi commented Jan 13, 2021 • edited Loading

sothawo left a comment

Choose a reason for hiding this comment

ffaoudi commented Jan 13, 2021

sothawo commented Jan 14, 2021

sothawo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sothawo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ffaoudi commented Jan 13, 2021 •

edited

Loading