Migrate client transports to Apache HttpClient / Core 5.x

[reta]

Is your feature request related to a problem? Please describe.
All OpenSearch clients (rest-client., RHLC, opensearch-java) use Apache HttpComponents / Core 4.x, which are already EOLed and superseded by Apache HttpClient / Core 5.x. The migration to the Apache HttpClient / Core 5.x is a necessary undertaking (not only to benefit from HTTP/2 support but also to rely on maintanable libraries).

Describe the solution you'd like
Migrate client transports to Apache HttpClient / Core 5.x

Describe alternatives you've considered
Stay on Apache HttpComponents / Core 4.x

Additional context
See please #4257

[reta]

@dblock @nknize @adnapibar any concerns / objections / risks? thanks!

[dblock]

I wonder whether our time is better spent deleting the client-side components from this repo and moving them into a separate component as legacy without this upgrade?

[reta]

Feasible, but what will we offer as a client component replacement?

[dblock]

Feasible, but what will we offer as a client component replacement?

I mean we need to decouple server from client. Users that want to talk to a server would consume opensearch-java which should contain everything it needs to talk to OpenSearch. And users that want to build extensions would consume opensearch-sdk-java. So maybe we should move REST clients into opensearch-java now (and delete all redundant parts)?

[reta]

Ah, sure, we could explore this path, but underlying transport (low level HTTP client) needs to be updated anyway, I think those two tracks could go in parallel.

[dblock]

Ah, sure, we could explore this path, but underlying transport (low level HTTP client) needs to be updated anyway, I think those two tracks could go in parallel.

I was just trying to explore ways where less work can be done, or we can divide and conquer.

[saratvemulapalli]

@reta just read up your PR upgrading to Core 5.x.
I am not aware how it works, but who uses low/high level client for OpenSearch ?

[reta]

@reta just read up your PR upgrading to Core 5.x. I am not aware how it works, but who uses low/high level client for OpenSearch ?

@saratvemulapalli sorry if I misinterpreted your question, the Rest Client and RHLC (Rest High Level Client) and https://github.com/Opensearch-project/Opensearch-java use Apache HttpClient under the hood. Now, different projects and application use either RHLC or opensearch-java in general.

[saratvemulapalli]

Thanks @reta I understand the first part. My question is tangential to this upgrade, I completely agree it makes sense to upgrade HttpClient.

My question mainly was, do customers use RHLC, why dont they use opensearch-java instead?
I see elastic deprecated it[1]

[1] https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/java-rest-high.html

[reta]

Thanks @reta I understand the first part. My question is tangential to this upgrade, I completely agree it makes sense to upgrade HttpClient.

My question mainly was, do customers use RHLC, why dont they use opensearch-java instead? I see elastic deprecated it[1]

[1] https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/java-rest-high.html

Yeah, opensearch-java uses RestClient [1] under the hood which consequently uses Apache HttpClient. AFAIK, we do not follow Elasticsearch with respect to deprecating RHLC. Why customers don't use opensearch-java instead RHLC - good question, I think the message from OpenSearch was not clear enough which one is preferred.

[1] https://github.com/opensearch-project/opensearch-java/blob/main/java-client/src/main/java/org/opensearch/client/transport/rest_client/RestClientTransport.java#L92

[saratvemulapalli]

Why customers don't use opensearch-java instead RHLC - good question, I think the message from OpenSearch was not clear enough which one is preferred.

Sounds like it needs more discussion, i'll open an issue.
I was curious how these clients are being used, looks like low level rest client is consumed by our Java client[1]

[1] https://github.com/opensearch-project/opensearch-java/blob/main/java-client/build.gradle.kts#L139

[reta]

@saratvemulapalli correct, opensearch-java -> rest client -> Apache HttpClient