add raw json requests docs #196

samuelorji · 2023-10-24T14:32:27Z

Description

Add documentation on how to make raw json / http requests using the client

Issues Resolved

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: samuel orji <awesomeorji@gmail.com>

codecov · 2023-10-24T14:46:23Z

Codecov Report

Merging #196 (8f2fb66) into main (13d0d4b) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #196   +/-   ##
=======================================
  Coverage   73.82%   73.82%           
=======================================
  Files         402      402           
  Lines       63738    63738           
=======================================
  Hits        47055    47055           
  Misses      16683    16683

Flag	Coverage Δ
integration	`73.82% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

dblock

Let's make this a bit more detailed. See opensearch-project/opensearch-py#542 for an example. Add a working sample into samples as well so we can make sure this code runs.

dblock · 2023-10-24T16:00:56Z

USER_GUIDE.md

@@ -7,6 +7,7 @@
    - [Add a Document to the Index](#add-a-document-to-the-index)
    - [Search for a Document](#search-for-a-document)
    - [Delete the Index](#delete-the-index)
+    - [Make raw json requests](#make-raw-json-requests)


Capitalize Titles

dblock · 2023-10-25T18:14:39Z

USER_GUIDE.md

@@ -108,6 +109,90 @@ client
    .await?;
 ```

+### Make Raw Json Requests
+
+To invoke an API that is not supported by the client, use `client.send` method to do so. see [examples/json](./opensearch/examples/json.rs) for a complete working example


Make "See ... ." a full sentence with a period.

dblock · 2023-10-25T18:15:42Z

USER_GUIDE.md

+    )
+    .await?;
+
+assert_eq!(response.status_code().as_u16(), 201_u16);


Do we really have to cast these in Rust, can't write assert_eq!(response.status_code(), 201)?

dblock · 2023-10-25T18:16:35Z

USER_GUIDE.md

+    )
+    .await?;
+
+assert_eq!(response.status_code().as_u16(), 200_u16);


Does this work? Looks like a GET request with a query string, whereas OpenSearch uses POST /.../_search usually.

Look at the response and print some search results?

According to the docs, both work. The http client we use supports GET requests with a body, either way, i've changed to a POST instead

dblock · 2023-10-25T18:16:50Z

USER_GUIDE.md

+        Option::<&String>::None,
+        None,
+    )
+    .await?;


Check response error code like in the other examples above.

opensearch/examples/json.rs

dblock · 2023-10-25T18:17:43Z

opensearch/examples/json.rs

+        )
+        .await?;
+
+    assert_eq!(response.status_code().as_u16(), 200_u16);


Look at the response and print some search results.

dblock

Looks like we call the samples directory samples in other clients, not examples.

Signed-off-by: samuel orji <awesomeorji@gmail.com>

samuelorji · 2023-10-25T19:36:37Z

Looks like we call the samples directory samples in other clients, not examples.

I'm not a Rust expert, but I think in Rust, the convention is to put samples in the examples folder

dblock

LGTM, leaving this to @Xtansia .

Xtansia

Overall looks good, just a couple suggestions to improve the code.

Xtansia · 2023-10-25T23:17:25Z

USER_GUIDE.md

@@ -108,6 +109,94 @@ client
    .await?;
 ```

+### Make Raw Json Requests
+
+To invoke an API that is not supported by the client, use `client.send` method to do so. See [examples/json](./opensearch/examples/json.rs) for a complete working example.


Nit: use the client.send method.

Suggested change

To invoke an API that is not supported by the client, use `client.send` method to do so. See [examples/json](./opensearch/examples/json.rs) for a complete working example.

To invoke an API that is not supported by the client, use the `client.send` method to do so. See [examples/json](./opensearch/examples/json.rs) for a complete working example.

Xtansia · 2023-10-25T23:21:12Z

USER_GUIDE.md

+let info: Value = client
+    .send(
+        Method::Get,
+        "/",
+        HeaderMap::new(),
+        Option::<&String>::None,
+        Option::<&String>::None,
+        None,
+    )
+    .await?
+    .json()
+    .await?;


When there's no body or query params needed, this can be simplified to just use the unit type () rather than &String. Also specifying the types on send::<> means you don't have to fully qualify Option::<>::None.

Suggested change

let info: Value = client

.send(

Method::Get,

"/",

HeaderMap::new(),

Option::<&String>::None,

Option::<&String>::None,

None,

)

.await?

.json()

.await?;

let info: Value = client

.send::<(), ()>(Method::Get, "/", HeaderMap::new(), None, None, None)

.await?

.json()

.await?;

Makes perfect sense, really struggled getting the types to fit 😄

Xtansia · 2023-10-25T23:45:05Z

USER_GUIDE.md

+let index_body = json!({
+      "settings": {
+          "index": {
+              "number_of_shards" : 4
+          }
+      }
+  });
+
+client
+    .send(
+        Method::Put,
+        "/movies",
+        HeaderMap::new(),
+        Option::<&String>::None,
+        Some(index_body.to_string()),
+        None,
+    )
+    .await?;


It's better to use the JsonBody type, rather than .to_string, and can work with any type that implements Serialize.

Suggested change

let index_body = json!({

"settings": {

"index": {

"number_of_shards" : 4

}

}

});

client

.send(

Method::Put,

"/movies",

HeaderMap::new(),

Option::<&String>::None,

Some(index_body.to_string()),

None,

)

.await?;

let index_body: JsonBody<_> = json!({

"settings": {

"index": {

"number_of_shards" : 4

}

}

})

.into();

client

.send(

Method::Put,

"/movies",

HeaderMap::new(),

Option::<&()>::None,

Some(index_body),

None,

)

.await?;

Xtansia · 2023-10-25T23:47:19Z

USER_GUIDE.md

+let q = "miller";
+
+let query = json!({
+    "size": 5,
+    "query": {
+        "multi_match": {
+            "query": q,
+            "fields": ["title^2", "director"]
+        }
+    }
+});
+client
+    .send(
+        Method::Post,
+        "/movies/_search",
+        HeaderMap::new(),
+        Option::<&String>::None,
+        Some(query.to_string()),
+        None,
+    )
+    .await?;


Similar to above regarding JsonBody

Suggested change

let q = "miller";

let query = json!({

"size": 5,

"query": {

"multi_match": {

"query": q,

"fields": ["title^2", "director"]

}

}

});

client

.send(

Method::Post,

"/movies/_search",

HeaderMap::new(),

Option::<&String>::None,

Some(query.to_string()),

None,

)

.await?;

let q = "miller";

let query: JsonBody<_> = json!({

"size": 5,

"query": {

"multi_match": {

"query": q,

"fields": ["title^2", "director"]

}

}

})

.into();

client

.send(

Method::Post,

"/movies/_search",

HeaderMap::new(),

Option::<&()>::None,

Some(query),

None,

)

.await?;

Xtansia · 2023-10-25T23:49:06Z

USER_GUIDE.md

+client
+  .send(
+      Method::Delete,
+      "/movies",
+      HeaderMap::new(),
+      Option::<&String>::None,
+      Option::<&String>::None,
+      None,
+  )
+  .await?;


Similar to above regarding using unit type ()

Suggested change

client

.send(

Method::Delete,

"/movies",

HeaderMap::new(),

Option::<&String>::None,

Option::<&String>::None,

None,

)

.await?;

client

.send::<(), ()>(

Method::Delete,

"/movies",

HeaderMap::new(),

None,

None,

None,

)

.await?;

Xtansia · 2023-10-25T23:56:08Z

opensearch/examples/json.rs

+            &format!("/{index_name}/_doc/{document_id}"),
+            HeaderMap::new(),
+            Some(&vec![("refresh", "true")]),
+            Some(index_body.to_string()),


This should reference the document not the index_body

Suggested change

Some(index_body.to_string()),

Some(document),

Xtansia · 2023-10-25T23:58:09Z

opensearch/examples/json.rs

+        )
+        .await?;
+
+    assert_eq!(create_index_response.status_code(), 200);


This should reference create_document_response not create_index_response, additionally putting a not yet existing document returns a 201 Created rather than 200

Suggested change

assert_eq!(create_index_response.status_code(), 200);

assert_eq!(create_document_response.status_code(), 201);

Xtansia · 2023-10-26T00:02:16Z

opensearch/examples/json.rs

+    let query = json!({
+        "size": 5,
+        "query": {
+            "multi_match": {
+                "query": q,
+                "fields": ["title^2", "director"]
+            }
+        }
+    });
+
+    let search_response = client
+        .send(
+            Method::Post,
+            "/movies/_search",
+            HeaderMap::new(),
+            Option::<&String>::None,
+            Some(query.to_string()),
+            None,
+        )
+        .await?;


Following the suggestions about JsonBody and (), also use format! for path to match other requests.

Suggested change

let query = json!({

"size": 5,

"query": {

"multi_match": {

"query": q,

"fields": ["title^2", "director"]

}

}

});

let search_response = client

.send(

Method::Post,

"/movies/_search",

HeaderMap::new(),

Option::<&String>::None,

Some(query.to_string()),

None,

)

.await?;

let query: JsonBody<_> = json!({

"size": 5,

"query": {

"multi_match": {

"query": q,

"fields": ["title^2", "director"]

}

}

})

.into();

let search_response = client

.send(

Method::Post,

&format!("/{index_name}/_search"),

HeaderMap::new(),

Option::<&()>::None,

Some(query),

None,

)

.await?;

Xtansia · 2023-10-26T00:03:52Z

opensearch/examples/json.rs

+    let delete_document_response = client
+        .send(
+            Method::Delete,
+            &format!("/{index_name}/_doc/{document_id}"),
+            HeaderMap::new(),
+            Option::<&String>::None,
+            Option::<&String>::None,
+            None,
+        )
+        .await?;


Following unit-type suggestion

Suggested change

let delete_document_response = client

.send(

Method::Delete,

&format!("/{index_name}/_doc/{document_id}"),

HeaderMap::new(),

Option::<&String>::None,

Option::<&String>::None,

None,

)

.await?;

let delete_document_response = client

.send::<(), ()>(

Method::Delete,

&format!("/{index_name}/_doc/{document_id}"),

HeaderMap::new(),

None,

None,

None,

)

.await?;

Xtansia · 2023-10-26T00:04:34Z

opensearch/examples/json.rs

+    let delete_response = client
+        .send(
+            Method::Delete,
+            &format!("/{index_name}"),
+            HeaderMap::new(),
+            Option::<&String>::None,
+            Option::<&String>::None,
+            None,
+        )
+        .await?;


Following unit-type suggestions:

Suggested change

let delete_response = client

.send(

Method::Delete,

&format!("/{index_name}"),

HeaderMap::new(),

Option::<&String>::None,

Option::<&String>::None,

None,

)

.await?;

let delete_response = client

.send::<(), ()>(

Method::Delete,

&format!("/{index_name}"),

HeaderMap::new(),

None,

None,

None,

)

.await?;

Xtansia · 2023-10-26T00:11:55Z

opensearch/examples/json.rs

+        .await?;
+
+    assert_eq!(search_response.status_code(), 200);
+


You can print the results from the search like so:

Suggested change

let search_result = search_response.json::<Value>().await?;

println!("Hits: {:#?}", search_result["hits"]["hits"].as_array().unwrap());

Signed-off-by: samuel orji <awesomeorji@gmail.com>

reta · 2023-10-26T13:01:47Z

USER_GUIDE.md

+#### GET
+The following example returns the server version information via `GET /`.
+```rust
+let info: Value = client


Not necessarily the issue, may be using Future combinators (use futures::TryFutureExt;) would be a bit more concise as an example:

Suggested change

let info: Value = client

let info: Value = client.send::<(), ()>(

Method::Get,

"/",

HeaderMap::new(),

None,

None,

None,

)

.and_then(|r| r.json::<Value>())

.await?;

opensearch-project/opensearch-rs#196 Signed-off-by: dblock <dblock@amazon.com>

samuelorji requested review from dblock, Xtansia, VachaShah and reta as code owners October 24, 2023 14:32

add raw json requests docs

de28ab4

Signed-off-by: samuel orji <awesomeorji@gmail.com>

samuelorji force-pushed the docs/ad-raw-json-docs branch from 33ad290 to de28ab4 Compare October 24, 2023 14:35

dblock requested changes Oct 24, 2023

View reviewed changes

samuelorji requested a review from dblock October 25, 2023 16:05

dblock reviewed Oct 25, 2023

View reviewed changes

review feedback

7a046c5

Signed-off-by: samuel orji <awesomeorji@gmail.com>

samuelorji force-pushed the docs/ad-raw-json-docs branch from b3e25ff to 7a046c5 Compare October 25, 2023 19:34

samuelorji requested a review from dblock October 25, 2023 19:37

dblock approved these changes Oct 25, 2023

View reviewed changes

Xtansia added the OSCI-2023 label Oct 25, 2023

Xtansia requested changes Oct 26, 2023

View reviewed changes

Xtansia reviewed Oct 26, 2023

View reviewed changes

review feedback

8f2fb66

Signed-off-by: samuel orji <awesomeorji@gmail.com>

samuelorji force-pushed the docs/ad-raw-json-docs branch from 142b493 to 8f2fb66 Compare October 26, 2023 09:27

samuelorji requested a review from Xtansia October 26, 2023 09:40

reta reviewed Oct 26, 2023

View reviewed changes

Xtansia approved these changes Oct 26, 2023

View reviewed changes

Xtansia merged commit 1041309 into opensearch-project:main Oct 26, 2023
32 checks passed

dblock added a commit to dblock/opensearch-rust-client-demo that referenced this pull request Oct 30, 2023

Replace high level requests with raw JSON.

c3eab66

opensearch-project/opensearch-rs#196 Signed-off-by: dblock <dblock@amazon.com>

dblock mentioned this pull request Oct 30, 2023

[FEATURE] Make raw JSON REST requests to OpenSearch #193

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add raw json requests docs #196

add raw json requests docs #196

samuelorji commented Oct 24, 2023

codecov bot commented Oct 24, 2023 •

edited

Loading

dblock left a comment

dblock Oct 24, 2023

dblock Oct 25, 2023

dblock Oct 25, 2023

dblock Oct 25, 2023

samuelorji Oct 25, 2023

dblock Oct 25, 2023

dblock Oct 25, 2023

dblock left a comment

samuelorji commented Oct 25, 2023

dblock left a comment

Xtansia left a comment

Xtansia Oct 25, 2023

Xtansia Oct 25, 2023

samuelorji Oct 26, 2023

Xtansia Oct 25, 2023

Xtansia Oct 25, 2023

Xtansia Oct 25, 2023

Xtansia Oct 25, 2023

Xtansia Oct 25, 2023

Xtansia Oct 26, 2023

Xtansia Oct 26, 2023

Xtansia Oct 26, 2023

Xtansia Oct 26, 2023

reta Oct 26, 2023

	To invoke an API that is not supported by the client, use `client.send` method to do so. See [examples/json](./opensearch/examples/json.rs) for a complete working example.
	To invoke an API that is not supported by the client, use the `client.send` method to do so. See [examples/json](./opensearch/examples/json.rs) for a complete working example.

	assert_eq!(create_index_response.status_code(), 200);
	assert_eq!(create_document_response.status_code(), 201);



	let search_result = search_response.json::<Value>().await?;
	println!("Hits: {:#?}", search_result["hits"]["hits"].as_array().unwrap());

add raw json requests docs #196

add raw json requests docs #196

Conversation

samuelorji commented Oct 24, 2023

Description

Issues Resolved

codecov bot commented Oct 24, 2023 • edited Loading

Codecov Report

dblock left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dblock left a comment

Choose a reason for hiding this comment

samuelorji commented Oct 25, 2023

dblock left a comment

Choose a reason for hiding this comment

Xtansia left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Oct 24, 2023 •

edited

Loading