From e09cc9e9044d697abbe1ff0812ad08ce9bbd6cfe Mon Sep 17 00:00:00 2001 From: Scott Chamberlain Date: Wed, 13 Sep 2017 08:09:12 -0700 Subject: [PATCH] remove vignettes for now as wont build on r-devel --- DESCRIPTION | 4 +- vignettes/elastic_intro.Rmd | 396 --------------------- vignettes/search.Rmd | 670 ------------------------------------ 3 files changed, 1 insertion(+), 1069 deletions(-) delete mode 100644 vignettes/elastic_intro.Rmd delete mode 100644 vignettes/search.Rmd diff --git a/DESCRIPTION b/DESCRIPTION index c41d5c6..f5c4879 100644 --- a/DESCRIPTION +++ b/DESCRIPTION @@ -14,7 +14,6 @@ Authors@R: person("Scott", "Chamberlain", role = c("aut", "cre"), email = "myrmecocystus@gmail.com") URL: https://github.com/ropensci/elastic BugReports: https://github.com/ropensci/elastic/issues -VignetteBuilder: knitr Imports: utils, methods, @@ -23,6 +22,5 @@ Imports: jsonlite (>= 1.1) Suggests: roxygen2 (>= 6.0.1), - testthat, - knitr + testthat RoxygenNote: 6.0.1 diff --git a/vignettes/elastic_intro.Rmd b/vignettes/elastic_intro.Rmd deleted file mode 100644 index 64d78a4..0000000 --- a/vignettes/elastic_intro.Rmd +++ /dev/null @@ -1,396 +0,0 @@ - - - - -elastic introduction -====== - -`elastic` is an R client for [Elasticsearch](https://www.elastic.co/products/elasticsearch). This vignette is an introduction to the package, while other vignettes dive into the details of various topics. - -## Installation - -You can install from CRAN (once the package is up there) - - -```r -install.packages("elastic") -``` - -Or the development version from GitHub - - -```r -install.packages("devtools") -devtools::install_github("ropensci/elastic") -``` - -Then load the package - - -```r -library("elastic") -``` - -## Elasticsearch info - -+ [Elasticsearch home page](https://www.elastic.co/) -+ [API docs](http://www.elastic.co/guide/en/elasticsearch/reference/current/index.html) - -## Install Elasticsearch - -* [Elasticsearch installation help](http://www.elastic.co/guide/en/elasticsearch/reference/current/_installation.html) - -__Unix (linux/osx)__ - -Replace `2.1.1` with the version you are working with. - -+ Download zip or tar file from Elasticsearch [see here for download](https://www.elastic.co/downloads), e.g., `curl -L -O https://download.elasticsearch.org/elasticsearch/release/org/elasticsearch/distribution/tar/elasticsearch/2.1.1/elasticsearch-2.1.1.tar.gz` -+ Extract: `tar -zxvf elasticsearch-2.1.1.tar.gz` -+ Move it: `sudo mv /path/to/elasticsearch-2.1.1 /usr/local` (replace version with your version) -+ Navigate to /usr/local: `cd /usr/local` -+ Delete symlinked `elasticsearch` directory: `rm -rf elasticsearch` -+ Add shortcut: `sudo ln -s elasticsearch-2.1.1 elasticsearch` (replace version with your version) - -On OSX, you can install via Homebrew: `brew install elasticsearch` - -__Windows__ - -Windows users can follow the above, but unzip the zip file instead of uncompressing the tar file. - -## Start Elasticsearch - -* Navigate to elasticsearch: `cd /usr/local/elasticsearch` -* Start elasticsearch: `bin/elasticsearch` - -I create a little bash shortcut called `es` that does both of the above commands in one step (`cd /usr/local/elasticsearch && bin/elasticsearch`). - -__Note:__ Windows users should run the `elasticsearch.bat` file - -## Initialize connection - -The function `connect()` is used before doing anything else to set the connection details to your remote or local elasticsearch store. The details created by `connect()` are written to your options for the current session, and are used by `elastic` functions. - - -```r -connect() -``` - -``` -#> transport: http -#> host: 127.0.0.1 -#> port: 9200 -#> path: NULL -#> username: NULL -#> password: -#> errors: simple -#> headers (names): NULL -``` - -On package load, your base url and port are set to `http://127.0.0.1` and `9200`, respectively. You can of course override these settings per session or for all sessions. - -## Get some data - -Elasticsearch has a bulk load API to load data in fast. The format is pretty weird though. It's sort of JSON, but would pass no JSON linter. I include a few data sets in `elastic` so it's easy to get up and running, and so when you run examples in this package they'll actually run the same way (hopefully). - -I have prepared a non-exported function useful for preparing the weird format that Elasticsearch wants for bulk data loads (see below). See `elastic:::make_bulk_plos` and `elastic:::make_bulk_gbif`. - -### Shakespeare data - -Elasticsearch provides some data on Shakespeare plays. I've provided a subset of this data in this package. Get the path for the file specific to your machine: - - -```r -shakespeare <- system.file("examples", "shakespeare_data.json", package = "elastic") -``` - -Then load the data into Elasticsearch: - - -```r -docs_bulk(shakespeare) -``` - -If you need some big data to play with, the shakespeare dataset is a good one to start with. You can get the whole thing and pop it into Elasticsearch (beware, may take up to 10 minutes or so.): - -```sh -curl -XGET https://www.elastic.co/guide/en/kibana/3.0/snippets/shakespeare.json > shakespeare.json -curl -XPUT localhost:9200/_bulk --data-binary @shakespeare.json -``` - -### Public Library of Science (PLOS) data - -A dataset inluded in the `elastic` package is metadata for PLOS scholarly articles. Get the file path, then load: - - -```r -plosdat <- system.file("examples", "plos_data.json", package = "elastic") -docs_bulk(plosdat) -``` - -### Global Biodiversity Information Facility (GBIF) data - -A dataset inluded in the `elastic` package is data for GBIF species occurrence records. Get the file path, then load: - - -```r -gbifdat <- system.file("examples", "gbif_data.json", package = "elastic") -docs_bulk(gbifdat) -``` - -GBIF geo data with a coordinates element to allow `geo_shape` queries - - -```r -gbifgeo <- system.file("examples", "gbif_geo.json", package = "elastic") -docs_bulk(gbifgeo) -``` - -### More data sets - -There are more datasets formatted for bulk loading in the `ropensci/elastic_data` GitHub repository. Find it at [https://github.com/ropensci/elastic_data](https://github.com/ropensci/elastic_data) - -## Search - -Search the `plos` index and only return 1 result - - -```r -Search(index="plos", size=1)$hits$hits -``` - -``` -#> [[1]] -#> [[1]]$`_index` -#> [1] "plos" -#> -#> [[1]]$`_type` -#> [1] "article" -#> -#> [[1]]$`_id` -#> [1] "0" -#> -#> [[1]]$`_score` -#> [1] 1 -#> -#> [[1]]$`_source` -#> [[1]]$`_source`$id -#> [1] "10.1371/journal.pone.0007737" -#> -#> [[1]]$`_source`$title -#> [1] "Phospholipase C-β4 Is Essential for the Progression of the Normal Sleep Sequence and Ultradian Body Temperature Rhythms in Mice" -``` - -Search the `plos` index, and the `article` document type, and query for _antibody_, limit to 1 result - - -```r -Search(index="plos", type="article", q="antibody", size=1)$hits$hits -``` - -``` -#> [[1]] -#> [[1]]$`_index` -#> [1] "plos" -#> -#> [[1]]$`_type` -#> [1] "article" -#> -#> [[1]]$`_id` -#> [1] "568" -#> -#> [[1]]$`_score` -#> [1] 4.165291 -#> -#> [[1]]$`_source` -#> [[1]]$`_source`$id -#> [1] "10.1371/journal.pone.0085002" -#> -#> [[1]]$`_source`$title -#> [1] "Evaluation of 131I-Anti-Angiotensin II Type 1 Receptor Monoclonal Antibody as a Reporter for Hepatocellular Carcinoma" -``` - -## Get documents - -Get document with `id=1` - - -```r -docs_get(index='plos', type='article', id=1) -``` - -``` -#> $`_index` -#> [1] "plos" -#> -#> $`_type` -#> [1] "article" -#> -#> $`_id` -#> [1] "1" -#> -#> $`_version` -#> [1] 1 -#> -#> $found -#> [1] TRUE -#> -#> $`_source` -#> $`_source`$id -#> [1] "10.1371/journal.pone.0098602" -#> -#> $`_source`$title -#> [1] "Population Genetic Structure of a Sandstone Specialist and a Generalist Heath Species at Two Levels of Sandstone Patchiness across the Strait of Gibraltar" -``` - -Get certain fields - - -```r -docs_get(index='plos', type='article', id=1, fields='id') -``` - -``` -#> $`_index` -#> [1] "plos" -#> -#> $`_type` -#> [1] "article" -#> -#> $`_id` -#> [1] "1" -#> -#> $`_version` -#> [1] 1 -#> -#> $found -#> [1] TRUE -``` - -## Get multiple documents at once - -Same index and type, different document ids - - -```r -docs_mget(index="plos", type="article", id=3:4) -``` - -``` -#> $docs -#> $docs[[1]] -#> $docs[[1]]$`_index` -#> [1] "plos" -#> -#> $docs[[1]]$`_type` -#> [1] "article" -#> -#> $docs[[1]]$`_id` -#> [1] "3" -#> -#> $docs[[1]]$`_version` -#> [1] 1 -#> -#> $docs[[1]]$found -#> [1] TRUE -#> -#> $docs[[1]]$`_source` -#> $docs[[1]]$`_source`$id -#> [1] "10.1371/journal.pone.0107756" -#> -#> $docs[[1]]$`_source`$title -#> [1] "The Effect of S-Adenosylmethionine on Cognitive Performance in Mice: An Animal Model Meta-Analysis" -#> -#> -#> -#> $docs[[2]] -#> $docs[[2]]$`_index` -#> [1] "plos" -#> -#> $docs[[2]]$`_type` -#> [1] "article" -#> -#> $docs[[2]]$`_id` -#> [1] "4" -#> -#> $docs[[2]]$`_version` -#> [1] 1 -#> -#> $docs[[2]]$found -#> [1] TRUE -#> -#> $docs[[2]]$`_source` -#> $docs[[2]]$`_source`$id -#> [1] "10.1371/journal.pone.0107758" -#> -#> $docs[[2]]$`_source`$title -#> [1] "Lactobacilli Inactivate Chlamydia trachomatis through Lactic Acid but Not H2O2" -``` - -Different indeces, types, and ids - - -```r -docs_mget(index_type_id=list(c("plos","article",1), c("gbif","record",1)))$docs[[1]] -``` - -``` -#> $`_index` -#> [1] "plos" -#> -#> $`_type` -#> [1] "article" -#> -#> $`_id` -#> [1] "1" -#> -#> $`_version` -#> [1] 1 -#> -#> $found -#> [1] TRUE -#> -#> $`_source` -#> $`_source`$id -#> [1] "10.1371/journal.pone.0098602" -#> -#> $`_source`$title -#> [1] "Population Genetic Structure of a Sandstone Specialist and a Generalist Heath Species at Two Levels of Sandstone Patchiness across the Strait of Gibraltar" -``` - -## Raw JSON data - -You can optionally get back raw `json` from `Search()`, `docs_get()`, and `docs_mget()` setting parameter `raw=TRUE`. - -For example: - - -```r -(out <- docs_mget(index="plos", type="article", id=5:6, raw=TRUE)) -``` - -``` -#> [1] "{\"docs\":[{\"_index\":\"plos\",\"_type\":\"article\",\"_id\":\"5\",\"_version\":1,\"found\":true,\"_source\":{\"id\":\"10.1371/journal.pone.0085123\",\"title\":\"MiR-21 Is under Control of STAT5 but Is Dispensable for Mammary Development and Lactation\"}},{\"_index\":\"plos\",\"_type\":\"article\",\"_id\":\"6\",\"_version\":1,\"found\":true,\"_source\":{\"id\":\"10.1371/journal.pone.0098600\",\"title\":\"Correction: Designing Mixed Species Tree Plantations for the Tropics: Balancing Ecological Attributes of Species with Landholder Preferences in the Philippines\"}}]}" -#> attr(,"class") -#> [1] "elastic_mget" -``` - -Then parse - - -```r -jsonlite::fromJSON(out) -``` - -``` -#> $docs -#> _index _type _id _version found _source.id -#> 1 plos article 5 1 TRUE 10.1371/journal.pone.0085123 -#> 2 plos article 6 1 TRUE 10.1371/journal.pone.0098600 -#> _source.title -#> 1 MiR-21 Is under Control of STAT5 but Is Dispensable for Mammary Development and Lactation -#> 2 Correction: Designing Mixed Species Tree Plantations for the Tropics: Balancing Ecological Attributes of Species with Landholder Preferences in the Philippines -``` diff --git a/vignettes/search.Rmd b/vignettes/search.Rmd deleted file mode 100644 index 8c1d8f3..0000000 --- a/vignettes/search.Rmd +++ /dev/null @@ -1,670 +0,0 @@ - - - - -elastic searching -====== - -## Load elastic - - -```r -library("elastic") -``` - -## The Search function - -The main interface to searching documents in your Elasticsearch store is the function `Search()`. I nearly always develop R software using all lowercase, but R has a function called `search()`, and I wanted to avoid collision with that function. - -`Search()` is an interface to both the HTTP search API (in which queries are passed in the URI of the request, meaning queries have to be relatively simple), as well as the POST API, or the Query DSL, in which queries are passed in the body of the request (so can be much more complex). - -There are a huge amount of ways you can search Elasticsearch documents - this tutorial covers some of them, and highlights the ways in which you interact with the R outputs. - -### Search an index - - -```r -out <- Search(index="shakespeare") -out$hits$total -``` - -``` -#> [1] 5000 -``` - - -```r -out$hits$hits[[1]] -``` - -``` -#> $`_index` -#> [1] "shakespeare" -#> -#> $`_type` -#> [1] "act" -#> -#> $`_id` -#> [1] "0" -#> -#> $`_score` -#> [1] 1 -#> -#> $`_source` -#> $`_source`$line_id -#> [1] 1 -#> -#> $`_source`$play_name -#> [1] "Henry IV" -#> -#> $`_source`$line_number -#> [1] "" -#> -#> $`_source`$speaker -#> [1] "" -#> -#> $`_source`$text_entry -#> [1] "ACT I" -``` - -### Search an index by type - - -```r -Search(index="shakespeare", type="act")$hits$hits[[1]] -``` - -``` -#> $`_index` -#> [1] "shakespeare" -#> -#> $`_type` -#> [1] "act" -#> -#> $`_id` -#> [1] "0" -#> -#> $`_score` -#> [1] 1 -#> -#> $`_source` -#> $`_source`$line_id -#> [1] 1 -#> -#> $`_source`$play_name -#> [1] "Henry IV" -#> -#> $`_source`$line_number -#> [1] "" -#> -#> $`_source`$speaker -#> [1] "" -#> -#> $`_source`$text_entry -#> [1] "ACT I" -``` - -### Return certain fields - - -```r -Search(index="shakespeare", , body = '{ - "_source": ["play_name", "speaker"] -}')$hits$hits[[1]] -``` - -``` -#> $`_index` -#> [1] "shakespeare" -#> -#> $`_type` -#> [1] "act" -#> -#> $`_id` -#> [1] "0" -#> -#> $`_score` -#> [1] 1 -#> -#> $`_source` -#> $`_source`$play_name -#> [1] "Henry IV" -#> -#> $`_source`$speaker -#> [1] "" -``` - - -### Paging - - -```r -Search(index="shakespeare", size=1, from=1)$hits -``` - -``` -#> $total -#> [1] 5000 -#> -#> $max_score -#> [1] 1 -#> -#> $hits -#> $hits[[1]] -#> $hits[[1]]$`_index` -#> [1] "shakespeare" -#> -#> $hits[[1]]$`_type` -#> [1] "line" -#> -#> $hits[[1]]$`_id` -#> [1] "14" -#> -#> $hits[[1]]$`_score` -#> [1] 1 -#> -#> $hits[[1]]$`_source` -#> $hits[[1]]$`_source`$line_id -#> [1] 15 -#> -#> $hits[[1]]$`_source`$play_name -#> [1] "Henry IV" -#> -#> $hits[[1]]$`_source`$speech_number -#> [1] 1 -#> -#> $hits[[1]]$`_source`$line_number -#> [1] "1.1.12" -#> -#> $hits[[1]]$`_source`$speaker -#> [1] "KING HENRY IV" -#> -#> $hits[[1]]$`_source`$text_entry -#> [1] "Did lately meet in the intestine shock" -``` - -### Queries - -Using the `q` parameter you can pass in a query, which gets passed in the URI of the query. This type of query is less powerful than the below query passed in the body of the request, using the `body` parameter. - - -```r -Search(index="shakespeare", type="act", q="speaker:KING HENRY IV")$hits$total -``` - -``` -#> [1] 9 -``` - -#### More complex queries - -Here, query for values from 10 to 20 in the field `line_id` - - -```r -Search(index="shakespeare", q="line_id:[10 TO 20]")$hits$total -``` - -``` -#> [1] 11 -``` - -### Get version number for each document - -Version number usually is not returned. - - -```r -sapply(Search(index="shakespeare", version=TRUE, size=2)$hits$hits, "[[", "_version") -``` - -``` -#> [1] 4 4 -``` - -### Get raw data - - -```r -Search(index="shakespeare", type="scene", raw=TRUE) -``` - -``` -#> [1] "{\"took\":1,\"timed_out\":false,\"_shards\":{\"total\":5,\"successful\":5,\"skipped\":0,\"failed\":0},\"hits\":{\"total\":34,\"max_score\":1.0,\"hits\":[{\"_index\":\"shakespeare\",\"_type\":\"scene\",\"_id\":\"646\",\"_score\":1.0,\"_source\":{\"line_id\":647,\"play_name\":\"Henry IV\",\"speech_number\":54,\"line_number\":\"\",\"speaker\":\"HOTSPUR\",\"text_entry\":\"SCENE I. Rochester. An inn yard.\"}},{\"_index\":\"shakespeare\",\"_type\":\"scene\",\"_id\":\"1829\",\"_score\":1.0,\"_source\":{\"line_id\":1830,\"play_name\":\"Henry IV\",\"speech_number\":74,\"line_number\":\"\",\"speaker\":\"MORTIMER\",\"text_entry\":\"SCENE II. London. The palace.\"}},{\"_index\":\"shakespeare\",\"_type\":\"scene\",\"_id\":\"2588\",\"_score\":1.0,\"_source\":{\"line_id\":2589,\"play_name\":\"Henry IV\",\"speech_number\":28,\"line_number\":\"\",\"speaker\":\"SIR WALTER BLUNT\",\"text_entry\":\"SCENE IV. York. The ARCHBISHOPS palace.\"}},{\"_index\":\"shakespeare\",\"_type\":\"scene\",\"_id\":\"3156\",\"_score\":1.0,\"_source\":{\"line_id\":3157,\"play_name\":\"Henry IV\",\"speech_number\":37,\"line_number\":\"\",\"speaker\":\"FALSTAFF\",\"text_entry\":\"SCENE V. Another part of the field.\"}},{\"_index\":\"shakespeare\",\"_type\":\"scene\",\"_id\":\"3870\",\"_score\":1.0,\"_source\":{\"line_id\":3871,\"play_name\":\"Henry VI Part 1\",\"speech_number\":5,\"line_number\":\"\",\"speaker\":\"CHARLES\",\"text_entry\":\"SCENE I. Before Orleans.\"}},{\"_index\":\"shakespeare\",\"_type\":\"scene\",\"_id\":\"4031\",\"_score\":1.0,\"_source\":{\"line_id\":4032,\"play_name\":\"Henry VI Part 1\",\"speech_number\":12,\"line_number\":\"\",\"speaker\":\"Captain\",\"text_entry\":\"SCENE III. Auvergne. The COUNTESSs castle.\"}},{\"_index\":\"shakespeare\",\"_type\":\"scene\",\"_id\":\"4294\",\"_score\":1.0,\"_source\":{\"line_id\":4295,\"play_name\":\"Henry VI Part 1\",\"speech_number\":47,\"line_number\":\"\",\"speaker\":\"PLANTAGENET\",\"text_entry\":\"SCENE V. The Tower of London.\"}},{\"_index\":\"shakespeare\",\"_type\":\"scene\",\"_id\":\"4923\",\"_score\":1.0,\"_source\":{\"line_id\":4924,\"play_name\":\"Henry VI Part 1\",\"speech_number\":24,\"line_number\":\"\",\"speaker\":\"CHARLES\",\"text_entry\":\"SCENE IV. Paris. The palace.\"}},{\"_index\":\"shakespeare\",\"_type\":\"scene\",\"_id\":\"4975\",\"_score\":1.0,\"_source\":{\"line_id\":4976,\"play_name\":\"Henry VI Part 1\",\"speech_number\":11,\"line_number\":\"\",\"speaker\":\"VERNON\",\"text_entry\":\"SCENE I. Paris. A hall of state.\"}},{\"_index\":\"shakespeare\",\"_type\":\"scene\",\"_id\":\"324\",\"_score\":1.0,\"_source\":{\"line_id\":325,\"play_name\":\"Henry IV\",\"speech_number\":62,\"line_number\":\"\",\"speaker\":\"PRINCE HENRY\",\"text_entry\":\"SCENE III. London. The palace.\"}}]}}" -``` - -### Curl debugging - -Common options are `verbose()`, `timeout()`, `progress()`, `config(followlocation=TRUE)`. - - -```r -library('httr') -out <- Search(index="shakespeare", type="line", config=verbose()) -``` - -### Query DSL searches - queries sent in the body of the request - -Pass in as an R list - - -```r -mapping_create("shakespeare", "act", update_all_types = TRUE, body = '{ - "properties": { - "text_entry": { - "type": "text", - "fielddata": true - } - } -}') -``` - -``` -#> $acknowledged -#> [1] TRUE -``` - -```r -aggs <- list(aggs = list(stats = list(terms = list(field = "text_entry")))) -Search(index="shakespeare", body=aggs)$hits$hits[[1]] -``` - -``` -#> $`_index` -#> [1] "shakespeare" -#> -#> $`_type` -#> [1] "act" -#> -#> $`_id` -#> [1] "0" -#> -#> $`_score` -#> [1] 1 -#> -#> $`_source` -#> $`_source`$line_id -#> [1] 1 -#> -#> $`_source`$play_name -#> [1] "Henry IV" -#> -#> $`_source`$line_number -#> [1] "" -#> -#> $`_source`$speaker -#> [1] "" -#> -#> $`_source`$text_entry -#> [1] "ACT I" -``` - -Or pass in as json query with newlines, easy to read - - -```r -aggs <- '{ - "aggs": { - "stats" : { - "terms" : { - "field" : "text_entry" - } - } - } -}' -Search(index="shakespeare", body=aggs)$hits$hits[[1]] -``` - -``` -#> $`_index` -#> [1] "shakespeare" -#> -#> $`_type` -#> [1] "act" -#> -#> $`_id` -#> [1] "0" -#> -#> $`_score` -#> [1] 1 -#> -#> $`_source` -#> $`_source`$line_id -#> [1] 1 -#> -#> $`_source`$play_name -#> [1] "Henry IV" -#> -#> $`_source`$line_number -#> [1] "" -#> -#> $`_source`$speaker -#> [1] "" -#> -#> $`_source`$text_entry -#> [1] "ACT I" -``` - -Or pass in collapsed json string - - -```r -aggs <- '{"aggs":{"stats":{"terms":{"field":"text_entry"}}}}' -Search(index="shakespeare", body=aggs)$hits$hits[[1]] -``` - -``` -#> $`_index` -#> [1] "shakespeare" -#> -#> $`_type` -#> [1] "act" -#> -#> $`_id` -#> [1] "0" -#> -#> $`_score` -#> [1] 1 -#> -#> $`_source` -#> $`_source`$line_id -#> [1] 1 -#> -#> $`_source`$play_name -#> [1] "Henry IV" -#> -#> $`_source`$line_number -#> [1] "" -#> -#> $`_source`$speaker -#> [1] "" -#> -#> $`_source`$text_entry -#> [1] "ACT I" -``` - -### Aggregations - -Histograms - - -```r -aggs <- '{ - "aggs": { - "latbuckets" : { - "histogram" : { - "field" : "decimalLatitude", - "interval" : 5 - } - } - } -}' -Search(index="gbif", body=aggs, size=0)$aggregations$latbuckets$buckets[1:3] -``` - -``` -#> [[1]] -#> [[1]]$key -#> [1] -35 -#> -#> [[1]]$doc_count -#> [1] 1 -#> -#> -#> [[2]] -#> [[2]]$key -#> [1] -30 -#> -#> [[2]]$doc_count -#> [1] 0 -#> -#> -#> [[3]] -#> [[3]]$key -#> [1] -25 -#> -#> [[3]]$doc_count -#> [1] 0 -``` - -### A bool query - - -```r -mmatch <- '{ - "query": { - "bool" : { - "must_not" : { - "range" : { - "speech_number" : { - "from" : 1, "to": 5 -}}}}}}' -sapply(Search(index="shakespeare", body=mmatch)$hits$hits, function(x) x$`_source`$speech_number) -``` - -``` -#> [[1]] -#> NULL -#> -#> [[2]] -#> [1] 6 -#> -#> [[3]] -#> [1] 7 -#> -#> [[4]] -#> [1] 7 -#> -#> [[5]] -#> [1] 7 -#> -#> [[6]] -#> [1] 8 -#> -#> [[7]] -#> [1] 8 -#> -#> [[8]] -#> [1] 9 -#> -#> [[9]] -#> [1] 9 -#> -#> [[10]] -#> [1] 10 -``` - -### Fuzzy query - -Fuzzy query on numerics - - -```r -fuzzy <- list(query = list(fuzzy = list(text_entry = "arms"))) -Search(index="shakespeare", body = fuzzy)$hits$total -``` - -``` -#> [1] 49 -``` - - -```r -fuzzy <- list(query = list(fuzzy = list(text_entry = list(value = "arms", fuzziness = 4)))) -Search(index="shakespeare", body=fuzzy)$hits$total -``` - -``` -#> [1] 617 -``` - -### Range query - -With numeric - - -```r -body <- list(query=list(range=list(decimalLongitude=list(gte=1, lte=3)))) -Search('gbif', body=body)$hits$total -``` - -``` -#> [1] 24 -``` - - -```r -body <- list(query=list(range=list(decimalLongitude=list(gte=2.9, lte=10)))) -Search('gbif', body=body)$hits$total -``` - -``` -#> [1] 166 -``` - -With dates - - -```r -body <- list(query=list(range=list(eventDate=list(gte="2012-01-01", lte="now")))) -Search('gbif', body=body)$hits$total -``` - -``` -#> [1] 899 -``` - - -```r -body <- list(query=list(range=list(eventDate=list(gte="2014-01-01", lte="now")))) -Search('gbif', body=body)$hits$total -``` - -``` -#> [1] 685 -``` - -### More-like-this query (more_like_this can be shortened to mlt) - - -```r -body <- '{ - "query": { - "more_like_this": { - "fields": ["abstract","title"], - "like_text": "and then", - "min_term_freq": 1, - "max_query_terms": 12 - } - } -}' -Search('plos', body=body)$hits$total -``` - -``` -#> [1] 488 -``` - - -```r -body <- '{ - "query": { - "more_like_this": { - "fields": ["abstract","title"], - "like_text": "cell", - "min_term_freq": 1, - "max_query_terms": 12 - } - } -}' -Search('plos', body=body)$hits$total -``` - -``` -#> [1] 58 -``` - - -### Highlighting - - -```r -body <- '{ - "query": { - "query_string": { - "query" : "cell" - } - }, - "highlight": { - "fields": { - "title": {"number_of_fragments": 2} - } - } -}' -out <- Search('plos', 'article', body=body) -out$hits$total -``` - -``` -#> [1] 58 -``` - - -```r -sapply(out$hits$hits, function(x) x$highlight$title[[1]])[8:10] -``` - -``` -#> [[1]] -#> NULL -#> -#> [[2]] -#> NULL -#> -#> [[3]] -#> NULL -``` - -### Scrolling search - instead of paging - - -```r -Search('shakespeare', q="a*")$hits$total -``` - -``` -#> [1] 2747 -``` - -```r -res <- Search(index = 'shakespeare', q="a*", time_scroll = "1m") -length(scroll(res$`_scroll_id`)$hits$hits) -``` - -``` -#> [1] 10 -``` - - -```r -res <- Search(index = 'shakespeare', q = "a*", time_scroll = "5m") -out <- list() -hits <- 1 -while (hits != 0) { - res <- scroll(res$`_scroll_id`) - hits <- length(res$hits$hits) - if (hits > 0) - out <- c(out, res$hits$hits) -} -length(out) -``` - -``` -#> [1] 2737 -``` - -Woohoo! Collected all 2737 documents in very little time.