This involves removing some features as Elasticsearch 2.0 has breaking public API changes.
Contributed by Timo Sulg and Josh Tilles.
There is now a clojurewerkz.elastisch.native.index/get-aliases function, which
can be used to retrieve all aliases for given indices (defaults to all aliases
for the cluster if no indices are provided).
Contributed by Mikey Griffiths.
You can now supply a separate upsert document by using the :upsert options key.
The upsert document is used if the document does not already exist.
Contributed by Henrik Lundahl.
You can now supply an upsert document by using the :upsert options key.
The upsert document is used if the document does not already exist.
Contributed by Henrik Lundahl.
Added helper function for a type query.
Contributed by @nivekuil.
The default clj-http options that Elastisch sets can now be overridden in two ways:
- on a per-connection basis by passing the options to
clojurewerkz.elastisch.rest/connect - and per-invocation by passing the options as arguments to individual function calls
This was achieved by changing the order in which the options-maps are merged.
GitHub issue: clojurewerkz/elastisch#200.
Contributed by @MerelyAPseudonym.
This change allows the source attribute to be explicitly included/excluded from search results.
Contributed by @KeeganMyers.
_version and _version_type are supported in bulk operations.
Contributed by Joe Einertson.
clojurewerks.elastisch.native.conversion/->search-request now accepts
:template and :params arguments for use with search templates
clojurewerks.elastish.native.document/create-search-template,
clojurewerks.elastish.native.document/delete-search-template,
clojurewerks.elastish.native.document/put-search-template,
clojurewerks.elastish.native.document/get-search-template,
Added to simplify CRUD operations on search templates. Each is a vardic
function that will interface with the ".scripts" index and "mustache" type
by default. Users can override the type parameter if they would like to write
scripts in a language other than mustache.
Contributed by @KeeganMyers
GitHub issue: #193.
Contributed by @dspiteself.
Options passed to elastisch.rest/connect can now include :throw-exceptions true, causing HTTP errors to throw exceptions.
Contributed by @loganmhb.
Added the key :sort to each hit from the search results in order to remain close to feature parity
Contributed by @KeeganMyers
Native client now uses :filter for filters, just like the REST one.
clojurewerkz.elastisch.native.document has new function suggest
for term autocompletion. It allows filter results by category and
geolocation:
(require '[clojurewerkz.elastisch.native.document :as doc])
(doc/suggest conn index-name :completion "e" {:context {:gender "female"}})
(doc/suggest conn index-name :fuzzy "esmor" {:fuzziness 1 :min-length 2})Contributed by Timo Sulg (@timgluz).
clojurewerkz.elastisch.native.conversion/->open-index-request and ->close-index-request
passed plain string to CloseIndexRequest constructor, but it expected values passed as array of string.
Fixed it with existing function conversion/->string-array and added missing tests for this usecase
into clojurewerkz.elastisch.native-api.indices-settings-test.
Contributed by Timo Sulg (@timgluz).
clojurewerkz.elastisch.native.conversion/->update-request arguments
updated in clojurewerkz.elastisch.native.document/update-with-script to reflect
recent changes.
Contributed by Michael Nussbaum.
clojurewerkz.elastisch.rest.index/stats has been updated for Elasticsearch 1.3.x
and later versions.
Contributed by Roman Pearah.
Bulk operation helper functions now include create.
Contributed by @nikopol.
In both native and rest apis, :ignore-unmapped may be set in the query by specifying
a sort field-name and option-map instead of order name with the query/sort function.
For example:
(require '[clojurewerkz.elastisch.native.document :as doc])
(require '[clojurewerkz.elastisch.query :as q])
(doc/search conn index type
{:query (q/query-string :query "software" :default_field "summary")
:sort (q/sort "unmapped-field-name" {:ignore_unmapped true
:order "asc"})})Contributed by @ryfow
In both native and rest apis, :ignore-unmapped may be set in the query by specifying
a sort field-name and option-map instead of order name with the query/sort function.
For example:
(require '[clojurewerkz.elastisch.native.document :as doc])
(require '[clojurewerkz.elastisch.query :as q])
(doc/search conn [index-name missing-index-name]
mapping-type
:query (q/match-all)
:ignore_unavailable true)Contributed by Joachim De Beule.
scan-and-scroll-seq provides an easier-to-use abstraction over ES's
scan and scroll API, wrapping scroll-seq and handling the
special-case first request.
Contributed by @loganmhb
Elastisch now depends on Elasticsearch Java client version 1.7.x.
clj-http dependency has been upgraded to version 2.0.x.
:removeaction now works with singular:indexkey- multiple aliases can be added with single
:addaction - alias can be removed from multiple indices with single
:removeaction
Contributed by @mnylen
Previously only mappings allowed keys to be keywords, now same works with index settings.
Contributed by @mnylen
Elastisch now supports scroll IDs larger than 4 KB.
Contributed by niko.
Native client now supports bulk operations with the same API as the REST one.
Contributed by
- Mitchel Kuijpers (Avisi)
- Michael Nussbaum and Jack Lund (Braintree)
clojurewerkz.elastisch.native.percolation/unregister-query arguments
were mistakenly swapped when delegating to the Java client.
Contributed by Stephen Muss.
Contributed by Jan Stępień (Stylefruits).
Native client now supports nesting in the following aggregations
histogramdate_histogramrangedate_rangeterms
Contributed by Mitchel Kuijpers (Avisi).
Elastisch now depends on Elasticsearch Java client version 1.4.x.
clj-http dependency has been upgraded to version 1.0.1.
Cheshire dependency has been upgraded to version 5.4.0.
Elastisch no longer officially supports Clojure 1.4. Most of the functionality still works well on that version but please don't file bugs specific to that version.
clojurewerkz.elastisch.native.index/update-aliases expects indices to be added to be specified in the
:indices key while the respective REST function uses :index. This can have unexpected results, namely
the creation of the respective alias for all indices. It is now possible to supply either :index or
:indices to the function.
GH issue: #108.
Contributed by Yannick Scherer (stylefruits)
clojurewerkz.elastisch.native.document/update-with-partial-doc is a new function
in the Native Client (existed before in the REST API) that performs
partial updates:
(require '[clojurewerkz.elastisch.native.document :as doc])
(doc/update-with-partial-doc conn "people" "person" "1" {:country "Sweden"})Note that the REST API should now be called without wrapping the document in a :doc key.
I.e. change {:doc {:field-to-update "Update"}} to {:field-to-update "Update"}.
Contributed by Henrik Lundahl.
clojurewerkz.elastisch.rest.index.create-template now supports
the :aliases option:
(require '[clojurewerkz.elastisch.rest.index :as idx])
(idx/create-template conn "accounts" {:template "account*" :settings {:index {:refresh_interval "60s"}} :aliases {:account-alias {}}})Contributed by Jeffrey Erikson.
clj-http dependency has been upgraded to version 1.0.x.
Updates and upserts now allow the retry-on-conflict option to be set.
This helps to work around Elasticsearch version conflicts.
GH issue: #119.
Contributed by Michael Nussbaum (Braintree).
clojurewerkz.elastisch.rest.bulk/bulk-index now filters out
all operation/option keys so that they don't get stored in the document
body.
GH issue: #116.
Contributed by Michael Nussbaum (Braintree).
Elasticsearch Multi Search REST API endpoint is sensitive to the trailing new line. When it is missing, the response contains one result too few.
Elastisch now makes sure to append a new line to Multi Search request bodies.
Native client's document/async-put no longer fails with an exception.
Contributed by Nikita Burtsev.
clj-time dependency has been upgraded to version 0.8.0.
Elastisch now depends on Elasticsearch native client version 1.3.x.
Child aggregations in single-bucket aggregations (i.e. "global") are no longer silently dropped.
Contributed by Yannick Scherer (StyleFruits).
Native client now has support for aggregations.
The API is the same as in the REST client.
Note that Elasticsearch 1.2 has 26 aggregations. Currently only the most commonly used ones are supported but support for more types will be added eventually. The supported types are:
- Avg
- Max
- Min
- Sum
- Stats
- Extended stats
- Cardinality, value count
- Percentiles
- Histogram
- Date Histogram
- Range
- Date Range
- Terms
- Missing
- Global
Native client now has support for multi-search.
The API is the same as in the REST client except that the functions are in the
clojurewerkz.elastisch.native.multi.
clojurewerkz.elastisch.native.document/upsert now accepts a map of extra options,
e.g. parent document ID:
(doc/upsert conn index-name index-type id doc {:parent parent-id})clojurewerkz.elastisch.query/terms is a newly added alias for clojurewerkz.elastisch.query/term
when used with a collection.
Contributed by Martin Klepsch.
Bug fixed in native client for removing aliases from indices and improved inline documentation. See aliases in the guide.
GH issue: #98.
Native client now supports (most of the) highlighting features the REST client does:
(require '[clojurewerkz.elastisch.native.document :as doc])
(require '[clojurewerkz.elastisch.query :as q])
(doc/search conn index type
{:query (q/query-string :query "software" :default_field "summary")
:highlight {:fields {:summary {}}}})It is now possible to specify clj-http options for REST API connections, e.g. to specify a timeout:
(esr/connect "http://127.0.0.1:9200/" {:conn-timeout 1000
:basic-auth ["username" "pa$$w0rd"]})Native client now supports source filtering just like the REST API client:
(doc/search conn index-name mapping-type
:query (q/match-all)
:sort {"first-name" "asc"}
:_source ["first-name" "age"])(doc/search conn index-name mapping-type
:query (q/match-all)
:sort {"first-name" "asc"}
:_source {"exclude" ["title" "country"
"planet" "biography"
"last-name" "username"]})GH issue: #73.
Previously a search would return either the source document, or specific fields and not both. There are certain circumstances where having both are beneficial, for example when searching for a child document and you want to include the parent ID:
(require '[clojurewerkz.elastisch.native.document :as esd])
(esd/search conn "index" "child-type" :query (q/match-all) :fields ["_parent"])The above would return the parent document ID in the :_parent field of each hit, but
would not return the document itself. You can now have both by:
(esd/search conn "index" "child-type" :query (q/match-all) :fields ["_parent" "_source"])Now the parent ID is in the :_parent field of each hit, and the matching document
will be in :_source as per a normal search.
Contributed by Ben Ashford.
clojurewerkz.elastisch.rest.document/update-with-partial-doc is a new function
that performs partial updates:
(require '[clojurewerkz.elastisch.rest.document :as doc])
(doc/update-with-partial-doc conn "people" "person" "1" {:country "India"})Contributed by Sandeep Jagtap.
Elasticsearch client has been upgraded to 1.2.x.
Native client now supports snapshotting (updated for Elasticsearch 1.2) with the same Clojure API as the REST client (all the usual API conventions apply).
Starting with Elastisch 2.0.0-rc1, connection (client) is no longer a shared
dynamic var but rather is an explicit argument that relevant API functions
accept.
Before the change:
(ns clojurewerkz.elastisch.docs.examples
(:require [clojurewerkz.elastisch.rest :as esr]
[clojurewerkz.elastisch.rest.index :as esi]
[clojurewerkz.elastisch.rest.document :as esd]))
(defn -main
[& args]
(esr/connect! "http://127.0.0.1:9200")
(let [mapping-types {"person" {:properties {:username {:type "string" :store "yes"}
:first-name {:type "string" :store "yes"}
:last-name {:type "string"}
:age {:type "integer"}
:title {:type "string" :analyzer "snowball"}
:planet {:type "string"}
:biography {:type "string" :analyzer "snowball" :term_vector "with_positions_offsets"}}}}
doc {:username "happyjoe" :first-name "Joe" :last-name "Smith" :age 30 :title "Teh Boss" :planet "Earth" :biography "N/A"}]
(esi/create "myapp2_development" :mappings mapping-types)
(esd/create "myapp2_development" "person" doc)))After the change:
(ns clojurewerkz.elastisch.docs.examples
(:require [clojurewerkz.elastisch.rest :as esr]
[clojurewerkz.elastisch.rest.index :as esi]
[clojurewerkz.elastisch.rest.document :as esd]))
(defn -main
[& args]
(let [conn (esr/connect "http://127.0.0.1:9200")
mapping-types {"person" {:properties {:username {:type "string" :store "yes"}
:first-name {:type "string" :store "yes"}
:last-name {:type "string"}
:age {:type "integer"}
:title {:type "string" :analyzer "snowball"}
:planet {:type "string"}
:biography {:type "string" :analyzer "snowball" :term_vector "with_positions_offsets"}}}}
doc {:username "happyjoe" :first-name "Joe" :last-name "Smith" :age 30 :title "Teh Boss" :planet "Earth" :biography "N/A"}]
(esi/create conn "myapp2_development" :mappings mapping-types)
(esd/create conn "myapp2_development" "person" doc)))Dynamic var reliance has been a major complaint of Clojure users for quite some time and 2.0 is the right time to fix this.
clojurewerkz.elastisch.rest.response/created? and clojurewerkz.elastisch.native.response/created?
were adapted for recent ES releases.
Contributed by Oliver McCormack (The Climate Corporation).
Elasticsearch client has been upgraded to 1.1.1.
Elastisch has tranditionally accepted options as (pseudo) keywrod arguments, e.g.
(doc/search index-name mapping-type :query (q/term :biography "say"))Starting with 2.0.0-beta4, passing a single map of arguments
is now also supported by nearly all document, index, admin and percolation
functions:
(doc/search index-name mapping-type {:query (q/term :biography "say")})As a new design rule, all new API elements (e.g. aggregations) will accept a single map of options.
GH issue: #59.
REST API client now supports percolation of existing documents:
(require '[clojurewerkz.elastisch.rest.percolation :as pcl])
(pcl/percolate-existing "articles" "article" "123")Elasticsearch client has been upgraded to 1.1.0.
Elastisch now depends on org.clojure/clojure version 1.6.0. It is
still compatible with Clojure 1.4 and if your project.clj depends on
a different version, it will be used, but 1.6 is the default now.
types-exists support in both rest and native clients:
(require '[clojurewerkz.elastisch.rest.index :as esi])
(esi/type-exists? "an-index" "a-type")Contributed by Halit Olali.
Elastisch 2.0 features multiple convenience functions for working with Elasticsearch aggregations.
clojurewerkz.elastisch.aggregation is a new namespace that contains
helper functions that produce various types of aggregations. Just like
clojurewerkz.elastisch.query, all of the functions return maps and
are optional.
clojurewerkz.elastisch.rest.response/aggregations-from is a new function
that returns aggregations from a search response:
(require '[clojurewerkz.elastisch.rest.document :as doc])
(require '[clojurewerkz.elastisch.query :as q])
(require '[clojurewerkz.elastisch.aggregation :as a])
(require '[clojurewerkz.elastisch.rest.response :refer [aggregations-from]])
(let [index-name "people"
mapping-type "person"
response (doc/search index-name mapping-type
:query (q/match-all)
:aggregations {:min_age (a/min "age")})
agg (aggregation-from response :min_age)]
(is (= {:value 22.0} agg)))Aggregations support is primarily focused on REST client at the moment.
clj-http dependency has been upgraded to version 0.9.1.
Main goal of Elastisch 2.0 is Elasticsearch 2.0 compatibility. This includes minor API changes (in line with Elasticsearch 1.0 API and terminology changes) and moderate internal modifications.
clojureworkz.elastisch.rest.admin/nodes-info and clojureworkz.elastisch.rest.admin/nodes-stats
are new administrative functions that provide access to Elasticsearch
cluster stats and node info.
Examples:
(require '[clojurewerkz.elastisch.rest.admin :as admin])
(admin/nodes-stats)
(admin/nodes-stats :nodes "_all")
(admin/nodes-stats :nodes ["node1" "node2"] ["indices" "os" "plugins"])
(admin/nodes-info)
(admin/nodes-info :nodes "_all")
(admin/nodes-info :nodes ["node1" "node2"] ["indices" "os" "plugins"])See Elasticsearch nodes stats documentation, nodes info page, and node specification page for more info.
Contributed by Joachim De Beule.
Added (clojureworkz.elastisch.rest.admin/cluster-state & {:as params})
Examples:
(require '[clojurewerkz.elastisch.rest.admin :as admin])
(admin/cluster-state)
(admin/cluster-state :filter_nodes true)See Elasticsearch documentation for more info.
Contributed by Joachim De Beule.
Added (clojureworkz.elastisch.rest.admin/cluster-health & {:as params})
Example:
(require '[clojurewerkz.elastisch.rest.admin :as admin])
(admin/cluster-health)
(admin/cluster-health :index "index1")
(admin/cluster-health :index ["index1","index2"])
(admin/cluster-health :index "index1" :pretty true :level "indices")See Elasticsearch documentation for more info.
Contributed by Joachim De Beule.
Added (doc/analyze text & {:as params})
See Elasticsearch documentation for more info.
Examples:
(require '[clojurewerkz.elastisch.rest.document :as doc])
(doc/analyze "foo bar baz")
(doc/analyze "foo bar baz" :index "some-index-name")
(doc/analyze "foo bar baz" :analyzer "whitespace")
(doc/analyze "foo bar baz" :tokenizer "keyword" :filters "lowercase")
(doc/analyze "foo bar baz" :index "some-index-name" :field "some-field-name")Contributed by Joachim De Beule
clojurewerkz.elastisch.query/query-string accepts a new option, :escape-with,
which is a function that performs escaping of special characters in query string
queries.
By default clojurewerkz.elastisch.escape/escape-query-string-characters is used.
Contributed by Ben Reinhart (Groupon).
Elastisch now depends on Elasticsearch native client version 1.0.1.
clj-http dependency has been upgraded to version 0.9.0.
The native document API now supports the same :version option for optimistic locking that the REST api does.
Contributed by Richie Vos (Groupon).
GH issues: #56.
Elastisch now uses Elasticsearch exceptions instead of generic ones in the native client.
Contributed by Richie Vos (Groupon).
GH issues: #54, #57.
Elastisch now depends on Elasticsearch native client version 0.90.8.
clojurewerkz.elastisch.native.document/update-with-script invoked without
script parameters no longer raises an exception.
Cheshire dependency has been updated to 5.3.1.
:sort option can now be a com.elasticsearch.search.sort.SortBuilder instance
and not just a string.
Contributed by Mark Wong-VanHaren.
Elastisch now depends on Elasticsearch native client version 0.90.7.
Elastisch no longer erroneously inserts _index and _type fields
into documents inserted via bulk API.
Contributed by Max Barnash.
clojurewerkz.elastisch.native.document/scroll-seq and
clojurewerkz.elastisch.rest.document/scroll-seq
are new functions that accept a search query response
and return a lazy sequence of paginated search results.
This makes working with result sets that require pagination much more natural:
(require '[clojurewerkz.elastisch.native.document :as doc])
(let [index-name "articles"
mapping-type "article"
res-seq (doc/scroll-seq
(doc/search index-name mapping-type
:query (q/term :title "Emptiness")
:search_type "query_then_fetch"
:scroll "1m"
:size 2))]
res-seq))Contributed by Max Barnash.
Native client now supports upserts of documents:
(require '[clojurewerkz.elastisch.native.document :as doc])
(doc/upsert "people" "person" "elastisch" {:name "Elastisch" :language "Clojure"})clj-http dependency has been upgraded to version 0.7.7.
Date histogram in the native client now includes :total field.
Contributed by Jim Dunn.
Native client now returns the same value in :fields and :_fields
keys in search hits. This makes it both backwards compatible with
earlier versions and the format Elasticsearch HTTP API uses.
Elastisch now depends on Elasticsearch native client version 0.90.5.
Bulk index and delete operations support _parent and _routing
keys.
Contributed by Baptiste Fontaine.
Elastisch now requires Clojure 1.4.
Cheshire dependency has been upgraded to version 5.2.0.
clj-http dependency has been upgraded to version 0.7.6.
Elastisch now depends on Elasticsearch native client version 0.90.3.
Elastisch now will not perform a bulk operation if its list of operations is empty.
Contributed by Baptiste Fontaine.
Elastisch now depends on Elasticsearch native client version 0.90.2.
clojurewerkz.elastisch.rest.document/search,
clojurewerkz.elastisch.rest.document/search-all-types,
clojurewerkz.elastisch.rest.document/count,
clojurewerkz.elastisch.rest.document/delete-by-query, and
clojurewerkz.elastisch.rest.document/delete-by-query-across-all-types
now accepts the :ignore_indices option:
(doc/search [index-name, missing-index-name,...] mapping-type :query (q/match-all)
:ignore_indices "missing")See also elasticsearch/guide/reference/api
Contributed by Joachim De Beule
Search queries that only retrieve a subset of fields using
the :fields option are now correctly converted to Clojure maps.
Contributed by Soren Macbeth.
Elastisch now depends on Elasticsearch native client version 0.90.1.
clojurewerkz.elastisch.native.document/search now accepts maps as
:search option values:
(doc/search index-name mapping-type :query (q/match-all)
:sort (array-map "title" "asc")This is identical to how the option works with the REST client.
clojurewerkz.elastisch.rest.document/update-with-script is a new function
that updates a document with a provided script:
(require '[clojurewerkz.elastisch.rest.document :as doc])
;; initializes a counter at 1
(doc/update-with-script index-name mapping-type "1"
"ctx._source.counter = 1")
;; increments the counter by 4
(doc/update-with-script index-name mapping-type "1"
"ctx._source.counter += inc"
{"inc" 4})clojurewerkz.elastisch.native.document/update-with-script is the native
client counterpart that takes the same arguments.
Native client is now over 50% faster on most commonly used operations thanks to much lower conversion overhead from Elasticsearch native client data structures to Clojure maps.
Contributed by Jon Pither.
Elastisch 1.1.0 includes a major new feature: native Elasticsearch client.
The client uses Elasticsearch's Java API, and can be used with
both transport and node clients.
Native client is more bandwidth efficient. It also can use SMILE (binary JSON format) to be more efficient on the wire.
Native client API in Elastisch is nearly identical to that of the REST API client
and resides in clojurewerkz.elastisch.native and clojurewerkz.elastisch.native.*
namespaces (similarly to how clojurewerkz.elastisch.rest clojurewerkz.elastisch.rest.*
namespaces are organized).
Transport client (used for TCP/remote connections) connections are set up using
clojurewerkz.elastisch.native/connect!. Note that you need to provide node
configuration that at least has cluster name in it:
(require '[clojurewerkz.elastisch.native :as es])
;; note that transport client uses port 9300 by default.
;; it also can connect to multiple cluster nodes
(es/connect! [["127.0.0.1" 9300]]
{"cluster.name" "elasticsearch_antares" })Cluster name and transport node addresses can be retrieved via HTTP API, for example:
curl http://localhost:9200/_cluster/nodes
{"ok":true,"cluster_name":"elasticsearch_antares","nodes":...}}
The Native client tries to be as close as possible to the existing REST client API.
For example, document operation functions in clojurewerkz.elastisch.native.document,
such as clojurewerkz.elastisch.native.document/create,
follow clojurewerkz.elastisch.rest.document function signatures as closely as
possible:
;; in the REPL
(require '[clojurewerkz.elastisch.native :as es])
(require '[clojurewerkz.elastisch.native.document :as doc])
(es/connect! [["127.0.0.1" 9300]]
{"cluster.name" "elasticsearch_antares" })
(doc/put index-name index-type id document)
(doc/get index-name index-type id)The same with returned results. Note, however, that ES transport client does have (very) minor differences with the REST API and it is not always possible for Elastisch to completely cover such differences.
Native client offers a choice of synchronous (blocking calling thread until a response is received) and asynchronous (returns a future) versions of multiple API operations:
;; in the REPL
(require '[clojurewerkz.elastisch.native :as es])
(require '[clojurewerkz.elastisch.native.document :as doc])
(es/connect! [["127.0.0.1" 9300]]
{"cluster.name" "elasticsearch_antares" })
(doc/put index-name index-type id document)
;; returns a response
(doc/get index-name index-type id)
;; returns a future that will eventually
;; contain a response
(doc/async-get index-name index-type id)One notable exception to this is administrative operations (such as opening or closing an index). The rationale for this is that they are rarely executed on the hot code path (e.g. in tight loops), so convenience and better error visibility is more important for them.
GH issues: #17, #18, #20.
Note that native Elasticsearch client currently relies on Elasticsearch 0.90.0.Beta1 client libraries and some operations will only work with that version.
Elastisch now depends on org.clojure/clojure version 1.5.0. It is still compatible with Clojure 1.3+ and if your project.clj depends
on 1.3 or 1.4, it will be used, but 1.5 is the default now.
We encourage all users to upgrade to 1.5, it is a drop-in replacement for the majority of projects out there.
Bulk requests are now supported. All the relevant code is in the clojurewerkz.elastisch.rest.bulk
namespace. Here is a small example of bulk document indexing using this new API:
(require '[clojurewerkz.elastisch.rest.bulk :as eb])
(eb/bulk (eb/bulk-index doc1 doc2 doc3) :refresh true)Contributed by Davie Moston.
Scroll queries are now easier to perform thanks to the new clojurewerkz.elastisch.rest.document/scroll
function that takes a scroll id and amount of time retrieved documents and related information
will be kept in memory for future retrieval. They are analogous to database cursors.
A short code example:
(require '[clojurewerkz.elastisch.rest.document :as doc])
(require '[clojurewerkz.elastisch.query :as q])
(require '[clojurewerkz.elastisch.rest.response :refer [hits-from]])
(let [index-name "articles"
mapping-type "article"
response (doc/search index-name mapping-type
:query (q/query-string :query "*")
:search_type "scan"
:scroll "1m"
:size 1)
scroll-id (:_scroll_id response)
scan-response (doc/scroll scroll-id :scroll "1m")
scan-hits (hits-from scan-response)]
(println scan-hits))Contributed by Davie Moston.
Cheshire dependency has been upgraded to version 5.1.1.
clj-http dependency has been upgraded to version 0.7.2.
clojurewerkz.elastisch.rest.document/count no longer ignores mapping types.
GH issue: #6.
clojurewerkz.elastisch.rest.document/count now correctly uses GET for requests without
the query part andPOST for request that have it.
GH issue: #5.
Elastisch will now URL-encode document ids.
clj-http dependency has been upgraded to version 0.5.5.
Cheshire dependency has been upgraded to version 4.0.3.
clojurewerkz.elastisch.rest.document/validate-query is a new function that implements support for the Validation API:
(require '[clojurewerkz.elastisch.rest.document :as doc])
(require '[clojurewerkz.elastisch.query :as q])
(require '[clojurewerkz.elastisch.rest.response :as r])
(let [response (doc/validate-query "myproduct_development" (q/field "latest-edit.author" "Thorwald") :explain true)]
(println response)
(println (r/valid? response)))Query validation does not execute the query.
clojurewerkz.elastisch.rest.percolation is a new namespace with functions that implement the Percolation API.
(require '[clojurewerkz.elastisch.rest.percolation :as pcl])
(require '[clojurewerkz.elastisch.rest.response :as r])
;; register a percolator query for the given index
(pcl/register-query "myapp" "sample_percolator" :query {:term {:title "search"}})
;; match a document against the percolator
(let [response (pcl/percolate "myapp" "sample_percolator" :doc {:title "You know, for search"})]
(println (r/ok? response))
;; print matches
(println (r/matches-from response)))
;; unregister the percolator
(pcl/unregister-query "myapp" "sample_percolator")clj-http dependency has been upgraded to version 0.5.2.
Elastisch now uses (and depends on) Cheshire for JSON serialization. clojure.data.json is no longer a dependency.
clojurewerkz.elastisch.rest.response/facets-from is a new convenience function that returns the facets section of a response.
The exact response format will vary between facet types and queries but it is always returned as an immutable map and has the same
structure as in the respective Elasticsearch response JSON document.
Elastisch now depends on org.clojure/clojure version 1.4.0. It is still compatible with Clojure 1.3 and if your project.clj depends
on 1.3, it will be used, but 1.4 is the default now.
We encourage all users to upgrade to 1.4, it is a drop-in replacement for the majority of projects out there.
Elasticsearch 0.19.9 renames Text Query to Match Query. Elastisch adapts by introducing clojurewerkz.elastisch.query/match that
is effectively an alias for clojurewerkz.elastisch.query/text (Elasticsearch still supports :text in the query DSL for backwards
compatibility).
Documentation guides were greatly improved.
clj-http dependency has been upgraded to version 0.5.1.
clojurewerkz.elastisch.query/text now takes two arguments, the first being the name of the field.
It was mistakenly hardcoded previously.¯
clojurewerkz.elastisch.rest.index/create-template, clojurewerkz.elastisch.rest.index/delete-template
and clojurewerkz.elastisch.rest.index/get-template are new functions that implement support for index templates:
(clojurewerkz.elastisch.rest.index/create-template "accounts" :template "account*" :settings {:index {:refresh_interval "60s"}})
(clojurewerkz.elastisch.rest.index/get-template "accounts")
(clojurewerkz.elastisch.rest.index/delete-template "accounts")clojurewerkz.elastisch.rest.index/update-aliases and clojurewerkz.elastisch.rest.index/get-aliases
are new functions that implement support for index aliases:
(clojurewerkz.elastisch.rest.index/update-aliases [{:add {:index "client0000001" :alias "alias1"}}
{:add {:index "client0000002" :alias "alias2"}}])
(clojurewerkz.elastisch.rest.index/get-aliases "client0000001")clojurewerkz.elastisch.rest.index/optimize is a function that optimizes an index:
(clojurewerkz.elastisch.rest.index/optimize "my-index" :refresh true :max_num_segments 48)clojurewerkz.elastisch.rest.index/flush is a function that flushes an index:
(clojurewerkz.elastisch.rest.index/flush "my-index" :refresh true)clojurewerkz.elastisch.rest.index/snapshot is a function that takes a snapshot of an index or multiple indexes:
(clojurewerkz.elastisch.rest.index/snapshot "my-index")clojurewerkz.elastisch.rest.index/clear-cache is a function that can be used to clear index caches:
(clojurewerkz.elastisch.rest.index/clear-cache "my-index" :filter true :field_data true)It takes the same options as documented in the Elasticsearch guide on the Clear Cache Index operation
clojurewerkz.elastisch.rest.index/status is a function that returns status an index or multple indexes:
(clojurewerkz.elastisch.rest.index/status "my-index" :recovery true :snapshot true)clojurewerkz.elastisch.rest.index/segments is a function that returns segments information for an index or multiple indexes:
(clojurewerkz.elastisch.rest.index/segments "my-index")clojurewerkz.elastisch.rest.index/stats is a function that returns statistics for an index or multiple indexes:
(clojurewerkz.elastisch.rest.index/stats "my-index" :docs true :store true :indexing true)It takes the same options as documented in the Elasticsearch guide on the Stats Index operation
clj-http dependency has been upgraded to version 0.5.0.
clojurewerkz.elastisch.rest.document/delete-by-query-across-all-types is a new function that searches across
one or more indexes and all mapping types:
(doc/delete-by-query-across-all-types index-name (q/term :username "esjoe"))clojurewerkz.elastisch.rest.document/delete-by-query-across-all-indexes-and-types is another new function that searches across all indexes and all mapping types:
(doc/delete-by-query-across-all-indexes-and-types (q/term :username "esjoe"))clojurewerkz.elastisch.rest.document/search-all-types is a new function that searches across
one or more indexes and all mapping types:
(doc/search-all-types ["customer1_index" "customer2_index"] :query (q/query-string :query "Austin" :default_field "title"))clojurewerkz.elastisch.rest.document/search-all-indexes-and-types is another new function that searches across all indexes and all mapping types:
(doc/search-all-indexes-and-types :query (q/query-string :query "Austin" :default_field "title"))It is now possible to create indexes without specifying mapping types: clojurewerkz.elastisch.rest.index/create
no longer requires :mapping to be passed.
Elastisch now uses clj-http 0.4.x.
HTTP/REST API namespaces are now grouped under clojurewerkz.elastisch.rest, for example, what used to be clojurewerkz.elastisch.document is now
clojurewerkz.elastisch.rest.document. This is done to leave room for Memcached transport support in the future.
Elastisch now supports Custom Filters Score queries.
Elastisch now supports Nested queries.
Elastisch now supports Indices queries.
Elastisch now supports Top Children queries.
Elastisch now supports Has Child queries.
Elastisch now supports Wildcard queries.
Elastisch now supports span queries:
Span queries are used for proximity search.
Elastisch now supports Query String queries.
Elastisch now supports More Like This Field queries.
clj-time dependency has been upgraded to version 0.4.1.
Elastisch now supports More Like This queries.
Elastisch now supports match all queries.
Elastisch now supports fuzzy (edit distance) queries.
Elastisch now supports fuzzy like this and fuzzy like this field queries.
Elastisch now supports prefix queries, see clojurewerkz.elastisch.query/prefix,
clojurewerkz.elastisch.query/field, clojurewerkz.elastisch.query/filtered, and clojurewerkz.elastisch.document/search to learn more.
clojurewerkz.elastisch.document/more-like-this provides access to the Elastic Search More Like This API for
documents and returns documents similar to a given one:
(doc/more-like-this "people" "person" "1" :min_term_freq 1 :min_doc_freq 1)Please note that :min_doc_freq and :min_term_freq parameters may be very important for small data sets.
If you observe responses with no results, try lowering them.
clojurewerkz.elastisch.document/count provides access to the Elastic Search count API
and is almost always used with a query, for example:
(doc/count "people" "person" (q/term :username "clojurewerkz"))clojurewerkz.elastisch.document/delete-by-query provides access to the Delete by query API of Elastic Search, for example:
(doc/delete-by-query "people" "person" (q/term :tag "mongodb"))clojurewerkz.elastisch.document/replace deletes a document by id and immediately adds a new one
with the same id.
clojurewerkz.elastisch.response was extracted from clojurewerkz.elastisch.utils
Elastisch now uses Leiningen 2.