Merge #2216 #2222 #2224 #2231

2216: v1.1: Add new filter rule `_geoBoundingBox` r=guimachiavelli a=guimachiavelli Closes #2201 2222: v1.1: Metrics r=guimachiavelli a=guimachiavelli Closes #2206 2224: v1.1: Update index size/count limits r=guimachiavelli a=guimachiavelli Closes #2203 2231: v1.1: Telemetry r=guimachiavelli a=guimachiavelli Closes #2202 Co-authored-by: gui machiavelli <hey@guimachiavelli.com> Co-authored-by: gui machiavelli <gui@meilisearch.com>
meilisearch · Mar 30, 2023 · 9937154 · 9937154
5 parents b0a7119 + 4824f73 + 5c7f094 + 25f3c10 + 92b0b09
commit 9937154
Show file tree

Hide file tree

Showing 11 changed files with 101 additions and 41 deletions.
diff --git a/.code-samples.meilisearch.yaml b/.code-samples.meilisearch.yaml
@@ -693,6 +693,11 @@ geosearch_guide_filter_usage_2: |-
     -X POST 'http://localhost:7700/indexes/restaurants/search' \
     -H 'Content-type:application/json' \
     --data-binary '{ "filter": "_geoRadius(45.472735, 9.184019, 2000) AND type = pizza" }'
+geosearch_guide_filter_usage_3: |-
+  curl \
+    -X POST 'http://localhost:7700/indexes/restaurants/search' \
+    -H 'Content-type:application/json' \
+    --data-binary '{ "filter": "_geoBoundingBox([45.494181, 9.214024], [45.449484, 9.179175])" }'
 geosearch_guide_sort_settings_1: |-
   curl \
     -X PUT 'http://localhost:7700/indexes/restaurants/settings/sortable-attributes' \

diff --git a/.vuepress/config.js b/.vuepress/config.js
@@ -289,6 +289,14 @@ module.exports = {
             },
           ],
         },
+        {
+          title: '🧪 Experimental',
+          path: '/learn/experimental/metrics.html',
+          collapsable: false,
+          children: [
+            '/learn/experimental/metrics',
+          ],
+        },
         {
           title: '👐 Contributing',
           path: '/learn/contributing/overview.html',

diff --git a/.vuepress/public/sample-template.yaml b/.vuepress/public/sample-template.yaml
@@ -102,6 +102,7 @@ search_parameter_guide_sort_1: |-
 geosearch_guide_filter_settings_1: |-
 geosearch_guide_filter_usage_1: |-
 geosearch_guide_filter_usage_2: |-
+geosearch_guide_filter_usage_3: |-
 geosearch_guide_sort_settings_1: |-
 geosearch_guide_sort_usage_1: |-
 geosearch_guide_sort_usage_2: |-

diff --git a/learn/advanced/filtering_and_faceted_search.md b/learn/advanced/filtering_and_faceted_search.md
@@ -342,23 +342,13 @@ If you only want recent `Planet of the Apes` movies that weren't directed by `Ti
 release_date > 1577884550 AND (NOT director = "Tim Burton" AND director EXISTS)
 ```
 
-### Filtering with `_geoRadius`
+### Filtering by geographic location
 
-If your documents contain `_geo` data, you can use the `_geoRadius` built-in filter rule to filter results according to their geographic position.
-
-`_geoRadius` establishes a circular area based on a central point and a radius. Results beyond this area will be excluded from your search. This filter rule requires three parameters: `lat`, `lng` and `distance_in_meters`.
-
-```
-_geoRadius(lat, lng, distance_in_meters)
-```
-
-`lat` and `lng` must be floating point numbers indicating a geographic position. `distance_in_meters` must be an integer indicating the radius covered by the `_geoRadius` filter.
-
-When using a <a id="downloadRestaurants" href="/restaurants.json" download="restaurants.json"> dataset of restaurants</a> containing geopositioning data, we can filter our search so it only includes places within two kilometers of our location:
+If your documents contain `_geo` data, you can filter results according to their geographic position using one of our built-in geosearch filter rules:
 
 <CodeSamples id="geosearch_guide_filter_usage_1" />
 
-[You can read more about filtering results with `_geoRadius` in our geosearch guide.](/learn/advanced/geosearch.md#filtering-results-with-georadius)
+[Read more about filtering results with `_geoRadius` and `_geoBoundingBox` in our geosearch guide.](/learn/advanced/geosearch.md#filtering-results-with-georadius-and-geoboundingbox)
 
 ### Filtering by nested fields
 

diff --git a/learn/advanced/geosearch.md b/learn/advanced/geosearch.md
@@ -109,45 +109,55 @@ If your dataset is formatted as CSV, the file header must have a `_geo` column.
 "3", "Artico Gelateria Tradizionale", "Via Dogana, 1, 20123 Milan, Italy", "ice cream", 10, "48.8826517,2.3352748"
 ```
 
-## Filtering results with `_geoRadius`
+## Filtering results with `_geoRadius` and `_geoBoundingBox`
 
-You can use `_geo` data to filter queries and make sure you only receive results located within a certain geographic area.
+You can use `_geo` data to filter queries so you only receive results located within a given geographic area.
 
 ### Configuration
 
 In order to filter results based on their location, you must add the `_geo` attribute to the `filterableAttributes` list:
 
 <CodeSamples id="geosearch_guide_filter_settings_1" />
 
-Note that Meilisearch will rebuild your index whenever you update `filterableAttributes`. Depending on the size of your dataset, this might take a considerable amount of time.
+Meilisearch will rebuild your index whenever you update `filterableAttributes`. Depending on the size of your dataset, this might take a considerable amount of time.
 
 [You can read more about configuring `filterableAttributes` in our dedicated filtering guide.](/learn/advanced/filtering_and_faceted_search.md#configuring-filters)
 
 ### Usage
 
-First, ensure your documents contain valid geolocation data and that you have added the `_geo` attribute to the `filterableAttributes` list. Then, you can use the [`filter` search parameter](/reference/api/search.md#filter) along with `_geoRadius`, a special filter rule, to ensure Meilisearch only returns results located within a specific geographic area.
+Use the [`filter` search parameter](/reference/api/search.md#filter) along with `_geoRadius` or `_geoBoundingBox`. These are special filter rules that ensure Meilisearch only returns results located within a specific geographic area.
 
-`_geoRadius` establishes a circular area based on a central point and a radius. Results beyond this area will be excluded from your search. This filter rule requires three parameters: `lat`, `lng` and `distance_in_meters`.
+### `_geoRadius`
+
+`_geoRadius` establishes a circular area based on a central point and a radius. This filter rule requires three parameters: `lat`, `lng` and `distance_in_meters`.
 
 ```
 _geoRadius(lat, lng, distance_in_meters)
 ```
 
-`lat` and `lng` must be floating point numbers indicating a geographic position. `distance_in_meters` must be an integer indicating the radius covered by the `_geoRadius` filter. If any of these three parameters are invalid or missing, Meilisearch will return an [`invalid_search_filter`](/reference/errors/error_codes.md#invalid-search-filter) error.
+`lat` and `lng` must be floating point numbers indicating a geographic position. `distance_in_meters` must be an integer indicating the radius covered by the `_geoRadius` filter.
 
-[You can read more about using `filter` in our dedicated guide.](/learn/advanced/filtering_and_faceted_search.md#using-filters)
+### `_geoBoundingBox`
 
-::: warning
-`_geo`, `_geoDistance`, and `_geoPoint` are not valid filter rules. Trying to use any of them with the `filter` search parameter will result in an [`invalid_search_filter`](/reference/errors/error_codes.md#invalid-search-filter) error.
-:::
+`_geoBoundingBox` establishes a rectangular area based on the coordinates for its top right and bottom left corners. This filter rule requires two arrays:
+
+```
+_geoBoundingBox([{lat}, {lng}], [{lat}, {lng}])
+```
+
+`lat` and `lng` must be floating point numbers indicating a geographic position. The first array indicates the geographic coordinates of the top right corner of the rectangular area. The second array indicates the coordinates of the bottom left corner of the rectangular area.
 
 ### Examples
 
-`_geoRadius` works like any other filter rule. Using our <a id="downloadRestaurants" href="/restaurants.json" download="restaurants.json">example dataset</a>, we can search for places to eat near the center of Milan:
+Using our <a id="downloadRestaurants" href="/restaurants.json" download="restaurants.json">example dataset</a>, we can search for places to eat near the center of Milan with `_geoRadius`:
 
 <CodeSamples id="geosearch_guide_filter_usage_1" />
 
-The results should look like this:
+We also make a similar query using `_geoBoundingBox`:
+
+<CodeSamples id="geosearch_guide_filter_usage_3" />
+
+In both cases, the results should look like this:
 
 ```json
 [
@@ -176,12 +186,10 @@ The results should look like this:
 ]
 ```
 
-It is also possible to use `_geoRadius` together with other filters. We can narrow down our previous search so it only includes pizzerias:
+It is also possible to combine `_geoRadius` and `_geoBoundingBox` with other filters. We can narrow down our previous search so it only includes pizzerias:
 
 <CodeSamples id="geosearch_guide_filter_usage_2" />
 
-The above command will only work if you have previously added `type` to `filterableAttributes`.
-
 ```json
 [
   {
@@ -198,6 +206,12 @@ The above command will only work if you have previously added `type` to `filtera
 ]
 ```
 
+The above command will only work if you have previously added `type` to `filterableAttributes`.
+
+::: warning
+`_geo`, `_geoDistance`, and `_geoPoint` are not valid filter rules. Trying to use any of them with the `filter` search parameter will result in an [`invalid_search_filter`](/reference/errors/error_codes.md#invalid-search-filter) error.
+:::
+
 ## Sorting results with `_geoPoint`
 
 You can use `_geo` data to sort results based on their distance from a specific location.

diff --git a/learn/advanced/known_limitations.md b/learn/advanced/known_limitations.md
@@ -115,12 +115,12 @@ user = 1 OR user = 2 […] OR user = 1500 OR user = 1501 […] OR user = 2000 OR
 
 ## Maximum database size
 
-**Limitation:** The maximum size of an index is 500GiB, and the maximum size of the task database is 10GiB.
+**Limitation:** Meilisearch supports a maximum index size of around 80TiB on Linux environments. For performance reasons, Meilisearch recommends keeping indexes under 2TiB.
 
-**Explanation:** Meilisearch allocates all the virtual memory it requires upfront. The maximum database size ensures instances can contain several large indexes without reaching operating system limits on the amount of virtual memory available to a single process.
+**Explanation:** Meilisearch can accommodate indexes of any size as long the combined size of active databases is below the maximum virtual address space the OS devotes to a single process. On 64-bit Linux, this limit is approximately 80TiB.
 
 ## Maximum number of indexes in an instance
 
-**Limitation:** A single Meilisearch instance can safely contain around 180 indexes in Linux and macOS environments, and around 15 indexes in Windows environments.
+**Limitation:** Meilisearch can accommodate an arbitrary number of indexes as long as their size does not exceed 2TiB. When dealing with larger indexes, Meilisearch can accommodate up to 20 indexes as long as their combined size does not exceed the OS's virtual address space limit.
 
-**Explanation:** Operating systems restrict the amount of virtual memory available to a single process. Since Meilisearch allocates the maximum index size for each index upfront, this effectively creates a limit on how many indexes an instance can contain. The values provided are indicative: because of the particularities of each specific setup, the actual maximum number of indexes per instance will vary from machine to machine. A Linux install might contain 200 indexes without issue, or return allocation failures at 181 indexes depending on the runtime environment.
+**Explanation:** While Meilisearch supports an arbitrary number of indexes under 2TiB, accessing hundreds of different databases in short periods of time might lead to decreased performance and should be avoided when possible.
diff --git a/learn/experimental/metrics.md b/learn/experimental/metrics.md
@@ -0,0 +1,13 @@
+# Metrics
+
+The `/metrics` endpoint exposes data compatible with [Prometheus](https://prometheus.io/). You can monitor this endpoint to gain valuable insight into Meilisearch's behavior and performance. This information can then be used to optimize an application's performance, troubleshoot issues, or improve the overall reliability of a system.
+
+[Visit the GitHub discussion to learn how to use this feature](https://github.com/meilisearch/product/discussions/625).
+
+Are you using this feature? Meilisearch wants to hear from you! Tell us about your experience in the [GitHub discussion](https://github.com/meilisearch/product/discussions/625). All feedback is useful and helps make Meilisearch better and easier-to-use.
+
+::: warning
+`/metrics` is an experimental feature. Experimental features are unstable: their API might significantly change and become incompatible between releases. Meilisearch does not recommend using experimental features in a production environment.
+
+Meilisearch makes experimental features available with the expectation they will become stable in future releases. However, it is not possible to guarantee when and if this will happen.
+:::
diff --git a/learn/what_is_meilisearch/comparison_to_alternatives.md b/learn/what_is_meilisearch/comparison_to_alternatives.md
@@ -127,7 +127,7 @@ Can't find a client you'd like us to support? [Submit your idea or vote for it](
 |  | Meilisearch | Algolia | Typesense | Elasticsearch |
 |---|:---:|:----:|:---:|:---:|
 | Maximum number of indexes | No limitation | 1000, increasing limit possible by contacting support | No limitation | No limitation |
-| Maximum index size | 100GiB default, configurable  | 128GB | Constrained by RAM | No limitation |
+| Maximum index size | 80TiB  | 128GB | Constrained by RAM | No limitation |
 | Maximum words per attribute | No limitation | No limitation | No limitation | No limitation |
 | Maximum document size | No limitation | 100KB, configurable | No limitation | 100KB default, configurable  |
 

diff --git a/learn/what_is_meilisearch/telemetry.md b/learn/what_is_meilisearch/telemetry.md
@@ -106,7 +106,7 @@ This list is liable to change with every new version of Meilisearch. It's not be
 | `infos.env`                                        | Value of `--env`/`MEILI_ENV`                                                                | production
 | `infos.db_path`                                    | `true` if `--db-path`/`MEILI_DB_PATH` is specified, otherwise `false`                       | true
 | `infos.import_dump`                                | `true` if `--import-dump` is specified, otherwise `false`                                   | true
-| `infos.dump_dir`                                  | `true` if `--dump-dir`/`MEILI_DUMP_DIR` is specified, otherwise `false`                   | true
+| `infos.dump_dir`                                  | `true` if `--dump-dir`/`MEILI_DUMP_DIR` is specified, otherwise `false`                      | true
 | `infos.ignore_missing_dump`                        | `true` if `--ignore-missing-dump` is activated, otherwise `false`                           | true
 | `infos.ignore_dump_if_db_exists`                   | `true` if `--ignore-dump-if-db-exists` is activated, otherwise `false`                      | true
 | `infos.import_snapshot`                            | `true` if `--import-snapshot` is specified, otherwise `false`                               | true
@@ -143,6 +143,7 @@ This list is liable to change with every new version of Meilisearch. It's not be
 | `requests.total_received`                          | Total number of received search requests                                                    | 3480
 | `sort.with_geoPoint`                               | `true` if the sort rule `_geoPoint` is specified, otherwise `false`                         | true
 | `sort.avg_criteria_number`                         | Average number of sort criteria among all search requests containing the `sort` parameter   | 2
+| `filter.with_geoBoundingBox`                       | `true` if the filter rule `_geoBoundingBox` is specified, otherwise `false`                 | false
 | `filter.with_geoRadius`                            | `true` if the filter rule `_geoRadius` is specified, otherwise `false`                      | false
 | `filter.most_used_syntax`                          | Most used filter syntax among all search requests containing the `filter` parameter         | string
 | `q.max_terms_number`                               | Highest number of terms given for the `q` parameter                                         | 5
@@ -152,7 +153,7 @@ This list is liable to change with every new version of Meilisearch. It's not be
 | `formatting.max_attributes_to_highlight`           | Maximum number of attributes to highlight                                                   | 100
 | `formatting.highlight_pre_tag`                     | `true` if `highlightPreTag` is specified, otherwise `false`                                 | false
 | `formatting.highlight_post_tag`                    | `true` if `highlightPostTag` is specified, otherwise `false`                                | false
-| `formatting.max_attributes_to_crop`                | Maximum number of attributes to crop | 100
+| `formatting.max_attributes_to_crop`                | Maximum number of attributes to crop                                                        | 100
 | `formatting.crop_length`                           | `true` if `cropLength` is specified, otherwise `false`                                      | false
 | `formatting.crop_marker`                           | `true` if `cropMarker` is specified, otherwise `false`                                      | false
 | `formatting.show_matches_position`                 | `true` if `showMatchesPosition` is used in this batch, otherwise `false`                    | false
@@ -181,7 +182,7 @@ This list is liable to change with every new version of Meilisearch. It's not be
 | `filtered_by_canceled_by`                          | `true` if tasks are filtered by the `canceledBy` query parameter, otherwise `false`         | false
 | `filtered_by_before_enqueued_at`                   | `true` if tasks are filtered by the `beforeEnqueuedAt` query parameter, otherwise `false`   | false
 | `filtered_by_after_enqueued_at`                    | `true` if tasks are filtered by the `afterEnqueuedAt` query parameter, otherwise `false`    | false
-| `filtered_by_before_started_at`                    | `true` if tasks are filtered by the `beforeStartedAt` query parameter, otherwise `false`    |  false
+| `filtered_by_before_started_at`                    | `true` if tasks are filtered by the `beforeStartedAt` query parameter, otherwise `false`    | false
 | `filtered_by_after_started_at`                     | `true` if tasks are filtered by the `afterStartedAt` query parameter, otherwise `false`     | false
 | `filtered_by_before_finished_at`                   | `true` if tasks are filtered by the `beforeFinishedAt` query parameter, otherwise `false`   | false
 | `filtered_by_after_finished_at`                    | `true` if tasks are filtered by the `afterFinishedAt` query parameter, otherwise `false`    | false
@@ -198,8 +199,14 @@ This list is liable to change with every new version of Meilisearch. It's not be
 | `stop_words.total`                                 | Number of stop words                                                                        | 3
 | `synonyms.total`                                   | Number of synonyms                                                                          | 3
 | `per_index_uid`                                    | `true` if the `uid` is used to fetch an index stat resource, otherwise `false`              | false
+| `searches.avg_search_count`                        | The average number of search queries received per call for the aggregated event             | 4.2
+| `searches.total_search_count`                      | The total number of search queries received for the aggregated event                        | 16023
+| `indexes.avg_distinct_index_count`                 | The average number of queried indexes received per call for the aggregated event            | 1.2
+| `indexes.total_distinct_index_count`               | The total number of distinct index queries for the aggregated event                       | 6023
+| `indexes.total_single_index`                       | The total number of calls when only one index is queried                                | 2007
 | `matching_strategy.most_used_strategy`             | Most used word matching strategy                                                            | last
 | `infos.with_configuration_file`                    | `true` if the instance is launched with a configuration file, otherwise `false`             | false
+| `infos.experimental_enable_metrics`                | `true` if `--experimental-enable-metrics`/`MEILI_EXPERIMENTAL_ENABLE_METRICS` is specified, otherwise `false`         | false
 | `swap_operation_number`                            | Number of swap operations                                                                   | 2
 | `pagination.most_used_navigation`                  | Most used search results navigation                                                         | estimated
 | `per_document_id`                                  | `true` if the `DELETE /indexes/:indexUid/documents/:documentUid` endpoint was used, otherwise `false` | false