Tasks API - Filter tasks list by `type`/`status`/`indexUid` #116

gmourier · 2022-02-21T09:53:47Z

Summary

Add filtering capabilities to the tasks endpoints to facilitate the management of an instance and its indexes, or a specific index.

This first iteration adds filters on the status, type and indexUid attributes of the task API resource.

Changes

Add filtering capabilities on type, status and indexUid for GET task lists endpoints.
It is possible to specify several values for a filter using the , character. e.g. ?status=enqueued,processing
Between two different filters, an AND operation is applied. e.g. ?status=enqueued&type=indexCreation is equivalent to status=enqueued AND type = indexCreation
GET /indexes/:indexUid/tasks is removed
- It can be replaced by GET /tasks?indexUid=:indexUid
GET /indexes/:indexUid/tasks/:taskUid is removed
- It's already replaced by GET /tasks/:taskUid
Task types documentPartial and documentAddition are renamed documentAdditionOrUpdate
Task type clearAll is renamed documentDeletion

Telemetry

Index Tasks Seen event is removed.
Add filtered_by_index_uid boolean property for Tasks Seen event.
Add filtered_by_type boolean property for Tasks Seen event.
Add filtered_by_statusboolean property for Tasks Seen event.

Motivation

Following the specification aiming to stabilize the task API resource, we want to give users the capability to refine the lists of task according to several criteria to find precise information more quickly.

bidoubiwa · 2022-03-21T14:26:03Z

Is it possible to filter on index ? Since using GET /indexes/:index/task/0 would cause to throw a 404 error when called directly after creating the index. If added, we could suggest to users to always use GET /tasks&index=movies as it would never return a 404.

Also, by doing so, we would avoid having to fetch all tasks and then filter them just to have the ones from one index. Which in turn if there are a lot of different indexes may result in the user having to paginate to be sure they fetched all the tasks

Exemple:

const tasks = await client.getTasks() // could contain a lot of tasks unrelated to the index
const indexTasks = tasks.filter(task => indexUid === "movies")

vs

const tasks = await client.getTasks({ indexUid: "movies" })

MarinPostma · 2022-03-22T08:25:23Z

Is it possible to filter on index ? Since using GET /indexes/:index/task/0 would cause to throw a 404 error when called directly after creating the index. If added, we could suggest to users to always use GET /tasks&index=movies as it would never return a 404.

Also, by doing so, we would avoid having to fetch all tasks and then filter them just to have the ones from one index. Which in turn if there are a lot of different indexes may result in the user having to paginate to be sure they fetched all the tasks

Exemple:
const tasks = await client.getTasks() // could contain a lot of tasks unrelated to the index
const indexTasks = tasks.filter(task => indexUid === "movies")
vs
const tasks = await client.getTasks({ indexUid: "movies" })

technically calling /indexes/:index/tasks is syntactic sugar for a filter on index. ~~The spec doesn't say anything about allowing filters on /indexes/:uid/tasks route, but I think it should be part of this.~~ my bad, just saw it :)

MarinPostma

This API feels a bit awkward to me TBH, we have on one end index and uid that are identified as resources, and status and type that are filters. It would be more consistent to have everything be filters, wouldn't it?

text/0061-error-format-and-definitions.md

text/0060-tasks-api.md

gmourier · 2022-04-14T14:01:16Z

@bidoubiwa, if we add a query parameter to filter tasks by indexUid, we could evaluate if we can remove the syntactic sugar /indexes/:index_uid/tasks and /indexes/:index_uid/tasks/:task_uid to simplify the number of endpoints, we will have to look at the usage with our analytics.

bidoubiwa · 2022-04-14T15:37:38Z

I'm all for removing /indexes/:index_uid/tasks/:task_uid as it leads to much to a possible 404. I'm curious about the analytics.

brunoocasali

I left some polemical comments 😬 😅

text/0060-tasks-api.md

brunoocasali · 2022-04-14T17:39:51Z

Oh, I think I came late to the party about the potential removal of the index/uid/tasks, @MarinPostma and @bidoubiwa already raised the question! 🐞

github-actions · 2022-04-21T14:04:00Z

🚨 Breaking API change detected:

Modified (26)

DELETE /indexes/{indexUid}/documents
- Response modified: 202
  - Body attribute added: type
DELETE /indexes/{indexUid}/documents/{documentId}
- Response modified: 202
  - Body attribute added: type
DELETE /indexes/{indexUid}/settings
- Response modified: 202
  - Body attribute added: type
DELETE /indexes/{indexUid}/settings/displayed-attributes
- Response modified: 202
  - Body attribute added: type
DELETE /indexes/{indexUid}/settings/distinct-attribute
- Response modified: 202
  - Body attribute added: type
DELETE /indexes/{indexUid}/settings/filterable-attributes
- Response modified: 202
  - Body attribute added: type
DELETE /indexes/{indexUid}/settings/ranking-rules
- Response modified: 202
  - Body attribute added: type
DELETE /indexes/{indexUid}/settings/searchable-attributes
- Response modified: 202
  - Body attribute added: type
DELETE /indexes/{indexUid}/settings/sortable-attributes
- Response modified: 202
  - Body attribute added: type
DELETE /indexes/{indexUid}/settings/stop-words
- Response modified: 202
  - Body attribute added: type
DELETE /indexes/{indexUid}/settings/synonyms
- Response modified: 202
  - Body attribute added: type
DELETE /indexes/{indexUid}/settings/typo-tolerance
- Response modified: 202
  - Body attribute added: type
GET /tasks
- Path parameters added: indexUid, status, type
POST /indexes/{indexUid}/documents
- Response modified: 202
  - Body attribute added: type
POST /indexes/{indexUid}/documents/delete-batch
- Response modified: 202
  - Body attribute added: type
POST /indexes/{indexUid}/settings
- Response modified: 202
  - Body attribute added: type
POST /indexes/{indexUid}/settings/displayed-attributes
- Response modified: 202
  - Body attribute added: type
POST /indexes/{indexUid}/settings/distinct-attribute
- Response modified: 202
  - Body attribute added: type
POST /indexes/{indexUid}/settings/filterable-attributes
- Response modified: 202
  - Body attribute added: type
POST /indexes/{indexUid}/settings/ranking-rules
- Response modified: 202
  - Body attribute added: type
POST /indexes/{indexUid}/settings/searchable-attributes
- Response modified: 202
  - Body attribute added: type
POST /indexes/{indexUid}/settings/sortable-attributes
- Response modified: 202
  - Body attribute added: type
POST /indexes/{indexUid}/settings/stop-words
- Response modified: 202
  - Body attribute added: type
POST /indexes/{indexUid}/settings/synonyms
- Response modified: 202
  - Body attribute added: type
POST /indexes/{indexUid}/settings/typo-tolerance
- Response modified: 202
  - Body attribute added: type
PUT /indexes/{indexUid}/documents
- Response modified: 202
  - Body attribute added: type

Removed (2)

[Breaking] GET /indexes/{indexUid}/tasks
[Breaking] GET /indexes/{indexUid}/tasks/{taskUid}

View documentation diff

Powered by Bump

gmourier · 2022-04-21T14:05:26Z

The specification is now updated with the latest information given in the comments.

2399: Update the tasks endpoints r=MarinPostma a=Kerollmops This PR wraps all the changes related to the `tasks` endpoints, it is related to #2377 but doesn't close it. I will create a new PR to work on [the seek-based pagination](meilisearch/specifications#115). I wanted to do something cool with Github: being able to merge multiple PR in this one, to help review changes one by one, unfortunately, Github doesn't allow creating empty PRs. I also struggled with git itself when it comes to merging things in the right order, so I decided that I would add all of the changes in this single PR. I will list the changes and references to the specs here. - [x] Tasks statuses and types must be case insensitive - [x] Tasks statuses, types and indexUid must accept the `*` selector - [ ] Rename the `TaskDetails` struct fields ## Changes - [ ] Add seek-based pagination following [the spec](meilisearch/specifications#115) - [x] Add filtering on the `/tasks` endpoint following [this spec](meilisearch/specifications#116) - [x] Add filtering capabilities on `type`, `status` and `indexUid` for `GET` `task` lists endpoints. - [x] It is possible to specify several values for a filter using the `,` character. e.g. `?status=enqueued,processing` - [x] Between two different filters, an AND operation is applied. e.g. `?status=enqueued&type=indexCreation` is equivalent to `status=enqueued AND type = indexCreation` - [x] Remove `GET /indexes/:indexUid/tasks`. It can be replaced by `GET /tasks?indexUid=:indexUid` - [x] Remove `GET /indexes/:indexUid/tasks/:taskUid`. - [x] Rename `uid` to `taskUid` in the `202 - Accepted` task response return by every asynchronous tasks (ex: index creation, document addition...) - [x] Rename some task properties - [x] `documentPartial`-> `documentAdditionOrUpdate` - [x] `documentAddition`-> `documentAdditionOrUpdate` - [x] `clearAll` -> `documentDeletion` Co-authored-by: Kerollmops <clement@meilisearch.com>

…uth instead of a changelog

… field to the 202 Response resource

* move filtering tasks by status/type parameter to task api spec * Update specification * Add details about case-sensitivy + rework error message * Introducing naming changes plus make the specification a source of truth instead of a changelog * Remove a future possibility being introduced * misc - replace createIndex to the right type and add the missing type field to the 202 Response resource

* Bump open-api.yml to v0.28 * Telemetry - Add `x-meilisearch-client` query parameter (#145) * Introduce the x-meilisearch-client query parameter * Update text/0034-telemetry-policies.md Co-authored-by: Bruno Casali <brunoocasali@gmail.com> * Update text/0034-telemetry-policies.md Co-authored-by: Bruno Casali <brunoocasali@gmail.com> Co-authored-by: Bruno Casali <brunoocasali@gmail.com> * GeoSearch — Support string type for `_geo` `lat` and `lng` fields (#83) * Update specification to support string type for _geo lat and lng fields * mention types mixing for _geo object * Tasks API - Rename `uid` to `taskUid` in the `202 - Accepted` Summarized Task Response (#144) * Rename 202 uid to taskUid * Update text/0060-tasks-api.md Co-authored-by: cvermand <33010418+bidoubiwa@users.noreply.github.com> Co-authored-by: cvermand <33010418+bidoubiwa@users.noreply.github.com> * Tasks API - Seek/Keyset based pagination (#115) * Move cursor based pagination spec to tasks API spec * remove pagination as a future capability * Clarify boundaries for limit query parameter * Update OpenApi specification * Remove limit field boundaries * Apply suggestions from code review Co-authored-by: Clément Renault <renault.cle@gmail.com> * Update open-api.yml and removes cursor term mentions * Update text/0060-tasks-api.md Co-authored-by: Tommy <68053732+dichotommy@users.noreply.github.com> * Remove route mention in seek-keyset pagination section Co-authored-by: Clément Renault <renault.cle@gmail.com> Co-authored-by: Tommy <68053732+dichotommy@users.noreply.github.com> * Tasks API - Filter tasks list by `type`/`status`/`indexUid` (#116) * move filtering tasks by status/type parameter to task api spec * Update specification * Add details about case-sensitivy + rework error message * Introducing naming changes plus make the specification a source of truth instead of a changelog * Remove a future possibility being introduced * misc - replace createIndex to the right type and add the missing type field to the 202 Response resource * Dumps API - Make dump creation an asynchronous task (#139) * wip * Make a dump creation a visible asynchronous task * Add precisions * Update open-api.yml * Add ommited type field for summarized task response * Add future possibilities * Apply suggestions from code review Co-authored-by: cvermand <33010418+bidoubiwa@users.noreply.github.com> * Precise that indexUid can be null * Precise priorization of dumpCreation task over other task types * Keep taskUid for 202 response * remove dumps.get API key action Co-authored-by: cvermand <33010418+bidoubiwa@users.noreply.github.com> * Search API - Remove/Rename confusing fields (#135) * Rename nbHits, remove exhaustive* boolean fields * Rename approximativeNbHits to estimatedTotalHits * Update open-api.yaml * Apply naming changes for facet distribution and showing matches position * Add a telemetry for facet distribution usage * API Guideline - Return list of API resources under a `results` array (#138) * Place list of documents under a results array on /documents * Add results array for indexes object list * Add the future of indexes pagination * Update open-api.yml * Fix typo * Apply suggestions from code review Co-authored-by: cvermand <33010418+bidoubiwa@users.noreply.github.com> * Add offset/limit pagination for indexes and API keys * Try to add multipe refs to a response object Co-authored-by: cvermand <33010418+bidoubiwa@users.noreply.github.com> * Remove name field (#140) * Documents API - `displayedAttributes` should not impact the documents API / Rename `attributesToRetrieve` to `fields` (#143) * Specifies that displayedAttributes setting does not impact the GET documents endpoint * Rename attributeToRetrieve to fields on /documents * Add a future possibily to rejectt a field from a document in the given response * Precise behavior details about fields query parameter * Add fields query parameter on GET /indexes/{index}/documents/{docId} * API Keys - Determinist API Keys + Security changes (#148) * Add an uid to make API Keys determinists, plus add a non-unique human readable name field to ease reading information * Describe errors for uid and name fields * Apply suggestions from code review Co-authored-by: Bruno Casali <brunoocasali@gmail.com> * misc: add precisions * Reorganize route descriptions * Update error_code when API Key already exists for a given uid * Apply suggestions from code review Co-authored-by: Many the fish <legendre.maxime.isn@gmail.com> * Add new keys actions, remove master-key changes, introduce a new error for immutable field and update tenant token * Update open-api spec * Update immutable_field error message * Apply suggestions from code review Co-authored-by: Many the fish <legendre.maxime.isn@gmail.com> * Mention that the Default Admin API Key can manage keys * Specify that the JWT Tenant Token must be enrypted with the API Key value * Update the spec regarding the description of the Admin API Key to be up-to-date * Add uid_or_key url param to update and delete a key * Update text/0085-api-keys.md Co-authored-by: Many the fish <legendre.maxime.isn@gmail.com> * Update text/0085-api-keys.md Co-authored-by: Many the fish <legendre.maxime.isn@gmail.com> Co-authored-by: Bruno Casali <brunoocasali@gmail.com> Co-authored-by: Many the fish <legendre.maxime.isn@gmail.com> Co-authored-by: Kerollmops <clement@meilisearch.com> * Geosearch - Enhance lat/lng format error messages (#149) * Update the geosearch error message * misc: organiser error message in the right specification Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com> * Introduces HTTP Verbs changesto be compliant regarding a Rest API (#152) * Telemetry - Replace `x-meilisearch-client` query parameter by `X-Meilisearch-Client` header (#150) * Removes x-meilisearch-client, replace it by a header * Remove capslock * fix typo (#151) * Mention telemetry.meilisearch.com (#153) * update ranking rules error message (#154) * Misc — Update dump versions compatibility table (#156) * Update dump table * Update text/0105-dumps-api.md * Settings API - Customize the hard limits for `pagination` and `faceting` (#157) * Introduces specification files * Update files name * branch telemetry * Update open-api.yml * Update text/0034-telemetry-policies.md Co-authored-by: Clément Renault <renault.cle@gmail.com> * update open-api.yml * Update text/157-faceting-setting-api.md Co-authored-by: Clément Renault <renault.cle@gmail.com> * Rename limitedTo to maxTotalHits * Specify order of returned facet Co-authored-by: Clément Renault <renault.cle@gmail.com> * Add dumpCreation task type to OpenAPI.yml * Tasks filtering params to be in query instead of path on OpenAPI spec Co-authored-by: Bruno Casali <brunoocasali@gmail.com> Co-authored-by: cvermand <33010418+bidoubiwa@users.noreply.github.com> Co-authored-by: Clément Renault <renault.cle@gmail.com> Co-authored-by: Tommy <68053732+dichotommy@users.noreply.github.com> Co-authored-by: Many the fish <legendre.maxime.isn@gmail.com> Co-authored-by: Kerollmops <clement@meilisearch.com> Co-authored-by: Tamo <tamo@meilisearch.com> Co-authored-by: ad hoc <postma.marin@protonmail.com> Co-authored-by: Clémentine Urquizar - curqui <clementine@meilisearch.com>

gmourier added the Ready For Review Feature specification must be reviewed. label Feb 21, 2022

gmourier mentioned this pull request Feb 21, 2022

Add status and type filters on task lists endpoints. #73

Closed

gmourier added the v0.28 label Feb 21, 2022

gmourier requested a review from MarinPostma March 1, 2022 16:51

gmourier force-pushed the add-filter-on-status-and-type-to-tasks-api-spec branch from 2b38651 to 6070409 Compare March 14, 2022 16:56

MarinPostma reviewed Mar 22, 2022

View reviewed changes

text/0061-error-format-and-definitions.md Show resolved Hide resolved

text/0060-tasks-api.md Outdated Show resolved Hide resolved

gmourier mentioned this pull request Mar 30, 2022

Release v0.28.0 #137

Merged

1 task

gmourier added In Progress Feature specification is in elaboration. Important changes can still occurs in the specification. and removed Ready For Review Feature specification must be reviewed. labels Apr 14, 2022

gmourier changed the title ~~Tasks API - Filter lists by type/status~~ Tasks API - Filter lists by type/status/index Apr 14, 2022

brunoocasali reviewed Apr 14, 2022

View reviewed changes

text/0060-tasks-api.md Outdated Show resolved Hide resolved

text/0060-tasks-api.md Outdated Show resolved Hide resolved

gmourier added Telemetry Update the telemetry collect. OpenAPI Update OpenAPI specification. Q3:2022 labels Apr 21, 2022

gmourier changed the title ~~Tasks API - Filter lists by type/status/index~~ Tasks API - Filter lists by type/status/indexUid Apr 21, 2022

gmourier changed the title ~~Tasks API - Filter lists by type/status/indexUid~~ Tasks API - Filter tasks list by type/status/indexUid Apr 21, 2022

gmourier self-assigned this Apr 21, 2022

gmourier force-pushed the add-filter-on-status-and-type-to-tasks-api-spec branch 3 times, most recently from 6817e5e to 58415df Compare April 21, 2022 14:03

gmourier added Ready For Review Feature specification must be reviewed. and removed In Progress Feature specification is in elaboration. Important changes can still occurs in the specification. labels Apr 21, 2022

gmourier requested review from brunoocasali, curquiza and MarinPostma April 21, 2022 14:05

curquiza mentioned this pull request May 5, 2022

Changes on /tasks endpoints meilisearch/meilisearch#2377

Closed

15 tasks

gmourier force-pushed the add-filter-on-status-and-type-to-tasks-api-spec branch from 58415df to 55b97b5 Compare May 16, 2022 11:29

Kerollmops mentioned this pull request May 17, 2022

Remove the tasks routes under the indexes resource meilisearch/meilisearch#2400

Closed

gmourier mentioned this pull request May 17, 2022

Tasks API - Seek/Keyset based pagination #115

Merged

gmourier force-pushed the add-filter-on-status-and-type-to-tasks-api-spec branch from 298681f to 1b65c27 Compare May 23, 2022 14:23

curquiza previously approved these changes May 24, 2022

View reviewed changes

Kerollmops mentioned this pull request May 25, 2022

Update the tasks endpoints meilisearch/meilisearch#2399

Merged

15 tasks

gmourier dismissed curquiza’s stale review via d310f5e May 25, 2022 10:36

Kerollmops mentioned this pull request Jun 3, 2022

/indexes/movies/tasks and /indexes/movies/tasks/:uid endpoints result in a 404 meilisearch/meilisearch#2458

Closed

guimachiavelli mentioned this pull request Jun 8, 2022

v0.28: Changes to Tasks API meilisearch/documentation#1706

Closed

gmourier added 6 commits July 7, 2022 12:12

move filtering tasks by status/type parameter to task api spec

34de968

Update specification

7789cd2

Add details about case-sensitivy + rework error message

8f50ff8

Introducing naming changes plus make the specification a source of tr…

c2df7e1

…uth instead of a changelog

Remove a future possibility being introduced

a0d688a

misc - replace createIndex to the right type and add the missing type…

4d3ebdd

… field to the 202 Response resource

gmourier force-pushed the add-filter-on-status-and-type-to-tasks-api-spec branch from 2afb0f3 to 4d3ebdd Compare July 7, 2022 10:29

gmourier merged commit a4af7d2 into develop Jul 7, 2022

gmourier deleted the add-filter-on-status-and-type-to-tasks-api-spec branch July 7, 2022 10:34

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Tasks API - Filter tasks list by `type`/`status`/`indexUid` #116

Tasks API - Filter tasks list by `type`/`status`/`indexUid` #116

gmourier commented Feb 21, 2022 •

edited

Loading

bidoubiwa commented Mar 21, 2022 •

edited

Loading

MarinPostma commented Mar 22, 2022 •

edited

Loading

MarinPostma left a comment

gmourier commented Apr 14, 2022 •

edited

Loading

bidoubiwa commented Apr 14, 2022

brunoocasali left a comment

brunoocasali commented Apr 14, 2022

github-actions bot commented Apr 21, 2022 •

edited

Loading

gmourier commented Apr 21, 2022

Tasks API - Filter tasks list by type/status/indexUid #116

Tasks API - Filter tasks list by type/status/indexUid #116

Conversation

gmourier commented Feb 21, 2022 • edited Loading

Summary

Changes

Telemetry

Motivation

bidoubiwa commented Mar 21, 2022 • edited Loading

MarinPostma commented Mar 22, 2022 • edited Loading

MarinPostma left a comment

Choose a reason for hiding this comment

gmourier commented Apr 14, 2022 • edited Loading

bidoubiwa commented Apr 14, 2022

brunoocasali left a comment

Choose a reason for hiding this comment

brunoocasali commented Apr 14, 2022

github-actions bot commented Apr 21, 2022 • edited Loading

Modified (26)

Removed (2)

gmourier commented Apr 21, 2022

Tasks API - Filter tasks list by `type`/`status`/`indexUid` #116

Tasks API - Filter tasks list by `type`/`status`/`indexUid` #116

gmourier commented Feb 21, 2022 •

edited

Loading

bidoubiwa commented Mar 21, 2022 •

edited

Loading

MarinPostma commented Mar 22, 2022 •

edited

Loading

gmourier commented Apr 14, 2022 •

edited

Loading

github-actions bot commented Apr 21, 2022 •

edited

Loading