Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add bulk operation for Engine and Indexers and use it for reindex operation #430

Merged

Conversation

alexander-schranz
Copy link
Member

@alexander-schranz alexander-schranz commented Oct 6, 2024

A new interface will support bulk operations. If the index does not support it we wall fallback to the normal save and delete functions:

    /**
     * @param iterable<array<string, mixed>> $saveDocuments
     * @param iterable<string> $deleteDocumentIdentifiers
     * @param array{return_slow_promise_result?: true} $options
     *
     * @return ($options is non-empty-array ? TaskInterface<array<string, mixed>> : null)
     */
    public function bulk(
        Index $index,
        iterable $saveDocuments,
        iterable $deleteDocumentIdentifiers,
        int $bulkSize = 100,
        array $options = [],
    ): TaskInterface|null;

fix #24

Adapters

Todo

  • Fix integration reindex tests
  • Fix Meilisearch lowest 784d084

@alexander-schranz alexander-schranz added features New feature or request SEAL Core Seal Core related issue labels Oct 6, 2024
@alexander-schranz alexander-schranz changed the title Add support for bulk save and delete via BulkableIndexerInterface Add bulk operation for Engine and Indexers and use it for reindex operation Oct 6, 2024
@alexander-schranz alexander-schranz added this to the 1.0.0 milestone Oct 6, 2024
@tacman
Copy link

tacman commented Oct 6, 2024

I think Meilisearch will be even easier to implement, and will see a big performance boost for adding documents.

@alexander-schranz alexander-schranz force-pushed the feature/bulk-operation branch 2 times, most recently from da3701d to f40e005 Compare October 6, 2024 16:58
@alexander-schranz alexander-schranz marked this pull request as ready for review October 8, 2024 15:22
@alexander-schranz alexander-schranz added Adapter: Elasticsearch Elasticsearch Adapter releated issue Adapter: Opensearch Opensearch Adapter related issue Adapter: Memory Memory Adapter related issue Adapter: Multi Multi Adapter related issue Adapter: Algolia Algolia Adapter related issue Adapter: Solr Solr Adapter related issue labels Oct 8, 2024
@alexander-schranz alexander-schranz added Adapter: Typesense Typesense Adapter related issue Adapter: RediSearch RediSearch Adapter related issue Adapter: Loupe Loupe Adapter related issue Adapter: ReadWrite ReadWrite Adapter related issue labels Oct 8, 2024
@alexander-schranz alexander-schranz merged commit efd87ac into PHP-CMSIG:0.6 Oct 8, 2024
30 of 32 checks passed
@alexander-schranz alexander-schranz deleted the feature/bulk-operation branch October 8, 2024 15:30
@tacman
Copy link

tacman commented Oct 8, 2024

Congrats and thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Adapter: Algolia Algolia Adapter related issue Adapter: Elasticsearch Elasticsearch Adapter releated issue Adapter: Loupe Loupe Adapter related issue Adapter: Memory Memory Adapter related issue Adapter: Multi Multi Adapter related issue Adapter: Opensearch Opensearch Adapter related issue Adapter: ReadWrite ReadWrite Adapter related issue Adapter: RediSearch RediSearch Adapter related issue Adapter: Solr Solr Adapter related issue Adapter: Typesense Typesense Adapter related issue features New feature or request SEAL Core Seal Core related issue
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Core] Bulk Action on Connection and Engine
2 participants