Set up nightly runs for vector search workloads

[VijayanB]

Is your feature request related to a problem? Please describe

Similar to https://opensearch.org/benchmarks/ , we would like to set up nightly runs that runs vector search workloads every night, publish performance metrics, and, make it available for public.

Describe the solution you'd like

Would like to see following night runs:

• Cohere 1M

  • Engine types: nmslib, faiss
  • space type: innerproduct
  • Cluster configuration:
    3 nodes ( 3 shards, 1 replica )
    4 CPU cores in each node
    32 GB memory in each node
  • Algo params
    hnsw_ef_construction : 100
    hnsw_ef_search : 100
    hnsw_m : 16

• Cohere 10 M

  • Engine types:nmslib, faiss
  • space type: innerproduct
  • Cluster configuration:
    3 nodes ( 3 shards, 1 replica )
    16 CPU cores in each node
    128 GB memory in each node
  • Algo params
    hnsw_ef_construction : 100
    hnsw_ef_search : 256
    hnsw_m : 16

Describe alternatives you've considered

No response

Additional context

Support to run those configuration should be available in opensearch-benchmarks-workloads. Hence, users can reproduce this easily without performing any additional steps

[jordarlu]

Hi, @rishabh6788 , @IanHoang, if you can help commnet on this issue, thanks!!

[rishabh6788]

Nightly benchmarks for VectorSearch workload has been scheduled. @VijayanB is currently reviewing metrics with the team and will confirm if he needs any more updates.

[bbarani]

@rishabh6788 @VijayanB Can you please provide an update? Is the dashboard ready for public use?

[VijayanB]

Created 6 dashboards for Vector Search. We are reviewing the metrics and identifying any new dashboards that can be added. It is already available here (Please check metrics starting March 28)

Nmslib 1M Cohere 768D
Nmslib 10 M Cohere 768D
Faiss 1M Cohere 768D
Faiss 10 M Cohere 768D
Lucene 1M Cohere 768D
Lucene 10 M Cohere 768D

[VijayanB]

This is available in website. Can this be closed?