Providing multiple dense vectors to Elasticsearch script params

Question

I’m using the kNN functionality in Elastic to retrieve the nearest neighbors given a certain vector. However, in contrast with searches based on terms in the query body, I can’t figure out how to do a single search request using multiple vectors.

Using the Python elasticsearch library I can do the following simple query while providing multiple ids to search in batch:

query_body = {
            "terms": {
                "transaction_id": id_list  # <--- this is a list with multiple ids
            }
        }
    
    
search_results = client.search(index=index, query=query_body)

However, when I am doing a kNN query, I can only do the following:

query_body = {
        "script_score": {
            "min_score": 0.80,
            "query": {"match_all": {}},
            "script": {
                "source": "cosineSimilarity(params.query_vector, 'vector') + 1.0",
                "params": {"query_vector": query_vector},
            },
        }
    }

search_results = client.search(index=index, query=query_body)

The above code only retrieves the neighbours for one vector at a time. So if I have 10 vectors that I retrieve by id using the first code snippet, I have to sequentially feed them into the second snippet one by one. Is there not a way to provide multiple vectors to the "params": {"query_vector" : <multiple_vectors_here>}?

Ideally, I would like to chain the calls to Elastic in one operation, but I’m also not sure if that is possible (for example with Multi search?

Thank you very much in advance!

Asked By: mabergerx

||

Source

Answer 1

Currently exact brute force KNN not support multipul query vector. You can read same here on discuss.elastic site.

You can use Approximate KNN search with should cluase as shown here.

Answered By: Sagar Patel

Providing multiple dense vectors to Elasticsearch script params

Question:

Answers: