Connecting to remote Elasticsearch server with python's Elasticsearch package

Question

I want to use a remote Elasticsearch server for my website.

I have used elastic.co/ cloud service to create a remote Elasticsearch server.
I can connect/ping remote Elasticsearch server using the following command (it is scrubbed of sensitive info):
curl -u username:password https://55555555555bb0c30d1cba4e9e6.us-central1.gcp.cloud.es.io:9243

After tying this command into terminal, I receive the following response:

{
  "name" : "instance-0000000001",
  "cluster_name" : "555555555555",
  "cluster_uuid" : "55555555555",
  "version" : {
    "number" : "7.10.2",
    "build_flavor" : "default",
    "build_type" : "docker",
    "build_hash" : "555555555555555555555555",
    "build_date" : "2021-01-13T00:42:12.435326Z",
    "build_snapshot" : false,
    "lucene_version" : "8.7.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

I am using the following Elasticsearch python client to integrate Elasticsearch with my website: https://elasticsearch-py.readthedocs.io/en/v7.10.1/index.html#

To initialize a local connection, I import the Elasticsearch class and initialize an instance of that class by pasting in the local url of my Elasticsearch server.

>>> from elasticsearch import Elasticsearch
>>> es = Elasticsearch('http://localhost:9200')
>>> es
<Elasticsearch([{'host': 'localhost', 'port': 9200}])>

Now I want to connect to my remote server using the same Elasticsearch class. To do this, I need to format the initialization of the Elasticsearch object with the info to connect to my remote elasticsearch server.

This is where I am having some trouble. The docstring for the Elasticsearch class is very opaque. In short, it is asking me to create a custom connection object and I have not been able to figure it out how to do it.

The documentation for the Elasticsearch object is a [little bit better][1] but still does not give an example involving a username and password.

I have the ability to create an API key, but I am not sure how to use it. An answer involving either an API key or answering how to connect using my username and password, would be very helpful.

Asked By: alluppercase

||

Source

Answer 1

You need to connect using TLS/SSL and Authentication as described in the documentation.

In your case you should use something like this.

from elasticsearch import Elasticsearch

es = Elasticsearch(
    ['5555555555bb0c30d1cba4e9e6.us-central1.gcp.cloud.es.io'],
    http_auth=('username', 'password'),
    scheme="https",
    port=9243,
)

Answered By: leandrojmp

Answer 2

I was using elasticsearch v8.6, this one worked for me.

es = Elasticsearch(cloud_id='ELASTIC_CLOUD_ID', basic_auth=('ELASTIC_USER', 'ELASTIC_PASSWORD'))

cloud_id is different from url,

Answered By: Rajesh Kumar

Connecting to remote Elasticsearch server with python's Elasticsearch package

Question:

Answers: