Test elasticsearch connection in Python

Question:

I am running this code in Python:

es = Elasticsearch([{'host': 'localhost', 'port': 9200}]).ping()
print (es)

if es:
    r = requests.get('http://localhost:9200')
    es.indices.delete(index='test_twitter', ignore=[400, 404])
    connected = True
    print('Connected to ES..')
else:
    print('Not connected to ES...')

When Elasticsearch is running, there is no problem with ping. The problem is when ES is not running, ping() is returning FALSE after a Traceback error is printed as below:

        HEAD http://localhost:9200/ [status:N/A request:2.007s]
Traceback (most recent call last):
ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x000001E2ECBD45F8>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it
HEAD http://localhost:9200/ [status:N/A request:2.003s]
Traceback (most recent call last):
ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x000001E2ECBD44E0>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it
HEAD http://localhost:9200/ [status:N/A request:2.016s]
Traceback (most recent call last):
ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x000001E2ECBD4668>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it
False
Not connected to ES...

Even if I try-except it like below, it does not work:

from elasticsearch.exceptions import ConnectionError
try:
    es = Elasticsearch([{'host': 'localhost', 'port': 9200}]).ping()
except ConnectionRefusedError:
    print ('Connection Errrorrr!')

print (es)

I want ping() to return false but without printing the error log, and idea how that can be done?

The full is code is below:

from __future__ import unicode_literals

import hug
from hug_middleware_cors import CORSMiddleware
import spacy
from elasticsearch import Elasticsearch
import json
import requests
import sys
sys.tracebacklimit = 0

MODELS = {
    'en_core_web_sm': spacy.load('en_core_web_sm'),
    # 'de_core_news_sm': spacy.load('de_core_news_sm'),
    # 'es_core_news_sm': spacy.load('es_core_news_sm'),
    # 'pt_core_news_sm': spacy.load('pt_core_news_sm'),
    # 'fr_core_news_sm': spacy.load('fr_core_news_sm'),
    # 'it_core_news_sm': spacy.load('it_core_news_sm'),
    # 'nl_core_news_sm': spacy.load('nl_core_news_sm')
}


def get_model_desc(nlp, model_name):
    """Get human-readable model name, language name and version."""
    lang_cls = spacy.util.get_lang_class(nlp.lang)
    lang_name = lang_cls.__name__
    model_version = nlp.meta['version']
    return '{} - {} (v{})'.format(lang_name, model_name, model_version)


@hug.get('/models')
def models():
    return {name: get_model_desc(nlp, name) for name, nlp in MODELS.items()}


@hug.post('/dep')
def dep(text: str, model: str, collapse_punctuation: bool=False,
        collapse_phrases: bool=False):
    """Get dependencies for displaCy visualizer."""
    nlp = MODELS[model]
    doc = nlp(text)
    if collapse_phrases:
        for np in list(doc.noun_chunks):
            np.merge(tag=np.root.tag_, lemma=np.root.lemma_,
                     ent_type=np.root.ent_type_)
    options = {'collapse_punct': collapse_punctuation}
    return spacy.displacy.parse_deps(doc, options)


@hug.post('/ent')
def ent(text: str, model: str):
    """Get entities for displaCy ENT visualizer."""
    nlp = MODELS[model]
    doc = nlp(text)

    for ent in doc.ents:
        if connected:
            es.index(index='test_twitter', doc_type='words', body={'tag': ent.text})
        else:
            print('text :')
            print(ent.text)
            print(ent.label_)
    # return [{'text': ent.text, 'start': ent.start_char, 'end': ent.end_char, 'label': ent.label_}
    #         for ent in doc.ents]


if __name__ == '__main__':

    try:
        es = Elasticsearch([{'host': 'localhost', 'port': 9200}]).ping()
    except ConnectionRefusedError:
        print ('Connection Errrorrr!')

    if es:
        es = Elasticsearch([{'host': 'localhost', 'port': 9200}])
        r = requests.get('http://localhost:9200')
        es.indices.delete(index='test_twitter', ignore=[400, 404])
        connected = True
        print('Connected to ES..')
    else:
        print('Not connected to ES...')

    import waitress
    app = hug.API(__name__)
    app.http.add_middleware(CORSMiddleware(app))
    waitress.serve(__hug_wsgi__, port=8000)
Asked By: Ahmad Hijazi

||

Answers:

"HonzaKral commented on Nov 9, 2017": The ping method deliberately catches all exceptions and only returns True/False whether it has been successful […]

"HonzaKral commented on Nov 9, 2017": The traceback is only logged using python’s standard logging module, to disable it configure your logging and set the proper logging level for the elasticsearch logger, shortcut, to disable all non-ERROR level messages is:

import logging
logging.getLogger("elasticsearch").setLevel(logging.CRITICAL)

ref: https://github.com/elastic/elasticsearch-py/issues/666

Answered By: Freddy

There is no way you can avoid the stacktrace you are getting in current scenario. The http_urllib3 deliberately logs this.

self.log_request_fail(method, full_url, url, body, time.time() - start, exception=e)
def log_request_fail(self, method, full_url, path, body, duration, status_code=None, response=None, exception=None):
  """ Log an unsuccessful API call.  """
  # do not log 404s on HEAD requests
  if method == 'HEAD' and status_code == 404:
    return logger.warning(
      '%s %s [status:%s request:%.3fs]', method, full_url,
      status_code or 'N/A', duration, exc_info=exception is not None
    )

The only way to tackle this would be by changing the logging level to anything above warning

Answered By: Ashay Dhavale

Since stacktrace is generated by urllib3, you can disable it like this:

import logging

logging.getLogger("urllib3").setLevel(logging.ERROR)
Answered By: Alex Medveshchek

Try

assert esCursor.ping(), "Elastic Search Connection establishment failed"
Answered By: ganna
Categories: questions Tags: ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.