Updated edxapp Elasticsearch config

[clintonb]

The urllib3 library (used by the Elasticsearch Python library) expects ports to be integers, not strings, when passed as kwargs. Thus, we set the port to be an integer in Ansible.

[jibsheet]

@MichaelRoytman is working in this area, I'll let him review/pull into another branch/etc

[clintonb]

Thanks for the feedback. I will merge this once I verify it works as expected. My Docker image is still building.

[clintonb]

This doesn't solve the problem. This is a known issue with Ansible + Jinja2:

• Integers passed around using jinja variable references are converted to strings ansible/ansible#9362
• int is not a number ansible/ansible#15249

[clintonb]

I am going to use a different method of specifying the host and port to workaround this issue.

[clintonb]

@MichaelRoytman please take another look. I have confirmed this solves the connection issues for Docker Devstack. Indexing is still broken, but that's another issue altogether that will be fixed by https://github.com/edx/devstack/issues/40.

[jibsheet]

Fascinating - we have the s/'9200'/9200/ change in the secure repo as part of Michael's tests and I thought it was working (without switching to the full url style).

The elasticsearch init that we use in edx-search is almost too flexible about arguments...

[clintonb]

@MichaelRoytman please review again.

[jibsheet]

@clintonb So, Michael ran into "9200" vs 9200 earlier and we were fixing it by overriding with something like this

EDXAPP_SEARCH_HOST: 'search-host-name'
EDXAPP_SEARCH_PORT: 443
EDXAPP_SEARCH_USE_SSL: true

EDXAPP_ELASTIC_SEARCH_CONFIG:
  - host: "{{ EDXAPP_SEARCH_HOST }}"
    port: "{{ EDXAPP_SEARCH_PORT }}"
    use_ssl: "{{ EDXAPP_SEARCH_USE_SSL }}"

Which connects and works for us?

[MichaelRoytman]

@clintonb If removing the quote around the port and keeping the dictionary works (Kevin's comment above), what does converting the dict to a list do? Or what is the advantage?

[clintonb]

Do your changes work without SSL (e.g. port 9200)? Simply converting from "9200" to 9200 did not work for me.

The major change here is passing a complete URL instead of a dictionary. The library itself knows how to parse URLs: https://github.com/elastic/elasticsearch-py/blob/master/elasticsearch/client/__init__.py#L18.

[clintonb]

I'm pleased to announce I was wrong! Removing the quotes from the port number resolve the problem. Clearly I tested with the wrong image/laptop. Updating the branch now.