async support

CONTRIBUTING.rst

@@ -25,22 +25,33 @@ The process for contributing to any of the Elasticsearch repositories is similar
    assure our users of the origin and continuing existence of the code. You only
    need to sign the CLA once.
 
-2. Run the test suite to ensure your changes do not break existing code:
+2. Many classes included in this library are offered in two versions, for
+   asynchronous and synchronous Python. When working with these classes, you only
+   need to make changes to the asynchronous code, located in the *_async*
+   subdirectory of the source tree. Once you've made your changes, run the
+   following command to automatically generate the corresponding synchronous code:
+
+    .. code:: bash
+
+        $ nox -rs format
+
+3. Run the test suite to ensure your changes do not break existing code:
 
     .. code:: bash
 
         $ nox -rs lint test
 
-3. Rebase your changes.
+4. Rebase your changes.
    Update your local repository with the most recent code from the main
    elasticsearch-dsl-py repository, and rebase your branch on top of the latest master
    branch. We prefer your changes to be squashed into a single commit.
 
-4. Submit a pull request. Push your local changes to your forked copy of the
+5. Submit a pull request. Push your local changes to your forked copy of the
    repository and submit a pull request. In the pull request, describe what your
    changes do and mention the number of the issue where discussion has taken
    place, eg “Closes #123″.  Please consider adding or modifying tests related to
-   your changes.
+   your changes. Include any generated files in the *_sync* subdirectory in your
+   pull request.
 
 Then sit back and wait. There will probably be discussion about the pull
 request and, if any changes are needed, we would love to work with you to get

docs/api.rst

@@ -11,39 +11,70 @@ Search
 ------
 
 .. autoclass:: Search
+   :inherited-members:
    :members:
 
 .. autoclass:: MultiSearch
+   :inherited-members:
+   :members:
+
+.. autoclass:: AsyncSearch
+   :inherited-members:
+   :members:
+
+.. autoclass:: AsyncMultiSearch
+   :inherited-members:
    :members:
 
 Document
 --------
 
 .. autoclass:: Document
+   :inherited-members:
+   :members:
+
+.. autoclass:: AsyncDocument
+   :inherited-members:
    :members:
 
 Index
 -----
 
 .. autoclass:: Index
+   :inherited-members:
+   :members:
+
+.. autoclass:: AsyncIndex
+   :inherited-members:
    :members:
 
 Faceted Search
 --------------
 
 .. autoclass:: FacetedSearch
+   :inherited-members:
+   :members:
+
+.. autoclass:: AsyncFacetedSearch
+   :inherited-members:
    :members:
 
 Update By Query 
 ----------------
+
 .. autoclass:: UpdateByQuery
-  :members:
+   :inherited-members:
+   :members:
+
+.. autoclass:: AsyncUpdateByQuery
+   :inherited-members:
+   :members:
 
 Mappings
 --------
 
-If you wish to create mappings manually you can use the ``Mapping`` class, for
-more advanced use cases, however, we recommend you use the :ref:`doc_type`
+If you wish to create mappings manually you can use the ``Mapping`` or ``AsyncMapping``
+classes, for more advanced use cases, however, we recommend you use the :ref:`doc_type`
 abstraction in combination with :ref:`index` (or :ref:`index-template`) to define
 index-level settings and properties. The mapping definition follows a similar
 pattern to the query dsl:

docs/asyncio.rst

@@ -0,0 +1,92 @@
+.. _asyncio:
+
+Using Asyncio with Elasticsearch DSL
+====================================
+
+The ``elasticsearch-dsl`` package supports async/await with `Asyncio <https://docs.python.org/3/library/asyncio.html>`__.
+To ensure that you have all the required dependencies, install the ``[async]`` extra:
+
+   .. code:: bash
+
+       $ python -m pip install elasticsearch-dsl[async]
+
+Connections
+-----------
+
+Use the ``async_connections`` module to manage your asynchronous connections.
+
+   .. code:: python
+
+       from elasticsearch_dsl import async_connections
+
+       async_connections.create_connection(hosts=['localhost'], timeout=20)
+
+All the options available in the ``connections`` module can be used with ``async_connections``.
+
+How to avoid 'Unclosed client session / connector' warnings on exit
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+These warnings come from the ``aiohttp`` package, which is used internally by the
+``AsyncElasticsearch`` client. They appear often when the application exits and
+are caused by HTTP connections that are open when they are garbage collected. To
+avoid these warnings, make sure that you close your connections.
+
+   .. code:: python
+
+       es = async_connections.get_connection()
+       await es.close()
+
+Search DSL
+----------
+
+Use the ``AsyncSearch`` class to perform asynchronous searches.
+
+   .. code:: python
+
+       from elasticsearch_dsl import AsyncSearch
+
+       s = AsyncSearch().query("match", title="python")
+       async for hit in s:
+           print(hit.title)
+
+Instead of using the ``AsyncSearch`` object as an asynchronous iterator, you can
+explicitly call the ``execute()`` method to get a ``Response`` object.
+
+    .. code:: python
+
+       s = AsyncSearch().query("match", title="python")
+       response = await s.execute()
+       for hit in response:
+           print(hit.title)
+
+An ``AsyncMultiSearch`` is available as well.
+
+   .. code:: python
+
+       from elasticsearch_dsl import AsyncMultiSearch
+
+       ms = AsyncMultiSearch(index='blogs')
+
+       ms = ms.add(AsyncSearch().filter('term', tags='python'))
+       ms = ms.add(AsyncSearch().filter('term', tags='elasticsearch'))
+
+       responses = await ms.execute()
+
+       for response in responses:
+           print("Results for query %r." % response.search.query)
+           for hit in response:
+               print(hit.title)
+
+Asynchronous Documents, Indexes, and more
+-----------------------------------------
+
+The ``Document``, ``Index``, ``IndexTemplate``, ``Mapping``, ``UpdateByQuery`` and
+``FacetedSearch`` classes all have asynchronous versions that use the same name
+with an ``Async`` prefix. These classes expose the same interfaces as the
+synchronous versions, but any methods that perform I/O are defined as coroutines.
+
+Auxiliary classes that do not perform I/O do not have asynchronous versions. The
+same classes can be used in synchronous and asynchronous applications.
+
+You can consult the :ref:`api` section for details about each specific
+method.

docs/index.rst

@@ -351,6 +351,7 @@ Contents
    persistence
    faceted_search
    update_by_query
+   asyncio
    api
    CONTRIBUTING
    Changelog

elasticsearch_dsl/__init__.py

@@ -18,14 +18,16 @@
 from . import connections
 from .aggs import A
 from .analysis import analyzer, char_filter, normalizer, token_filter, tokenizer
-from .document import Document, InnerDoc, MetaField
+from .document import AsyncDocument, Document
+from .document_base import InnerDoc, MetaField
 from .exceptions import (
     ElasticsearchDslException,
     IllegalOperation,
     UnknownDslObject,
     ValidationException,
 )
 from .faceted_search import (
+    AsyncFacetedSearch,
     DateHistogramFacet,
     Facet,
     FacetedResponse,
@@ -76,11 +78,11 @@
     construct_field,
 )
 from .function import SF
-from .index import Index, IndexTemplate
-from .mapping import Mapping
+from .index import AsyncIndex, AsyncIndexTemplate, Index, IndexTemplate
+from .mapping import AsyncMapping, Mapping
 from .query import Q
-from .search import MultiSearch, Search
-from .update_by_query import UpdateByQuery
+from .search import AsyncMultiSearch, AsyncSearch, MultiSearch, Search
+from .update_by_query import AsyncUpdateByQuery, UpdateByQuery
 from .utils import AttrDict, AttrList, DslBase
 from .wrappers import Range
 
@@ -89,6 +91,14 @@
 __versionstr__ = ".".join(map(str, VERSION))
 __all__ = [
     "A",
+    "AsyncDocument",
+    "AsyncFacetedSearch",
+    "AsyncIndex",
+    "AsyncIndexTemplate",
+    "AsyncMapping",
+    "AsyncMultiSearch",
+    "AsyncSearch",
+    "AsyncUpdateByQuery",
     "AttrDict",
     "AttrList",
     "Binary",

elasticsearch_dsl/_async/__init__.py

@@ -0,0 +1,16 @@
+#  Licensed to Elasticsearch B.V. under one or more contributor
+#  license agreements. See the NOTICE file distributed with
+#  this work for additional information regarding copyright
+#  ownership. Elasticsearch B.V. licenses this file to you under
+#  the Apache License, Version 2.0 (the "License"); you may
+#  not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#
+# 	http://www.apache.org/licenses/LICENSE-2.0
+#
+#  Unless required by applicable law or agreed to in writing,
+#  software distributed under the License is distributed on an
+#  "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+#  KIND, either express or implied.  See the License for the
+#  specific language governing permissions and limitations
+#  under the License.