Implement __await__ on operation classes

[sjlongland]

So, been thinking about this for a long while now. We have the necessary bits for asynchronous operations, and yes, it's on my to-do list to write an async HTTP client implementation. Modern ipython (and probably Jupyter notebook too) supports the await keyword.

It'd be real nice to be able to run:

res = await client.his_read_frame(…etc…)

Turns out, this can be done, with a little work on our end:
https://docs.python.org/3.5/library/collections.abc.html#collections.abc.Awaitable

We'd need to implement (in Python 3.5+ only):

class HaystackOperation(object):
    if CAN_PYTHON_AWAIT:
        def __await__(self):
            future = Future()
            def _on_done(*args, **kwargs):
                try:
                    future.set_result(self.result)
                except Exception as ex:
                    future.set_exception(ex)
            self.done_sig.connect(_on_done)
            return future

This is untested of course, and we'd need to write some code to detect if we're on Python 3.5+ since this won't work on anything older. That has the potential though to make a much nicer experience for users, particularly interactive users.

[sjlongland]

Ohhh so much nicer!

In [3]: points = await session.find_entity('point and cmd and wshHubRef->wshEui64=="00124b0018e4237f"')
DEBUG:urllib3.connectionpool:Resetting dropped connection: ….on.widesky.cloud
DEBUG:urllib3.connectionpool:https://….on.widesky.cloud:443 "GET /widesky/api/read?filter=point+and+cmd+and+wshHubRef-%3EwshEui64%3D%3D%2200124b0018e4237f%22 HTTP/1.1" 200 None
DEBUG:pyhaystack.client.WideskyHaystackSession.find_entity:Received grid: <Grid>
        Version: 2.0
        Columns:
                id
                cmd
                cur
                dis
                equipRef
                fqname
                kind
                name
                point
                siteRef
                writable
                writeLevel
                writeStatus
                writeVal
                wsgRealTimeChangeOfState
                wsgRealTimeInterval
                wsgRealTimeOffset
                wshHubRef
                wshPoint
        Row    0:
        id=Ref('926f9ad8-0585-4cd0-b4ea-90232609c7e9', 'Control point', True)
        cmd=MARKER
        cur=MARKER
        dis='Control point'
        equipRef=Ref('99c689cc-66d4-4db6-b159-25ef0699578a', '00124b0018e4237f H-Bridge on port 1', True)
        fqname='ws.hub237fhb1.ctl'
        kind='Bool'
        name='ctl'
        point=MARKER
        siteRef=Ref('70624619-de61-4f6c-aaf7-014443123f39', 'WideSky Hub "Test Site"', True)
        writable=MARKER
        writeLevel=17.0
        writeStatus='ok'
        writeVal=True
        wsgRealTimeChangeOfState=60.0
        wsgRealTimeInterval=60.0
        wsgRealTimeOffset=0.0
        wshHubRef=Ref('ba0a5a46-cef9-4d4f-841e-0fcb99e192cd', 'Hub 00124b0018e4237f (@The Gap)', True)
        wshPoint=16.0
        Row    1:
        id=Ref('4b93cbba-db3d-418c-a229-322016ef30ef', 'Control point', True)
        cmd=MARKER
        cur=MARKER
        dis='Control point'
        equipRef=Ref('61b67447-cff3-421b-8e87-7952918bf996', '00124b0018e4237f H-Bridge on port 2', True)
        fqname='ws.hub237fhb2.ctl'
        kind='Bool'
        name='ctl'
        point=MARKER
        siteRef=Ref('70624619-de61-4f6c-aaf7-014443123f39', 'WideSky Hub "Test Site"', True)
        writable=MARKER
        writeLevel=17.0
        writeStatus='ok'
        writeVal=True
        wsgRealTimeChangeOfState=60.0
        wsgRealTimeInterval=60.0
        wsgRealTimeOffset=0.0
        wshHubRef=Ref('ba0a5a46-cef9-4d4f-841e-0fcb99e192cd', 'Hub 00124b0018e4237f (@The Gap)', True)
        wshPoint=14.0
        Row    2:
        id=Ref('d3fd360a-81e0-4235-a2f2-dcb423fbebb5', 'Control point', True)
        cmd=MARKER
        cur=MARKER
        dis='Control point'
        equipRef=Ref('7f4e8800-8294-4e05-93e5-4b3646ad378f', '00124b0018e4237f Relay on port 1', True)
        fqname='ws.hub237frl1.ctl'
        kind='Bool'
        name='ctl'
        point=MARKER
        siteRef=Ref('70624619-de61-4f6c-aaf7-014443123f39', 'WideSky Hub "Test Site"', True)
        writable=MARKER
        writeLevel=17.0
        writeStatus='ok'
        writeVal=True
        wsgRealTimeChangeOfState=60.0
        wsgRealTimeInterval=60.0
        wsgRealTimeOffset=0.0
        wshHubRef=Ref('ba0a5a46-cef9-4d4f-841e-0fcb99e192cd', 'Hub 00124b0018e4237f (@The Gap)', True)
        wshPoint=17.0
        Row    3:
        id=Ref('6b05ac19-f649-41e5-8607-73f7140da841', 'Control point', True)
        cmd=MARKER
        cur=MARKER
        dis='Control point'
        equipRef=Ref('f82990c2-f3a3-41be-aef3-b1fc6f834e89', '00124b0018e4237f Relay on port 2', True)
        fqname='ws.hub237frl2.ctl'
        kind='Bool'
        name='ctl'
        point=MARKER
        siteRef=Ref('70624619-de61-4f6c-aaf7-014443123f39', 'WideSky Hub "Test Site"', True)
        writable=MARKER
        writeLevel=17.0
        writeStatus='ok'
        writeVal=True
        wsgRealTimeChangeOfState=60.0
        wsgRealTimeInterval=60.0
        wsgRealTimeOffset=0.0
        wshHubRef=Ref('ba0a5a46-cef9-4d4f-841e-0fcb99e192cd', 'Hub 00124b0018e4237f (@The Gap)', True)
        wshPoint=15.0
</Grid>

Now, an asynchronous HTTP client and we're all set.

[ChristianTremblay]

Pushing me down the canyon of async/await... :-)

[sjlongland]

On 25/11/20 2:48 am, Christian Tremblay wrote: Pushing me down the canyon of async/await... :-)

Well… it's got to be a more natural solution than:

op = doSomething() op.wait() res = op.result

Next step then would be to implement an asynchronous HTTP client. I'd be interested to see if `await` works in newer versions of Jupyter Notebook. Support is in `ipython` v6 and later. There's also support for Tornado, so Python 2.7 users who want async action can `yield op.future` to do the same thing with Tornado v4.

[ChristianTremblay]

Joking. It's awesome. Really.

But I haven't yet be able to wrap my head around async... I'll need some help ;-)

[ChristianTremblay]

I'll keep this open and tag myself as documentation would probably be a good way for me to learn.... If I can find some time.