Http structured cloudevents (cloudevents#47)

* Moved fields out of base & structured support base._ce_required_fields and base._ce_optional_fields were moved into event classes v03 and v1. http_events.CloudEvent class now looks for fieldnames in either headers or data, and can automatically determine whether this is a binary or structured event. Signed-off-by: Curtis Mason <cumason@google.com> * testing structured Signed-off-by: Curtis Mason <cumason@google.com> * added tests for structured events Signed-off-by: Curtis Mason <cumason@google.com> * Added test valid structured cloudevents Signed-off-by: Curtis Mason <cumason@google.com> * Created default headers arg in CloudEvent Signed-off-by: Curtis Mason <cumason@google.com> * Added http_events.py sample code Signed-off-by: Curtis Mason <cumason@google.com> * removed ../python-event-requests Signed-off-by: Curtis Mason <cumason@google.com> * README.md nit Signed-off-by: Curtis Mason <cumason@google.com> * client.py nit Signed-off-by: Curtis Mason <cumason@google.com> * comment nits Signed-off-by: Curtis Mason <cumason@google.com> * created __getitem__ in CloudEvent Signed-off-by: Curtis Mason <cumason@google.com> * sample nits Signed-off-by: Curtis Mason <cumason@google.com> * fixed structured empty data issue Signed-off-by: Curtis Mason <cumason@google.com> * Added CloudEvent to README Signed-off-by: Curtis Mason <cumason@google.com> * added http_msg to CloudEvent Signed-off-by: Curtis Mason <cumason@google.com> * implemented ToRequest in CloudEvent Signed-off-by: Curtis Mason <cumason@google.com> * testing more specversions Signed-off-by: Curtis Mason <cumason@google.com> * Added sample code to README.md Signed-off-by: Curtis Mason <cumason@google.com> * modified sample code Signed-off-by: Curtis Mason <cumason@google.com> * added datavalidation to changelog Signed-off-by: Curtis Mason <cumason@google.com> * updated README Signed-off-by: Curtis Mason <cumason@google.com> * README adjustment Signed-off-by: Curtis Mason <cumason@google.com> * ruler 80 adjustment on http_events Signed-off-by: Curtis Mason <cumason@google.com> * style and renamed ToRequest to to_request Signed-off-by: Curtis Mason <cumason@google.com> * lint fix Signed-off-by: Curtis Mason <cumason@google.com> * fixed self.binary typo Signed-off-by: Curtis Mason <cumason@google.com> * CHANGELOG adjustment Signed-off-by: Curtis Mason <cumason@google.com> * rollback CHANGELOG Signed-off-by: Curtis Mason <cumason@google.com> * Added documentation to to_request Signed-off-by: Curtis Mason <cumason@google.com> * README.md adjustment Signed-off-by: Curtis Mason <cumason@google.com> * renamed event_handler to event_version Signed-off-by: Curtis Mason <cumason@google.com> * inlined field_name_modifier Signed-off-by: Curtis Mason <cumason@google.com> * renamed test body data Signed-off-by: Curtis Mason <cumason@google.com> * removed unnecessary headers from test Signed-off-by: Curtis Mason <cumason@google.com> * removed field_name_modifier and fixed e.g. in client.py Signed-off-by: Curtis Mason <cumason@google.com> * pylint fix Signed-off-by: Curtis Mason <cumason@google.com>
cumason123 · Jul 22, 2020 · efed0f7 · efed0f7
1 parent d73731e
commit efed0f7
Show file tree

Hide file tree

Showing 12 changed files with 464 additions and 143 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -65,4 +65,4 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 [#22]: https://github.com/cloudevents/sdk-python/pull/22
 [#23]: https://github.com/cloudevents/sdk-python/pull/23
 [#25]: https://github.com/cloudevents/sdk-python/pull/25
-[#27]: https://github.com/cloudevents/sdk-python/pull/27
+[#27]: https://github.com/cloudevents/sdk-python/pull/27
diff --git a/README.md b/README.md
@@ -14,6 +14,57 @@ This SDK current supports the following versions of CloudEvents:
 
 Package **cloudevents** provides primitives to work with CloudEvents specification: https://github.com/cloudevents/spec.
 
+Sending CloudEvents:
+
+### Binary HTTP CloudEvent
+
+```python
+from cloudevents.sdk.http_events import CloudEvent
+import requests
+
+
+# This data defines a binary cloudevent
+headers = {
+    "Content-Type": "application/json",
+    "ce-specversion": "1.0",
+    "ce-type": "README.sample.binary",
+    "ce-id": "binary-event",
+    "ce-time": "2018-10-23T12:28:22.4579346Z",
+    "ce-source": "README",
+}
+data = {"message": "Hello World!"}
+
+event = CloudEvent(data, headers=headers)
+headers, body = event.to_request()
+
+# POST
+requests.post("<some-url>", json=body, headers=headers)
+```
+
+### Structured HTTP CloudEvent
+
+```python
+from cloudevents.sdk.http_events import CloudEvent
+import requests
+
+
+# This data defines a structured cloudevent
+data = {
+    "specversion": "1.0",
+    "type": "README.sample.structured",
+    "id": "structured-event",
+    "source": "README",
+    "data": {"message": "Hello World!"}
+}
+event = CloudEvent(data)
+headers, body = event.to_request()
+
+# POST
+requests.post("<some-url>", json=body, headers=headers)
+```
+
+### Event base classes usage
+
 Parsing upstream structured Event from HTTP request:
 
 ```python
@@ -68,22 +119,6 @@ event = m.FromRequest(
 )
 ```
 
-Creating a minimal CloudEvent in version 0.1:
-
-```python
-from cloudevents.sdk.event import v1
-
-event = (
-    v1.Event()
-    .SetContentType("application/json")
-    .SetData('{"name":"john"}')
-    .SetEventID("my-id")
-    .SetSource("from-galaxy-far-far-away")
-    .SetEventTime("tomorrow")
-    .SetEventType("cloudevent.greet.you")
-)
-```
-
 Creating HTTP request from CloudEvent:
 
 ```python

diff --git a/cloudevents/sdk/event/base.py b/cloudevents/sdk/event/base.py
@@ -16,21 +16,6 @@
 import json
 import typing
 
-_ce_required_fields = {
-    'id',
-    'source',
-    'type',
-    'specversion'
-}
-
-
-_ce_optional_fields = {
-    'datacontenttype',
-    'schema',
-    'subject',
-    'time'
-}
-
 
 # TODO(slinkydeveloper) is this really needed?
 class EventGetterSetter(object):

diff --git a/cloudevents/sdk/event/v03.py b/cloudevents/sdk/event/v03.py
@@ -17,6 +17,21 @@
 
 
 class Event(base.BaseEvent):
+    _ce_required_fields = {
+        'id',
+        'source',
+        'type',
+        'specversion'
+    }
+
+    _ce_optional_fields = {
+        'datacontentencoding',
+        'datacontenttype',
+        'schemaurl',
+        'subject',
+        'time'
+    }
+
     def __init__(self):
         self.ce__specversion = opt.Option("specversion", "0.3", True)
         self.ce__id = opt.Option("id", None, True)

diff --git a/cloudevents/sdk/event/v1.py b/cloudevents/sdk/event/v1.py
@@ -17,6 +17,20 @@
 
 
 class Event(base.BaseEvent):
+    _ce_required_fields = {
+        'id',
+        'source',
+        'type',
+        'specversion'
+    }
+
+    _ce_optional_fields = {
+        'datacontenttype',
+        'dataschema',
+        'subject',
+        'time'
+    }
+
     def __init__(self):
         self.ce__specversion = opt.Option("specversion", "1.0", True)
         self.ce__id = opt.Option("id", None, True)

diff --git a/cloudevents/sdk/http_events.py b/cloudevents/sdk/http_events.py
@@ -13,12 +13,14 @@
 #    under the License.
 import copy
 
+import io
+
 import json
 import typing
 
+from cloudevents.sdk import converters
 from cloudevents.sdk import marshaller
 
-from cloudevents.sdk.event import base
 from cloudevents.sdk.event import v03, v1
 
 
@@ -30,12 +32,14 @@ class CloudEvent():
 
     def __init__(
             self,
-            headers: dict,
-            data: dict,
-            data_unmarshaller: typing.Callable = lambda x: x
+            data: typing.Union[dict, None],
+            headers: dict = {},
+            data_unmarshaller: typing.Callable = lambda x: x,
     ):
         """
         Event HTTP Constructor
+        :param data: a nullable dict to be stored inside Event.
+        :type data: dict or None
         :param headers: a dict with HTTP headers
             e.g. {
                 "content-type": "application/cloudevents+json",
@@ -45,65 +49,118 @@ def __init__(
                 "ce-specversion": "0.2"
             }
         :type headers: dict
-        :param data: a dict to be stored inside Event
-        :type data: dict
         :param binary: a bool indicating binary events
         :type binary: bool
         :param data_unmarshaller: callable function for reading/extracting data
         :type data_unmarshaller: typing.Callable
         """
+        self.required_attribute_values = {}
+        self.optional_attribute_values = {}
+        if data is None:
+            data = {}
+
         headers = {key.lower(): value for key, value in headers.items()}
         data = {key.lower(): value for key, value in data.items()}
-        event_version = CloudEvent.detect_event_version(headers, data)
-        if CloudEvent.is_binary_cloud_event(headers):
-
-            # Headers validation for binary events
-            for field in base._ce_required_fields:
-                ce_prefixed_field = f"ce-{field}"
-
-                # Verify field exists else throw TypeError
-                if ce_prefixed_field not in headers:
-                    raise TypeError(
-                        "parameter headers has no required attribute {0}"
-                        .format(
-                            ce_prefixed_field
-                        ))
-
-                if not isinstance(headers[ce_prefixed_field], str):
-                    raise TypeError(
-                        "in parameter headers attribute "
-                        "{0} expected type str but found type {1}".format(
-                            ce_prefixed_field, type(headers[ce_prefixed_field])
-                        ))
-
-            for field in base._ce_optional_fields:
-                ce_prefixed_field = f"ce-{field}"
-                if ce_prefixed_field in headers and not \
-                        isinstance(headers[ce_prefixed_field], str):
-                    raise TypeError(
-                        "in parameter headers attribute "
-                        "{0} expected type str but found type {1}".format(
-                            ce_prefixed_field, type(headers[ce_prefixed_field])
-                        ))
 
-        else:
-            # TODO: Support structured CloudEvents
-            raise NotImplementedError
+        # returns an event class depending on proper version
+        event_version = CloudEvent.detect_event_version(headers, data)
+        self.isbinary = CloudEvent.is_binary_cloud_event(
+            event_version,
+            headers
+        )
 
-        self.headers = copy.deepcopy(headers)
-        self.data = copy.deepcopy(data)
         self.marshall = marshaller.NewDefaultHTTPMarshaller()
         self.event_handler = event_version()
-        self.marshall.FromRequest(
+
+        self.__event = self.marshall.FromRequest(
             self.event_handler,
-            self.headers,
-            self.data,
+            headers,
+            io.BytesIO(json.dumps(data).encode()),
             data_unmarshaller
         )
 
+        # headers validation for binary events
+        for field in event_version._ce_required_fields:
+
+            # prefixes with ce- if this is a binary event
+            fieldname = f"ce-{field}" if self.isbinary else field
+
+            # fields_refs holds a reference to where fields should be
+            fields_refs = headers if self.isbinary else data
+
+            fields_refs_name = 'headers' if self.isbinary else 'data'
+
+            # verify field exists else throw TypeError
+            if fieldname not in fields_refs:
+                raise TypeError(
+                    f"parameter {fields_refs_name} has no required "
+                    f"attribute {fieldname}."
+                )
+
+            elif not isinstance(fields_refs[fieldname], str):
+                raise TypeError(
+                    f"in parameter {fields_refs_name}, {fieldname} "
+                    f"expected type str but found type "
+                    f"{type(fields_refs[fieldname])}."
+                )
+
+            else:
+                self.required_attribute_values[f"ce-{field}"] = \
+                    fields_refs[fieldname]
+
+        for field in event_version._ce_optional_fields:
+            fieldname = f"ce-{field}" if self.isbinary else field
+            if (fieldname in fields_refs) and not \
+                    isinstance(fields_refs[fieldname], str):
+                raise TypeError(
+                    f"in parameter {fields_refs_name}, {fieldname} "
+                    f"expected type str but found type "
+                    f"{type(fields_refs[fieldname])}."
+                )
+            else:
+                self.optional_attribute_values[f"ce-{field}"] = field
+
+        # structured data is inside json resp['data']
+        self.data = copy.deepcopy(data) if self.isbinary else \
+            copy.deepcopy(data.get('data', {}))
+
+        self.headers = {
+            **self.required_attribute_values,
+            **self.optional_attribute_values
+        }
+
+    def to_request(
+        self,
+        data_unmarshaller: typing.Callable = lambda x: json.loads(
+            x.read()
+            .decode('utf-8')
+        )
+    ) -> (dict, dict):
+        """
+        Returns a tuple of HTTP headers/body dicts representing this cloudevent
+
+        :param data_unmarshaller: callable function used to read the data io
+        object
+        :type data_unmarshaller: typing.Callable
+        :returns: (http_headers: dict, http_body: dict)
+        """
+        converter_type = converters.TypeBinary if self.isbinary else \
+            converters.TypeStructured
+
+        headers, data = self.marshall.ToRequest(
+            self.__event,
+            converter_type,
+            data_unmarshaller
+        )
+        data = data if self.isbinary else data_unmarshaller(data)['data']
+        return headers, data
+
+    def __getitem__(self, key):
+        return self.data if key == 'data' else self.headers[key]
+
     @staticmethod
-    def is_binary_cloud_event(headers):
-        for field in base._ce_required_fields:
+    def is_binary_cloud_event(event_version, headers):
+        for field in event_version._ce_required_fields:
             if f"ce-{field}" not in headers:
                 return False
         return True