diff --git a/articles/iot-hub/iot-hub-devguide-direct-methods.md b/articles/iot-hub/iot-hub-devguide-direct-methods.md index 3604f288c0297..44a48de67b693 100644 --- a/articles/iot-hub/iot-hub-devguide-direct-methods.md +++ b/articles/iot-hub/iot-hub-devguide-direct-methods.md @@ -28,7 +28,7 @@ Refer to [Cloud-to-device communication guidance](iot-hub-devguide-c2d-guidance. Direct methods are implemented on the device and may require zero or more inputs in the method payload to correctly instantiate. You invoke a direct method through a service-facing URI (`{iot hub}/twins/{device id}/methods/`). A device receives direct methods through a device-specific MQTT topic (`$iothub/methods/POST/{method name}/`) or through AMQP links (the `IoThub-methodname` and `IoThub-status` application properties). > [!NOTE] -> When you invoke a direct method on a device, property names and values can only contain US-ASCII printable alphanumeric, except any in the following set: ``{'$', '(', ')', '<', '>', '@', ',', ';', ':', '\', '"', '/', '[', ']', '?', '=', '{', '}', SP, HT}``. +> When you invoke a direct method on a device, property names and values can only contain US-ASCII printable alphanumeric, except any in the following set: ``{'$', '(', ')', '<', '>', '@', ',', ';', ':', '\', '"', '/', '[', ']', '?', '=', '{', '}', SP, HT}`` > Direct methods are synchronous and either succeed or fail after the timeout period (default: 30 seconds, settable up to 3600 seconds). Direct methods are useful in interactive scenarios where you want a device to act if and only if the device is online and receiving commands. For example, turning on a light from a phone. In these scenarios, you want to see an immediate success or failure so the cloud service can act on the result as soon as possible. The device may return some message body as a result of the method, but it isn't required for the method to do so. There is no guarantee on ordering or any concurrency semantics on method calls. diff --git a/articles/iot-hub/iot-hub-devguide-jobs.md b/articles/iot-hub/iot-hub-devguide-jobs.md index 3d085d2d88ce4..8e5ae4374bbca 100644 --- a/articles/iot-hub/iot-hub-devguide-jobs.md +++ b/articles/iot-hub/iot-hub-devguide-jobs.md @@ -12,7 +12,7 @@ ms.author: dobett # Schedule jobs on multiple devices -Azure IoT Hub enables a number of building blocks like [device twin properties and tags][lnk-twin-devguide] and [direct methods][lnk-dev-methods]. Typically, back-end apps enable device administrators and operators to update and interact with IoT devices in bulk and at a scheduled time. Jobs execute device twin updates and direct methods against a set of devices at a scheduled time. For example, an operator would use a back-end app that initiates and tracks a job to reboot a set of devices in building 43 and floor 3 at a time that would not be disruptive to the operations of the building. +Azure IoT Hub enables a number of building blocks like [device twin properties and tags](iot-hub-devguide-device-twins.md) and [direct methods](iot-hub-devguide-direct-methods.md). Typically, back-end apps enable device administrators and operators to update and interact with IoT devices in bulk and at a scheduled time. Jobs execute device twin updates and direct methods against a set of devices at a scheduled time. For example, an operator would use a back-end app that initiates and tracks a job to reboot a set of devices in building 43 and floor 3 at a time that would not be disruptive to the operations of the building. [!INCLUDE [iot-hub-basic](../../includes/iot-hub-basic-whole.md)] @@ -23,14 +23,18 @@ Consider using jobs when you need to schedule and track progress any of the foll * Invoke direct methods ## Job lifecycle -Jobs are initiated by the solution back end and maintained by IoT Hub. You can initiate a job through a service-facing URI (`{iot hub}/jobs/v2/{device id}/methods/?api-version=2016-11-14`) and query for progress on an executing job through a service-facing URI (`{iot hub}/jobs/v2/?api-version=2016-11-14`). To refresh the status of running jobs once a job is initiated, run a job query. + +Jobs are initiated by the solution back end and maintained by IoT Hub. You can initiate a job through a service-facing URI (`{iot hub}/jobs/v2/{device id}/methods/?api-version=2016-11-14`) and query for progress on an executing job through a service-facing URI (`{iot hub}/jobs/v2/?api-version=2016-11-14`). To refresh the status of running jobs once a job is initiated, run a job query. > [!NOTE] -> When you initiate a job, property names and values can only contain US-ASCII printable alphanumeric, except any in the following set: `$ ( ) < > @ , ; : \ " / [ ] ? = { } SP HT`. +> When you initiate a job, property names and values can only contain US-ASCII printable alphanumeric, except any in the following set: `$ ( ) < > @ , ; : \ " / [ ] ? = { } SP HT` +> ## Jobs to execute direct methods -The following snippet shows the HTTPS 1.1 request details for executing a [direct method][lnk-dev-methods] on a set of devices using a job: +The following snippet shows the HTTPS 1.1 request details for executing a [direct method](iot-hub-devguide-direct-methods.md) on a set of devices using a job: + + ``` PUT /jobs/v2/?api-version=2016-11-14 Authorization: @@ -50,6 +54,7 @@ The following snippet shows the HTTPS 1.1 request details for executing a [direc startTime: , // as an ISO-8601 date string maxExecutionTimeInSeconds: } + ``` The query condition can also be on a single device ID or on a list of device IDs as shown in the following examples: @@ -58,12 +63,15 @@ queryCondition = "deviceId = 'MyDevice1'" queryCondition = "deviceId IN ['MyDevice1','MyDevice2']" queryCondition = "deviceId IN ['MyDevice1'] ``` -[IoT Hub Query Language][lnk-query] covers IoT Hub query language in additional detail. +[IoT Hub Query Language](iot-hub-devguide-query-language.md) covers IoT Hub query language in additional detail. ## Jobs to update device twin properties + The following snippet shows the HTTPS 1.1 request details for updating device twin properties using a job: + ``` PUT /jobs/v2/?api-version=2016-11-14 + Authorization: Content-Type: application/json; charset=utf-8 Request-Id: @@ -77,22 +85,27 @@ The following snippet shows the HTTPS 1.1 request details for updating device tw startTime: , // as an ISO-8601 date string maxExecutionTimeInSeconds: // format TBD } + ``` ## Querying for progress on jobs + The following snippet shows the HTTPS 1.1 request details for querying for jobs: + ``` GET /jobs/v2/query?api-version=2016-11-14[&jobType=][&jobStatus=][&pageSize=][&continuationToken=] Authorization: Content-Type: application/json; charset=utf-8 Request-Id: User-Agent: / + ``` + +The continuationToken is provided from the response. -The continuationToken is provided from the response. - -You can query for the job execution status on each device using the [IoT Hub query language for device twins, jobs, and message routing][lnk-query]. +You can query for the job execution status on each device using the [IoT Hub query language for device twins, jobs, and message routing](iot-hub-devguide-query-language.md). ## Jobs Properties + The following list shows the properties and corresponding descriptions, which can be used when querying for jobs or job results. | Property | Description | @@ -119,28 +132,21 @@ The following list shows the properties and corresponding descriptions, which ca | | **deviceJobStatistics.pendingCount**: Number of devices that are pending to run the job. | ### Additional reference material + Other reference topics in the IoT Hub developer guide include: -* [IoT Hub endpoints][lnk-endpoints] describes the various endpoints that each IoT hub exposes for run-time and management operations. -* [Throttling and quotas][lnk-quotas] describes the quotas that apply to the IoT Hub service and the throttling behavior to expect when you use the service. -* [Azure IoT device and service SDKs][lnk-sdks] lists the various language SDKs you can use when you develop both device and service apps that interact with IoT Hub. -* [IoT Hub query language for device twins, jobs, and message routing][lnk-query] describes the IoT Hub query language. Use this query language to retrieve information from IoT Hub about your device twins and jobs. -* [IoT Hub MQTT support][lnk-devguide-mqtt] provides more information about IoT Hub support for the MQTT protocol. +* [IoT Hub endpoints](iot-hub-devguide-endpoints.md) describes the various endpoints that each IoT hub exposes for run-time and management operations. -## Next steps -To try out some of the concepts described in this article, see the following IoT Hub tutorial: +* [Throttling and quotas](iot-hub-devguide-quotas-throttling.md) describes the quotas that apply to the IoT Hub service and the throttling behavior to expect when you use the service. + +* [Azure IoT device and service SDKs](iot-hub-devguide-sdks.md) lists the various language SDKs you can use when you develop both device and service apps that interact with IoT Hub. + +* [IoT Hub query language for device twins, jobs, and message routing](iot-hub-devguide-query-language.md) describes the IoT Hub query language. Use this query language to retrieve information from IoT Hub about your device twins and jobs. -* [Schedule and broadcast jobs][lnk-jobs-tutorial] +* [IoT Hub MQTT support](iot-hub-mqtt-support.md) provides more information about IoT Hub support for the MQTT protocol. - +## Next steps + +To try out some of the concepts described in this article, see the following IoT Hub tutorial: -[lnk-endpoints]: iot-hub-devguide-endpoints.md -[lnk-quotas]: iot-hub-devguide-quotas-throttling.md -[lnk-sdks]: iot-hub-devguide-sdks.md -[lnk-query]: iot-hub-devguide-query-language.md -[lnk-devguide-mqtt]: iot-hub-mqtt-support.md -[lnk-jobs-tutorial]: iot-hub-node-node-schedule-jobs.md -[lnk-c2d-methods]: quickstart-control-device-node.md -[lnk-dev-methods]: iot-hub-devguide-direct-methods.md -[lnk-get-started-twin]: iot-hub-node-node-twin-getstarted.md -[lnk-twin-devguide]: iot-hub-devguide-device-twins.md +* [Schedule and broadcast jobs](iot-hub-node-node-schedule-jobs.md) \ No newline at end of file diff --git a/articles/iot-hub/iot-hub-devguide-messages-c2d.md b/articles/iot-hub/iot-hub-devguide-messages-c2d.md index 3f924e65dbfb6..3f0d5a9caaf24 100644 --- a/articles/iot-hub/iot-hub-devguide-messages-c2d.md +++ b/articles/iot-hub/iot-hub-devguide-messages-c2d.md @@ -12,7 +12,7 @@ ms.author: dobett # Send cloud-to-device messages from IoT Hub -To send one-way notifications to the device app from your solution back end, send cloud-to-devices messages from your IoT hub to your device. For a discussion of other cloud-to-devices options supported by IoT Hub, see [Cloud-to-device communications guidance][lnk-c2d-guidance]. +To send one-way notifications to the device app from your solution back end, send cloud-to-devices messages from your IoT hub to your device. For a discussion of other cloud-to-devices options supported by IoT Hub, see [Cloud-to-device communications guidance](iot-hub-devguide-c2d-guidance.md). [!INCLUDE [iot-hub-basic](../../includes/iot-hub-basic-whole.md)] @@ -28,24 +28,26 @@ To guarantee at-least-once message delivery, IoT Hub persists cloud-to-device me The following diagram shows the lifecycle state graph for a cloud-to-device message in IoT Hub. -![Cloud-to-device message lifecycle][img-lifecycle] +![Cloud-to-device message lifecycle](./media/iot-hub-devguide-messages-c2d/lifecycle.png) When the IoT Hub service sends a message to a device, the service sets the message state to **Enqueued**. When a device wants to *receive* a message, IoT Hub *locks* the message (by setting the state to **Invisible**), which allows other threads on the device to start receiving other messages. When a device thread completes the processing of a message, it notifies IoT Hub by *completing* the message. IoT Hub then sets the state to **Completed**. A device can also choose to: * *Reject* the message, which causes IoT Hub to set it to the **Dead lettered** state. Devices that connect over the MQTT protocol cannot reject cloud-to-device messages. + * *Abandon* the message, which causes IoT Hub to put the message back in the queue, with the state set to **Enqueued**. Devices that connect over the MQTT protocol cannot abandon cloud-to-device messages. A thread could fail to process a message without notifying IoT Hub. In this case, messages automatically transition from the **Invisible** state back to the **Enqueued** state after a *visibility (or lock) timeout*. The default value of this timeout is one minute. -The **max delivery count** property on IoT Hub determines the maximum number of times a message can transition between the **Enqueued** and **Invisible** states. After that number of transitions, IoT Hub sets the state of the message to **Dead lettered**. Similarly, IoT Hub sets the state of a message to **Dead lettered** after its expiration time (see [Time to live][lnk-ttl]). +The **max delivery count** property on IoT Hub determines the maximum number of times a message can transition between the **Enqueued** and **Invisible** states. After that number of transitions, IoT Hub sets the state of the message to **Dead lettered**. Similarly, IoT Hub sets the state of a message to **Dead lettered** after its expiration time (see [Time to live](#message-expiration-time-to-live)). -The [How to send cloud-to-device messages with IoT Hub][lnk-c2d-tutorial] shows you how to send cloud-to-device messages from the cloud and receive them on a device. +The [How to send cloud-to-device messages with IoT Hub](iot-hub-csharp-csharp-c2d.md) shows you how to send cloud-to-device messages from the cloud and receive them on a device. Typically, a device completes a cloud-to-device message when the loss of the message does not affect the application logic. For example, when the device has persisted the message content locally or has successfully executed an operation. The message could also carry transient information, whose loss would not impact the functionality of the application. Sometimes, for long-running tasks, you can: * Complete the cloud-to-device message after persisting the task description in local storage. + * Notify the solution back end with one or more device-to-cloud messages at various stages of progress of the task. ## Message expiration (time to live) @@ -55,7 +57,7 @@ Every cloud-to-device message has an expiration time. This time is set by one of * The **ExpiryTimeUtc** property in the service. * IoT Hub using the default *time to live* specified as an IoT Hub property. -See [Cloud-to-device configuration options][lnk-c2d-configuration]. +See [Cloud-to-device configuration options](#cloud-to-device-configuration-options). A common way to take advantage of message expiration and avoid sending messages to disconnected devices, is to set short time to live values. This approach achieves the same result as maintaining the device connection state, while being more efficient. When you request message acknowledgements, IoT Hub notifies you which devices are: @@ -74,7 +76,7 @@ When you send a cloud-to-device message, the service can request the delivery of If **Ack** is **full**, and you don't receive a feedback message, it means that the feedback message expired. The service can't know what happened to the original message. In practice, a service should ensure that it can process the feedback before it expires. The maximum expiry time is two days, which leaves time to get the service running again if a failure occurs. -As explained in [Endpoints][lnk-endpoints], IoT Hub delivers feedback through a service-facing endpoint (**/messages/servicebound/feedback**) as messages. The semantics for receiving feedback are the same as for cloud-to-device messages. Whenever possible, message feedback is batched in a single message, with the following format: +As explained in [Endpoints](iot-hub-devguide-endpoints.md), IoT Hub delivers feedback through a service-facing endpoint (**/messages/servicebound/feedback**) as messages. The semantics for receiving feedback are the same as for cloud-to-device messages. Whenever possible, message feedback is batched in a single message, with the following format: | Property | Description | | ------------ | ----------- | @@ -125,21 +127,10 @@ Each IoT hub exposes the following configuration options for cloud-to-device mes | feedback.ttlAsIso8601 | Retention for service-bound feedback messages. | ISO_8601 interval up to 2D (minimum 1 minute). Default: 1 hour. | | feedback.maxDeliveryCount |Maximum delivery count for feedback queue. | 1 to 100. Default: 100. | -For more information about how to set these configuration options, see [Create IoT hubs][lnk-portal]. +For more information about how to set these configuration options, see [Create IoT hubs](iot-hub-create-through-portal.md). ## Next steps -For information about the SDKs you can use to receive cloud-to-device messages, see [Azure IoT SDKs][lnk-sdks]. - -To try out receiving cloud-to-device messages, see the [Send cloud-to-device][lnk-c2d-tutorial] tutorial. - -[img-lifecycle]: ./media/iot-hub-devguide-messages-c2d/lifecycle.png +For information about the SDKs you can use to receive cloud-to-device messages, see [Azure IoT SDKs](iot-hub-devguide-sdks.md). -[lnk-portal]: iot-hub-create-through-portal.md -[lnk-c2d-guidance]: iot-hub-devguide-c2d-guidance.md -[lnk-endpoints]: iot-hub-devguide-endpoints.md -[lnk-sdks]: iot-hub-devguide-sdks.md -[lnk-ttl]: #message-expiration-time-to-live -[lnk-c2d-configuration]: #cloud-to-device-configuration-options -[lnk-lifecycle]: #message-lifecycle -[lnk-c2d-tutorial]: iot-hub-csharp-csharp-c2d.md +To try out receiving cloud-to-device messages, see the [Send cloud-to-device](iot-hub-csharp-csharp-c2d.md) tutorial. \ No newline at end of file diff --git a/articles/iot-hub/iot-hub-devguide-messages-construct.md b/articles/iot-hub/iot-hub-devguide-messages-construct.md index cd4059dd229fb..7b23c76a0a675 100644 --- a/articles/iot-hub/iot-hub-devguide-messages-construct.md +++ b/articles/iot-hub/iot-hub-devguide-messages-construct.md @@ -12,15 +12,18 @@ ms.author: asrastog # Create and read IoT Hub messages -To support seamless interoperability across protocols, IoT Hub defines a common message format for all device-facing protocols. This message format is used for both [device-to-cloud routing](https://docs.microsoft.com/azure/iot-hub/iot-hub-devguide-messages-d2c) and [cloud-to-device](https://docs.microsoft.com/azure/iot-hub/iot-hub-devguide-messages-c2d) messages. +To support seamless interoperability across protocols, IoT Hub defines a common message format for all device-facing protocols. This message format is used for both [device-to-cloud routing](iot-hub-devguide-messages-d2c.md) and [cloud-to-device](iot-hub-devguide-messages-c2d.md) messages. [!INCLUDE [iot-hub-basic](../../includes/iot-hub-basic-partial.md)] -IoT Hub implements device-to-cloud messaging using a streaming messaging pattern. IoT Hub's device-to-cloud messages are more like [Event Hubs](https://docs.microsoft.com/azure/event-hubs/) *events* than [Service Bus](https://docs.microsoft.com/azure/service-bus-messaging/) *messages* in that there is a high volume of events passing through the service that can be read by multiple readers. +IoT Hub implements device-to-cloud messaging using a streaming messaging pattern. IoT Hub's device-to-cloud messages are more like [Event Hubs](/azure/event-hubs/) *events* than [Service Bus](/azure/service-bus-messaging/) *messages* in that there is a high volume of events passing through the service that can be read by multiple readers. An IoT Hub message consists of: + * A predetermined set of *system properties* as listed below. + * A set of *application properties*. A dictionary of string properties that the application can define and access, without needing to deserialize the message body. IoT Hub never modifies these properties. + * An opaque binary body. Property names and values can only contain ASCII alphanumeric characters, plus ```{'!', '#', '$', '%, '&', "'", '*', '+', '-', '.', '^', '_', '`', '|', '~'}``` when you send device-to-cloud messages using the HTTPS protocol or send cloud-to-device messages. @@ -28,11 +31,14 @@ Property names and values can only contain ASCII alphanumeric characters, plus ` Device-to-cloud messaging with IoT Hub has the following characteristics: * Device-to-cloud messages are durable and retained in an IoT hub's default **messages/events** endpoint for up to seven days. + * Device-to-cloud messages can be at most 256 KB, and can be grouped in batches to optimize sends. Batches can be at most 256 KB. + * IoT Hub does not allow arbitrary partitioning. Device-to-cloud messages are partitioned based on their originating **deviceId**. -* As explained in the [Control access to IoT Hub](https://docs.microsoft.com/azure/iot-hub/iot-hub-devguide-security) section, IoT Hub enables per-device authentication and access control. -For more information about how to encode and decode messages sent using different protocols, see [Azure IoT SDKs](https://docs.microsoft.com/azure/iot-hub/iot-hub-devguide-sdks). +* As explained in [Control access to IoT Hub](iot-hub-devguide-security.md), IoT Hub enables per-device authentication and access control. + +For more information about how to encode and decode messages sent using different protocols, see [Azure IoT SDKs](iot-hub-devguide-sdks.md). The following table lists the set of system properties in IoT Hub messages. @@ -40,15 +46,16 @@ The following table lists the set of system properties in IoT Hub messages. | --- | --- | --- | | MessageId |A user-settable identifier for the message used for request-reply patterns. Format: A case-sensitive string (up to 128 characters long) of ASCII 7-bit alphanumeric characters + `{'-', ':',’.', '+', '%', '_', '#', '*', '?', '!', '(', ')', ',', '=', '@', ';', '$', '''}`. | Yes | | Sequence number |A number (unique per device-queue) assigned by IoT Hub to each cloud-to-device message. | No for C2D messages; yes otherwise. | -| To |A destination specified in [Cloud-to-Device][lnk-c2d] messages. | No for C2D messages; yes otherwise. | +| To |A destination specified in [Cloud-to-Device](iot-hub-devguide-c2d-guidance.md) messages. | No for C2D messages; yes otherwise. | | ExpiryTimeUtc |Date and time of message expiration. | Yes | -| EnqueuedTime |Date and time the [Cloud-to-Device][lnk-c2d] message was received by IoT Hub. | No for C2D messages; yes otherwise. | +| EnqueuedTime |Date and time the [Cloud-to-Device](iot-hub-devguide-c2d-guidance.md) message was received by IoT Hub. | No for C2D messages; yes otherwise. | | CorrelationId |A string property in a response message that typically contains the MessageId of the request, in request-reply patterns. | Yes | | UserId |An ID used to specify the origin of messages. When messages are generated by IoT Hub, it is set to `{iot hub name}`. | No | -| Ack |A feedback message generator. This property is used in cloud-to-device messages to request IoT Hub to generate feedback messages as a result of the consumption of the message by the device. Possible values: **none** (default): no feedback message is generated, **positive**: receive a feedback message if the message was completed, **negative**: receive a feedback message if the message expired (or maximum delivery count was reached) without being completed by the device, or **full**: both positive and negative. For more information, see [Message feedback][lnk-feedback]. | Yes | +| Ack |A feedback message generator. This property is used in cloud-to-device messages to request IoT Hub to generate feedback messages as a result of the consumption of the message by the device. Possible values: **none** (default): no feedback message is generated, **positive**: receive a feedback message if the message was completed, **negative**: receive a feedback message if the message expired (or maximum delivery count was reached) without being completed by the device, or **full**: both positive and negative. + | Yes | | ConnectionDeviceId |An ID set by IoT Hub on device-to-cloud messages. It contains the **deviceId** of the device that sent the message. | No for D2C messages; yes otherwise. | -| ConnectionDeviceGenerationId |An ID set by IoT Hub on device-to-cloud messages. It contains the **generationId** (as per [Device identity properties][lnk-device-properties]) of the device that sent the message. | No for D2C messages; yes otherwise. | -| ConnectionAuthMethod |An authentication method set by IoT Hub on device-to-cloud messages. This property contains information about the authentication method used to authenticate the device sending the message. For more information, see [Device to cloud anti-spoofing][lnk-antispoofing]. | No for D2C messages; yes otherwise. | +| ConnectionDeviceGenerationId |An ID set by IoT Hub on device-to-cloud messages. It contains the **generationId** (as per [Device identity properties](iot-hub-devguide-identity-registry.md#device-identity-properties)) of the device that sent the message. | No for D2C messages; yes otherwise. | +| ConnectionAuthMethod |An authentication method set by IoT Hub on device-to-cloud messages. This property contains information about the authentication method used to authenticate the device sending the message. | No for D2C messages; yes otherwise. | | CreationTimeUtc | Date and time the message was created on a device. A device must set this value explicitly. | Yes | ## Message size @@ -69,7 +76,7 @@ To avoid device spoofing in device-to-cloud messages, IoT Hub stamps all message * **ConnectionDeviceGenerationId** * **ConnectionAuthMethod** -The first two contain the **deviceId** and **generationId** of the originating device, as per [Device identity properties](https://docs.microsoft.com/azure/iot-hub/iot-hub-devguide-identity-registry#device-identity-properties). +The first two contain the **deviceId** and **generationId** of the originating device, as per [Device identity properties](iot-hub-devguide-identity-registry.md#device-identity-properties). The **ConnectionAuthMethod** property contains a JSON serialized object, with the following properties: @@ -83,5 +90,6 @@ The **ConnectionAuthMethod** property contains a JSON serialized object, with th ## Next steps -* For information about message size limits in IoT Hub, see [IoT Hub quotas and throttling](https://docs.microsoft.com/azure/iot-hub/iot-hub-devguide-quotas-throttling). -* To learn how to create and read IoT Hub messages in various programming languages, see the [Quickstarts](https://docs.microsoft.com/azure/iot-hub/quickstart-send-telemetry-node). \ No newline at end of file +* For information about message size limits in IoT Hub, see [IoT Hub quotas and throttling](iot-hub-devguide-quotas-throttling.md). + +* To learn how to create and read IoT Hub messages in various programming languages, see the [Quickstarts](quickstart-send-telemetry-node.md). \ No newline at end of file diff --git a/articles/iot-hub/iot-hub-devguide-messages-d2c.md b/articles/iot-hub/iot-hub-devguide-messages-d2c.md index fd3a5108342bf..063962ecaa114 100644 --- a/articles/iot-hub/iot-hub-devguide-messages-d2c.md +++ b/articles/iot-hub/iot-hub-devguide-messages-d2c.md @@ -1,6 +1,6 @@ --- title: Understand Azure IoT Hub message routing | Microsoft Docs -description: Developer guide - how to use message routing to send device-to-cloud messages. Includes information about sending both telemetry and non-telemtry data. +description: Developer guide - how to use message routing to send device-to-cloud messages. Includes information about sending both telemetry and non-telemetry data. author: ash2017 manager: briz ms.service: iot-hub @@ -18,23 +18,26 @@ Message routing enables you to send messages from your devices to cloud services * **Sending device telemetry messsages as well as events** namely, device lifecycle events, and device twin change events to the built-in-endpoint and custom endpoints. Learn about [routing endpoints](##routing-endpoints). -* **Filtering data before routing it to various endpoints** by applying rich queries. Message routing allows you to query on the message properties and message body as well as device twin tags and device twin properties. Learn more about using [queries in message routing](../iot-hub/iot-hub-devguide-routing-query-syntax.md). +* **Filtering data before routing it to various endpoints** by applying rich queries. Message routing allows you to query on the message properties and message body as well as device twin tags and device twin properties. Learn more about using [queries in message routing](iot-hub-devguide-routing-query-syntax.md). IoT Hub needs write access to these service endpoints for message routing to work. If you configure your endpoints through the Azure portal, the necessary permissions are added for you. Make sure you configure your services to support the expected throughput. When you first configure your IoT solution, you may need to monitor your additional endpoints and make any necessary adjustments for the actual load. -The IoT Hub defines a [common format](../iot-hub/iot-hub-devguide-messages-construct.md) for all device-to-cloud messaging for interoperatbility across protocols. If a message matches multiple routes that point to the same endpoint, IoT Hub delivers message to that endpoint only once. Therefore, you don't need to configure deduplication on your Service Bus queue or topic. In partitioned queues, partition affinity guarantees message ordering. Use this tutorial to learn how to [configure message routing] (https://docs.microsoft.com/azure/iot-hub/tutorial-routing). +The IoT Hub defines a [common format](iot-hub-devguide-messages-construct.md) for all device-to-cloud messaging for interoperatbility across protocols. If a message matches multiple routes that point to the same endpoint, IoT Hub delivers message to that endpoint only once. Therefore, you don't need to configure deduplication on your Service Bus queue or topic. In partitioned queues, partition affinity guarantees message ordering. Use this tutorial to learn how to [configure message routing] (tutorial-routing.md). ## Routing endpoints -An IoT hub has a default built-in-endpoint (**messages/events**) that is compatible with Event Hubs. You can create [custom endpoints](https://docs.microsoft.com/azure/iot-hub/iot-hub-devguide-endpoints#custom-endpoints) to route messages to by linking other services in your subscription to the IoT Hub. IoT Hub currently supports the following services as custom endpoints: +An IoT hub has a default built-in-endpoint (**messages/events**) that is compatible with Event Hubs. You can create [custom endpoints](iot-hub-devguide-endpoints.md#custom-endpoints) to route messages to by linking other services in your subscription to the IoT Hub. IoT Hub currently supports the following services as custom endpoints: ### Built-in endpoint -You can use standard [Event Hubs integration and SDKs](https://docs.microsoft.com/azure/iot-hub/iot-hub-devguide-messages-read-builtin) to receive device-to-cloud messages from the built-in endpoint (**messages/events**). Note that once a Route is created, data stops flowing to the built-in-endpoint, unless a Route is created to that endpoint. + +You can use standard [Event Hubs integration and SDKs](iot-hub-devguide-messages-read-builtin.md) to receive device-to-cloud messages from the built-in endpoint (**messages/events**). Note that once a Route is created, data stops flowing to the built-in-endpoint unless a Route is created to that endpoint. ### Azure Blob Storage + IoT Hub only supports writing data to Azure Blob Storage in the [Apache Avro](http://avro.apache.org/) format. IoT Hub batches messages and writes data to a blob whenever the batch reaches a certain size or a certain amount of time has elapsed. IoT Hub defaults to the following file naming convention: + ``` {iothub}/{partition}/{YYYY}/{MM}/{DD}/{HH}/{mm} ``` @@ -42,47 +45,63 @@ IoT Hub defaults to the following file naming convention: You may use any file naming convention, however you must use all listed tokens. IoT Hub will write to an empty blob if there is no data to write. ### Service Bus Queues and Service Bus Topics + Service Bus queues and topics used as IoT Hub endpoints must not have **Sessions** or **Duplicate Detection** enabled. If either of those options are enabled, the endpoint appears as **Unreachable** in the Azure portal. ### Event Hubs + Apart from the built-in-Event Hubs compatible endpoint, you can also route data to custom endpoints of type Event Hubs. When you use routing and custom endpoints, messages are only delivered to the built-in endpoint if they don't match any rules. To deliver messages to the built-in endpoint and custom endpoints, add a route that sends messages to the events endpoint. ## Reading data that has been routed -You can configure a route by following this [tutorial](https://docs.microsoft.com/azure/iot-hub/tutorial-routing). + +You can configure a route by following this [tutorial](tutorial-routing.md). Use the following tutorials to learn how to read message from an endpoint. -* Reading from [Built-in-endpoint](https://docs.microsoft.com/azure/iot-hub/quickstart-send-telemetry-node) -* Reading from [Blob storage](https://docs.microsoft.com/azure/storage/blobs/storage-blob-event-quickstart) -* Reading from [Event Hubs](https://docs.microsoft.com/azure/event-hubs/event-hubs-dotnet-standard-getstarted-send) -* Reading from [Service Bus Queues](https://docs.microsoft.com/azure/service-bus-messaging/service-bus-dotnet-get-started-with-queues) +* Reading from [Built-in-endpoint](quickstart-send-telemetry-node.md) + +* Reading from [Blob storage](../storage/blobs/storage-blob-event-quickstart.md) + +* Reading from [Event Hubs](../event-hubs/event-hubs-dotnet-standard-getstarted-send.md) + +* Reading from [Service Bus Queues](../service-bus-messaging/service-bus-dotnet-get-started-with-queues.md) + * Read from [Service Bus Topics](https://docs.microsoft.com/azure/service-bus-messaging/service-bus-dotnet-how-to-use-topics-subscriptions) ## Fallback route -The fallback route sends all the messages that don't satisfy query conditions on any of the existing routes to the built-in-Event Hubs (**messages/events**), that is compatible with [Event Hubs](https://docs.microsoft.com/azure/event-hubs/). If message routing is turned on, you can enable the fallback route capability. Note that once a route is created, data stops flowing to the built-in-endpoint, unless a route is created to that endpoint. If there are no routes to the built-in-endpoint and a fallback route is enabled, only messages that don't match any query conditions on routes will be sent to the built-in-endpoint. Also, if all existing routes are deleted, fallback route must be enabled to receive all data at the built-in-endpoint. -You can enable/disable the fallback route in the Azure Portal->Message Routing blade. You can also use Azure Resource Manager for [FallbackRouteProperties](https://docs.microsoft.com/rest/api/iothub/iothubresource/createorupdate#fallbackrouteproperties) to use a custom endpoint for fallback route. +The fallback route sends all the messages that don't satisfy query conditions on any of the existing routes to the built-in-Event Hubs (**messages/events**), that is compatible with [Event Hubs](/azure/event-hubs/). If message routing is turned on, you can enable the fallback route capability. Note that once a route is created, data stops flowing to the built-in-endpoint, unless a route is created to that endpoint. If there are no routes to the built-in-endpoint and a fallback route is enabled, only messages that don't match any query conditions on routes will be sent to the built-in-endpoint. Also, if all existing routes are deleted, fallback route must be enabled to receive all data at the built-in-endpoint. + +You can enable/disable the fallback route in the Azure Portal->Message Routing blade. You can also use Azure Resource Manager for [FallbackRouteProperties](/rest/api/iothub/iothubresource/createorupdate#fallbackrouteproperties) to use a custom endpoint for fallback route. ## Non-telemetry events + In addition to device telemetry, message routing also enables sending device twin change events and device lifecycle events. For example, if a route is created with data source set to **device twin change events**, IoT Hub sends messages to the endpoint that contain the change in the device twin. Similarly, if a route is created with data source set to **device lifecycle events**, IoT Hub will send a message indicating whether the device was deleted or created. + [IoT Hub also integrates wtih Azure Event Grid](iot-hub-event-grid.md) to publish device events to support real time integrations and automation of workflows based on these events. See key [differences between message routing and Event Grid](iot-hub-event-grid-routing-comparison.md) to learn which works best for your scenario. ## Testing routes -When you create a new route or edit an existing route, you should test the route query with a sample message. You can test individual routes or test all routes at once and no messages are routed to the endpoints during the test. Azure Portal, Azure Resource Manager, Azure PowerShell and Azure CLI can be used for testing. Outcomes help identify whether the sample message matched the query, message did not match the query or test couldn't run because the sample message or query syntax are incorrect. To learn more, see [Test Route](https://docs.microsoft.com/rest/api/iothub/iothubresource/testroute) and [Test all routes](https://docs.microsoft.com/rest/api/iothub/iothubresource/testallroutes). + +When you create a new route or edit an existing route, you should test the route query with a sample message. You can test individual routes or test all routes at once and no messages are routed to the endpoints during the test. Azure Portal, Azure Resource Manager, Azure PowerShell, and Azure CLI can be used for testing. Outcomes help identify whether the sample message matched the query, message did not match the query or test couldn't run because the sample message or query syntax are incorrect. To learn more, see [Test Route](/rest/api/iothub/iothubresource/testroute) and [Test all routes](/rest/api/iothub/iothubresource/testallroutes). ## Latency + When you route device-to-cloud telemetry messages using built-in endpoints, there is a slight increase in the end-to-end latency after the creation of the first route. In most cases, the average increase in latency is less than 500ms. You can monitor the latency using **Routing: message latency for messages/events** or **d2c.endpoints.latency.builtIn.events** IoT Hub metric. Creating or deleting any route after the first one does not impact the end-to-end latency. ## Monitoring and troubleshooting -IoT Hub provides several routing and endpoint related metrics to give you an overview of the health of your hub and messages sent. You can combine information from multiple metrics to identify root cause for issues. For example use metric **Routing: telemetry messages dropped** or **d2c.telemetry.egress.dropped** to identify the number of messages that were dropped when they didn't match queries on any of the routes and fallback route was disabled. [IoT Hub metrics](https://docs.microsoft.com/azure/iot-hub/iot-hub-metrics) lists all metrics that are enabled by default for your IoT Hub. -Using the **routes** diagnostic logs in Azure Monitor [diagnostic settings](https://docs.microsoft.com/azure/iot-hub/iot-hub-monitor-resource-health), you can tracks errors that occur during evaluation of a routing query and endpoint health as perceived by IoT Hub, for example when an endpoint is dead. These diagnostic logs can be sent to Log Analytics, Event Hubs, or Azure Storage for custom processing. +IoT Hub provides several routing and endpoint related metrics to give you an overview of the health of your hub and messages sent. You can combine information from multiple metrics to identify root cause for issues. For example use metric **Routing: telemetry messages dropped** or **d2c.telemetry.egress.dropped** to identify the number of messages that were dropped when they didn't match queries on any of the routes and fallback route was disabled. [IoT Hub metrics](iot-hub-metrics.md) lists all metrics that are enabled by default for your IoT Hub. + +Using the **routes** diagnostic logs in Azure Monitor [diagnostic settings](../iot-hub/iot-hub-monitor-resource-health.md), you can tracks errors that occur during evaluation of a routing query and endpoint health as perceived by IoT Hub, for example when an endpoint is dead. These diagnostic logs can be sent to Log Analytics, Event Hubs, or Azure Storage for custom processing. ## Next steps -* To learn how to create Message Routes, see the [Process IoT Hub device-to-cloud messages using routes](../iot-hub/tutorial-routing.md) tutorial. -* The [Quickstarts](https://docs.microsoft.com/azure/iot-hub/quickstart-send-telemetry-node) show you how to send device-to-cloud messages from simulated devices. -* For information about the SDKs you can use to send device-to-cloud messages, see [Azure IoT SDKs](../iot-hub/iot-hub-devguide-sdks.md). + +* To learn how to create Message Routes, see [Process IoT Hub device-to-cloud messages using routes](tutorial-routing.md). + +* [How to send device-to-cloud messages](quickstart-send-telemetry-node.md) + +* For information about the SDKs you can use to send device-to-cloud messages, see [Azure IoT SDKs](iot-hub-devguide-sdks.md). diff --git a/articles/iot-hub/iot-hub-devguide-messages-read-builtin.md b/articles/iot-hub/iot-hub-devguide-messages-read-builtin.md index fcf32e81fd9d5..016850f82d0f3 100644 --- a/articles/iot-hub/iot-hub-devguide-messages-read-builtin.md +++ b/articles/iot-hub/iot-hub-devguide-messages-read-builtin.md @@ -12,32 +12,35 @@ ms.author: dobett # Read device-to-cloud messages from the built-in endpoint -By default, messages are routed to the built-in service-facing endpoint (**messages/events**) that is compatible with [Event Hubs][lnk-event-hubs]. This endpoint is currently only exposed using the [AMQP][lnk-amqp] protocol on port 5671. An IoT hub exposes the following properties to enable you to control the built-in Event Hub-compatible messaging endpoint **messages/events**. +By default, messages are routed to the built-in service-facing endpoint (**messages/events**) that is compatible with [Event Hubs](http://azure.microsoft.com/documentation/services/event-hubs/ +). This endpoint is currently only exposed using the [AMQP](https://www.amqp.org/) protocol on port 5671. An IoT hub exposes the following properties to enable you to control the built-in Event Hub-compatible messaging endpoint **messages/events**. | Property | Description | | ------------------- | ----------- | -| **Partition count** | Set this property at creation to define the number of [partitions][lnk-event-hub-partitions] for device-to-cloud event ingestion. | +| **Partition count** | Set this property at creation to define the number of [partitions](../event-hubs/event-hubs-features.md#partitions) for device-to-cloud event ingestion. | | **Retention time** | This property specifies how long in days messages are retained by IoT Hub. The default is one day, but it can be increased to seven days. | IoT Hub also enables you to manage consumer groups on the built-in device-to-cloud receive endpoint. If you are using [message routing](iot-hub-devguide-messages-d2c.md) and the [fallback route](iot-hub-devguide-messages-d2c.md#fallback-route) is enabled, all messages that do not match a query on any route are written to the built-in endpoint. If you disable this fallback route, messages that do not match any query are dropped. -You can modify the retention time, either programmatically using the [IoT Hub resource provider REST APIs][lnk-resource-provider-apis], or with the [Azure portal][lnk-management-portal]. +You can modify the retention time, either programmatically using the [IoT Hub resource provider REST APIs](/rest/api/iothub/iothubresource), or with the [Azure portal](https://portal.azure.com). IoT Hub exposes the **messages/events** built-in endpoint for your back-end services to read the device-to-cloud messages received by your hub. This endpoint is Event Hub-compatible, which enables you to use any of the mechanisms the Event Hubs service supports for reading messages. ## Read from the built-in endpoint -When you use the [Azure Service Bus SDK for .NET][lnk-servicebus-sdk] or the [Event Hubs - Event Processor Host][lnk-eventprocessorhost], you can use any IoT Hub connection strings with the correct permissions. Then use **messages/events** as the Event Hub name. +When you use the [Azure Service Bus SDK for .NET](https://www.nuget.org/packages/WindowsAzure.ServiceBus) or the [Event Hubs - Event Processor Host](..//event-hubs/event-hubs-dotnet-standard-getstarted-receive-eph.md), you can use any IoT Hub connection strings with the correct permissions. Then use **messages/events** as the Event Hub name. When you use SDKs (or product integrations) that are unaware of IoT Hub, you must retrieve an Event Hub-compatible endpoint and Event Hub-compatible name: -1. Sign in to the [Azure portal][lnk-management-portal] and navigate to your IoT hub. -1. Click **Built-in endpoints**. -1. The **Events** section contains the following values: **Event Hub-compatible endpoint**, **Event Hub-compatible name**, **Partitions**, **Retention time**, and **Consumer groups**. +1. Sign in to the [Azure portal](https://portal.azure.com) and navigate to your IoT hub. - ![Device-to-cloud settings][img-eventhubcompatible] +2. Click **Built-in endpoints**. + +3. The **Events** section contains the following values: **Event Hub-compatible endpoint**, **Event Hub-compatible name**, **Partitions**, **Retention time**, and **Consumer groups**. + + ![Device-to-cloud settings](./media/iot-hub-devguide-messages-read-builtin/eventhubcompatible.png) The IoT Hub SDK requires the IoT Hub endpoint name, which is **messages/events** as shown under **Endpoints**. @@ -57,20 +60,10 @@ The SDKs and integrations that you can use with Event Hub-compatible endpoints t ## Next steps -* For more information about IoT Hub endpoints, see [IoT Hub endpoints][lnk-endpoints]. -* The [Quickstarts][lnk-get-started] show you how to send device-to-cloud messages from simulated devices and read the messages from the built-in endpoint. For more detail, see the [Process IoT Hub device-to-cloud messages using routes][lnk-d2c-tutorial] tutorial. -* If you want to route your device-to-cloud messages to custom endpoints, see [Use message routes and custom endpoints for device-to-cloud messages][lnk-custom]. - -[img-eventhubcompatible]: ./media/iot-hub-devguide-messages-read-builtin/eventhubcompatible.png - -[lnk-custom]: iot-hub-devguide-messages-read-custom.md -[lnk-get-started]: quickstart-send-telemetry-node.md -[lnk-endpoints]: iot-hub-devguide-endpoints.md -[lnk-resource-provider-apis]: https://docs.microsoft.com/rest/api/iothub/iothubresource -[lnk-event-hubs]: http://azure.microsoft.com/documentation/services/event-hubs/ -[lnk-management-portal]: https://portal.azure.com -[lnk-d2c-tutorial]: tutorial-routing.md -[lnk-event-hub-partitions]: ../event-hubs/event-hubs-features.md#partitions -[lnk-servicebus-sdk]: https://www.nuget.org/packages/WindowsAzure.ServiceBus -[lnk-eventprocessorhost]: https://docs.microsoft.com/azure/event-hubs/event-hubs-dotnet-standard-getstarted-receive-eph -[lnk-amqp]: https://www.amqp.org/ +* For more information about IoT Hub endpoints, see [IoT Hub endpoints](iot-hub-devguide-endpoints.md). + +* The [Quickstarts](quickstart-send-telemetry-node.md) show you how to send device-to-cloud messages from simulated devices and read the messages from the built-in endpoint. + +For more detail, see the [Process IoT Hub device-to-cloud messages using routes](tutorial-routing.md) tutorial. + +* If you want to route your device-to-cloud messages to custom endpoints, see [Use message routes and custom endpoints for device-to-cloud messages](iot-hub-devguide-messages-read-custom.md). \ No newline at end of file diff --git a/articles/iot-hub/media/iot-hub-devguide-messages-read-builtin/eventhubcompatible.png b/articles/iot-hub/media/iot-hub-devguide-messages-read-builtin/eventhubcompatible.png index f4b0e2ae3fae6..3f8430a908fdd 100644 Binary files a/articles/iot-hub/media/iot-hub-devguide-messages-read-builtin/eventhubcompatible.png and b/articles/iot-hub/media/iot-hub-devguide-messages-read-builtin/eventhubcompatible.png differ