gcoap: rebase networking on sock

[kb2ma]

gcoap has been based on the GNRC stack, but sock is the preferred, stack-agnostic API for applications. This PR updates gcoap to use it.

Highlights:

• Modified gcoap.c _event_loop() to create a remote-less sock. Listen on this sock indefinitely if not expecting a response. Otherwise, periodically poll for an IPC msg to timeout waiting for a response. I left a debug message in _listen() for now to make it easy to see when sock_udp_recv() is called.
• To send a message, use the new gcoap_req_send2(), with the sock remote endpoint parameter. Post a msg to the sock mbox to interrupt listening, to set the response timeout. gcoap_req_send() now is deprecated, and simply creates the sock endpoint from the provided address and port, and forwards to gcoap_req_send2().
• Moved gcoap.c to sys/net. Moved and renamed coap.h to net/application_layer/coap/gcoap.h. Kept net/gnrc/application_layer/coap/coap.h as a shim that references the new header file. I need to mark it as deprecated though.

Let me know if there is a better way to reach the goal. Otherwise, it needs a little polishing and documentation update. I also need to figure out what the 'g' in gcoap stands for now. :-)

Thanks to @miri64 for the reference in the code docs to her thesis and defense slides. They helped a lot to understand the networking stack rationale.

[miri64]

I probably can find this out reading the code, but can you also highlight how this approach now differs from using pure nanocoap+sock? AFAIK it also brings client and server functionalities with it, so what does this implementation now bring that the other solution does not.

[kb2ma]

Rebased and compressed commits. Updated the overview documentation to focus on gcoap features .

For reviewers, the core changes are in commit eb6b87e. You'll see the documentation updates in gcoap.h, and the implementation changes in gcoap.c.

Thanks for asking about gcoap vs. nanocoap. This is similar to the issue #6038 that @cgundogan raised about the number of CoAP examples. So, maybe we can reuse this response there. @kaspar030, please jump in if I have described nanocoap incorrectly. I'm just trying to provide a clear comparison. :-)

The description below is copied from the gcoap Status page. It's kind of long -- focus on the bullets if it's too much.

Features

In general, gcoap provides a higher-level interface to CoAP than nanocoap. The main differences are gcoap's single CoAP messaging hub approach, and also a consistent, well-documented API for client requests and server responses.

Single CoAP messaging hub:

1. For server use, allows an external module/application to register handlers for an array of incoming request paths.
2. For client use, supports waiting for a response to multiple outstanding requests.
3. Runs in a dedicated thread and supports non-blocking timeout handling for a request. A single instance handles both server and client use.
4. In effect, provides a single listening port, which supports RFC 6282 UDP port compression.

In other words, gcoap provides a single, reusable access point for CoAP messaging. On the server side, I was inspired by the Python microframeworks like Bottle or Flask. gcoap provides gcoap_register_listener() for an application to register resources. In contrast, as a server nanocoap appears to be designed for a server per application. All resource callbacks must be specified within a single const array in the nanocoap_server example in RIOT. On the client side, gcoap tracks a configurable number of open requests in an array. In contrast, nanocoap is focused on handling one request at a time in the nanocoap_get() function in nanocoap_sock.c.

gcoap's timeout capability for a client request provides for lightweight confirmable messaging without specifically supporting the CON message type yet. nanocoap already provides CON messaging.

Simple high-level API

1. A simple, no-payload request/response is a single function call.
2. A payload-based request/response includes an ...init(), write the payload, and ...finish().
3. Consistent API between request and response.
4. User is never expected to write a CoAP option, at the cost of an extra memmove().

The contrast between gcoap and nanocoap APIs is not that great. However, gcoap is well-documented in the RIOT Modules>Networking>CoAP source documentation (moved there in this PR). nanocoap recently added a coap_reply_simple(), which also writes the content format option.

[kaspar030]

please jump in if I have described nanocoap incorrectly. I'm just trying
to provide a clear comparison. :-)

You summed it up pretty well. ;)

I designed nanocoap to be pretty minimal. So it'll never have any
register methods like gcoap has.

My plans for the future of nanocoap are (so far):

• add blockwise transfers
• add a more generic client request function (allow POST/PUT/DELETE/...,
  implement non-piggybacked ACK/data, ...)
• implement OBSERVE for both client and server
• allow one handler entry to handle a whole subtree (e.g., /saul/*)
• allow more than one server context
• obviously, support DTLS

[kb2ma]

I refined the introductory documentation in gcoap.h, squashed, and rebased. I also have been testing the implementation, and it works as expected. I think it's ready to merge, so I removed the WIP in the title. Please remove that label, too.

[kb2ma]

@miri64, is there an issue blocking merge of this PR? FWIW, I created some functional tests with Pexpect to convince myself that gcoap works with ongoing client and server exchanges.

[miri64]

@miri64, is there an issue blocking merge of this PR?

At the moment just time to review it. Will try to find some.

[miri64]

Here and below: I don't understand your indentation pattern. But since we seem to have no convention about this, I'm fine in general

[kb2ma]

The main goal is readability. If the declaration extends beyond approx. 80 chars, I right-align the following parameters below the last one on the first line. I think this block-like look is easier to read than wrapping the following parameters starting under the first parameter on the first line.

[miri64]

Is this file still required? I would completely remove it.

[kb2ma]

No, coap.h is not required any longer -- it's only there for backwards compatibility. I didn't want to break code for existing users out of the box. I included a 'deprecated' comment above, but I'm happy to remove the file. I don't expect that there are hordes of gcoap users, but is there a policy on when to deprecate?

FWIW, for the same reason I also retained gcoap_req_send(), which uses an ipv6_addr_t and an int port for the destination. I created gcoap_req_send2(), which uses a sock_udp_ep_t. Let me know if I should make any changes there.

[kb2ma]

@miri64, I removed net/gnrc/coap.h in commit ef2d660. I'm glad you brought up this subject. You'll see in that commit that I also changed several #include's that referenced the old header file. I should have done that with the original commits.

I also would appreciate any insights on deprecation policy, as I mentioned above in my inline responses to your review. Maybe it would be a better question for the riot-devel list.

[miri64]

Still WIP? Otherwise, I'm fine with the code and want to test sometime this week (most likely at the interim H&A on Wednesday).

[kb2ma]

Glad to hear it! No longer WIP, as mentioned above. I'll keep in mind that you might be testing Wednesday so I can respond quickly if questions arise.

[PeterKietzmann]

@miri64 any news?

[miri64]

@miri64 any news?

Sorry, didn't think about testing this at H'n'A. Will do, but needs rebase ;-)

[miri64]

I was able to send GET requests and receive replies from another examples/gcoap application. Since this only changes how gcoap accesses GNRC this is sufficient enough for me

[kb2ma]

Excellent! In a few hours I will rebase and verify all is well. Unless I hear otherwise, I do not plan to squash the latest commit, which removes the obsolete coap.h include file. Leaving this commit will help make the change explicit to any current gcoap users.

[kb2ma]

Murdock and CI are happy now. Rebased, fixed and squashed a few issues:

• Fixed trailing underscores in header guard for gcoap.h (PR *: remove trailing underscores from header guards #6407). I became aware of this PR due to a conflict when rebasing. I fixed it because gcoap.h was created with this PR.
• Replaced use of SEC_IN_USEC with US_PER_SEC, per PR timex: unambiguous time conversion macros #6399.
• Fixed a minor variable scope issue flagged by cppcheck.

[miri64]

and go