Ros2 service crystal #71

ross-desmond · 2018-11-28T08:47:05Z

Added generate_permissions file based on node graph api. Enables security of services through sros policy yaml and definition file of how the policy yaml is constructed.

ros2_security_helper enables cmake macro to generate security files through colcon build for development and deployment. Note, this does not get built due to the structure of sros 2. OSRF please let me know where this macro should live to enable security by default when developing on ROS2
e.g:
find_package(ros2_security_helper REQUIRED)
ros2_secure_node(NODES minimal_publisher minimal_subscriber minimal_service)

Generate an sros2 yaml permissions file with the permissions of every visible node on the dds network. Add custom service security to sros2 Example: run the minimal_publisher_lambda node Execute: `ros2 security generate_permissions node_policies.yaml` It will create the following file in the current directory: ``` /minimal_publisher: services: /minimal_publisher/describe_parameters: allows: - request - reply . . . topics: /parameter_events: allows: - publish - subscribe /topic: allows: - publish ``` cr https://code.amazon.com/reviews/CR-3943967

Issue: services and actions are not considered in the policy yaml definition. Solution: Add ipc types (services and actions) Issue: access values are strings with p, s, or ps in order. This is not descriptive and difficult to scale should more permissions become necessary. Solution: Either change the parsing of the string or change the yaml to be more flexible and descriptive for users. The proposed changes include: * Access value is a list, not a string * Add ipc types such as actions and services * Access values are no longer shorthand p or s, but publish/subscribe Amend policy definition with verbose ROS ipc types

Add security macro for automagically generating public and private keys for authentication and encryption. custom macro use `ros2_secure_node(NODES node_name_1 node_name_2 ...)` cr https://code.amazon.com/reviews/CR-3517594

mjcarroll · 2018-11-28T18:56:26Z

@ross-desmond I think that we need to get the underlying pull requests in for the node graph API before we can considering getting this merged in to sros2. Did you make any progress on the other implementations outlined in this comment? ros2/rcl#333 (comment)

ross-desmond · 2018-11-28T19:14:47Z

They are on the way and will be available at the latest friday. Ill ensure all our pr's are at the top of master. Would it be best to ping out at that time?

…

On Wed, Nov 28, 2018, 10:56 AM Michael Carroll ***@***.*** wrote: @ross-desmond <https://github.com/ross-desmond> I think that we need to get the underlying pull requests in for the node graph API before we can considering getting this merged in to sros2. Did you make any progress on the other implementations outlined in this comment? ros2/rcl#333 (comment) <ros2/rcl#333 (comment)> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#71 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AqOeTBztDphCBiM1xsOjkcm270ciDdFoks5uztxcgaJpZM4Y3HnL> .

mjcarroll · 2018-11-28T19:15:19Z

Yes please! On Wed, Nov 28, 2018 at 1:14 PM Ross Desmond <notifications@github.com> wrote:

…

They are on the way and will be available at the latest friday. Ill ensure all our pr's are at the top of master. Would it be best to ping out at that time? On Wed, Nov 28, 2018, 10:56 AM Michael Carroll ***@***.*** wrote: > @ross-desmond <https://github.com/ross-desmond> I think that we need to > get the underlying pull requests in for the node graph API before we can > considering getting this merged in to sros2. Did you make any progress on > the other implementations outlined in this comment? ros2/rcl#333 (comment) > <ros2/rcl#333 (comment)> > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#71 (comment)>, or mute > the thread > < https://github.com/notifications/unsubscribe-auth/AqOeTBztDphCBiM1xsOjkcm270ciDdFoks5uztxcgaJpZM4Y3HnL > > . > — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#71 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AARElVP0C8w5iDnlm8MG6TeZYouA3zllks5uzuCogaJpZM4Y3HnL> .

mjcarroll · 2018-11-29T14:59:22Z

@ruffsl I'm just now getting up to speed on sros2 stuff after transitioning it from Mikael, would you mind putting a second set of eyes on this for me?

ross-desmond · 2018-12-05T00:17:25Z

General comment: In the future, is anyone opposed to using jinja for templates?

wjwwood · 2018-12-05T01:14:20Z

General comment: In the future, is anyone opposed to using jinja for templates?

As opposed to? Or for which task?

ross-desmond · 2018-12-05T01:27:36Z

General comment: In the future, is anyone opposed to using jinja for templates?

As opposed to? Or for which task?

sros2 has all the xml code in the api.py, it would be nice to have template files that python inserts the correct data to. This is just a though, it will not change this PR

clalancette · 2018-12-05T01:30:21Z

sros2 has all the xml code in the api.py, it would be nice to have template files that python inserts the correct data to. This is just a though, it will not change this PR

I won't go too far down this road here, but my concerns would be whether a) jinja is supported on macOS, Windows, and Linux, and b) adding in yet another templating system. We already have empy (which is non-specific to XML), and I find templating systems difficult to master. Adding more of them doesn't seem like a recipe for maintainable code down the road.

wjwwood · 2018-12-05T01:31:06Z

sros2 has all the xml code in the api.py, it would be nice to have template files that python inserts the correct data to. This is just a though, it will not change this PR

That would be fine with me. We do already have a dependency on EmPy which is a template engine, so using that would avoid a new dependency, but I don't feel strongly about that. +1 for moving the file text out of the python files (but later though).

ruffsl · 2018-12-05T09:13:40Z

I won't go too far down this road here, but my concerns would be whether a) jinja is supported on macOS, Windows, and Linux, and b) adding in yet another templating system. We already have empy (which is non-specific to XML), and I find templating systems difficult to master. Adding more of them doesn't seem like a recipe for maintainable code down the road.

+1 for not adding yet another PL specific templating library. I'd suggest we formalize the permission transform to be agnostic of any programming implementation, to be more formally typed, and transparent to promote standardization of how ROS2 permissions are generated for DDS, or any other transport implementation in the future for that matter.

I've created a branch that gives an example using program language agnostic transform for converting ROS2 permission profile markup into transport specific permission XML file. Users could then ecopertate the eXtensible Stylesheet into their own tools to consistently translate the permissions as a known finite transformation. I just spent an hour or so on it, but its relatively simple to support, and I can extend it into this PR if there is interest. See #72 for current diff.

ross-desmond · 2018-12-07T00:32:14Z

Is this planned for a merge at some point? Is there any help I can provide?

jacobperron

@ross-desmond Since this doesn't appear to break any existing API, it is okay for this to be merged and released post Crystal launch (which is why it hasn't received much attention yet). I've left a preliminary review, but since I'm not up-to-speed with sros2, it would be better to get more eyes on.

jacobperron · 2018-12-07T00:46:54Z

sros2/api/__init__.py

        # we have some policies to add !
        for topic_name, policy in topic_dict.items():
            tags = []
-            if policy['allow'] == 'ps':


Is this change necessary? I think it would be better not include it to help keep the PR succinct.

It would help keep the PR succinct, but then there would be different ways of describing the security for services(-request, -response) vs topics (ps). It makes the parsing and description of the elements more verbose and robust.

sros2/api/__init__.py

examples/sample_policy.yaml

examples/policy_definition.txt

ross-desmond · 2019-01-03T22:07:31Z

Thank you for the review @jacobperron.

Just a reminder the security_helpers folder in sros2 may want to be split out to another package as it does not build under the current sros2 package structure. If we believe this helper macro should stay in sros2 then I'll need to restructure to build sros2 python modules and the helper macro, thoughts?

jacobperron · 2019-01-08T19:02:31Z

Just a reminder the security_helpers folder in sros2 may want to be split out to another package as it does not build under the current sros2 package structure. If we believe this helper macro should stay in sros2 then I'll need to restructure to build sros2 python modules and the helper macro, thoughts?

IMO, the cmake package can remain in this repo (but remain a separate package) as it matches the repo description. However, I would consider remaining the package to something like sros2_cmake, since its main purpose is providing cmake macros for interacting with ROS 2 security.
I would restructure this repo so that the two packages are at top along with the general docs about sros2:

sros2
  |_ sros2 (python lib)
  |_ test
  |_ package.xml
  |_ setup.py
  |_ ... etc
sros2_cmake
  |_ cmake
    |_ ros2_create_keystore.cmake
    |_ ros2_secure_node.cmake
  |_ CMakeLists.txt
  |_ package.xml
  |_ ... etc
CONTRIBUTING.md
LICENSE
README.md
SROS2_Linux.md
SROS2_MacOS.md
SROS2_Windows.md

I'd like to get some other thoughts on this proposal.

If we go this approach, I think it would be better if the cmake package is added in a separate PR as it is distinct from the CLI changes and is a larger change.

examples/policy_definition.md

CHANGELOG.rst

ros2_security_helper/CMakeLists.txt

* Adds check to allow "/" or no "/" in permissions file * Renames policy definition file to markdown

ross-desmond · 2019-01-09T02:35:42Z

Fixing flake8 errors, incoming

mjcarroll · 2019-01-09T20:31:58Z

CI up to SROS2:

Linux
Linux-aarch64
macOS
Windows

ross-desmond · 2019-01-09T22:49:49Z

Just a reminder the security_helpers folder in sros2 may want to be split out to another package as it does not build under the current sros2 package structure. If we believe this helper macro should stay in sros2 then I'll need to restructure to build sros2 python modules and the helper macro, thoughts?

IMO, the cmake package can remain in this repo (but remain a separate package) as it matches the repo description. However, I would consider remaining the package to something like sros2_cmake, since its main purpose is providing cmake macros for interacting with ROS 2 security.
I would restructure this repo so that the two packages are at top along with the general docs about sros2:
sros2
  |_ sros2 (python lib)
  |_ test
  |_ package.xml
  |_ setup.py
  |_ ... etc
sros2_cmake
  |_ cmake
    |_ ros2_create_keystore.cmake
    |_ ros2_secure_node.cmake
  |_ CMakeLists.txt
  |_ package.xml
  |_ ... etc
CONTRIBUTING.md
LICENSE
README.md
SROS2_Linux.md
SROS2_MacOS.md
SROS2_Windows.md
I'd like to get some other thoughts on this proposal.

If we go this approach, I think it would be better if the cmake package is added in a separate PR as it is distinct from the CLI changes and is a larger change.

I'm in for this, I'll remove it from this package, @ruffsl how does that sound?

ruffsl · 2019-01-10T00:01:09Z

I'm in for this, I'll remove it from this package, @ruffsl how does that sound?

I'd be fine with that restructuring. I'd still like to recommend the use of stylesheets to compile the xml permissions, rather than a yaml implementation specific approach. I'd welcome feedback on #72 , and could rebase if the repo is restructured as above.

ross-desmond · 2019-01-10T17:50:37Z

I'm in for this, I'll remove it from this package, @ruffsl how does that sound?

I'd be fine with that restructuring. I'd still like to recommend the use of stylesheets to compile the xml permissions, rather than a yaml implementation specific approach. I'd welcome feedback on #72 , and could rebase if the repo is restructured as above.

Awesome, I'm happy to move to XML as well. In the meantime, using generate_permissions with create_permission will have the same result as with json and xml. Let's get the xml in for the next patch release?

jacobperron

Linux
Linux-aarch64
macOS
Windows

ross-desmond added 3 commits November 28, 2018 00:35

cmake security macro

597f1bd

Add security macro for automagically generating public and private keys for authentication and encryption. custom macro use `ros2_secure_node(NODES node_name_1 node_name_2 ...)` cr https://code.amazon.com/reviews/CR-3517594

tfoote added the in review Waiting for review (Kanban column) label Nov 28, 2018

jacobperron reviewed Dec 7, 2018

View reviewed changes

jacobperron reviewed Jan 8, 2019

View reviewed changes

examples/policy_definition.md Outdated Show resolved Hide resolved

CHANGELOG.rst Outdated Show resolved Hide resolved

ros2_security_helper/CMakeLists.txt Outdated Show resolved Hide resolved

clalancette mentioned this pull request Jan 9, 2019

ROS 2 Crystal Clemmys Patch Release 1 ros2/ros2#638

Closed

ross-desmond added 2 commits January 9, 2019 02:11

Addresses comments from pull-71

3c4f981

* Adds check to allow "/" or no "/" in permissions file * Renames policy definition file to markdown

Modifies policy definition tables

81189cf

ross-desmond force-pushed the ros2_service_crystal branch from 513f0b3 to 81189cf Compare January 9, 2019 02:19

Fixes flake8 and copyright errors

a3d9de2

Removes security helper for separate pull request

7a67e2f

jacobperron approved these changes Jan 10, 2019

View reviewed changes

ross-desmond mentioned this pull request Jan 11, 2019

SROS2 cmake restructure #74

Merged

jacobperron merged commit 9efd4e0 into ros2:master Jan 11, 2019

jacobperron removed the in review Waiting for review (Kanban column) label Jan 11, 2019

ross-desmond mentioned this pull request Jan 11, 2019

Sros2 cmake macros #75

Merged

mikaelarguedas mentioned this pull request Feb 22, 2019

[Issue 79] Fix permissions tutorial #81

Closed

mikaelarguedas mentioned this pull request Jun 13, 2019

Sros2 fixups ros2/ros2_documentation#281

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Ros2 service crystal #71

Ros2 service crystal #71

ross-desmond commented Nov 28, 2018

mjcarroll commented Nov 28, 2018

ross-desmond commented Nov 28, 2018 via email

mjcarroll commented Nov 28, 2018 via email

mjcarroll commented Nov 29, 2018

ross-desmond commented Dec 5, 2018 •

edited

Loading

wjwwood commented Dec 5, 2018

ross-desmond commented Dec 5, 2018

clalancette commented Dec 5, 2018

wjwwood commented Dec 5, 2018

ruffsl commented Dec 5, 2018

ross-desmond commented Dec 7, 2018

jacobperron left a comment

jacobperron Dec 7, 2018

ross-desmond Jan 3, 2019

ross-desmond commented Jan 3, 2019

jacobperron commented Jan 8, 2019

ross-desmond commented Jan 9, 2019

mjcarroll commented Jan 9, 2019

ross-desmond commented Jan 9, 2019

ruffsl commented Jan 10, 2019

ross-desmond commented Jan 10, 2019

jacobperron left a comment

Ros2 service crystal #71

Ros2 service crystal #71

Conversation

ross-desmond commented Nov 28, 2018

mjcarroll commented Nov 28, 2018

ross-desmond commented Nov 28, 2018 via email

mjcarroll commented Nov 28, 2018 via email

mjcarroll commented Nov 29, 2018

ross-desmond commented Dec 5, 2018 • edited Loading

wjwwood commented Dec 5, 2018

ross-desmond commented Dec 5, 2018

clalancette commented Dec 5, 2018

wjwwood commented Dec 5, 2018

ruffsl commented Dec 5, 2018

ross-desmond commented Dec 7, 2018

jacobperron left a comment

Choose a reason for hiding this comment

jacobperron Dec 7, 2018

Choose a reason for hiding this comment

ross-desmond Jan 3, 2019

Choose a reason for hiding this comment

ross-desmond commented Jan 3, 2019

jacobperron commented Jan 8, 2019

ross-desmond commented Jan 9, 2019

mjcarroll commented Jan 9, 2019

ross-desmond commented Jan 9, 2019

ruffsl commented Jan 10, 2019

ross-desmond commented Jan 10, 2019

jacobperron left a comment

Choose a reason for hiding this comment

ross-desmond commented Dec 5, 2018 •

edited

Loading