Skip to content

Commit

Permalink
add tutorial for using security enclaves with ConfigMap. (#43)
Browse files Browse the repository at this point in the history
Signed-off-by: Tomoya.Fujita <tomoya.fujita825@gmail.com>
  • Loading branch information
fujitatomoya authored Apr 26, 2024
1 parent 464a851 commit 7a04ce1
Show file tree
Hide file tree
Showing 7 changed files with 333 additions and 0 deletions.
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,7 @@ This environment is very much useful to try or test your container images or ser
- [ROS 2 GUI Display Access](./docs/ROS2_Deployment_Intermediate.md#ros-2-gui-display-access)
- [ROS 2 Zero Copy Data Sharing](./docs/ROS2_Deployment_Intermediate.md#ros-2-zero-copy-data-sharing)
- [ROS 2 Fast-DDS Discovery Server](./docs/ROS2_Deployment_Intermediate.md#ros-2-fast-dds-discovery-server)
- [ROS 2 Security Enclaves with ConfigMap](./docs/ROS2_Deployment_Intermediate.md#ros-2-security-enclaves-with-configmap)
- [Robotics System General](./docs/Robotics_System_General.md)
- [Reserve Resources for Host System](./docs/Robotics_System_General.md#reserve-resources-for-host-system)
- [Assign Resources to Pods/Containers](./docs/Robotics_System_General.md#assign-resources-to-podscontainers)
Expand Down
2 changes: 2 additions & 0 deletions docker/Dockerfile.rolling
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ RUN apt-get update \
net-tools \
iputils-ping \
dnsutils \
# ROS 2 Security
libssl-dev \
# install full packages including GUI
ros-${ROS_DISTRO}-desktop \
# install cyclonedds
Expand Down
195 changes: 195 additions & 0 deletions docs/ROS2_Deployment_Intermediate.md
Original file line number Diff line number Diff line change
Expand Up @@ -171,3 +171,198 @@ deployment.apps/ros2-listener-1 created
deployment.apps/ros2-listener-2 created
deployment.apps/ros2-listener-3 created
```

### ROS 2 Security Enclaves with ConfigMap

[Secured ROS 2 / SROS2](https://github.com/ros2/sros2) provides the tools and instructions to use ROS2 on top of DDS-Security.
Before going through this tutorial, it is highly recommended to read the [Introducing ROS 2 Security](https://docs.ros.org/en/rolling/Tutorials/Advanced/Security/Introducing-ros2-security.html) to understand how to enable security enclaves for ROS 2 application.

That is good to have authentication and access permission to establish the secured connection using ROS 2.
On the other hand this generate the another problem, that is how to manage the security enclaves to bind the distributed nodes in the ROS 2 system.

It is likely that ROS 2 system scales up to hundreds nodes, that are connected via network interfaces and distributed system.
In this case, we need to be able to distribute the appropriate security enclaves to the physical systems where the ROS 2 applications are running.
This will be operating cost for managing/updating security data, secured storage support, and we have to make sure the system where nodes are starting up has the security enclaves, otherwise nodes cannot start up.

This burden can be solved in a second with Kubernetes `Configuration Map`.
This tutorial tells how to convert the security enclaves into `Configuration Map`, and bind the appropriate security enclaves to the ROS 2 application when it starts.
This does not matter where the security enclaves are, nor where ROS 2 application starts up.
Kubernetes takes care of deployment and configuration map, to start the ROS 2 application with specific configuration map dynamically.

![ROS 2 Security with Configuration Map](./../images/ros2_security_enclaves.png)

**see deployment description [ROS 2 Secured App Deployment](./../yaml/ros2-fastdds-discovery-server.yaml) and [ROS 2 Security Configuration Map](./../yaml/ros2-fastdds-discovery-server-apps.yaml)**

- Generate Security Enclaves

Usually security enclaves are generated by security administrator based on the security requirement for each ROS 2 application node. How to configure authentication and access permission is dependent on the requirement, so here it expects that talker and listener security enclaves are generated along with [Setting up security](https://docs.ros.org/en/rolling/Tutorials/Advanced/Security/Introducing-ros2-security.html) by user already.

The following security enclaves are generated.

```bash
./demo_keystore/enclaves/talker_listener/talker
./demo_keystore/enclaves/talker_listener/talker/identity_ca.cert.pem
./demo_keystore/enclaves/talker_listener/talker/cert.pem
./demo_keystore/enclaves/talker_listener/talker/permissions.xml
./demo_keystore/enclaves/talker_listener/talker/governance.p7s
./demo_keystore/enclaves/talker_listener/talker/key.pem
./demo_keystore/enclaves/talker_listener/talker/permissions.p7s
./demo_keystore/enclaves/talker_listener/talker/permissions_ca.cert.pem
./demo_keystore/enclaves/talker_listener/listener
./demo_keystore/enclaves/talker_listener/listener/identity_ca.cert.pem
./demo_keystore/enclaves/talker_listener/listener/cert.pem
./demo_keystore/enclaves/talker_listener/listener/permissions.xml
./demo_keystore/enclaves/talker_listener/listener/governance.p7s
./demo_keystore/enclaves/talker_listener/listener/key.pem
./demo_keystore/enclaves/talker_listener/listener/permissions.p7s
./demo_keystore/enclaves/talker_listener/listener/permissions_ca.cert.pem
```

- Copy security enclaves

A few security files are symbolic links, those links cannot be loaded into ConfigMap data, so copy all files at once.

```bash
### Copy talker enclave files
mkdir talker_enclaves
cp -L demo_keystore/enclaves/talker_listener/talker/* talker_enclaves/

### Copy listener enclave files
mkdir listener_enclaves
cp -L demo_keystore/enclaves/talker_listener/listener/* listener_enclaves/
```

- Create ConfigMap from Security Enclaves

Let's create ConfigMap from actual files without symbolic links from the temporary directory as following.

```bash
### Create talker enclaves
kubectl create configmap talker-enclaves --from-file=./talker_enclaves
configmap/talker-enclaves created
### Create linstener enclaves
kubectl create configmap listener-enclaves --from-file=./listener_enclaves
configmap/listener-enclaves created
```

We can describe the ConfigMap to see what is inside of it.
As we can see below, ConfigMap is generated as `Data` contents from each files.
Now we have security enclaves ConfigMap data is ready to bind to the application container.

```bash
kubectl describe configmap talker-enclaves
Name: talker-enclaves
Namespace: default
Labels: <none>
Annotations: <none>

Data
====
cert.pem:
----
-----BEGIN CERTIFICATE-----
MIIBQTCB6aADAgECAhRsN6ffbQQ6/GhPaTXm3xZbWYgK9TAKBggqhkjOPQQDAjAS
MRAwDgYDVQQDDAdzcm9zMkNBMB4XDTI0MDQxNjA0NTMyMloXDTM0MDQxNTA0NTMy
MlowIjEgMB4GA1UEAwwXL3RhbGtlcl9saXN0ZW5lci90YWxrZXIwWTATBgcqhkjO
PQIBBggqhkjOPQMBBwNCAATLKY5Ej2RyAeGDUZkl7Hu58/Me+N/emHkkdrh8M0jY
cFMwaDG0qz8DiVeqaBi1dqsVeTXfZSkzsy7omv4Ypb1Low0wCzAJBgNVHRMEAjAA
MAoGCCqGSM49BAMCA0cAMEQCIGoSaKVtEQXpSFCNjm2PAwB1/rYj+SHgR5O37APc
1h6+AiAA437cnnom8Up/DhlJjZmXfxevtFwjfy2WLk+j8pbFhw==
-----END CERTIFICATE-----

key.pem:
----
-----BEGIN PRIVATE KEY-----
MIGHAgEAMBMGByqGSM49AgEGCCqGSM49AwEHBG0wawIBAQQgEHHPJ9VvW6dq38AF
npoHYa8d0L/s6y3DNofanILskuahRANCAATLKY5Ej2RyAeGDUZkl7Hu58/Me+N/e
mHkkdrh8M0jYcFMwaDG0qz8DiVeqaBi1dqsVeTXfZSkzsy7omv4Ypb1L
-----END PRIVATE KEY-----

permissions.p7s:
----
MIME-Version: 1.0
Content-Type: multipart/signed; protocol="application/x-pkcs7-signature"; micalg="sha-256"; boundary="----0A5A21336BCFA2412598220B9633A26B"

This is an S/MIME signed message
...<snip>
```

- ROS 2 Security Environmental Variables

To use ROS 2 security enclaves, the following environmental variables must be set properly.
You can see more details for [Configure Environment Variable](https://docs.ros.org/en/rolling/Tutorials/Advanced/Security/Introducing-ros2-security.html#configure-environment-variables).

```bash
ROS_SECURITY_KEYSTORE=/etc/demo_keystore
ROS_SECURITY_ENABLE=true
ROS_SECURITY_STRATEGY=Enforce
```

Since this is not specific security enclaves data, we can also use another ConfigMap to bind these environmental variable to the application container at startup.
Let`s use [ROS 2 Security Environment Configuration](./../yaml/ros2-security-config.yaml) to create ConfigMap in the cluster.

```bash
### Create Security Environment ConfigMap
kubectl apply -f ./yaml/ros2-security-config.yaml
configmap/security-env-config created

### Check
kubectl describe configmap security-env-config
Name: security-env-config
Namespace: default
Labels: <none>
Annotations: <none>

Data
====
ROS_SECURITY_KEYSTORE:
----
/etc/demo_keystore
ROS_SECURITY_STRATEGY:
----
Enforce
ROS_SECURITY_ENABLE:
----
true

BinaryData
====

Events: <none>
```

- Deploy ROS 2 application with security enclaves and environmental variables

Then all we have to is to start the ROS 2 application with ConfigMap created above to start the secured ROS 2 communication.
**see deployment description [ROS 2 Secured Sample](./../yaml/ros2-security-sample.yaml) for more details.**

```bash
kubectl apply -f yaml/ros2-security-sample.yaml
deployment.apps/ros2-secured-talker created
deployment.apps/ros2-secured-listener created
```

you can see the log from application pod, that tells that security enclaves are enabled.

```bash
kubectl get pods
NAME READY STATUS RESTARTS AGE
ros2-secured-listener-78784df45f-jpdvt 1/1 Running 0 85s
ros2-secured-talker-6b694cbf6f-5p8t2 1/1 Running 0 85s

kubectl logs ros2-secured-listener-78784df45f-jpdvt
[INFO] [1714115341.611394483] [rcl]: Found security directory: /etc/demo_keystore/enclaves/talker_listener/listener
[INFO] [1714115342.633319407] [listener]: I heard: [Hello World: 1]
...<snip>
```

Besides that, we can jump in the container and use `ros2cli` to see the ROS 2 system.
Since `ros2cli` is not bound to security enclaves, it cannot discover any nodes or topics as following.
This also indicates that security authentication and access permission works just as expected.

```bash
root@ros2-secured-listener-78784df45f-jpdvt:/# ros2 node list
root@ros2-secured-listener-78784df45f-jpdvt:/# ros2 topic list
/parameter_events
/rosout
```
Binary file added images/ros2_security_enclaves.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
37 changes: 37 additions & 0 deletions scripts/replace_simlink.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
#!/bin/bash

### enable debug
#set -exvfC
set -e

# Directory to process
DIR="$1"

# Ensure the directory is provided as an argument
if [ -z "$DIR" ]; then
echo "Usage: $0 <directory>"
exit 1
fi

# Check if the provided directory exists
if [ ! -d "$DIR" ]; then
echo "Error: Directory '$DIR' does not exist."
exit 1
fi

# Find all symbolic links in the directory (not recursively)
find "$DIR" -maxdepth 1 -type l | while read -r symlink; do
# Get the actual file to which the symbolic link points
target_file=$(readlink -f "$symlink")

# If the target file doesn't exist, warn and skip
if [ ! -e "$target_file" ]; then
echo "Warning: Symbolic link '$symlink' points to a non-existent file."
continue
fi

# Copy the content of the original file to the location of the symbolic link
cp "$target_file" "$symlink" && \
echo "Replaced symbolic link '$symlink' with the actual file." || \
echo "Error: Failed to replace symbolic link '$symlink'."
done
9 changes: 9 additions & 0 deletions yaml/ros2-security-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: security-env-config
namespace: default
data:
ROS_SECURITY_KEYSTORE: /etc/demo_keystore
ROS_SECURITY_ENABLE: "true"
ROS_SECURITY_STRATEGY: Enforce
89 changes: 89 additions & 0 deletions yaml/ros2-security-sample.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: ros2-secured-talker
spec:
replicas: 1
selector:
matchLabels:
app: ros2-secured-talker
template:
metadata:
labels:
app: ros2-secured-talker
spec:
containers:
- image: tomoyafujita/ros:rolling
command: ["/bin/bash", "-c"]
# Bind the security environmental variable
envFrom:
- configMapRef:
name: security-env-config
# Start the application with security option
args: ["source /opt/ros/$ROS_DISTRO/setup.bash && ros2 run demo_nodes_cpp talker --ros-args --enclave /talker_listener/talker"]
imagePullPolicy: IfNotPresent
tty: true
name: ros2-secured-talker
# Bind the volume
volumeMounts:
- name: talker-security
mountPath: /etc/demo_keystore/enclaves/talker_listener/talker
# Create volume to bind from ConfigMap
volumes:
- name: talker-security
configMap:
name: talker-enclaves
tolerations:
- key: node-role.kubernetes.io/master
operator: Exists
effect: NoSchedule
- key: node-role.kubernetes.io/control-plane
operator: Exists
effect: NoSchedule
restartPolicy: Always

---

apiVersion: apps/v1
kind: Deployment
metadata:
name: ros2-secured-listener
spec:
replicas: 1
selector:
matchLabels:
app: ros2-secured-listener
template:
metadata:
labels:
app: ros2-secured-listener
spec:
containers:
- image: tomoyafujita/ros:rolling
command: ["/bin/bash", "-c"]
# Bind the security environmental variable
envFrom:
- configMapRef:
name: security-env-config
# Start the application with security option
args: ["source /opt/ros/$ROS_DISTRO/setup.bash && ros2 run demo_nodes_cpp listener --ros-args --enclave /talker_listener/listener"]
imagePullPolicy: IfNotPresent
tty: true
name: ros2-secured-listener
# Bind the volume
volumeMounts:
- name: listener-security
mountPath: /etc/demo_keystore/enclaves/talker_listener/listener
# Create volume to bind from ConfigMap
volumes:
- name: listener-security
configMap:
name: listener-enclaves
tolerations:
- key: node-role.kubernetes.io/master
operator: Exists
effect: NoSchedule
- key: node-role.kubernetes.io/control-plane
operator: Exists
effect: NoSchedule
restartPolicy: Always

0 comments on commit 7a04ce1

Please sign in to comment.