Skip to content

Commit

Permalink
Isaac ROS 0.30.0 (DP3)
Browse files Browse the repository at this point in the history
  • Loading branch information
@jaiveersinghNV
jaiveersinghNV committed Apr 6, 2023
1 parent d2027fd commit b876656
Show file tree
Hide file tree
Showing 21 changed files with 526 additions and 101 deletions.
4 changes: 4 additions & 0 deletions .gitattributes
View file
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
# Ignore Python files in linguist
*.py linguist-detectable=false

# Images
*.gif filter=lfs diff=lfs merge=lfs -text
*.jpg filter=lfs diff=lfs merge=lfs -text
Expand All @@ -17,5 +20,6 @@
*.so.* filter=lfs diff=lfs merge=lfs -text

# ROS Bags
**/resources/**/*.zstd filter=lfs diff=lfs merge=lfs -text
**/resources/**/*.db3 filter=lfs diff=lfs merge=lfs -text
**/resources/**/*.yaml filter=lfs diff=lfs merge=lfs -text
45 changes: 27 additions & 18 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,25 +4,33 @@

## Overview

This ROS2 node uses the NVIDIA GPU-accelerated AprilTags library to detect AprilTags in images and publish their poses, IDs, and additional metadata. This has been tested on ROS2 (Humble) and should build and run on x86_64 and aarch64 (Jetson). It is modeled after and comparable to the ROS2 node for [CPU AprilTags detection](https://github.com/christianrauch/apriltag_ros.git).
Isaac ROS AprilTag contains a ROS 2 package for detection of [AprilTags](https://april.eecs.umich.edu/software/apriltag#:~:text=AprilTag%20is%20a%20visual%20fiducial,tags%20relative%20to%20the%20camera.), a type of fiducial marker that provides a point of reference or measure. AprilTag detections are GPU-accelerated for high performance.

For more information on the Isaac GEM that this node is based off of, see the latest Isaac SDK documentation [here](https://docs.nvidia.com/isaac/packages/fiducials/doc/apriltags.html).
<div align="center"><img alt="graph of nodes with AprilTag" src="resources/isaac_ros_apriltag_nodegraph.png" width="800px"/></div>

For more information on AprilTags themselves, including the paper and the reference CPU implementation, click [here](https://april.eecs.umich.edu/software/apriltag.html).
A common graph of nodes connects from an input camera through rectify and resize to AprilTag. Rectify warps the input camera image into a rectified, undistorted output image; this node may not be necessary if the camera driver provides rectified camera images. Resize is often used to downscale higher resolution cameras into the desired resolution for AprilTags if needed. The input resolution to AprilTag is selected by the required detection distance for the application, as a minimum number of pixels are required to perform an AprilTag detection and classification. For example, an 8mp input image of 3840×2160 may be much larger than necessary and a 4:1 downscale to 1920x1080 could make more efficienct use of compute resources and satisfy the required detection distance of the application. Each of the green nodes in the above diagram is GPU accelerated, allowing for a high-performance compute graph from Argus Camera to ApriTag. For USB and Ethernet cameras, the graph is accelerated from Rectify through AprilTag

<div align="center"><img alt="visual of AprilTag output message information" src="resources/apriltagdetection_message_illustration.png" width="700px"/></div>

As illustrated above, detections are provided in an output array for the number of AprilTag detections in the input image. Each entry in the array contains the ID (two-dimensional bar code) for the AprilTag, the four corners ((x0, y0), (x1, y1), (x2, y2), (x3, y3)) and center (x, y) of the input image, and the pose of the AprilTag.

> **Note**: This package is a GPU-accelerated drop-in replacement for the [CPU version of ROS Apriltag](https://github.com/christianrauch/apriltag_ros)
<!-- Split blockquote -->
> **Note**: For more information, including the paper and the reference CPU implementation, refer to the [AprilTag page](https://april.eecs.umich.edu/software/apriltag.html)
### Isaac ROS NITROS Acceleration

This package is powered by [NVIDIA Isaac Transport for ROS (NITROS)](https://developer.nvidia.com/blog/improve-perception-performance-for-ros-2-applications-with-nvidia-isaac-transport-for-ros/), which leverages type adaptation and negotiation to optimize message formats and dramatically accelerate communication between participating nodes.

## Performance

The performance results of benchmarking the prepared pipelines in this package on supported platforms are below:
The following table summarizes the per-platform performance statistics of sample graphs that use this package, with links included to the full benchmark output. These benchmark configurations are taken from the [Isaac ROS Benchmark](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark#list-of-isaac-ros-benchmarks) collection, based on the [`ros2_benchmark`](https://github.com/NVIDIA-ISAAC-ROS/ros2_benchmark) framework.

| Pipeline | AGX Orin | Orin Nano | x86_64 w/ RTX 3060 Ti |
| ------------------------ | ------------------ | ---------------- | --------------------- |
| Isaac ROS Apriltag(720p) | 248 fps <br> 5.5ms | 82 fps <br> 14ms | 600 fps <br> 2ms |
| Sample Graph | Input Size | AGX Orin | Orin NX | Orin Nano 8GB | x86_64 w/ RTX 3060 Ti |
| -------------------------------------------------------------------------------------------------------------------------------- | ---------- | -------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ |
| [AprilTag Node](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/scripts//isaac_ros_apriltag_node.py) | 720p | [108 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_apriltag_node-agx_orin.json)<br>11 ms | [65.5 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_apriltag_node-orin_nx.json)<br>17 ms | [47.1 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_apriltag_node-orin_nano_8gb.json)<br>23 ms | [242 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_apriltag_node-x86_64_rtx_3060Ti.json)<br>4.7 ms |
| [AprilTag Graph](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/scripts//isaac_ros_apriltag_graph.py) | 720p | [117 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_apriltag_graph-agx_orin.json)<br>12 ms | [70.5 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_apriltag_graph-orin_nx.json)<br>17 ms | [51.9 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_apriltag_graph-orin_nano_8gb.json)<br>24 ms | [270 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_apriltag_graph-x86_64_rtx_3060Ti.json)<br>5.0 ms |

These data have been collected per the methodology described [here](https://github.com/NVIDIA-ISAAC-ROS/.github/blob/main/profile/performance-summary.md#methodology).

## Table of Contents

Expand Down Expand Up @@ -50,28 +58,28 @@ These data have been collected per the methodology described [here](https://gith
- [ROS Topics Published](#ros-topics-published)
- [Troubleshooting](#troubleshooting)
- [Isaac ROS Troubleshooting](#isaac-ros-troubleshooting)
- [Updates](#updates)
- [Updates](#updates)

## Latest Update

Update 2022-10-19: Updated OSS licensing
Update 2023-04-05: Source available GXF extensions

## Supported Platforms

This package is designed and tested to be compatible with ROS2 Humble running on [Jetson](https://developer.nvidia.com/embedded-computing) or an x86_64 system with an NVIDIA GPU.
This package is designed and tested to be compatible with ROS 2 Humble running on [Jetson](https://developer.nvidia.com/embedded-computing) or an x86_64 system with an NVIDIA GPU.

> **Note**: Versions of ROS2 earlier than Humble are **not** supported. This package depends on specific ROS2 implementation features that were only introduced beginning with the Humble release.
> **Note**: Versions of ROS 2 earlier than Humble are **not** supported. This package depends on specific ROS 2 implementation features that were only introduced beginning with the Humble release.
| Platform | Hardware | Software | Notes |
| -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Jetson | [Jetson Orin](https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-orin/) <br> [Jetson Xavier](https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-agx-xavier/) | [JetPack 5.0.2](https://developer.nvidia.com/embedded/jetpack) | For best performance, ensure that [power settings](https://docs.nvidia.com/jetson/archives/r34.1/DeveloperGuide/text/SD/PlatformPowerAndPerformance.html) are configured appropriately. |
| x86_64 | NVIDIA GPU | [Ubuntu 20.04+](https://releases.ubuntu.com/20.04/) <br> [CUDA 11.6.1+](https://developer.nvidia.com/cuda-downloads) |
| Platform | Hardware | Software | Notes |
| -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Jetson | [Jetson Orin](https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-orin/) <br> [Jetson Xavier](https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-agx-xavier/) | [JetPack 5.1.1](https://developer.nvidia.com/embedded/jetpack) | For best performance, ensure that [power settings](https://docs.nvidia.com/jetson/archives/r34.1/DeveloperGuide/text/SD/PlatformPowerAndPerformance.html) are configured appropriately. |
| x86_64 | NVIDIA GPU | [Ubuntu 20.04+](https://releases.ubuntu.com/20.04/) <br> [CUDA 11.8+](https://developer.nvidia.com/cuda-downloads) |

### Docker

To simplify development, we strongly recommend leveraging the Isaac ROS Dev Docker images by following [these steps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common/blob/main/docs/dev-env-setup.md). This will streamline your development environment setup with the correct versions of dependencies on both Jetson and x86_64 platforms.

> **Note:** All Isaac ROS Quickstarts, tutorials, and examples have been designed with the Isaac ROS Docker images as a prerequisite.
> **Note**: All Isaac ROS Quickstarts, tutorials, and examples have been designed with the Isaac ROS Docker images as a prerequisite.
## Quickstart

Expand Down Expand Up @@ -231,10 +239,11 @@ ros2 launch isaac_ros_apriltag isaac_ros_apriltag.launch.py --ros-args -p size:=

For solutions to problems with Isaac ROS, please check [here](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common/blob/main/docs/troubleshooting.md).

# Updates
## Updates

| Date | Changes |
| ---------- | --------------------------------------------------------------------------------------- |
| 2023-04-05 | Source available GXF extensions |
| 2022-10-19 | Updated OSS licensing |
| 2022-08-31 | Update to be compatible with JetPack 5.0.2 |
| 2022-06-30 | Update to use NITROS for improved performance |
Expand Down
17 changes: 10 additions & 7 deletions docs/tutorial-isaac-sim.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,9 @@

## Overview

This tutorial walks you through a pipeline to estimate the 6DOF pose of [AprilTags](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_apriltag) using images from Isaac Sim.
This tutorial walks you through a graph to estimate the 6DOF pose of [AprilTags](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_apriltag) using images from Isaac Sim.

Last validated with [Isaac Sim 2022.2.1](https://docs.omniverse.nvidia.com/app_isaacsim/app_isaacsim/release_notes.html#id1)

## Tutorial Walkthrough

Expand All @@ -24,21 +26,22 @@ This tutorial walks you through a pipeline to estimate the 6DOF pose of [AprilTa
source install/setup.bash
```

4. Launch the pre-composed pipeline launchfile:
4. Launch the pre-composed graph launchfile:

```bash
ros2 launch isaac_ros_apriltag isaac_ros_apriltag_isaac_sim_pipeline.launch.py
```

5. Install and launch Isaac Sim following the steps in the [Isaac ROS Isaac Sim Setup Guide](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common/blob/main/docs/isaac-sim-sil-setup.md)
6. Open up the Isaac ROS Common USD scene (using the "content" window) located at:
6. Open up the Isaac ROS Common USD scene (using the *Content* tab) located at:

`omniverse://localhost/NVIDIA/Assets/Isaac/2022.1/Isaac/Samples/ROS2/Scenario/carter_warehouse_apriltags_worker.usd`
```text
http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/2022.2.1/Isaac/Samples/ROS2/Scenario/carter_warehouse_apriltags_worker.usd
```

Wait for it to load completely.
> **Note:** To use a different server, replace `localhost` with `<your_nucleus_server>`
And wait for it to load completely.
7. Press **Play** to start publishing data from Isaac Sim.
<div align="center"><img src="../resources/Isaac_sim_april_tag.png" width="800px"/></div>
<div align="center"><img src="../resources/Isaac_sim_play.png" width="800px"/></div>

8. In a separate terminal, run RViz to visualize the AprilTag detections:

Expand Down
10 changes: 5 additions & 5 deletions docs/tutorial-usb-cam.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,13 @@

## Overview

This tutorial walks you through a pipeline to estimate the 6DOF pose of [AprilTags](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_apriltag) using images streamed from a USB camera.
This tutorial walks you through a graph to estimate the 6DOF pose of [AprilTags](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_apriltag) using images streamed from a USB camera.

## Tutorial Walkthrough

1. Complete the [Quickstart section](../README.md#quickstart) in the main README.
2. Calibrate the camera following instructions in the [camera calibration tutorial](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common/blob/main/docs/camera-calibration.md). For step 11, use `isaac_ros_apriltag` in place of `<isaac_ros_package>`
3. Clone the ROS2 usb_cam package
3. Clone the ROS 2 usb_cam package

```bash
cd ~/workspaces/isaac_ros-dev/src && git clone -b ros2 https://github.com/ros-drivers/usb_cam
Expand Down Expand Up @@ -50,6 +50,6 @@ This tutorial walks you through a pipeline to estimate the 6DOF pose of [AprilTa
rviz2 -d /workspaces/isaac_ros-dev/src/isaac_ros_apriltag/isaac_ros_apriltag/rviz/usb_cam.rviz
```

> **Note:** Your camera vendor may offer a specific ROS2-compatible camera driver package and you can use that in place of the usb_cam package.

> **Important:** If you are using a different camera driver package ensure that the camera stream publishes `Image` and `CameraInfo` pairs to the topics `/image` and `/camera_info`, respectively.
> **Note**: Your camera vendor may offer a specific ROS 2-compatible camera driver package and you can use that in place of the usb_cam package.
<!-- Split blockquote -->
> **Important**: If you are using a different camera driver package ensure that the camera stream publishes `Image` and `CameraInfo` pairs to the topics `/image` and `/camera_info`, respectively.
Loading

0 comments on commit b876656

Please sign in to comment.