Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

REP-2008 - ROS 2 Hardware Acceleration Architecture and Conventions #324

Merged
merged 25 commits into from
Dec 15, 2022
Merged

REP-2008 - ROS 2 Hardware Acceleration Architecture and Conventions #324

merged 25 commits into from
Dec 15, 2022

Conversation

vmayoral
Copy link
Contributor

@vmayoral vmayoral commented Aug 25, 2021 •
edited
Loading

Accepted: Document available at https://ros.org/reps/rep-2008.html.

This REP proposal describes the architectural pillars and conventions required to introduce hardware acceleration in ROS 2 in a scalable and technology-agnostic manner. A set of categories meant to classify supported hardware acceleration solutions is provided. Inclusion in a category is based on the fulfillment of the hardware acceleration capabilities.

Steps for the REP taken from REP 1:

Signed-off-by: Víctor Mayoral Vilches v.mayoralv@gmail.com

@vmayoral
Initial draft of REP-2008
75dd482 
ROS 2 Hardware Acceleration Architecture and Conventions

Signed-off-by: Víctor Mayoral Vilches <v.mayoralv@gmail.com>
@ros-discourse
Copy link

This pull request has been mentioned on ROS Discourse. There might be relevant details there:

https://discourse.ros.org/t/rep-2008-rfc-ros-2-hardware-acceleration-architecture-and-conventions/22026/1

@ros-discourse
Copy link

This pull request has been mentioned on ROS Discourse. There might be relevant details there:

https://discourse.ros.org/t/rep-2008-rfc-ros-2-hardware-acceleration-architecture-and-conventions/22026/3

@ros-discourse
Copy link

This pull request has been mentioned on ROS Discourse. There might be relevant details there:

https://discourse.ros.org/t/rep-2008-rfc-ros-2-hardware-acceleration-architecture-and-conventions/22026/6

@ros-discourse
Copy link

This pull request has been mentioned on ROS Discourse. There might be relevant details there:

https://discourse.ros.org/t/announcing-the-hardware-acceleration-wg-meeting-1/20826/22

@vmayoral
Copy link
Contributor Author

vmayoral commented Sep 2, 2021

@wjwwood @tfoote @clalancette @kscottz, any chance we could get input on this?

@ros-discourse
Copy link

This pull request has been mentioned on ROS Discourse. There might be relevant details there:

https://discourse.ros.org/t/rep-2008-rfc-ros-2-hardware-acceleration-architecture-and-conventions/22026/8

vmayoral
vmayoral commented Sep 2, 2021
View reviewed changes
rep-2008.rst Outdated Show resolved Hide resolved
This was referenced Sep 27, 2021
Closed
Closed
@ros-discourse
Copy link

This pull request has been mentioned on ROS Discourse. There might be relevant details there:

https://discourse.ros.org/t/kria-robotics-stack-krs-alpha-release-a-ros-2-superset-for-industry/22462/1

@vmayoral
Copy link
Contributor Author

Any chance to get a first reaction on this? anyone from Open Robotics?

gbalke
gbalke reviewed Sep 29, 2021
View reviewed changes
Copy link
Contributor

@gbalke gbalke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First off, thanks Victor for pioneering this huge effort, it's definitely a contribution of epic proportions! 💯

Overall, looks pretty well thought out and the demo speaks for itself! The implementation is very streamlined for the user and I can definitely see this as a huge value add. There's obviously a ton of work required to get this sort of system up and running from the vendor side so I'd really be interested in hearing from another such vendor if this is generic enough to be able to add support for their systems (I'm sure it is)! I can understand if that might be difficult to get though 😢.

I'm extremely excited to see more hardware acceleration in ROS!

rep-2008.rst Outdated Show resolved Hide resolved
rep-2008.rst Outdated Show resolved Hide resolved
rep-2008.rst Outdated Show resolved Hide resolved
rep-2008.rst Outdated Show resolved Hide resolved
@vmayoral @gbalke
Update rep-2008.rst
86f2d77 
Co-authored-by: Greg Balke <11024792+gbalke@users.noreply.github.com>
@vmayoral
Copy link
Contributor Author

vmayoral commented Oct 1, 2021

@gbalke thanks for the review. I'm glad this initial proposal is sound. Core objective we pursued is to be inclusive, as much as possible and in alignment to the discussions held at https://discourse.ros.org/t/rep-2008-rfc-ros-2-hardware-acceleration-architecture-and-conventions/22026.

There's obviously a ton of work required to get this sort of system up and running from the vendor side so I'd really be interested in hearing from another such vendor if this is generic enough to be able to add support for their systems (I'm sure it is)! I can understand if that might be difficult to get though 😢.

This is a fair ask. Can you clarify exactly what's your expectation in here though? Please note I need to report internally on specific requirements, so that we can allocate resources if appropriate or ask for help/support from third parties.

E.g. one roadmap idea we've looking at for a while is supporting AMD to create ament_rocm package (similar to ament_vitis but for AMD's ROCm) and a matching acceleration_firmware_* package, so that we can validate the proposed architecture with AMD general purpose GPUs for workstations (PCIe-based). Would this be enough validation? If not, what specifically are you looking for, and from whom?

Besides this, is there anything else Open Robotics requires to move this forward into an official REP?

@gbalke
Copy link
Contributor

gbalke commented Oct 1, 2021 •
edited
Loading

This is a fair ask. Can you clarify exactly what's your expectation in here though? Please note I need to report internally on specific requirements, so that we can allocate resources if appropriate or ask for help/support from third parties.

E.g. one roadmap idea we've looking at for a while is supporting AMD to create ament_rocm package (similar to ament_vitis but for AMD's ROCm) and a matching acceleration_firmware_* package, so that we can validate the proposed architecture with AMD general purpose GPUs for workstations (PCIe-based). Would this be enough validation? If not, what specifically are you looking for, and from whom?

What you've suggested definitely addresses my request though I'm imagining a separate team from your own doing the implementation (to avoid bias). The "separate team" part is the reason why I said it might be difficult. If you can manage to walk them through the process and everything goes relatively smoothly, I think that'll be a good beta test for the design you've put together. I also image you'll find bugs on your own that way 😄. It doesn't have to be with AMD but that could work! I think you mentioned @pimartos was already working on a new board (though it still uses the Xilinx Ultrascale+ chipset so it has relatively small coverage).

Besides this, is there anything else Open Robotics requires to move this forward into an official REP?

I'm working on getting you an answer for this! I'm not currently a maintainer of the REP process so I'll have to rally the "top brass" 😃.

@vmayoral
Copy link
Contributor Author

vmayoral commented Oct 4, 2021

All right, let's work then towards ros-acceleration/community#1.

I'm working on getting you an answer for this! I'm not currently a maintainer of the REP process so I'll have to rally the "top brass" 😆.

Thanks @gbalke, let me know if you have further input then. ros-acceleration/community#1 is indeed tackling the same silicon, but a different vendor and firmware (this is often going to be the case since the number of silicon vendors is significantly lower than the number of embedded solution vendors).

For now, I'll work supporting the folks at ros-acceleration/community#1. I agree it's indeed a good validation and I'm sure we'll collect good input to simplify the process further.

@gbalke
Copy link
Contributor

gbalke commented Oct 7, 2021 •
edited
Loading

@vmayoral the feedback I got is mostly in-line with what I said earlier but a bit more strict. The interest of the REP process is to establish confidence in a system that many people agree on being useful. At the moment, the ros-acceleration WG is very new and not much has been done in terms of community input. This REP has interest but needs more proof of general viability before moving to a vote. I think if you can fully address my question, the code base will likely have the sort of activity that we are looking for! It may be best to focus the next HWG on the discussion of the REP and maybe have some commitment from other members to vet out the code-base or extend with their own toolchains. I think while it's a definite win in terms of adding to ROS, until it sees community support, it will be challenging to turn into an official REP.

@vmayoral
Copy link
Contributor Author

vmayoral commented Oct 8, 2021

Understood, thanks @gbalke, appreciate the effort and clarity.

While we push towards the objectives set, would it make sense to take an intermediate step and make an article about hardware acceleration at https://design.ros2.org/? This would help bringing community attention to this PR.

@wjwwood
Copy link
Contributor

wjwwood commented Oct 8, 2021

While we push towards the objectives set, would it make sense to take an intermediate step and make an article about hardware acceleration at https://design.ros2.org/? This would help bringing community attention to this PR.

I would not recommend this. Keep with the REP process.

@clalancette clalancette mentioned this pull request Nov 4, 2021
Merged
@vmayoral vmayoral changed the title Initial draft of REP-2008 - ROS 2 Hardware Acceleration Architecture and Conventions REP-2008 - ROS 2 Hardware Acceleration Architecture and Conventions Nov 22, 2021
@ros-discourse
Copy link

This pull request has been mentioned on ROS Discourse. There might be relevant details there:

https://discourse.ros.org/t/adaptive-ros-2-node-computations-and-hardware-acceleration-contributions/23197/1

@ros-discourse
Copy link

This pull request has been mentioned on ROS Discourse. There might be relevant details there:

https://discourse.ros.org/t/new-working-group-proposal-hardware-interface-working-group/23774/3

This was referenced Jan 24, 2022
Closed
Closed
@ros-discourse
Copy link

This pull request has been mentioned on ROS Discourse. There might be relevant details there:

https://discourse.ros.org/t/proposal-for-ros-2-hardware-acceleration-working-group-hawg/20112/31

@ros-discourse
Copy link

This pull request has been mentioned on ROS Discourse. There might be relevant details there:

https://discourse.ros.org/t/new-working-group-proposal-ai-integration-working-group/23996/8

@ros-discourse
Copy link

This pull request has been mentioned on ROS Discourse. There might be relevant details there:

https://discourse.ros.org/t/hardware-acceleration-wg-meeting-6/24207/2

@ros-discourse
Copy link

This pull request has been mentioned on ROS Discourse. There might be relevant details there:

https://discourse.ros.org/t/feedback-on-rep-2008-ros2-acceleration-kernels-with-build-integration/27872/3

vmayoral
vmayoral commented Nov 2, 2022
View reviewed changes
rep-2008.rst Outdated Show resolved Hide resolved
@vmayoral
Add pointers to REP-2007 and REP-2009
4732bff 
Following from community feedback, these changes hint ROS users that inter-Node data exchange can be further optimized by following leveraging REP-2007 and REP-2009.

It must be noted though that each accelerator (and associated host CPUs) may presents various other opportunities which are specific to that architecture. Such aspects shouldn't be discarded and would often lead to best optimizations.
@ros-discourse
Copy link

This pull request has been mentioned on ROS Discourse. There might be relevant details there:

https://discourse.ros.org/t/feedback-on-rep-2008-ros2-acceleration-kernels-with-build-integration/27872/6

@IoTDan
Copy link
Contributor

IoTDan commented Nov 4, 2022 via email

@vmayoral @IoTDan
Adjust text to make it OS-agnostic
71120c9 
Removed pointers to OS-specific tracers and
pushed tracing and benchmarking related topics
to REP-2014.

Signed-off-by: Víctor Mayoral Vilches <v.mayoralv@gmail.com>
Co-authored-by: Daniel Rosenstein <danrose@microsoft.com>
@ros-discourse
Copy link

This pull request has been mentioned on ROS Discourse. There might be relevant details there:

https://discourse.ros.org/t/ros-2-tsc-meeting-minutes-2022-10-13/28213/1

henningkayser
henningkayser reviewed Nov 17, 2022
View reviewed changes
rep-2008.rst Outdated Show resolved Hide resolved
rep-2008.rst Outdated Show resolved Hide resolved
vmayoral and others added 2 commits November 18, 2022 14:20
@vmayoral @henningkayser
Update rep-2008.rst
337bb49 
Co-authored-by: Henning Kayser <henningkayser@picknik.ai>
@vmayoral @henningkayser
Update rep-2008.rst
e370808 
Co-authored-by: Henning Kayser <henningkayser@picknik.ai>
@ros-discourse
Copy link

This pull request has been mentioned on ROS Discourse. There might be relevant details there:

https://discourse.ros.org/t/ros-2-tsc-meeting-minutes-2022-11-17/28457/1

@ros-discourse
Copy link

This pull request has been mentioned on ROS Discourse. There might be relevant details there:

https://discourse.ros.org/t/rep-2008-final-review-and-draft-vote-ros-2-hardware-acceleration-architecture-and-conventions/28544/1

@ros-discourse
Copy link

This pull request has been mentioned on ROS Discourse. There might be relevant details there:

https://discourse.ros.org/t/robotperf-benchmarks-the-benchmarking-suite-to-evaluate-robotics-computing-performance-using-ros-2/28742/1

@clalancette
Copy link
Contributor

Based on the voting result from the Hardware Acceleration Working Group (22 for, 0 against), this REP is approved as a Draft and will be merged as-is. Updates are of course welcome in follow-up PRs.

@clalancette clalancette merged commit 6c87966 into ros-infrastructure:master Dec 15, 2022
@ros-discourse
Copy link

This pull request has been mentioned on ROS Discourse. There might be relevant details there:

https://discourse.ros.org/t/ros-2-tsc-meeting-minutes-2022-12-15/28840/1

@ros-discourse
Copy link

This pull request has been mentioned on ROS Discourse. There might be relevant details there:

https://discourse.ros.org/t/ros-2-hardware-acceleration-working-group-2022-dissemination-report-and-feedback-request/29260/1

@vmayoral vmayoral mentioned this pull request Jan 16, 2023
Closed
@ros-discourse
Copy link

This pull request has been mentioned on ROS Discourse. There might be relevant details there:

https://discourse.ros.org/t/call-for-help-maintainership-of-the-ros-cross-compile-tool/26511/8

@vmayoral vmayoral mentioned this pull request Jan 4, 2024
Open
@ros-discourse
Copy link

This pull request has been mentioned on ROS Discourse. There might be relevant details there:

https://discourse.ros.org/t/ros-2-hardware-acceleration-working-group-2023-summary-and-new-directions-for-2024/35454/1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@soallak soallak soallak left review comments

@fujitatomoya fujitatomoya fujitatomoya left review comments

@gbalke gbalke gbalke left review comments

@clalancette clalancette clalancette left review comments

@henningkayser henningkayser Awaiting requested review from henningkayser

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.