Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[sensors] Encapsulate VTK image readers and writers #20026

Merged

Conversation

jwnimmer-tri
Copy link
Collaborator

@jwnimmer-tri jwnimmer-tri commented Aug 19, 2023

Towards #19945 and #16502. In particular, the new unit test here will serve as a backstop against regressions in the reader/writer code.


This change is Reviewable

@jwnimmer-tri jwnimmer-tri added priority: medium release notes: none This pull request should not be mentioned in the release notes labels Aug 19, 2023
@jwnimmer-tri jwnimmer-tri marked this pull request as ready for review August 19, 2023 15:06
@jwnimmer-tri jwnimmer-tri marked this pull request as draft August 19, 2023 19:34
@jwnimmer-tri
Copy link
Collaborator Author

\CC @SeanCurtis-TRI @sammy-tri @zachfang FYI. I still want to double-check the testing for missing cases, but overall this is the direction I'm aiming.

The call site code is somewhat improved by this, but the real win here is the additional tests. The new tests are part of what will help convince me that our pending upgrade to VTK will be sound.

I also have a WIP branch that further encapsulates image reading and writing, when the source / destination is specifically a sensors::Image instead of just a raw pixel buffer, specifically encapsulating the alpha (un)padding, bgr swizzling, etc. That one isn't ready for prime time yet.

@jwnimmer-tri jwnimmer-tri force-pushed the vtk-image-reader-writer branch 3 times, most recently from 68a501f to 287327c Compare August 21, 2023 23:07
Copy link
Collaborator Author

@jwnimmer-tri jwnimmer-tri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 6 of 10 files at r1, 5 of 5 files at r2, 5 of 5 files at r3, all commit messages.
Reviewable status: needs platform reviewer assigned, needs at least two assigned reviewers

@jwnimmer-tri jwnimmer-tri marked this pull request as ready for review August 21, 2023 23:08
@jwnimmer-tri jwnimmer-tri force-pushed the vtk-image-reader-writer branch 2 times, most recently from 9982651 to eefea33 Compare August 22, 2023 17:51
Copy link
Collaborator Author

@jwnimmer-tri jwnimmer-tri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 1 files at r5, all commit messages.
Reviewable status: needs platform reviewer assigned, needs at least two assigned reviewers

@jwnimmer-tri jwnimmer-tri force-pushed the vtk-image-reader-writer branch from eefea33 to 289c5a0 Compare August 22, 2023 19:45
@jwnimmer-tri
Copy link
Collaborator Author

Okay, I think this is ready for feature review.

+@SeanCurtis-TRI you have right of first refusal, I think.

Copy link
Contributor

@SeanCurtis-TRI SeanCurtis-TRI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:LGTM:

+@xuchenhan-tri for platform review, please.

Reviewed 3 of 10 files at r1, 3 of 5 files at r2, 4 of 5 files at r3, 1 of 1 files at r6, all commit messages.
Reviewable status: 4 unresolved discussions, LGTM missing from assignee xuchenhan-tri(platform)


systems/sensors/lcm_image_array_to_images.cc line 67 at r6 (raw file):

      image->width() * image->height() * image->kPixelSize) {
    drake::log()->error(
        fmt::format("Malformed output decoding incoming LCM {} image", format));

BTW Why the injection of fmt::format? log()->error() is already equipped to handle the format string.


systems/sensors/vtk_image_reader_writer.h line 19 at r6 (raw file):

namespace internal {

// (Internal use only) This file provides helper functions that encapsulate

BTW This kind of comment seems redundant when you have namespace internal { just two lines higher.


systems/sensors/vtk_image_reader_writer.cc line 75 at r6 (raw file):

}

/* Trampolies VTK progress events into a Drake-specific callback. */

nit typo

Suggestion:

Trampolines

systems/sensors/vtk_image_reader_writer.cc line 136 at r6 (raw file):

  // Add an observer to the writer that copies the encoded image into the
  // `output` buffer.
  auto observer = vtkSmartPointer<VtkProgressObserver>::New();

BTW An equivalent way to do this:

vtkNew<VtkProgressObserver> observer;

@jwnimmer-tri jwnimmer-tri force-pushed the vtk-image-reader-writer branch from 289c5a0 to 6ae44c5 Compare August 23, 2023 11:19
Copy link
Collaborator Author

@jwnimmer-tri jwnimmer-tri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 1 files at r6, 3 of 3 files at r7, all commit messages.
Reviewable status: 1 unresolved discussion, LGTM missing from assignee xuchenhan-tri(platform)


systems/sensors/lcm_image_array_to_images.cc line 67 at r6 (raw file):

Previously, SeanCurtis-TRI (Sean Curtis) wrote…

BTW Why the injection of fmt::format? log()->error() is already equipped to handle the format string.

Oops. Shrapnel from the future version of this that uses DiagnosticPolicy.


systems/sensors/vtk_image_reader_writer.cc line 136 at r6 (raw file):

Previously, SeanCurtis-TRI (Sean Curtis) wrote…

BTW An equivalent way to do this:

vtkNew<VtkProgressObserver> observer;

I need a shared ptr though, to attach it to the writer and return it. I think your line only creates a unique ptr?

Copy link
Contributor

@xuchenhan-tri xuchenhan-tri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm: nice!

Reviewed 2 of 10 files at r1, 3 of 5 files at r2, 2 of 5 files at r3, 1 of 1 files at r6, 3 of 3 files at r7, all commit messages.
Reviewable status: 1 unresolved discussion

Copy link
Contributor

@SeanCurtis-TRI SeanCurtis-TRI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 3 of 3 files at r7, all commit messages.
Reviewable status: :shipit: complete! all discussions resolved, LGTM from assignees SeanCurtis-TRI(platform),xuchenhan-tri(platform)


systems/sensors/vtk_image_reader_writer.cc line 136 at r6 (raw file):

Previously, jwnimmer-tri (Jeremy Nimmer) wrote…

I need a shared ptr though, to attach it to the writer and return it. I think your line only creates a unique ptr?

In today's episode of "Strange but True", we discuss vtk's "smart pointers"...

While it's natural to want to think of vtkNew and vtkSmartPointer as pre-C++11 versions of unique_ptr and shared_ptr (the names invite this), they are not.

vtkNew is not a unique pointer. Not at all. And while vtkSmartPointer is kind of like a shared pointer, it doesn't maintain the reference counter, that lives inside the object itself. vtkNew<Foo> merely takes care of calling Foo:New for you. You can implicitly create a vtkSmartPointer from a vtkNew and both remain valid pointers to the underlying object (both accounting for a tick in the reference counter). In fact, it was only in 2019 (after I suggested it), that they even created move semantics to create a vtkSmartPointer from vtkNew&&.

So, as counter-intuitive as it may seem, vtkNew does exactly what you need it to do. Particularly as all you're doing is adding it as an observer to writer. That said, vtkSmartPointer isn't wrong, just overly verbose.

The file format is now a first-class citizen, passed as an argument,
and reading or writing from files or a memory buffer are now equally
well supported.

Switch LCM image receiver system to use the VTK PNG decoder instead of
custom code.

Solve the VTK TIFF upside down dilemma noted in a prior glTF render
engine TODO comment.

Add acceptance test for write-and-readback.
@jwnimmer-tri jwnimmer-tri force-pushed the vtk-image-reader-writer branch from 6ae44c5 to ea4552a Compare August 25, 2023 15:00
Copy link
Collaborator Author

@jwnimmer-tri jwnimmer-tri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 1 files at r8, all commit messages.
Reviewable status: :shipit: complete! all discussions resolved, LGTM from assignees SeanCurtis-TRI(platform),xuchenhan-tri(platform)

@jwnimmer-tri jwnimmer-tri merged commit 441e2e4 into RobotLocomotion:master Aug 25, 2023
@jwnimmer-tri jwnimmer-tri deleted the vtk-image-reader-writer branch August 25, 2023 15:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
priority: medium release notes: none This pull request should not be mentioned in the release notes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants