Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor ImageManip node #444

Merged
merged 14 commits into from
Jul 11, 2022
Merged

Refactor ImageManip node #444

merged 14 commits into from
Jul 11, 2022

Conversation

OanaMariaVatavu
Copy link
Contributor

@OanaMariaVatavu OanaMariaVatavu commented Apr 5, 2022

Features added:

  1. Input RGB/BGR, planar/interleaved, YUV/NV12, RAW8 & GRAY8 formats
  2. Color convert to any of the above formats
  3. Resize, scale, warp, ... completely arbitrarily
  4. Adds image flipping (shorthand)
  5. Adds proper thumbnailing with background color selection (which is also correctly color converted to specified output type)
  6. Adds capability to provide a completely custom mesh for the warp engine (ImageManip setWarpMesh function)
  7. Supports RAW16 (uint16) format (eg: depth,...) but without color conversion capability currently
    Note: Planar formats are faster to process (eg. interleaved RGB/BGR or NV12 are more taxing on memory transfers because of how the internal conversion to planar gets made). Converting to interleaved formats is without any penalties.

Known issues(related to the current version, not to the mainline one):

New issues discovered at debug:

  • Unproperly aligning the maximum size of the allocated Warp descriptors
  • YUV420/NV12 -> RGB conversion produces artefacts
  • setKeepAspectRatio API deprecated
  • large resize unsupported on YUV420/NV12 format; e.g. 1080p -> 224x224
  • CMX memory corruption when output format is YUV420/NV12

ToDo

  • Adding support for 4K input resolution
  • Adding support for FP16 -> U8 conversion
  • Optimize input interleaved->planar conversions
  • Checks for invalid messages being sent to eg. ImageManip.inputImage / ImageManip.inputConfig / etc.

…actor

2. Update FW
   - Merge remote-tracking branch 'origin/develop' into image_manip_refactor
   - [BUG FIXED] YUV420/NV12 -> RGB conversion produces artefacts(for classification scenario)
   - [BUG FIXED] large resize unsupported on YUV420/NV12 format(for classification scenario)
   - [BUG FIXED] CMX memory corruption when output format is YUV420/NV12
   - [FEATURE ADDED] Adding support for 4K input/output resolution
    [API maintained]
        [setKeepAspectRatio]
            - resize with keeping aspect ratio + crop the input field of view to match the output aspect ratio
            - using the "setKeepAspectRatio" API in conjunction with the "setResizeThumbnail" API, the "setKeepAspectRatio" API will not have any effect because the "setResizeThumbnail" API has priority
            - keepAspectRatio config is True by default

    [bugs fixed]
        [artifacts/memory coruption]
            Properly extract the chroma rect
            Properly compute the luma/chroma cached memory size
            Input interleaved -> planar conversion + rect rotation
        [hang]
            input rectangle outside frame
@alex-luxonis alex-luxonis force-pushed the image_manip_refactor branch from 0b23c73 to 75d9906 Compare June 17, 2022 00:40
@themarpe themarpe merged commit f55eda1 into develop Jul 11, 2022
@themarpe themarpe deleted the image_manip_refactor branch July 11, 2022 17:25
@Luxonis-Brandon
Copy link
Contributor

Woohoo!!!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants