Remove lidar code in nuImages #442
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
holger-motional
changed the title
holger-motional
added a commit
that referenced
this pull request
Aug 11, 2020
* nuImages v0.1 (#372) * Implemented lazy loading * Implement rendering of images with annotations * Disentangle requirements, better rendering * Explicitly init tables * Fix bug with explicitly assigned tables * Fix instance mask parsing * Fixed rendering * Remove unnecessary matplotlib commands * Remove explorer * Implemented list_attributes * list_cameras() and better lazy loading * Implement list_category * Improve tutorial, implement missing functions * Fix case when mask is empty, fix attribute names, change color map * Overhauled tutorial * Cleanup * Split readme into nuImages and nuScenes parts * Typo * Address review comments, add split to version * Render another image * nuImages new schema (#427) * Increment version number to v1.0 * Update table names * Installation instructions * Improve tutorial * Replace with global colormap, update tutorial * Patch all methods for new schema * Add detailed attribute tests * Added test for complete attributes * Add tests for foreign keys * Added schema figures * Add missing requirement * Skip long unit tests * Manual test alternative * Fix test issues and setup for all versions * Completed tests * Better error message * Show sample_token instead * Readd init shortcut for tutorial * Sort schema alphabetically * Add new table from nuScenes-lidarseg * Rename image to sample * Auto-format * Add new method list_sample_content * Add missing ego vehicle color * Remove nbr_points field from lidarseg table * Bugfix, select only the jpg * Typo * Modify render_image to take sample_data tokens * Start integrating depth methods * Rework depth rendering * Rename function * Fix one more * Don't init sample_to_key_frame_map * Fix lambda expression * Fix inconsistent naming * Fix indexing bug * Select all lidar tokens * Fix lidar extension * Add depth rendering to tutorial * Rework rendering of next image * Return projected points as 2d, set tutorial to val split, spelling * Workaround for wrong intrinsics format * Fix projection bug * Standardize out_path options * Remove intrinsics workaround, align tutorials, add schema to tutorial * Remove deprecated nuim.sample_to_key_frame method * Typo * Use get_sample_content, simplify table loading * Add test for prev/next pointers * Fix typo in driveable * Adjust test for new database fields * Fix sorting bug * Change output string * Output message * Add script to render random images to disk * Fix bug, improve output messages * Fix wrong path, remove title for better visibility, disable lazy * Debug numerical overflow, improve render script, filter by camera, align render_depth output size * Avoid overflow and remove special cases * Simplify depth completion options * Cleanup * Implemented pointcloud rendering * Color by height * Adjust distort settings, add shortcut function * Reorganize and add method for trajectory * Fancy rendering method for trajectories * Cleanup * Rename tutorials, remove the word basic * Move all docs to docs folder and rename some * Throw error if sweeps does not exist * Revamp tutorial * Add arguments in render, fix bugs * Use typing throughout, improve documentation * Improve installation instructions * Auto-clean and updated comments * More cleanup * Address some review comments * Rename variable * Remove deprecated output argument * Add new functions to tutorial * FIX CRITICAL BUG IN MASK_DECODE, Add list_anns, dynamic bow_line_width, change rendering order, improve tutorial * nuImages videos (#432) * Fix automatic test discover and disable test outputs * Add nuImages schema * Add function to render videos * Unified image and video rendering * Further improvements to render script * Fix bug around unchanged mode * Check number of sample_datas * Add new function to tutorial * Add sort order argument to list_categories * Fix sorting * Garbage collection to avoid memory overflow * Format * Reorganize render_images, add render_rare_classes * Rename has_rider attribute in test * Minor fix for tests on test set * Replace val with mini * Address review comments * Fix merge conflicts, remove warnings, change rendered car * Fix test for mini * Remove todo * Added filtering options to various scripts * Updated render_depth method (v2) (#437) * Separate methods for dense and sparse depth * More parameters for dense depth * Print velocity, fix fisheye * Workaround when no key_lidar_token is known * Fix previous modification * Filter by fileformat * Handle missing lidar * Same for depth_sparse * Update nuImages documentation, add ability to get segmentation masks, improve listing and rendering functions (#438) * Shift section of lazy loading * Remove sentence in README saying that files will not be available for public download * Inform user in tutorial that with_category must be True in order to set We can also set with_attributes=True * Inform user in tutorial that with_category must be True in order to set with_attributes=True * Section on setup before section on verifying install in installation.md * Shift lazy description back and just state at intro that it will be discussed later * Improve docstring and add assertion for render_image, allow list_attributes to be sorted * Change to in list_cameras * Change cones and barriers from black to gray in tutorial * Change foreground to object and background to surface * Allow user to choose to draw only boxes, surfaces, both, or all in render_image * Update tutorial to mention with_annotations options in render_image * Give user abiity to adjust font size in render_image * Add get_segmentation method * Classes are 1-indexed in masks * Address comments for nuimages and util * Address comments for tutorial * Tidy up tutorial * More content for instructions, improve structure of instructions * Address comments * Address some comments for instructions * Address more comments on instructions * Tidy up instructions a bit * Edits from proofreading * Re-introduce fonts_valid param in get_font * Align class indices in lidarseg and get_semgentation * Remove class descriptions, add attributes to instructions * Only include added attributes * Add link to attributes in instructions * Tidy up somemore * Proofread * Proofread again * Proofread more * Address more comments on instructions * Do not assume category.json is sorted when creating index map in get_segmentation * Add links for images for instructions * Fix list_anns * Add samples images for surface classes * Remove images * Add links to images * Address comments for phrasing * Added empty line before last image * Attempt to resolve requirements issue * Revert * Copy entire requirements folder * Fix wrong folder Co-authored-by: Holger Caesar <holger@nutonomy.com> * Resolve some warnings, adapt to new render function arguments * Add log name to the output path * Use future ego pose * Debug output * Don't do ego compensation * Remove lidar code in nuImages (#442) * Purge depth from code * Update documentation * Simplify variable names * Wording * Wording * Fix wrong assertion * Typo * Install pycocotools from conda-forge (#441) * install pycocotools from conda-forge * less verbose output * Minor corrections * Address review comments Co-authored-by: whyekit-aptiv <62535720+whyekit-aptiv@users.noreply.github.com> Co-authored-by: Valentyn Klindukh <valentyn.klindukh@nutonomy.com>
holger-motional
added a commit
that referenced
this pull request
Sep 1, 2020
* Merge lidarseg_v1.0 into nuscenes_v2.0 (#434) * nuScenes-lidarseg (#343) * initial commit for NuScenes-lidarseg * add function to list lidarseg categories and create colormap * integrated lidarseg function into nuscenes * print number of lidarseg annotations * Render lidarseg labels in image * Improve error message when pointsensor is not lidar * Calculate stats for lidarseg * Assertion for nsweeps to display lidarseg labels * Render only points in image which are labeled with classes the user chooses * Modularize function to filter colormap * Modularize function for generating colors for lidarseg classes * Clean up example usages * Allow user to filter and show desired lidarseg labels for render_sample_data * Save figure from render_pointcloud_in_image without excessive border * Render scene with pointclouds and lidarseg labels * Shift conditional statements in render_sample_data() * Check that lidar pointcloud is a keyframe * Render all camera channels with pointclouds and lidarseg labels for all scenes * Allow user to show lidarseg labels for render_sample() * Hacks to render for VOs * Added scene to filename for easy retrieval for VOs * Render scenes for VOs per class * Updated rendering of scenes for cameras to output videos * Function to convert plt scatter plot to cv2 * Tidy up examples * Print stats for lidarseg keyframe * Enable sorting of counts * Tweak verbosity for render_sample * Include class index when printing sample stats * Assert that path for video given by user ends with .avi * Enable showing of legend when mapping pc to image * Style edits * nuScenes-lidarseg initial tutorial * Cleanup, move methods out of tutorial, copyright * Cleanup * Remove empty cell * Throw errors rather than warnings * Address some comments on style. * Add full stops to comments * Tidy up utils * Removed TODOs * Initial draft of lidarseg tutorial * Added gifs for tutorial * Tidy up tutorial * Add setup instructions to tutorial * Edit style for one assertion * Changed show_lidarseg_labels default value in render_sample * Change type to np.ndarray * Fix some typos * Remove version from lidarseg.json * Bugfix * Unbugfix * Don't add category_lidarseg to table_names * Comment out dangerous commands Co-authored-by: Holger Caesar <holger@nutonomy.com> * View predictions using lidarseg devkit (#396) * Allow user to view preds using render_pointcloud_in_image * Allow users to get stats of predictions for a sample * Add visualization for user's preds to relevant functions * Updated tutorial * Fixed some typos in tutorial * Comment out code which may crash notebook * Fix double # * Get lidarseg file name from json instead, default coloring to depth if empty json * Change show_lidarseg_preds to lidarseg_preds_bin_path * Amend doc string for lidarseg_preds_bin_path * Amend render video functions to take folder name for preds * Amend tutorial * Remove gifs from folder * Clear outputs from tutorial * Update documentations for lidarseg (#404) * Update docs for lidarseg * Fix some typos * Address comments for docs * Update folder structure description in notebook * Automatically determine which lidarseg classes are present in a pointcloud projected onto an image (#410) * Auto find liarseg labels present in projected pcl * Add assertion to check num of bin files equals num of lidarseg records * Address comments * Adjust legend for aesthetics * Improve aesthetics for render_scene_with_pointclouds_for_all_cameras (#412) * Flip back cams horizontally for aesthetics * Explicitly set margins to zero and turn axes off * Prevent final frames from showing up in notebook if user stops render * Allow render_scene_with_pointclouds_for_all_cameras to output frames as images * Update indices in tutorial examples * Style changes * Clear memory in render_camera_channel_with_pointclouds if user stops rendering * Graceful exit if users stops rendering * Change random seed for color scheme * Style edits * Enable bboxes to be plotted with lidarseg (#418) * Include rendering of lidarseg with bboxes for render_scene_with_pointclouds_for_all_cameras * Add option to render bboxes in lidarseg videos * Style edits * Rename methods * Renamed argument to be more similar to nuScenes * Rename show_lidarseg_labels to show_lidarseg * Added comment on videos and images * Rewording * Making plotting of bboxes with lidarseg more modular * Remove option to save renders as images for render_scene_lidarseg * Use render_mode instead of render_if_no_points * Style edits * Update notebook * Style edits for notebook * Remove render_if_no_points for render_pointcloud_in_image * Style edits * Update file names for renders * Save image even for verbose=False * Made plot_points_and_bboxes * Update coloring * Update colormap * Choose nicer colors for some classes * Deep copy for colormap * Reduce bbox line width and update colormap * One line per arg * Update colormap * Remove unused method for arbitrary colormap * Remove leading 0 for filename * One colormap to rule them all * Update error msg for checking colormap to colors conversion * Add dpi as an arg for rendering lidarseg scenes * Amend docstring in get_colormap() * Make colormap DRY Co-authored-by: Holger Caesar <holger@nutonomy.com> * Remove render_if_no_points argument from lidarseg tutorial (#423) * Remove render_if_no_points arg and section * Rephrasing * Single category.json for both nuScenes and nuScenes-lidarseg (#424) * Change 'label' to 'name' when loading category json; check if preds folder exists * Check in various functions that lidarseg is installed * Shorten assertion statement * Check that new version of category.json is used, if loading lidarseg * Check only for 'index' in category records * nutonomy green for driveable surface (#425) * Improve lidarseg rendering methods, update tutorial, add unit tests (#429) * Initial commit to address comments * Fix bug where passing np.array instead of array into filter_lidarseg_labels throws an error * Rephrase some parts of tutorial * Include legend for render_sample_data * Add arg for show_lidarseg_legend in render_sample_data * Adjust aesthetics of legend in render_sample_data * Update colormap * Make creating legend modular * Ensure only labels in pc is included in legend * Explain verbose * Check that filter_lidarseg_labels is either a list or np.array * Allow user to list stats by class index * Update docstrings * Update tutorial to demo sort_by * Add unit tests * Make painting of labels modular * Add docstrings * Add lidarseg annotation instructions, update lidarseg tutorial intro (#431) * Initial commit * Add links from classes to examples * Link from examples to class definitions using Top * Remove extra white line * Include script to render lidarseg histogram for split (#433) * Initial commit * Arrange order of seaborn in requirements.txt * Add docstrings * Add more docstrings * Remove seaborn from requirements * Address comments * Add horizontal gridlines * Replicate look and feel of seaborn without needing package * Style edits * Add type in truncate_class_name Co-authored-by: Holger Caesar <holger@nutonomy.com> * Align indices in nuImages and nuScenes-lidarseg (#440) * Order colormap * Do not assume any ordering in colormap * nuImages v1.0 (#435) * nuImages v0.1 (#372) * Implemented lazy loading * Implement rendering of images with annotations * Disentangle requirements, better rendering * Explicitly init tables * Fix bug with explicitly assigned tables * Fix instance mask parsing * Fixed rendering * Remove unnecessary matplotlib commands * Remove explorer * Implemented list_attributes * list_cameras() and better lazy loading * Implement list_category * Improve tutorial, implement missing functions * Fix case when mask is empty, fix attribute names, change color map * Overhauled tutorial * Cleanup * Split readme into nuImages and nuScenes parts * Typo * Address review comments, add split to version * Render another image * nuImages new schema (#427) * Increment version number to v1.0 * Update table names * Installation instructions * Improve tutorial * Replace with global colormap, update tutorial * Patch all methods for new schema * Add detailed attribute tests * Added test for complete attributes * Add tests for foreign keys * Added schema figures * Add missing requirement * Skip long unit tests * Manual test alternative * Fix test issues and setup for all versions * Completed tests * Better error message * Show sample_token instead * Readd init shortcut for tutorial * Sort schema alphabetically * Add new table from nuScenes-lidarseg * Rename image to sample * Auto-format * Add new method list_sample_content * Add missing ego vehicle color * Remove nbr_points field from lidarseg table * Bugfix, select only the jpg * Typo * Modify render_image to take sample_data tokens * Start integrating depth methods * Rework depth rendering * Rename function * Fix one more * Don't init sample_to_key_frame_map * Fix lambda expression * Fix inconsistent naming * Fix indexing bug * Select all lidar tokens * Fix lidar extension * Add depth rendering to tutorial * Rework rendering of next image * Return projected points as 2d, set tutorial to val split, spelling * Workaround for wrong intrinsics format * Fix projection bug * Standardize out_path options * Remove intrinsics workaround, align tutorials, add schema to tutorial * Remove deprecated nuim.sample_to_key_frame method * Typo * Use get_sample_content, simplify table loading * Add test for prev/next pointers * Fix typo in driveable * Adjust test for new database fields * Fix sorting bug * Change output string * Output message * Add script to render random images to disk * Fix bug, improve output messages * Fix wrong path, remove title for better visibility, disable lazy * Debug numerical overflow, improve render script, filter by camera, align render_depth output size * Avoid overflow and remove special cases * Simplify depth completion options * Cleanup * Implemented pointcloud rendering * Color by height * Adjust distort settings, add shortcut function * Reorganize and add method for trajectory * Fancy rendering method for trajectories * Cleanup * Rename tutorials, remove the word basic * Move all docs to docs folder and rename some * Throw error if sweeps does not exist * Revamp tutorial * Add arguments in render, fix bugs * Use typing throughout, improve documentation * Improve installation instructions * Auto-clean and updated comments * More cleanup * Address some review comments * Rename variable * Remove deprecated output argument * Add new functions to tutorial * FIX CRITICAL BUG IN MASK_DECODE, Add list_anns, dynamic bow_line_width, change rendering order, improve tutorial * nuImages videos (#432) * Fix automatic test discover and disable test outputs * Add nuImages schema * Add function to render videos * Unified image and video rendering * Further improvements to render script * Fix bug around unchanged mode * Check number of sample_datas * Add new function to tutorial * Add sort order argument to list_categories * Fix sorting * Garbage collection to avoid memory overflow * Format * Reorganize render_images, add render_rare_classes * Rename has_rider attribute in test * Minor fix for tests on test set * Replace val with mini * Address review comments * Fix merge conflicts, remove warnings, change rendered car * Fix test for mini * Remove todo * Added filtering options to various scripts * Updated render_depth method (v2) (#437) * Separate methods for dense and sparse depth * More parameters for dense depth * Print velocity, fix fisheye * Workaround when no key_lidar_token is known * Fix previous modification * Filter by fileformat * Handle missing lidar * Same for depth_sparse * Update nuImages documentation, add ability to get segmentation masks, improve listing and rendering functions (#438) * Shift section of lazy loading * Remove sentence in README saying that files will not be available for public download * Inform user in tutorial that with_category must be True in order to set We can also set with_attributes=True * Inform user in tutorial that with_category must be True in order to set with_attributes=True * Section on setup before section on verifying install in installation.md * Shift lazy description back and just state at intro that it will be discussed later * Improve docstring and add assertion for render_image, allow list_attributes to be sorted * Change to in list_cameras * Change cones and barriers from black to gray in tutorial * Change foreground to object and background to surface * Allow user to choose to draw only boxes, surfaces, both, or all in render_image * Update tutorial to mention with_annotations options in render_image * Give user abiity to adjust font size in render_image * Add get_segmentation method * Classes are 1-indexed in masks * Address comments for nuimages and util * Address comments for tutorial * Tidy up tutorial * More content for instructions, improve structure of instructions * Address comments * Address some comments for instructions * Address more comments on instructions * Tidy up instructions a bit * Edits from proofreading * Re-introduce fonts_valid param in get_font * Align class indices in lidarseg and get_semgentation * Remove class descriptions, add attributes to instructions * Only include added attributes * Add link to attributes in instructions * Tidy up somemore * Proofread * Proofread again * Proofread more * Address more comments on instructions * Do not assume category.json is sorted when creating index map in get_segmentation * Add links for images for instructions * Fix list_anns * Add samples images for surface classes * Remove images * Add links to images * Address comments for phrasing * Added empty line before last image * Attempt to resolve requirements issue * Revert * Copy entire requirements folder * Fix wrong folder Co-authored-by: Holger Caesar <holger@nutonomy.com> * Resolve some warnings, adapt to new render function arguments * Add log name to the output path * Use future ego pose * Debug output * Don't do ego compensation * Remove lidar code in nuImages (#442) * Purge depth from code * Update documentation * Simplify variable names * Wording * Wording * Fix wrong assertion * Typo * Install pycocotools from conda-forge (#441) * install pycocotools from conda-forge * less verbose output * Minor corrections * Address review comments Co-authored-by: whyekit-aptiv <62535720+whyekit-aptiv@users.noreply.github.com> Co-authored-by: Valentyn Klindukh <valentyn.klindukh@nutonomy.com> * nuScenes 2.0 tutorial revamp (#446) * Add list_sample_data * Fix indexing bug * Fix type * Fix columns * Type conversion * Type conversion * Fix bug with different length of values and bins * By default, use standard font size * Workaround for mini split * Fix swapped dimensions * Add nuim.list_sample_data_histogram() * More verbose * Make matplotlib style attributes case-sensitive * Fix rendering issue in map expansion * Change figure alignment in prediction tutorial * nuImages data export (#447) * Add boilerplate * Add export script * Print status * Typo * Add mini split to all * Rename files * Skip only individual files * Fix file paths in mini * Remove redundant path * Add assert_download for nuImages * Clarification on past sweeps * VERY SLOW RENDERING with alpa composite * Simplify alpha composite * Cleanup * Update links and silence test * Update more links * Update FAQ * Fix requirements for new pip package (#449) * nuScenes v2.0 alpha (#452) * Fix broken link in readme * aummy typo * Add a note on bicycle rcks * Fix render_rare_classes, additional checks for object_tokens in render_image * Fix tutorials link * Address comments from nuScenes-lidarseg alpha testing (#451) * Fix broken link in readme * aummy typo * Add a note on bicycle rcks * Fix render_rare_classes, additional checks for object_tokens in render_image * Amend classes shown in tutorial * Add note for MacOSX users * Shift matplotlib macOSX tip to installaton.md instead * Updated backend instructions Co-authored-by: Holger Caesar <holger@nutonomy.com> * Update requirements * Address review comments * Address review comments * Address review comments Co-authored-by: whyekit-aptiv <62535720+whyekit-aptiv@users.noreply.github.com> Co-authored-by: Valentyn Klindukh <valentyn.klindukh@nutonomy.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR removes all handling of lidar data in nuImages.