-
Notifications
You must be signed in to change notification settings - Fork 644
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* Fix automatic test discover and disable test outputs * Add nuImages schema * Add function to render videos * Unified image and video rendering * Further improvements to render script * Fix bug around unchanged mode * Check number of sample_datas * Add new function to tutorial * Add sort order argument to list_categories * Fix sorting * Garbage collection to avoid memory overflow * Format * Reorganize render_images, add render_rare_classes * Rename has_rider attribute in test * Minor fix for tests on test set * Replace val with mini * Address review comments
- Loading branch information
1 parent
ac4bbb7
commit 71c0f2b
Showing
15 changed files
with
559 additions
and
134 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| TODO: Coming soon! |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,6 +1,164 @@ | ||
| nuImages schema | ||
| ========== | ||
| This document describes the database schema used in nuImages. | ||
| All annotations and meta data (including calibration, maps, vehicle coordinates etc.) are covered in a relational database. | ||
| The database tables are listed below. | ||
| Every row can be identified by its unique primary key `token`. | ||
| Foreign keys such as `sample_token` may be used to link to the `token` of the table `sample`. | ||
| Please refer to the [tutorial](https://www.nuscenes.org/nuimages#tutorial) for an introduction to the most important database tables. | ||
|
|
||
| TODO: Coming soon! | ||
|  | ||
|
|
||
|  | ||
| attribute | ||
| --------- | ||
| An attribute is a property of an instance that can change while the category remains the same. | ||
| Example: a vehicle being parked/stopped/moving, and whether or not a bicycle has a rider. | ||
| The attributes in nuImages are a superset of those in nuScenes. | ||
| ``` | ||
| attribute { | ||
| "token": <str> -- Unique record identifier. | ||
| "name": <str> -- Attribute name. | ||
| "description": <str> -- Attribute description. | ||
| } | ||
| ``` | ||
|
|
||
| calibrated_sensor | ||
| --------- | ||
| Definition of a particular sensor (lidar/camera, but no radar) as calibrated on a particular vehicle. | ||
| All extrinsic parameters are given with respect to the ego vehicle body frame. | ||
| All camera images come undistorted and rectified. | ||
| ``` | ||
| calibrated_sensor { | ||
| "token": <str> -- Unique record identifier. | ||
| "sensor_token": <str> -- Foreign key pointing to the sensor type. | ||
| "translation": <float> [3] -- Coordinate system origin in meters: x, y, z. | ||
| "rotation": <float> [4] -- Coordinate system orientation as quaternion: w, x, y, z. | ||
| "camera_intrinsic": <float> [3, 3] -- Intrinsic camera calibration. Empty for sensors that are not cameras. | ||
| "camera_distortion": <float> [5 or 6] -- Camera calibration parameters. We use the 5 parameter camera convention of the CalTech camera calibration toolbox, that is also used in OpenCV. Only for fish-eye lenses in CAM_BACK do we use the 6th parameter. | ||
| } | ||
| ``` | ||
|
|
||
| category | ||
| --------- | ||
| Taxonomy of object categories (e.g. vehicle, human). | ||
| Subcategories are delineated by a period (e.g. `human.pedestrian.adult`). | ||
| The categories in nuImages are the same as in the nuScenes (w/o lidarseg), plus `flat.driveable_surface`. | ||
| ``` | ||
| category { | ||
| "token": <str> -- Unique record identifier. | ||
| "name": <str> -- Category name. Subcategories indicated by period. | ||
| "description": <str> -- Category description. | ||
| } | ||
| ``` | ||
|
|
||
| ego_pose | ||
| --------- | ||
| Ego vehicle pose at a particular timestamp. Given with respect to global coordinate system of the log's map. | ||
| The ego_pose is the output of a lidar map-based localization algorithm described in our paper. | ||
| The localization is 2-dimensional in the x-y plane. | ||
| Warning: nuImages is collected from almost 500 logs with different maps versions. | ||
| Therefore the coordinates **should not be compared across logs** or rendered on the semantic maps of nuScenes. | ||
| ``` | ||
| ego_pose { | ||
| "token": <str> -- Unique record identifier. | ||
| "translation": <float> [3] -- Coordinate system origin in meters: x, y, z. Note that z is always 0. | ||
| "rotation": <float> [4] -- Coordinate system orientation as quaternion: w, x, y, z. | ||
| "timestamp": <int> -- Unix time stamp. | ||
| "rotation_rate": <float> [3] -- The angular velocity vector (x, y, z) of the vehicle in rad/s. This is expressed in the ego vehicle frame. | ||
| "acceleration": <float> [3] -- Acceleration vector (x, y, z) in the ego vehicle frame in m/s/s. The z value is close to the gravitational acceleration `g = 9.81 m/s/s`. | ||
| "speed": <float> -- The speed of the ego vehicle in the driving direction in m/s. | ||
| } | ||
| ``` | ||
|
|
||
| log | ||
| --------- | ||
| Information about the log from which the data was extracted. | ||
| ``` | ||
| log { | ||
| "token": <str> -- Unique record identifier. | ||
| "logfile": <str> -- Log file name. | ||
| "vehicle": <str> -- Vehicle name. | ||
| "date_captured": <str> -- Date (YYYY-MM-DD). | ||
| "location": <str> -- Area where log was captured, e.g. singapore-onenorth. | ||
| } | ||
| ``` | ||
|
|
||
| object_ann | ||
| --------- | ||
| The annotation of a foreground object (car, bike, pedestrian) in an image. | ||
| Each foreground object is annotated with a 2d box, a 2d instance mask and category-specific attributes. | ||
| ``` | ||
| object_ann { | ||
| "token": <str> -- Unique record identifier. | ||
| "sample_data_token": <str> -- Foreign key pointing to the sample data, which must be a keyframe image. | ||
| "category_token": <str> -- Foreign key pointing to the object category. | ||
| "attribute_tokens": <str> [n] -- Foreign keys. List of attributes for this annotation. | ||
| "bbox": <int> [4] -- Annotated amodal bounding box. Given as [xmin, ymin, xmax, ymax]. | ||
| "mask": <RLE> -- Run length encoding of instance mask using the pycocotools package. | ||
| } | ||
| ``` | ||
|
|
||
| sample_data | ||
| --------- | ||
| A sensor data e.g. image or lidar pointcloud. Note that we don't have radar in nuImages. | ||
| Sample_data covers all sensor data, regardless of whether it is a keyframe or not. | ||
| For every keyframe image or lidar, we also include up to 6 past and 6 future sweeps at 2 Hz. | ||
| We can navigate between consecutive lidar or camera sample_datas using the `prev` and `next` pointers. | ||
| Only keyframe (sample) images are annotated. | ||
| The sample timestamp is inherited from the keyframe camera sample_data timestamp. | ||
| ``` | ||
| sample_data { | ||
| "token": <str> -- Unique record identifier. | ||
| "sample_token": <str> -- Foreign key. Sample to which this sample_data is associated. | ||
| "ego_pose_token": <str> -- Foreign key. | ||
| "calibrated_sensor_token": <str> -- Foreign key. | ||
| "filename": <str> -- Relative path to data-blob on disk. | ||
| "fileformat": <str> -- Data file format. | ||
| "width": <int> -- If the sample data is an image, this is the image width in pixels. | ||
| "height": <int> -- If the sample data is an image, this is the image height in pixels. | ||
| "timestamp": <int> -- Unix time stamp. | ||
| "is_key_frame": <bool> -- True if sample_data is part of key_frame, else False. | ||
| "next": <str> -- Foreign key. Sample data from the same sensor that follows this in time. Empty if end of scene. | ||
| "prev": <str> -- Foreign key. Sample data from the same sensor that precedes this in time. Empty if start of scene. | ||
| } | ||
| ``` | ||
|
|
||
| sample | ||
| --------- | ||
| A sample is an annotated keyframe selected from a large pool of images in a log. | ||
| Every sample has up to 13 lidar sample_datas and 13 camera sample_datas corresponding to it. | ||
| These include the actual lidar and camera keyframe sample_datas, which can be accessed via the `key_*_token` fields. | ||
| ``` | ||
| sample { | ||
| "token": <str> -- Unique record identifier. | ||
| "timestamp": <int> -- Unix time stamp. | ||
| "log_token": <str> -- Foreign key pointing to the log. | ||
| "key_camera_token": <str> -- Foreign key of the sample_data corresponding to the camera keyframe. | ||
| "key_lidar_token": <str> -- Foreign key of the sample_data corresponding to the lidar keyframe. | ||
| } | ||
| ``` | ||
|
|
||
| sensor | ||
| --------- | ||
| --------- | ||
| A specific sensor type. | ||
| ``` | ||
| sensor { | ||
| "token": <str> -- Unique record identifier. | ||
| "channel": <str> -- Sensor channel name. | ||
| "modality": <str> {camera, lidar} -- Sensor modality. Supports category(ies) in brackets. | ||
| } | ||
| ``` | ||
|
|
||
| surface_ann | ||
| --------- | ||
| The annotation of a background object (driveable surface) in an image. | ||
| Each background object is annotated with a 2d semantic segmentation mask. | ||
| ``` | ||
| surface_ann { | ||
| "token": <str> -- Unique record identifier. | ||
| "sample_data_token": <str> -- Foreign key pointing to the sample data, which must be a keyframe image. | ||
| "category_token": <str> -- Foreign key pointing to the surface category. | ||
| "mask": <RLE> -- Run length encoding of segmentation mask using the pycocotools package. | ||
| } | ||
| ``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.