-
Notifications
You must be signed in to change notification settings - Fork 41
Batch rendering
Please see ssc for overview of the SSC scripts.
render.js --id <id> - Renders an asset using an asset id.
render-file.js --input <filename> - Renders an asset from file.
scn2img.js --limit <n> --cameras <camfile> --id <sceneId> - Renders a scene using viewpoints specified in a (gaps) camera file.
Options can be specified in a config file using --config_file <filename>.
The config file can be used to specify how to load the asset as well as views to generate. Common options to specify for asset loading is the defaultUp and defaultFront that specify how the asset should be oriented so the up/front aligns with the STK world up (Y) and front (-Z).
Examples of config files can be found in ssc/config directory.
Example of rendering with fully specified camera position in scene coordinate frame
ssc/config/render_model_normalized.json
{
"assetInfo": {
"defaultUp": [ 0, 1, 0 ], // Up vector for asset
"defaultFront": [ -1, 0, 0], // Front vector for asset
"defaultUnit": 1, // unit (in meters) at which asset was modeled
"materialSidedness": "Front",
"useVertexColors": true,
"computeNormals": true
},
"view": {
"coordinate_frame": "scene", // Specify "scene" to use scene coordinate frame vs default world coordinate frame
"position": [0, 0.5, -1], // camera position
"target": [0, 0, 0], // lookAt of camera
"up": [0, 1, 0], // up axis for camera
"near": 0.01, // near
"far": 100, // far
"fov": 90 // vertical field of view (horizontal field of view is computed based on aspect ratio)
}
}
Example of rendering with automatically placed camera
ssc/config/render_scan.json
{
"assetInfo": {
"defaultUp": [ 0, 0, 1 ], // Up vector for asset
"defaultFront": [ -1, 0, 0], // Front vector for asset
"defaultUnit": 1, // unit (in meters) at which asset was modeled
"materialSidedness": "Front",
"useVertexColors": true,
"computeNormals": true
},
"view": {
"phi": 0, // rotation from front (longitude in rad)
"theta": 0.78539816339, // angleFromHorizontal (latitude in rad)
"distanceScale": 1.5 // how far to be (multiplier of max bounding box dimension)
}
}
# Rendering options
--use_ambient_occlusion [flag] Use ambient occlusion or not
--ambient_occlusion_type <type> Type of ambient occlusion to use (ssao-old, ssao, sao, edl)
--outline_color <color> Color for outlines
--use_highlight_outline [flag]'. Use outline shader or not for highlighting
--highlight_outline_color <color>. Color for highlight outlines
--use_antialias_pass [flag] Use extra antialiasing pass
--antialias_type <type> Type of antialias to use (ssaa,fxaa)
# Image options
--width <width> Image width [default: 1000]
--height <height> Image height [default: 1000]
--compress_png [flag] Compress PNG output using pngquant
--flipxy <fliptype> Flip xy when generating image (default flip about 'upper_left' corner, use 'lower_left' to flip about lower left corner)
# Coloring and semantic segmentation options
--color_by <color_by> Recoloring scheme
--color <color> Color when coloring by color
--encode_index [flag] Encode color index directly
--write_index [flag] Output index to file
--index <filename> Input index to use for consistent encoding
--object_index <filename> Input index to use for object ids
--restrict_to_color_index [flag] Restrict coloring to index
# Output image options
--output_image_encoding <encoding> What encoding to use for output image
--convert_pixels <target_type> Target type for pixels (uint8,uint16)
# Automatic resize options
--center [flag] Center so scene is at origin
--normalize_size <type> What to normalize (diagonal,max)
--normalize_size_to <target> What to normalize the size to
# Render canonical views (left/right/bottom/top/front/back)
--render_all_views [flag] Render all canonical views
# Turn table options
--render_turntable [flag] Render a sequence of turntable images and a video
--turntable_step <degrees> Degrees to step for each turntable step [default: 1]
--framerate <frames per secs> Frames per second for turntable video [default: 25]
--tilt <tilt> Default tilt (from horizontal) in degrees [default: 60]
--skip_video [flag] If true, skips generation of turntable video
# View parameters
--view_index <view_index> Which view to render [0-7 corresponding to default/left/right/bottom/top/front/back]
--use_lights [flag] Whether to use lights
--use_shadows [flag] Whether to use shadows
--use_physical_lights [flag] Use physical lights or not
--use_directional_lights [flag] Use directional lights or not
Note that for light and shadow effects, your asset (scene or model) must have lights specified. Otherwise, the scene will be dark and gloomy. There is a known issue with using THREE.js for rendering shadows in that it will use too many uniforms (possibly exceeding what is supported by the GPU) and causing nothing to be rendered.
Use --render_turntable to generate turntable views of the asset. A separate png is output for each view, and an mp4 video is created using ffmpeg. Specify --skip_video true to not output the mp4 video (if you don't have ffmpeg installed). Use --turntable_step <degrees>, --framerate <frames per secs> to control the speed of the rotation and --tilt <degrees> to control the angle from horizontal.
Use --view_index <number> to specify one of the canonical views.
left = 1
right = 2
bottom = 3
top = 4
front = 5
back = 6
To specify a custom view, create an config file and specify details of the view parameter in the config file
...
"view": {
"coordinate_frame": "scene", // Specify "scene" to use scene coordinate frame vs default world coordinate frame
"position": [0, 0.5, -1], // camera position
"target": [0, 0, 0], // lookat of camera (can also be specified as lookat)
"up": [0, 1, 0], // up axis for camera
"near": 0.01, // near
"far": 100, // far
"fov": 90 // vertical field of view (horizontal field of view is computed based on aspect ratio)
}
By default, a perspective camera is created. If you want, you can also specify a orthographic camera:
...
"view": {
"type": "orthographic",
"left": ...,
"right": ...,
"top": ...,
"bottom": ...,
...
}
isEquirectangular can be used to specify a equirectangular camera.
If you want to generate multiple views of the same scene, you can specify the camera viewpoints in a separate file that is passed in.
scn2img.js can handle many different camera specifications and can render either using --id or --path. To specify the file with camera viewpoints use --camera <filename>
The main supported format for the --cameras <filename> is json or jsonl files with a camera record having the following fields:
{ "id": "cam_id", "camera": { "type": ..., "position": ..., "target": ..., "up": ... }}
If specified (and no name field is specified within the camera), the id field is used to specify the filename.
Example with orthographic camera:
{ "id": "cam0", "camera": { "position": [-0.008218,1.013098,-0.345625], "lookat": [0,0,0], "type": "orthographic" }}
For convenience, other specifications for the camera is also supported.
Example with extrinsics matrix (with direction specifying the camera direction convention, the STK uses the OpenGL context of -z being the camera direction (see https://learnopengl.com/Getting-started/Camera)):
{"id": "scene0474_00-738", "camera": {"extrinsics": [0.962244, 0.06998, -0.263037, 3.794199, -0.270179, 0.362725, -0.891871, 3.017831, 0.032997, 0.929265, 0.367937, 1.413183, 0.0, -0.0, -0.0, 1.0], "isRowMajor": true, "direction": [0, 0, -1]}}
Currently scn2img.js also supports the following (deprecated) formats for --camera <filename>
- Camera conf file (from ScanNet)
- Camera views in a text format compatible with the GAPS software
Note that it is also possible to use the scn2img.js, render.js and render-file.js with camera and multiple view information specified in a config file.
See ssc/config/render_lfd.json and ssc/config/lfd.jsonl for example.
For generating screen shots, it's convenient to have automatically computed camera views based on bounding box of the scene.
...
"view": {
"cameraPositionStrategy": "positionByDistance", // How to position the camera
"phi": 0, // rotation from front (longitude in rad)
"theta": 0.78539816339, // angleFromHorizontal (latitude in rad)
"distanceScale": 1.5 // how far to be (multiplier of max bounding box dimension)
}
How the camera is positioned is controlled by the cameraPositionStrategy
positionToFit (default) - position camera so object will fit in image (for screenshots)
positionByDistance - position camera by distance away from axis aligned bounding box
positionByCentroid - position camera by distance away from centroid
positionToFixedResolution - position camera to give a fixed resolution
You can also try to have the camera be optimized to look at specific objects
--view_target_ids <object_ids>. # Which objects to look at (use object to include all objects)
--find_good_views [flag] # Whether to optimize for good views or not (applies if --view_target_ids is set)
Use --view_target_ids object to find an view of all objects (excludes Room and architecture).
Camera type and parameters can be specified using a config file.
...
"camera": {
"type": "perspective", // Specify "orthographic" for orthographic camera
// Should specify "left", "right", "top", "bottom" for orthographic camera
"near": 0.01, // near
"far": 100, // far
"fov": 90 // vertical field of view (horizontal field of view is computed based on aspect ratio)
}
Use --color_by to color the model/scene in exciting new ways. By default, the colors used will be colors selected for human interpretation of the scene. To generate semantic segmentation images for training, use the --encode_index option along with the --write_index <filename> to export the mapping from an encoded RGB png to the labels. Existing mappings can be imported using the --index option.
The index of the object instance is encoded as a integer in the RGB (with R as highest bits and B as the lowest) channels in the image (the image itself is RGBA). The index starts from 0 (for unknown) and goes up (the R will never be reached). When encoding object parts, the part index is encoded in the R channel.
If the pixel is outside of the scene, it has a alpha value of 0, otherwise the alpha is 255.
objectId - Color object by unique instance id
objectMaterialId - Color object parts by unique instance id and material
objectPartId - Color object parts by unique instance id and part id
objectPartLabel - Color object parts by part label
modelId - Color object by model id
material - Color object by material
objectType - Color object by object type
category - Color object by object type (please use `objectType` instead)
roomId - Color objects by room id
roomType - Color objects by room type
roomSurface - Color objects by room surface
depth
normal
wireframe
color - Color everything a single color
If using --color_by color, you must specify the color to use with --color <color>
If you are working with a single object (vs a scene), the above options may not be supported.
If you have custom attributes on your mesh, you can also use color_by the vertex or face attribute, with the color specifying the vertex or face attribute to use.
vertexAttribute
faceAttribute
# Rendering using the asset id and color by category
./render.js --id <id> --color_by category
# Rendering using the asset id and color by single color
./render.js --id <id> --color_by color --color '#fef9ed'
# Rendering from a file with specified config file
./render-file.js --input <filename> --config_file <config_filename>
- Home
- Main
- Annotators
- Assets
- Batch processing
- Development
- File formats
- Rendering
- Scene Tools
- Voxels