Skip to content

Commit

Permalink
Renames VIEWPORT_ENABLED to OFFSCREEN_RENDERING for clarity (#167)
Browse files Browse the repository at this point in the history
# Description

- Made SimulationContext correctly report RenderMode for streaming
- Changed language about HEADLESS in RenderMode to instead be NO_GUI,
because headlessness still requires rendering when livestreaming is
active, whereas NO_GUI is unambiguous. Also did some reordering of
docstrings for clarity.
- Removed unnecessary RENDER and VIEWPORT class vars from LauncherApp,
now that this signal is being taken from SimulationContext.

I will propagate this change to all standalone scripts/update docs once
I have conceptual approval.

Fixes #90, #145 

## Type of change

- New feature (non-breaking change which adds functionality)
- This change requires a documentation update

## Checklist

- [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with
`./orbit.sh --format`
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] I have updated the changelog and the corresponding version in the
extension's `config/extension.toml` file

---------

Signed-off-by: AutonomousHansen <50837800+AutonomousHansen@users.noreply.github.com>
Co-authored-by: Mayank Mittal <mittalma@leggedrobotics.com>
  • Loading branch information
hhansen-bdai and Mayankm96 committed Dec 22, 2023
1 parent fdf304b commit 9af48a1
Show file tree
Hide file tree
Showing 20 changed files with 254 additions and 205 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@
**/output/*
**/outputs/*
**/videos/*
docker/artifacts/
*.tmp

# Isaac-Sim packman
Expand Down
70 changes: 41 additions & 29 deletions docs/source/tutorials_envs/02_wrappers.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,14 +15,25 @@ For example, here is how you would wrap an environment to enforce that reset is

.. code-block:: python
"""Launch Isaac Sim Simulator first."""
from omni.isaac.orbit.app import AppLauncher
# launch omniverse app in headless mode
app_launcher = AppLauncher(headless=True)
simulation_app = app_launcher.app
"""Rest everything follows."""
import gym
import omni.isaac.orbit_envs # noqa: F401
from omni.isaac.orbit_envs.utils import load_default_env_cfg
# create base environment
cfg = load_default_env_cfg("Isaac-Reach-Franka-v0")
env = gym.make("Isaac-Reach-Franka-v0", cfg=cfg, render=True)
env = gym.make("Isaac-Reach-Franka-v0", cfg=cfg)
# wrap environment to enforce that reset is called before step
env = gym.wrappers.OrderEnforcing(env)
Expand All @@ -41,23 +52,12 @@ To use the wrapper, you need to first install ``ffmpeg``. On Ubuntu, you can ins
sudo apt-get install ffmpeg
The :class:`IsaacEnv` supports different rendering modes: "human" and "rgb_array". The "human" mode
is used when you want to render the environment to the screen. The "rgb_array" mode is used when you want
to get the rendered image as a numpy array. However, the "rgb_array" mode is allowed only when viewport
is enabled. This can be done by setting the ``viewport`` argument to ``True`` when creating the environment.

.. code-block:: python
import gym
import omni.isaac.orbit_envs # noqa: F401
from omni.isaac.orbit_envs.utils import load_default_env_cfg
The :class:`omni.isaac.orbit.envs.RlEnv` supports the rendering modes:

# create base environment
cfg = load_default_env_cfg("Isaac-Reach-Franka-v0")
env = gym.make("Isaac-Reach-Franka-v0", cfg=cfg, render=True, viewport=True)
# wrap environment to enforce that reset is called before step
env = gym.wrappers.OrderEnforcing(env)
* **"human"**: When you want to render the environment to the screen. It does not return any image.
* **"rgb_array"**: When you want to get the rendered image as a numpy array. It returns the image as a numpy array.
This mode is only possible when viewport is enabled, i.e. either the GUI window is open or off-screen rendering flag
is enabled.

.. attention::

Expand All @@ -67,40 +67,52 @@ is enabled. This can be done by setting the ``viewport`` argument to ``True`` wh
We notice the following performance in different rendering modes with the ``Isaac-Reach-Franka-v0`` environment
using an RTX 3090 GPU:

* Headless execution without viewport enabled: ~65,000 FPS
* Headless execution with viewport enabled: ~57,000 FPS
* Non-headless execution (i.e. with GUI): ~13,000 FPS
* No GUI execution without off-screen rendering enabled: ~65,000 FPS
* No GUI execution with off-screen enabled: ~57,000 FPS
* GUI execution with full rendering: ~13,000 FPS


The viewport camera used for rendering is the default camera in the scene called ``"/OmniverseKit_Persp"``.
The camera's pose and image resolution can be configured through the
:class:`omni.isaac.orbit_env.isaac_env_cfg.ViewerCfg` class.
:class:`omni.isaac.orbit.envs.base_env_cfg.ViewerCfg` class.


.. dropdown:: :fa:`eye,mr-1` Default parameters of the :class:`ViewerCfg` in the ``isaac_env_cfg.py`` file:
.. dropdown:: :fa:`eye,mr-1` Default parameters of the :class:`ViewerCfg` in the ``base_env_cfg.py`` file:

.. literalinclude:: ../../../source/extensions/omni.isaac.orbit_envs/omni/isaac/orbit_envs/isaac_env_cfg.py
.. literalinclude:: ../../../source/extensions/omni.isaac.orbit/omni/isaac/orbit/envs/base_env_cfg.py
:language: python
:lines: 37-52
:lines: 23-38
:linenos:
:lineno-start: 37
:lineno-start: 31


After adjusting the parameters, you can record videos by wrapping the environment with the
:class:`gym.wrappers.RecordVideo <gym:RecordVideo>` wrapper. As an example, the following code
records a video of the ``Isaac-Reach-Franka-v0`` environment for 200 steps, and saves it in the
``videos`` folder at a step interval of 1500 steps.
:class:`gym.wrappers.RecordVideo <gym:RecordVideo>` wrapper and enabling the off-screen rendering
flag. As an example, the following code records a video of the ``Isaac-Reach-Franka-v0`` environment
for 200 steps, and saves it in the ``videos`` folder at a step interval of 1500 steps.

.. code:: python
"""Launch Isaac Sim Simulator first."""
from omni.isaac.orbit.app import AppLauncher
# launch omniverse app in headless mode with off-screen rendering
app_launcher = AppLauncher(headless=True, offscreen_render=True)
simulation_app = app_launcher.app
"""Rest everything follows."""
import gym
# adjust camera resolution and pose
env_cfg.viewer.resolution = (640, 480)
env_cfg.viewer.eye = (1.0, 1.0, 1.0)
env_cfg.viewer.lookat = (0.0, 0.0, 0.0)
# create isaac-env instance
env = gym.make(task_name, cfg=env_cfg, render=headless, viewport=True)
env = gym.make(task_name, cfg=env_cfg)
# wrap for video recording
video_kwargs = {
"video_folder": "videos",
Expand Down
2 changes: 1 addition & 1 deletion source/extensions/omni.isaac.orbit/config/extension.toml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
[package]

# Note: Semantic Versioning is used: https://semver.org/
version = "0.9.19"
version = "0.9.20"

# Description
title = "ORBIT framework for Robot Learning"
Expand Down
16 changes: 16 additions & 0 deletions source/extensions/omni.isaac.orbit/docs/CHANGELOG.rst
Original file line number Diff line number Diff line change
@@ -1,6 +1,20 @@
Changelog
---------

0.9.20 (2023-10-03)
~~~~~~~~~~~~~~~~~~~

Fixed
^^^^^

* Changed naming in :class:`omni.isaac.orbit.sim.SimulationContext.RenderMode` to use ``NO_GUI_OR_RENDERING``
and ``NO_RENDERING`` instead of ``HEADLESS`` for clarity.
* Changed :class:`omni.isaac.orbit.sim.SimulationContext` to be capable of handling livestreaming and
offscreen rendering.
* Changed :class:`omni.isaac.orbit.app.AppLauncher` envvar ``VIEWPORT_RECORD`` to the more descriptive
``OFFSCREEN_RENDER``.


0.9.19 (2023-10-25)
~~~~~~~~~~~~~~~~~~~

Expand Down Expand Up @@ -173,6 +187,8 @@ Fixed
* Fixed the boundedness of class objects that register callbacks into the simulator.
These include devices, :class:`AssetBase`, :class:`SensorBase` and :class:`CommandGenerator`.
The fix ensures that object gets deleted when the user deletes the object.


0.9.7 (2023-09-26)
~~~~~~~~~~~~~~~~~~

Expand Down
Loading

0 comments on commit 9af48a1

Please sign in to comment.