[Question] Seeking Assistance for Resolving Configuration Error in SKRL Training with Franka Lift Environment

[XInyuSong000]

Question

Hi, I attempted to integrate a camera sensor into the Franka Lift environment as outlined here: Integration Guide. The observation space seems to be functioning correctly, as I can successfully execute the command ./orbit.sh -p source/standalone/environments/random_agent.py --task Isaac-Lift-Franka-v0 --num_envs 2 without encountering any errors.

However, issues arose when I tried using SKRL to test the environment. In the torch_lift_franka_ppo.py file, I modified the model to use CNN for image input and adjusted several hyperparameters accordingly:

memory = RandomMemory(memory_size=4, num_envs=env.num_envs, device=device)

cfg["rollouts"] = 4  # memory_size
cfg["learning_epochs"] = 5
cfg["mini_batches"] = 2

When attempting to run SKRL training with two environments using the command ./orbit.sh -p torch_lift_franka_ppo.py --num_envs 2 --headless, I encountered the following error:

./orbit.sh -p torch_lift_franka_ppo.py --num_envs 2 --headless
...
...
Traceback (most recent call last):
  File " torch_lift_franka_ppo.py", line 244, in <module>
    trainer.train()
  File "/home/wuyou/.local/share/ov/pkg/isaac_sim-2022.2.1/kit/python/lib/python3.7/site-packages/skrl/trainers/torch/sequential.py", line 77, in train
    self.single_agent_train()
  File "/home/wuyou/.local/share/ov/pkg/isaac_sim-2022.2.1/kit/python/lib/python3.7/site-packages/skrl/trainers/torch/base.py", line 193, in single_agent_train
    self.agents.post_interaction(timestep=timestep, timesteps=self.timesteps)
  File "/home/wuyou/.local/share/ov/pkg/isaac_sim-2022.2.1/kit/python/lib/python3.7/site-packages/skrl/agents/torch/ppo/ppo.py", line 294, in post_interaction
    self._update(timestep, timesteps)
  File "/home/wuyou/.local/share/ov/pkg/isaac_sim-2022.2.1/kit/python/lib/python3.7/site-packages/skrl/agents/torch/ppo/ppo.py", line 384, in _update
    _, next_log_prob, _ = self.policy.act({"states": sampled_states, "taken_actions": sampled_actions}, role="policy")
  File "torch_lift_franka_ppo.py", line 87, in act
    return GaussianMixin.act(self, inputs, role)
  File "/home/wuyou/.local/share/ov/pkg/isaac_sim-2022.2.1/kit/python/lib/python3.7/site-packages/skrl/models/torch/gaussian.py", line 160, in act
    log_prob = self._g_distribution[role].log_prob(inputs.get("taken_actions", actions))
  File "/home/wuyou/.local/share/ov/pkg/isaac_sim-2022.2.1/extscache/omni.pip.torch-1_13_1-0.1.4+104.2.lx64/torch-1-13-1/torch/distributions/normal.py", line 79, in log_prob
    self._validate_sample(value)
  File "/home/wuyou/.local/share/ov/pkg/isaac_sim-2022.2.1/extscache/omni.pip.torch-1_13_1-0.1.4+104.2.lx64/torch-1-13-1/torch/distributions/distribution.py", line 283, in _validate_sample
    format(actual_shape, expected_shape))
ValueError: Value is not broadcastable with batch_shape+event_shape: torch.Size([4, 8]) vs torch.Size([2, 8]).

It appears that training only initiates successfully when both mini_batches and rollouts are set to the same number. Could there be an oversight in my environment configuration that is causing this error?

[Mayankm96]

@Toni-SM Do you have an idea on this?

[Mayankm96]

Closing this issue since it is not related to Orbit library itself. Please open an issue on skrl to receive a faster response from the team there.