Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes the image feature extractor in observations #1340

Open
wants to merge 16 commits into
base: main
Choose a base branch
from

Conversation

Dhoeller19
Copy link
Collaborator

@Dhoeller19 Dhoeller19 commented Oct 30, 2024

Description

  • Fixes the image feature extractor in observations
  • Adds missing dependencies in setup.py

Type of change

  • Bug fix (non-breaking change which fixes an issue)

Checklist

  • I have run the pre-commit checks with ./isaaclab.sh --format
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • I have updated the changelog and the corresponding version in the extension's config/extension.toml file
  • I have added my name to the CONTRIBUTORS.md or my name already exists there

@glvov-bdai
Copy link
Collaborator

Hi, this is fixed by this earlier open PR: #1323

Copy link
Collaborator

@glvov-bdai glvov-bdai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM sorry about the device hiccup. I think this PR should include the normalization fixes from #1323 or #1323 should be merged first. It also addresses the device issue

Mayankm96 and others added 3 commits October 30, 2024 14:52
Signed-off-by: Mayank Mittal <12863862+Mayankm96@users.noreply.github.com>
Copy link
Collaborator

@jsmith-bdai jsmith-bdai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for fixing @Mayankm96 @Dhoeller19!

Sorry again for not noticing the test breakage

Copy link
Collaborator

@glvov-bdai glvov-bdai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good would just update to support more resnet model sizes than just ResNet50

@glvov-bdai
Copy link
Collaborator

Overall looks good, I am able to train resnet and theia.

One note; it may be good to change the feature extraction yaml for RL games cartpole to use 512 or 1024 envs by default, as well as to have a max epoch length of 200 instead of 5k. Can also take care of this in a separate PR but I find these defaults lead to better convergence and less overcomputation respectively; currently the default env count is a little low with a max epoch length of 5k

@glvov-bdai
Copy link
Collaborator

LGTM! 👈 🤠 👉

@glvov-bdai
Copy link
Collaborator

glvov-bdai commented Oct 30, 2024

@Mayankm96 @Dhoeller19

I suggest that we delete the ground plane for the cartpole environments, or at least the feature extraction ones. It's not needed, and it leads to behavior where it seems like the feature extraction based cartpole doesn't work when you open up the viewport. Feature extraction cartpole works where the ground plane is not visible.

Alternatively, we could extend the ground plane to cover the whole field (from 100 by 100 to 10k by 10k), but this slows down convergence, leads to worse performance, and I believe this suffers from rendering issues. When looking at the ground plane, the lines seem to shift around a lot like in the issues reported in #1074 , which I argue is likely introducing a lot of high frequency noise

This makes sense as most of the environments do not see the ground plane, and it is a pretty visually distinct feature. As the encoders are frozen, they can not adapt, and I guess the MLP agent doesn't adapt either, as most of the gradient information it is using is originating from environments that do not seee the ground plane.

All of the following I trained with the default settings in this PR (1024 envs, 200 epochs) with Theia TIny.

Limited Ground Plane (current, most work when ground plane is not visible):

Screencast.from.10-30-2024.04.37.37.PM.webm

No Ground Plane (What I recommend):

Screencast.from.10-30-2024.05.13.15.PM.webm

Extended Ground Plane (Poor performance):

Screencast.from.10-30-2024.04.59.34.PM.webm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants