Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Aloha env and ACT policy #11

Merged
merged 7 commits into from
Mar 12, 2024
Merged

Conversation

Cadene
Copy link
Collaborator

@Cadene Cadene commented Mar 8, 2024

  • Add Aloha env + dataset of 4 tasks (2 humans 2 scripted)
    • Add abstract env and all envs inherite from it
  • Add ACT policy
  • Add Aloha tests + push dataset test artefacts
  • Fix normalization + Add some tests all envs and datasets are expected to load uint8
  • Improve github test workflow (test on CPU)
  • Trained a policy able to achieve the task

video_25000_3e1f1b1028486b536b67.mp4

video_34500_4dd290deed8c33ca4480.mp4

@Cadene Cadene force-pushed the user/rcadene/2024_03_06_aloha_env branch from 55b1824 to d98b435 Compare March 8, 2024 12:08
@Cadene Cadene changed the title [WIP] Add aloha env [WIP] Add Aloha env and ACT policy Mar 10, 2024
@Cadene Cadene force-pushed the user/rcadene/2024_03_06_aloha_env branch 3 times, most recently from d6279da to 4c37b77 Compare March 11, 2024 18:52
@Cadene Cadene added the CI label Mar 11, 2024
WIP Aloha env tests pass

Rendering works (fps look fast tho? TODO action bounding is too wide [-1,1])

Update README

Copy past from act repo

Remove download.py add a WIP for Simxarm

Remove download.py add a WIP for Simxarm

Add act yaml (TODO: try train.py)

Training can runs (TODO: eval)

Add tasks without end_effector that are compatible with dataset, Eval can run (TODO: training and pretrained model)

Add AbstractEnv, Refactor AlohaEnv, Add rendering_hook in env, Minor modifications, (TODO: Refactor Pusht and Simxarm)

poetry lock

fix bug in compute_stats for action normalization

fix more bugs in normalization

fix training

fix import

PushtEnv inheriates AbstractEnv, Improve factory Normalization

Add _make_env to EnvAbstract

Add call_rendering_hooks to pusht env

SimxarmEnv inherites from AbstractEnv (NOT TESTED)

Add aloha tests artifacts + update pusht stats

fix image normalization: before env was in [0,1] but dataset in [0,255], and now both in [0,255]

Small fix on simxarm

Add next to obs

Add top camera to Aloha env (TODO: make it compatible with set of cameras)

Add top camera to Aloha env (TODO: make it compatible with set of cameras)
@Cadene Cadene force-pushed the user/rcadene/2024_03_06_aloha_env branch from 0d4cea7 to 9d00203 Compare March 12, 2024 10:53
@Cadene Cadene merged commit 8c56770 into main Mar 12, 2024
1 check passed
@Cadene Cadene changed the title [WIP] Add Aloha env and ACT policy Add Aloha env and ACT policy Mar 12, 2024
@aliberts aliberts deleted the user/rcadene/2024_03_06_aloha_env branch April 27, 2024 08:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants