Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[RLlib; Offline RL] Store episodes in state form. #47294

Merged
merged 20 commits into from
Sep 10, 2024
Merged

[RLlib; Offline RL] Store episodes in state form. #47294

merged 20 commits into from
Sep 10, 2024
+176 −30

Commits on Aug 23, 2024

  1. Added 'get_state' and 'from_state' to 'InfiniteLookbackBuffer' and mo… 

    …dified 'SingleAgentEpisode's 'get_state' and 'set_state' to convert to and from lookback buffers.

Signed-off-by: simonsays1980 <simon.zehnder@gmail.com>
    @simonsays1980
    simonsays1980 committed Aug 23, 2024
    Configuration menu
    Copy the full SHA
    7f7decb View commit details
    Browse the repository at this point in the history

Commits on Aug 26, 2024

  1. Added serialization for 'SingleAgentEpisode' objects via 'msgpack'. A… 

    …dded '__eq__' method to 'InfiniteLookBackBuffer' together with 'get/set_state' methods. Tested serialization and desrialization with recording offline data and in unit tests.

Signed-off-by: simonsays1980 <simon.zehnder@gmail.com>
    @simonsays1980
    simonsays1980 committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    9cd30ae View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into offline-store-episodes-in-state-form

    @simonsays1980
    simonsays1980 committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    7c5f67a View commit details
    Browse the repository at this point in the history

  3. Added 'msgpack' and 'msgpack_numpy' to the 'rllib-test-requirements.txt. 

    Signed-off-by: simonsays1980 <simon.zehnder@gmail.com>
    @simonsays1980
    simonsays1980 committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    37d37a0 View commit details
    Browse the repository at this point in the history

Commits on Aug 27, 2024

  1. Merge branch 'master' into offline-store-episodes-in-state-form

    @simonsays1980
    simonsays1980 committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    13e4d71 View commit details
    Browse the repository at this point in the history

  2. Changed some code in response to @sven1977's review. 

    Signed-off-by: simonsays1980 <simon.zehnder@gmail.com>
    @simonsays1980
    simonsays1980 committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    8fb8a38 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'master' into offline-store-episodes-in-state-form

    @simonsays1980
    simonsays1980 committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    b1a924a View commit details
    Browse the repository at this point in the history

Commits on Aug 30, 2024

  1. Modified comment. 

    Signed-off-by: simonsays1980 <simon.zehnder@gmail.com>
    @simonsays1980
    simonsays1980 committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    54b5d2d View commit details
    Browse the repository at this point in the history

  2. Added 'msgpack' and 'msgpack_numpy' to the 'autodoc_mock_imports'. 

    Signed-off-by: simonsays1980 <simon.zehnder@gmail.com>
    @simonsays1980
    simonsays1980 committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    ffeb244 View commit details
    Browse the repository at this point in the history

  3. Merged master. 

    Signed-off-by: simonsays1980 <simon.zehnder@gmail.com>
    @simonsays1980
    simonsays1980 committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    0e9260f View commit details
    Browse the repository at this point in the history

  4. Fixed name error for 'msgpack-numpy' in requirements. 

    Signed-off-by: simonsays1980 <simon.zehnder@gmail.com>
    @simonsays1980
    simonsays1980 committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    bcb9fb7 View commit details
    Browse the repository at this point in the history

Commits on Sep 2, 2024

  1. Merge branch 'master' into offline-store-episodes-in-state-form

    @simonsays1980
    simonsays1980 committed Sep 2, 2024
    Configuration menu
    Copy the full SHA
    cdfea4a View commit details
    Browse the repository at this point in the history

Commits on Sep 4, 2024

  1. Added 'msgpack' and 'msgpack-numpy' to the 'setup.py'. 

    Signed-off-by: simonsays1980 <simon.zehnder@gmail.com>
    @simonsays1980
    simonsays1980 committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    0426889 View commit details
    Browse the repository at this point in the history

Commits on Sep 6, 2024

  1. Merge branch 'master' into offline-store-episodes-in-state-form

    @simonsays1980
    simonsays1980 committed Sep 6, 2024
    Configuration menu
    Copy the full SHA
    5d6a213 View commit details
    Browse the repository at this point in the history

  2. Removed dependencies on 'msgpack_numpy' from 'setup.py'. Modified imp… 

    …ort path for 'PPO' in 'test_usage_stats.py' as it was failing due to missing package 'msgpack_numpy'.

Signed-off-by: simonsays1980 <simon.zehnder@gmail.com>
    @simonsays1980
    simonsays1980 committed Sep 6, 2024
    Configuration menu
    Copy the full SHA
    c6a862d View commit details
    Browse the repository at this point in the history

Commits on Sep 9, 2024

  1. Merge branch 'master' into offline-store-episodes-in-state-form

    @simonsays1980
    simonsays1980 committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    8ca8411 View commit details
    Browse the repository at this point in the history

  2. fix 

    Signed-off-by: sven1977 <svenmika1977@gmail.com>
    @sven1977
    sven1977 committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    53ba4f5 View commit details
    Browse the repository at this point in the history

  3. LINT 

    Signed-off-by: sven1977 <svenmika1977@gmail.com>
    @sven1977
    sven1977 committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    bc79bc8 View commit details
    Browse the repository at this point in the history

  4. fix 

    Signed-off-by: sven1977 <svenmika1977@gmail.com>
    @sven1977
    sven1977 committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    2cb5a7c View commit details
    Browse the repository at this point in the history

Commits on Sep 10, 2024

  1. Merged master. 

    Signed-off-by: simonsays1980 <simon.zehnder@gmail.com>
    @simonsays1980
    simonsays1980 committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    9be3266 View commit details
    Browse the repository at this point in the history