Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

MARO v0.3: a new design of RL Toolkit, CLI refactorization, and corresponding updates. #539

Merged
merged 579 commits into from
Jun 1, 2022
Merged

MARO v0.3: a new design of RL Toolkit, CLI refactorization, and corresponding updates. #539

merged 579 commits into from
Jun 1, 2022
This pull request is big! We’re only showing the most recent 250 commits.

Commits on May 27, 2021

  1. refined proxy coding style

    ysqyang committed May 27, 2021
    Configuration menu
    Copy the full SHA
    3456b43 View commit details
    Browse the repository at this point in the history

Commits on May 28, 2021

  1. merge v0.2_rl_refinement to get rl toolkit updates

    @Jinyu-W
    Jinyu-W committed May 28, 2021
    Configuration menu
    Copy the full SHA
    fe96364 View commit details
    Browse the repository at this point in the history

  2. updated images and refined doc

    ysqyang committed May 28, 2021
    Configuration menu
    Copy the full SHA
    fed8467 View commit details
    Browse the repository at this point in the history

  3. updated images

    ysqyang committed May 28, 2021
    Configuration menu
    Copy the full SHA
    03bcb41 View commit details
    Browse the repository at this point in the history

Commits on May 31, 2021

  1. Merge branch 'master' into v0.2

    @Jinyu-W
    Jinyu-W committed May 31, 2021
    Configuration menu
    Copy the full SHA
    6ef102f View commit details
    Browse the repository at this point in the history

  2. Merge branch 'v0.2' into v0.2_rl_refinement

    @Jinyu-W
    Jinyu-W committed May 31, 2021
    Configuration menu
    Copy the full SHA
    12ba200 View commit details
    Browse the repository at this point in the history

  3. merge v0.2_rl_refinment

    @Jinyu-W
    Jinyu-W committed May 31, 2021
    Configuration menu
    Copy the full SHA
    fda6ab5 View commit details
    Browse the repository at this point in the history

  4. updated CIM-AC example

    ysqyang committed May 31, 2021
    Configuration menu
    Copy the full SHA
    f044471 View commit details
    Browse the repository at this point in the history

Commits on Jun 1, 2021

  1. refined proxy retry logic

    ysqyang committed Jun 1, 2021
    Configuration menu
    Copy the full SHA
    1692871 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'v0.2_rl_refinement' of github.com:microsoft/maro into v… 

    …0.2_rl_refinement
    ysqyang committed Jun 1, 2021
    Configuration menu
    Copy the full SHA
    e3cbe14 View commit details
    Browse the repository at this point in the history

  3. call policy update only for AbsCorePolicy

    @Jinyu-W
    Jinyu-W committed Jun 1, 2021
    Configuration menu
    Copy the full SHA
    2a3e90d View commit details
    Browse the repository at this point in the history

Commits on Jun 2, 2021

  1. add limitation of AbsCorePolicy in Actor.collect()

    @Jinyu-W
    Jinyu-W committed Jun 2, 2021
    Configuration menu
    Copy the full SHA
    2e925ab View commit details
    Browse the repository at this point in the history

  2. modify the supply_chain example to use the new rl toolkit architecture

    @Jinyu-W
    Jinyu-W committed Jun 2, 2021
    Configuration menu
    Copy the full SHA
    e3710e3 View commit details
    Browse the repository at this point in the history

  3. refined actor to return only experiences for policies that received n… 

    …ew experiences
    ysqyang committed Jun 2, 2021
    Configuration menu
    Copy the full SHA
    03c26b8 View commit details
    Browse the repository at this point in the history

  4. fix MsgKey issue in rollout_manager

    @Jinyu-W
    Jinyu-W committed Jun 2, 2021
    Configuration menu
    Copy the full SHA
    aff0f44 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'v0.2_rl_refinement' into v0.2_sc_0506_updated

    @Jinyu-W
    Jinyu-W committed Jun 2, 2021
    Configuration menu
    Copy the full SHA
    fba2ccf View commit details
    Browse the repository at this point in the history

  6. fix typo in learner

    @Jinyu-W
    Jinyu-W committed Jun 2, 2021
    Configuration menu
    Copy the full SHA
    8c05562 View commit details
    Browse the repository at this point in the history

  7. Merge branch 'v0.2_rl_refinement' into v0.2_sc_0506_updated

    @Jinyu-W
    Jinyu-W committed Jun 2, 2021
    Configuration menu
    Copy the full SHA
    ce155d4 View commit details
    Browse the repository at this point in the history

Commits on Jun 3, 2021

  1. call exit function for parallel rollout manager

    @Jinyu-W
    Jinyu-W committed Jun 3, 2021
    Configuration menu
    Copy the full SHA
    cc0c555 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'v0.2_rl_refinement' into v0.2_sc_0506_updated

    @Jinyu-W
    Jinyu-W committed Jun 3, 2021
    Configuration menu
    Copy the full SHA
    74ede26 View commit details
    Browse the repository at this point in the history

  3. update supply chain example distributed training scripts

    @Jinyu-W
    Jinyu-W committed Jun 3, 2021
    Configuration menu
    Copy the full SHA
    034e5bb View commit details
    Browse the repository at this point in the history

  4. 1. moved exploration scheduling to rollout manager; 2. fixed bug in l… 

    …r schedule registration in core model; 3. added parallel policy manager prorotype
    ysqyang committed Jun 3, 2021
    Configuration menu
    Copy the full SHA
    13d7d9b View commit details
    Browse the repository at this point in the history

  5. fixed merge conflicts

    ysqyang committed Jun 3, 2021
    Configuration menu
    Copy the full SHA
    5337aa9 View commit details
    Browse the repository at this point in the history

  6. reformat render

    @Jinyu-W
    Jinyu-W committed Jun 3, 2021
    Configuration menu
    Copy the full SHA
    6dbfd36 View commit details
    Browse the repository at this point in the history

  7. fix supply chain business engine action type problem

    @Jinyu-W
    Jinyu-W committed Jun 3, 2021
    Configuration menu
    Copy the full SHA
    146eeb6 View commit details
    Browse the repository at this point in the history

  8. reset supply chain example render figsize from 4 to 3

    @Jinyu-W
    Jinyu-W committed Jun 3, 2021
    Configuration menu
    Copy the full SHA
    a80f6c3 View commit details
    Browse the repository at this point in the history

  9. Add render to all modes of supply chain example

    @Jinyu-W
    Jinyu-W committed Jun 3, 2021
    Configuration menu
    Copy the full SHA
    731bc59 View commit details
    Browse the repository at this point in the history

  10. fix or policy typos

    @Jinyu-W
    Jinyu-W committed Jun 3, 2021
    Configuration menu
    Copy the full SHA
    ebe5065 View commit details
    Browse the repository at this point in the history

Commits on Jun 4, 2021

  1. 1. added parallel policy manager prototype; 2. used training ep for e… 

    …valuation episodes
    ysqyang committed Jun 4, 2021
    Configuration menu
    Copy the full SHA
    0549a14 View commit details
    Browse the repository at this point in the history

Commits on Jun 9, 2021

  1. refined parallel policy manager

    ysqyang committed Jun 9, 2021
    Configuration menu
    Copy the full SHA
    023a9d3 View commit details
    Browse the repository at this point in the history

  2. updated rl/__init__/py

    ysqyang committed Jun 9, 2021
    Configuration menu
    Copy the full SHA
    5a57e01 View commit details
    Browse the repository at this point in the history

  3. fixed lint issues and CIM local learner bugs

    ysqyang committed Jun 9, 2021
    Configuration menu
    Copy the full SHA
    1f62f3a View commit details
    Browse the repository at this point in the history

  4. deleted unwanted supply_chain test files

    ysqyang committed Jun 9, 2021
    Configuration menu
    Copy the full SHA
    6208d86 View commit details
    Browse the repository at this point in the history

  5. revised default config for cim-dqn

    ysqyang committed Jun 9, 2021
    Configuration menu
    Copy the full SHA
    11ca4be View commit details
    Browse the repository at this point in the history

Commits on Jun 10, 2021

  1. removed test_store.py as it is no longer needed

    ysqyang committed Jun 10, 2021
    Configuration menu
    Copy the full SHA
    36d4178 View commit details
    Browse the repository at this point in the history

Commits on Jun 11, 2021

  1. 1. changed Actor class to rollout_worker function; 2. renamed algorit… 

    …hm to algorithms
    ysqyang committed Jun 11, 2021
    Configuration menu
    Copy the full SHA
    0fab08b View commit details
    Browse the repository at this point in the history

  2. updated figures

    ysqyang committed Jun 11, 2021
    Configuration menu
    Copy the full SHA
    3b5faeb View commit details
    Browse the repository at this point in the history

  3. removed unwanted import

    ysqyang committed Jun 11, 2021
    Configuration menu
    Copy the full SHA
    7911162 View commit details
    Browse the repository at this point in the history

Commits on Jun 15, 2021

  1. refactored CIM-DQN example

    ysqyang committed Jun 15, 2021
    Configuration menu
    Copy the full SHA
    4f2182f View commit details
    Browse the repository at this point in the history

Commits on Jun 16, 2021

  1. added MultiProcessRolloutManager and MultiProcessTrainingManager

    ysqyang committed Jun 16, 2021
    Configuration menu
    Copy the full SHA
    2b1541b View commit details
    Browse the repository at this point in the history

Commits on Jun 17, 2021

  1. updated doc

    ysqyang committed Jun 17, 2021
    Configuration menu
    Copy the full SHA
    6392fcf View commit details
    Browse the repository at this point in the history

Commits on Jun 18, 2021

  1. lint issue fix

    ysqyang committed Jun 18, 2021
    Configuration menu
    Copy the full SHA
    5089f7c View commit details
    Browse the repository at this point in the history

  2. lint issue fix

    ysqyang committed Jun 18, 2021
    Configuration menu
    Copy the full SHA
    41a7b27 View commit details
    Browse the repository at this point in the history

  3. fixed import formatting

    ysqyang committed Jun 18, 2021
    Configuration menu
    Copy the full SHA
    35cf25a View commit details
    Browse the repository at this point in the history

  4. [Feature] Prioritized Experience Replay (#355) 

    * added prioritized experience replay

* deleted unwanted supply_chain test files

* fixed import order

* import fix

* fixed lint issues

* fixed import formatting

* added note in docstring that rank-based PER has yet to be implemented

Co-authored-by: ysqyang <v-yangqi@microsoft.com>
    @ysqyang
    ysqyang and ysqyang authored Jun 18, 2021
    Configuration menu
    Copy the full SHA
    ceadf4f View commit details
    Browse the repository at this point in the history

  5. rm AbsDecisionGenerator

    ysqyang committed Jun 18, 2021
    Configuration menu
    Copy the full SHA
    248d1e4 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'v0.2_rl_refinement' of github.com:microsoft/maro into v… 

    …0.2_rl_refinement
    ysqyang committed Jun 18, 2021
    Configuration menu
    Copy the full SHA
    721d91b View commit details
    Browse the repository at this point in the history

  7. small fixes

    ysqyang committed Jun 18, 2021
    Configuration menu
    Copy the full SHA
    85e304a View commit details
    Browse the repository at this point in the history

  8. bug fix

    ysqyang committed Jun 18, 2021
    Configuration menu
    Copy the full SHA
    2601970 View commit details
    Browse the repository at this point in the history

Commits on Jun 20, 2021

  1. reorganized training folder structure

    ysqyang committed Jun 20, 2021
    Configuration menu
    Copy the full SHA
    f72e884 View commit details
    Browse the repository at this point in the history

  2. fixed lint issues

    ysqyang committed Jun 20, 2021
    Configuration menu
    Copy the full SHA
    4f4d5bb View commit details
    Browse the repository at this point in the history

  3. fixed lint issues

    ysqyang committed Jun 20, 2021
    Configuration menu
    Copy the full SHA
    96b9cce View commit details
    Browse the repository at this point in the history

Commits on Jun 21, 2021

  1. policy manager refined

    ysqyang committed Jun 21, 2021
    Configuration menu
    Copy the full SHA
    78c225a View commit details
    Browse the repository at this point in the history

  2. lint fix

    ysqyang committed Jun 21, 2021
    Configuration menu
    Copy the full SHA
    9acae80 View commit details
    Browse the repository at this point in the history

  3. restructured CIM-dqn sync code

    ysqyang committed Jun 21, 2021
    Configuration menu
    Copy the full SHA
    424cabb View commit details
    Browse the repository at this point in the history

Commits on Jun 22, 2021

  1. added policy version index and used it as a measure of experience sta… 

    …leness
    ysqyang committed Jun 22, 2021
    Configuration menu
    Copy the full SHA
    18f73f2 View commit details
    Browse the repository at this point in the history

  2. lint issue fix

    ysqyang committed Jun 22, 2021
    Configuration menu
    Copy the full SHA
    49d93c2 View commit details
    Browse the repository at this point in the history

  3. lint issue fix

    ysqyang committed Jun 22, 2021
    Configuration menu
    Copy the full SHA
    bc96c5e View commit details
    Browse the repository at this point in the history

  4. switched log_dir and proxy_kwargs order

    ysqyang committed Jun 22, 2021
    Configuration menu
    Copy the full SHA
    1bb4b56 View commit details
    Browse the repository at this point in the history

Commits on Jun 23, 2021

  1. cim example refinement

    ysqyang committed Jun 23, 2021
    Configuration menu
    Copy the full SHA
    20c6385 View commit details
    Browse the repository at this point in the history

Commits on Jun 28, 2021

  1. eval schedule sorted only when it's a list

    ysqyang committed Jun 28, 2021
    Configuration menu
    Copy the full SHA
    42c24ab View commit details
    Browse the repository at this point in the history

  2. eval schedule sorted only when it's a list

    ysqyang committed Jun 28, 2021
    Configuration menu
    Copy the full SHA
    8db90d5 View commit details
    Browse the repository at this point in the history

  3. update sc env wrapper

    @Jinyu-W
    Jinyu-W committed Jun 28, 2021
    Configuration menu
    Copy the full SHA
    81f574a View commit details
    Browse the repository at this point in the history

  4. added docker scripts for cim-dqn

    ysqyang committed Jun 28, 2021
    Configuration menu
    Copy the full SHA
    5ad21e4 View commit details
    Browse the repository at this point in the history

Commits on Jun 29, 2021

  1. Merge branch 'master' into v0.2

    @Jinyu-W
    Jinyu-W committed Jun 29, 2021
    Configuration menu
    Copy the full SHA
    bb25e71 View commit details
    Browse the repository at this point in the history

  2. refactored example folder structure and added workflow templates

    ysqyang committed Jun 29, 2021
    Configuration menu
    Copy the full SHA
    a56d4c2 View commit details
    Browse the repository at this point in the history

  3. fixed merge conflicts

    ysqyang committed Jun 29, 2021
    Configuration menu
    Copy the full SHA
    2525327 View commit details
    Browse the repository at this point in the history

Commits on Jun 30, 2021

  1. fixed lint issues

    ysqyang committed Jun 30, 2021
    Configuration menu
    Copy the full SHA
    f427b07 View commit details
    Browse the repository at this point in the history

  2. fixed lint issues

    ysqyang committed Jun 30, 2021
    Configuration menu
    Copy the full SHA
    b8dc7e4 View commit details
    Browse the repository at this point in the history

  3. fixed template bugs

    ysqyang committed Jun 30, 2021
    Configuration menu
    Copy the full SHA
    92a51da View commit details
    Browse the repository at this point in the history

  4. removed unused imports

    ysqyang committed Jun 30, 2021
    Configuration menu
    Copy the full SHA
    31b68f3 View commit details
    Browse the repository at this point in the history

  5. refactoring sc in progress

    ysqyang committed Jun 30, 2021
    Configuration menu
    Copy the full SHA
    bab8128 View commit details
    Browse the repository at this point in the history

  6. simplified cim meta

    ysqyang committed Jun 30, 2021
    Configuration menu
    Copy the full SHA
    f964924 View commit details
    Browse the repository at this point in the history

  7. updated sc code

    ysqyang committed Jun 30, 2021
    Configuration menu
    Copy the full SHA
    f9ccf2a View commit details
    Browse the repository at this point in the history

  8. fixed build.sh path bug

    ysqyang committed Jun 30, 2021
    Configuration menu
    Copy the full SHA
    5ad3e54 View commit details
    Browse the repository at this point in the history

  9. refined sc and template code

    ysqyang committed Jun 30, 2021
    Configuration menu
    Copy the full SHA
    916d8ad View commit details
    Browse the repository at this point in the history

  10. template refinement

    ysqyang committed Jun 30, 2021
    Configuration menu
    Copy the full SHA
    06c1cd3 View commit details
    Browse the repository at this point in the history

  11. fixed merge conflicts

    ysqyang committed Jun 30, 2021
    Configuration menu
    Copy the full SHA
    c17557c View commit details
    Browse the repository at this point in the history

Commits on Jul 1, 2021

  1. deleted obsolete svgs

    ysqyang committed Jul 1, 2021
    Configuration menu
    Copy the full SHA
    ff76caa View commit details
    Browse the repository at this point in the history

  2. merged with remote

    ysqyang committed Jul 1, 2021
    Configuration menu
    Copy the full SHA
    4842d16 View commit details
    Browse the repository at this point in the history

  3. updated learner logs

    ysqyang committed Jul 1, 2021
    Configuration menu
    Copy the full SHA
    35e55a7 View commit details
    Browse the repository at this point in the history

  4. minor edits

    ysqyang committed Jul 1, 2021
    Configuration menu
    Copy the full SHA
    ae1e93f View commit details
    Browse the repository at this point in the history

  5. refactored templates for easy merge with async PR

    ysqyang committed Jul 1, 2021
    Configuration menu
    Copy the full SHA
    04c53e6 View commit details
    Browse the repository at this point in the history

  6. added component names for rollout manager and policy manager

    ysqyang committed Jul 1, 2021
    Configuration menu
    Copy the full SHA
    1315f04 View commit details
    Browse the repository at this point in the history

  7. fixed incorrect position to add last episode to eval schedule

    ysqyang committed Jul 1, 2021
    Configuration menu
    Copy the full SHA
    de40647 View commit details
    Browse the repository at this point in the history

  8. added max_lag option in templates

    ysqyang committed Jul 1, 2021
    Configuration menu
    Copy the full SHA
    360240f View commit details
    Browse the repository at this point in the history

  9. formatting edit in docker_compose_yml script

    ysqyang committed Jul 1, 2021
    Configuration menu
    Copy the full SHA
    315e85f View commit details
    Browse the repository at this point in the history

  10. moved local learner and early stopper outside sync_tools

    ysqyang committed Jul 1, 2021
    Configuration menu
    Copy the full SHA
    953c873 View commit details
    Browse the repository at this point in the history

  11. refactored rl toolkit folder structure

    ysqyang committed Jul 1, 2021
    Configuration menu
    Copy the full SHA
    ed9d44a View commit details
    Browse the repository at this point in the history

Commits on Jul 2, 2021

  1. refactored rl toolkit folder structure

    ysqyang committed Jul 2, 2021
    Configuration menu
    Copy the full SHA
    d2b433e View commit details
    Browse the repository at this point in the history

  2. moved env_wrapper and agent_wrapper inside rl/learner

    ysqyang committed Jul 2, 2021
    Configuration menu
    Copy the full SHA
    9f799d4 View commit details
    Browse the repository at this point in the history

  3. refined scripts

    ysqyang committed Jul 2, 2021
    Configuration menu
    Copy the full SHA
    f8cccca View commit details
    Browse the repository at this point in the history

  4. modified sc imports according to changes in rl toolkit folder structure

    ysqyang committed Jul 2, 2021
    Configuration menu
    Copy the full SHA
    a4491a7 View commit details
    Browse the repository at this point in the history

  5. fixed typo in script

    ysqyang committed Jul 2, 2021
    Configuration menu
    Copy the full SHA
    0906577 View commit details
    Browse the repository at this point in the history

  6. changes needed for running sc

    ysqyang committed Jul 2, 2021
    Configuration menu
    Copy the full SHA
    a13322b View commit details
    Browse the repository at this point in the history

  7. removed unwanted imports

    ysqyang committed Jul 2, 2021
    Configuration menu
    Copy the full SHA
    8ec0282 View commit details
    Browse the repository at this point in the history

  8. Merge branch 'v0.2_rl_refinement' into v0.2_rl_refinement_sc

    ysqyang committed Jul 2, 2021
    Configuration menu
    Copy the full SHA
    56af26a View commit details
    Browse the repository at this point in the history

  9. config change for testing sc scenario

    ysqyang committed Jul 2, 2021
    Configuration menu
    Copy the full SHA
    894c376 View commit details
    Browse the repository at this point in the history

Commits on Jul 4, 2021

  1. changes for perf testing

    ysqyang committed Jul 4, 2021
    Configuration menu
    Copy the full SHA
    743e9f3 View commit details
    Browse the repository at this point in the history

Commits on Jul 5, 2021

  1. Asynchronous Training (#364) 

    * remote inference code draft

* changed actor to rollout_worker and updated init files

* removed unwanted import

* updated inits

* more async code

* added async scripts

* added async training code & scripts for CIM-dqn

* changed async to async_tools to avoid conflict with python keyword

* reverted unwanted change to dockerfile

* added doc for policy server

* addressed PR comments and fixed a bug in docker_compose_yml.py

* fixed lint issue

* resolved PR comment

* resolved merge conflicts

* added async templates

* added proxy.close() for actor and policy_server

* fixed incorrect position to add last episode to eval schedule

* reverted unwanted changes

* added missing async files

* rm unwanted echo in kill.sh

Co-authored-by: ysqyang <v-yangqi@microsoft.com>
    @ysqyang
    ysqyang and ysqyang authored Jul 5, 2021
    Configuration menu
    Copy the full SHA
    8e97adc View commit details
    Browse the repository at this point in the history

  2. renamed sync to synchronous and async to asynchronous to avoid confli… 

    …ct with keyword
    ysqyang committed Jul 5, 2021
    Configuration menu
    Copy the full SHA
    fac6006 View commit details
    Browse the repository at this point in the history

  3. fixed merge conflicts

    ysqyang committed Jul 5, 2021
    Configuration menu
    Copy the full SHA
    0004dfe View commit details
    Browse the repository at this point in the history

  4. added missing policy version increment in LocalPolicyManager

    ysqyang committed Jul 5, 2021
    Configuration menu
    Copy the full SHA
    60a7423 View commit details
    Browse the repository at this point in the history

  5. Merge remote-tracking branch 'origin/v0.2_rl_refinement' into v0.2_rl… 

    …_refinement_sc
    ysqyang committed Jul 5, 2021
    Configuration menu
    Copy the full SHA
    c004697 View commit details
    Browse the repository at this point in the history

  6. refined rollout manager recv logic

    ysqyang committed Jul 5, 2021
    Configuration menu
    Copy the full SHA
    a163554 View commit details
    Browse the repository at this point in the history

  7. removed a debugging print

    ysqyang committed Jul 5, 2021
    Configuration menu
    Copy the full SHA
    803faad View commit details
    Browse the repository at this point in the history

  8. moved supply_chain inside examples/rl

    ysqyang committed Jul 5, 2021
    Configuration menu
    Copy the full SHA
    0c10f36 View commit details
    Browse the repository at this point in the history

Commits on Jul 6, 2021

  1. added sleep in distributed launcher to avoid hanging

    ysqyang committed Jul 6, 2021
    Configuration menu
    Copy the full SHA
    34b47a5 View commit details
    Browse the repository at this point in the history

Commits on Jul 7, 2021

  1. updated api doc and rl toolkit doc

    ysqyang committed Jul 7, 2021
    Configuration menu
    Copy the full SHA
    c41ca35 View commit details
    Browse the repository at this point in the history

  2. refined dynamic imports using importlib

    yaqiu committed Jul 7, 2021
    Configuration menu
    Copy the full SHA
    a2244b5 View commit details
    Browse the repository at this point in the history

Commits on Jul 8, 2021

  1. Merge branch 'master' into v0.2

    Jinyu Wang authored and Jinyu Wang committed Jul 8, 2021
    Configuration menu
    Copy the full SHA
    edf9df4 View commit details
    Browse the repository at this point in the history

  2. 1. moved policy update triggers to policy manager; 2. added version c… 

    …ontrol in policy manager
    yaqiu committed Jul 8, 2021
    Configuration menu
    Copy the full SHA
    740efa7 View commit details
    Browse the repository at this point in the history

  3. fixed a few bugs and updated cim RL example

    yaqiu committed Jul 8, 2021
    Configuration menu
    Copy the full SHA
    c278693 View commit details
    Browse the repository at this point in the history

  4. fixed a few more bugs

    yaqiu committed Jul 8, 2021
    Configuration menu
    Copy the full SHA
    455751a View commit details
    Browse the repository at this point in the history

  5. resolved merge conflicts

    yaqiu committed Jul 8, 2021
    Configuration menu
    Copy the full SHA
    ef50957 View commit details
    Browse the repository at this point in the history

Commits on Jul 9, 2021

  1. Merge remote-tracking branch 'origin/v0.2' into v0.2_rl_refinement

    yaqiu committed Jul 9, 2021
    Configuration menu
    Copy the full SHA
    9a04a99 View commit details
    Browse the repository at this point in the history

  2. added agent wrapper instantiation to workflows

    yaqiu committed Jul 9, 2021
    Configuration menu
    Copy the full SHA
    746f0f9 View commit details
    Browse the repository at this point in the history

  3. added agent wrapper instantiation to workflows

    yaqiu committed Jul 9, 2021
    Configuration menu
    Copy the full SHA
    18cd676 View commit details
    Browse the repository at this point in the history

  4. removed abs_block and added max_prob option for DiscretePolicyNet and… 

    … DiscreteACNet
    yaqiu committed Jul 9, 2021
    Configuration menu
    Copy the full SHA
    c5cf9df View commit details
    Browse the repository at this point in the history

  5. fixed incorrect get_ac_policy signature for CIM

    yaqiu committed Jul 9, 2021
    Configuration menu
    Copy the full SHA
    1f3b590 View commit details
    Browse the repository at this point in the history

  6. moved exploration inside core policy

    yaqiu committed Jul 9, 2021
    Configuration menu
    Copy the full SHA
    dd017d3 View commit details
    Browse the repository at this point in the history

Commits on Jul 11, 2021

  1. added state to exploration call to support context-dependent exploration

    yaqiu committed Jul 11, 2021
    Configuration menu
    Copy the full SHA
    98d0961 View commit details
    Browse the repository at this point in the history

  2. updated sc example according to RL toolkit changes

    yaqiu committed Jul 11, 2021
    Configuration menu
    Copy the full SHA
    17f1655 View commit details
    Browse the repository at this point in the history

  3. separated non_rl_policy_index and rl_policy_index in workflows

    yaqiu committed Jul 11, 2021
    Configuration menu
    Copy the full SHA
    bbb6ba7 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'v0.2_rl_refinement' into v0.2_rl_refinement_sc

    yaqiu committed Jul 11, 2021
    Configuration menu
    Copy the full SHA
    c70105d View commit details
    Browse the repository at this point in the history

  5. modified sc example code according to workflow changes

    yaqiu committed Jul 11, 2021
    Configuration menu
    Copy the full SHA
    f004fba View commit details
    Browse the repository at this point in the history

  6. modified sc example code according to workflow changes

    yaqiu committed Jul 11, 2021
    Configuration menu
    Copy the full SHA
    2be9114 View commit details
    Browse the repository at this point in the history

Commits on Jul 12, 2021

  1. added replay_agent_ids parameter to get_env_func for RL examples

    yaqiu committed Jul 12, 2021
    Configuration menu
    Copy the full SHA
    9b04ad5 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'v0.2_rl_refinement' into v0.2_rl_refinement_sc

    yaqiu committed Jul 12, 2021
    Configuration menu
    Copy the full SHA
    c004323 View commit details
    Browse the repository at this point in the history

  3. fixed a few bugs

    yaqiu committed Jul 12, 2021
    Configuration menu
    Copy the full SHA
    700b149 View commit details
    Browse the repository at this point in the history

  4. added maro/simulator/scenarios/supply_chain as bind mount

    yaqiu committed Jul 12, 2021
    Configuration menu
    Copy the full SHA
    b9afaef View commit details
    Browse the repository at this point in the history

Commits on Jul 14, 2021

  1. added post-step, post-collect, post-eval and post-update callbacks

    yaqiu committed Jul 14, 2021
    Configuration menu
    Copy the full SHA
    87066c9 View commit details
    Browse the repository at this point in the history

  2. fixed lint issues

    yaqiu committed Jul 14, 2021
    Configuration menu
    Copy the full SHA
    f0a29ef View commit details
    Browse the repository at this point in the history

  3. fixed lint issues

    yaqiu committed Jul 14, 2021
    Configuration menu
    Copy the full SHA
    56fd2d6 View commit details
    Browse the repository at this point in the history

Commits on Jul 15, 2021

  1. fixed some bugs

    yaqiu committed Jul 15, 2021
    Configuration menu
    Copy the full SHA
    cb533fa View commit details
    Browse the repository at this point in the history

  2. moved instantiation of policy manager inside simple learner

    yaqiu committed Jul 15, 2021
    Configuration menu
    Copy the full SHA
    d2d66cd View commit details
    Browse the repository at this point in the history

  3. Merge branch 'v0.2_rl_refinement' into v0.2_rl_refinement_sc

    yaqiu committed Jul 15, 2021
    Configuration menu
    Copy the full SHA
    513ca40 View commit details
    Browse the repository at this point in the history

  4. fixed env_wrapper get_reward signature

    yaqiu committed Jul 15, 2021
    Configuration menu
    Copy the full SHA
    07fba7a View commit details
    Browse the repository at this point in the history

  5. minor edits

    yaqiu committed Jul 15, 2021
    Configuration menu
    Copy the full SHA
    a9e6b11 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'v0.2_rl_refinement' into v0.2_rl_refinement_sc

    yaqiu committed Jul 15, 2021
    Configuration menu
    Copy the full SHA
    1d5c242 View commit details
    Browse the repository at this point in the history

  7. removed get_eperience kwargs from env_wrapper

    yaqiu committed Jul 15, 2021
    Configuration menu
    Copy the full SHA
    8a84b11 View commit details
    Browse the repository at this point in the history

  8. Merge branch 'v0.2_rl_refinement' into v0.2_rl_refinement_sc

    yaqiu committed Jul 15, 2021
    Configuration menu
    Copy the full SHA
    2cc0f7b View commit details
    Browse the repository at this point in the history

  9. 1. renamed step_callback to post_step in env_wrapper; 2. added get_ev… 

    …al_env_func to RL workflows
    yaqiu committed Jul 15, 2021
    Configuration menu
    Copy the full SHA
    ec338fb View commit details
    Browse the repository at this point in the history

  10. Merge branch 'v0.2_rl_refinement' into v0.2_rl_refinement_sc

    yaqiu committed Jul 15, 2021
    Configuration menu
    Copy the full SHA
    8f00dc7 View commit details
    Browse the repository at this point in the history

  11. added rollout exp disribution option in RL examples

    yaqiu committed Jul 15, 2021
    Configuration menu
    Copy the full SHA
    1c94b62 View commit details
    Browse the repository at this point in the history

  12. fixed merge conflicts

    yaqiu committed Jul 15, 2021
    Configuration menu
    Copy the full SHA
    2b04cc0 View commit details
    Browse the repository at this point in the history

Commits on Jul 16, 2021

  1. Merge branch 'v0.2_sc_0506_updated' into v0.2_sc

    @lihuoran
    lihuoran committed Jul 16, 2021
    Configuration menu
    Copy the full SHA
    36092c2 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'v0.2_sc' into v0.2_rl_refinement_sc

    @lihuoran
    lihuoran committed Jul 16, 2021
    Configuration menu
    Copy the full SHA
    b4f5afa View commit details
    Browse the repository at this point in the history

  3. removed unwanted files

    yaqiu committed Jul 16, 2021
    Configuration menu
    Copy the full SHA
    4252a0c View commit details
    Browse the repository at this point in the history

  4. 1. made logger internal in learner; 2 removed logger creation in abs … 

    …classes
    yaqiu committed Jul 16, 2021
    Configuration menu
    Copy the full SHA
    c2c1c62 View commit details
    Browse the repository at this point in the history

  5. fixed merge conflicts

    yaqiu committed Jul 16, 2021
    Configuration menu
    Copy the full SHA
    be82de3 View commit details
    Browse the repository at this point in the history

  6. checked out supply chain test files from v0.2_sc

    yaqiu committed Jul 16, 2021
    Configuration menu
    Copy the full SHA
    0a05184 View commit details
    Browse the repository at this point in the history

Commits on Jul 19, 2021

  1. 1. added missing model.eval() to choose_action; 2.added entropy featu… 

    …res to AC
    yaqiu committed Jul 19, 2021
    Configuration menu
    Copy the full SHA
    c7bca77 View commit details
    Browse the repository at this point in the history

  2. fixed a bug in ac entropy

    yaqiu committed Jul 19, 2021
    Configuration menu
    Copy the full SHA
    81245dd View commit details
    Browse the repository at this point in the history

  3. abbreviated coefficient to coeff

    yaqiu committed Jul 19, 2021
    Configuration menu
    Copy the full SHA
    e56e9b8 View commit details
    Browse the repository at this point in the history

Commits on Jul 22, 2021

  1. removed -dqn from job name in rl example config

    yaqiu committed Jul 22, 2021
    Configuration menu
    Copy the full SHA
    072d9de View commit details
    Browse the repository at this point in the history

  2. added tmp patch to dev.df

    yaqiu committed Jul 22, 2021
    Configuration menu
    Copy the full SHA
    103eb40 View commit details
    Browse the repository at this point in the history

  3. renamed image name for running rl examples

    yaqiu committed Jul 22, 2021
    Configuration menu
    Copy the full SHA
    9c5e135 View commit details
    Browse the repository at this point in the history

Commits on Jul 28, 2021

  1. added get_loss interface for core policies

    yaqiu committed Jul 28, 2021
    Configuration menu
    Copy the full SHA
    d96aa44 View commit details
    Browse the repository at this point in the history

Commits on Jul 30, 2021

  1. added policy manager in rl_toolkit.rst

    yaqiu committed Jul 30, 2021
    Configuration menu
    Copy the full SHA
    1f37369 View commit details
    Browse the repository at this point in the history

  2. 1. env_wrapper bug fix; 2. policy manager update logic refinement

    yaqiu committed Jul 30, 2021
    Configuration menu
    Copy the full SHA
    12ac058 View commit details
    Browse the repository at this point in the history

Commits on Aug 3, 2021

  1. refactored policy and algorithms

    yaqiu committed Aug 3, 2021
    Configuration menu
    Copy the full SHA
    fc14e66 View commit details
    Browse the repository at this point in the history

Commits on Aug 5, 2021

  1. policy interface redesigned

    ysqyang committed Aug 5, 2021
    Configuration menu
    Copy the full SHA
    7702eba View commit details
    Browse the repository at this point in the history

Commits on Aug 8, 2021

  1. refined policy interfaces

    ysqyang committed Aug 8, 2021
    Configuration menu
    Copy the full SHA
    704c17f View commit details
    Browse the repository at this point in the history

  2. fixed typo

    yaqiu committed Aug 8, 2021
    Configuration menu
    Copy the full SHA
    56a54cb View commit details
    Browse the repository at this point in the history

Commits on Aug 9, 2021

  1. fixed bugs in refactored policy interface

    yaqiu committed Aug 9, 2021
    Configuration menu
    Copy the full SHA
    0b57d70 View commit details
    Browse the repository at this point in the history

  2. fixed some bugs

    yaqiu committed Aug 9, 2021
    Configuration menu
    Copy the full SHA
    cad2872 View commit details
    Browse the repository at this point in the history

Commits on Aug 11, 2021

  1. refactoring in progress

    yaqiu committed Aug 11, 2021
    Configuration menu
    Copy the full SHA
    3ba96d4 View commit details
    Browse the repository at this point in the history

Commits on Aug 17, 2021

  1. policy interface and policy manager redesigned

    yaqiu committed Aug 17, 2021
    Configuration menu
    Copy the full SHA
    5f6c47c View commit details
    Browse the repository at this point in the history

Commits on Aug 18, 2021

  1. 1. fixed bugs in ac and pg; 2. fixed bugs rl workflow scripts

    yaqiu committed Aug 18, 2021
    Configuration menu
    Copy the full SHA
    cb8a355 View commit details
    Browse the repository at this point in the history

  2. fixed bug in distributed policy manager

    yaqiu committed Aug 18, 2021
    Configuration menu
    Copy the full SHA
    f0222a7 View commit details
    Browse the repository at this point in the history

  3. fixed lint issues

    yaqiu committed Aug 18, 2021
    Configuration menu
    Copy the full SHA
    c0a8480 View commit details
    Browse the repository at this point in the history

  4. fixed lint issues

    yaqiu committed Aug 18, 2021
    Configuration menu
    Copy the full SHA
    3a10544 View commit details
    Browse the repository at this point in the history

  5. added scipy in setup

    yaqiu committed Aug 18, 2021
    Configuration menu
    Copy the full SHA
    026bcd3 View commit details
    Browse the repository at this point in the history

Commits on Aug 19, 2021

  1. 1. trimmed rollout manager code; 2. added option to docker scripts

    yaqiu committed Aug 19, 2021
    Configuration menu
    Copy the full SHA
    00df5d8 View commit details
    Browse the repository at this point in the history

Commits on Aug 20, 2021

  1. updated api doc for policy manager

    yaqiu committed Aug 20, 2021
    Configuration menu
    Copy the full SHA
    8619408 View commit details
    Browse the repository at this point in the history

Commits on Aug 23, 2021

  1. 1. simplified rl/learning code structure; 2. fixed bugs in rl example… 

    … docker script
    yaqiu committed Aug 23, 2021
    Configuration menu
    Copy the full SHA
    ca7b0d9 View commit details
    Browse the repository at this point in the history

  2. 1. simplified rl example structure; 2. fixed lint issues

    yaqiu committed Aug 23, 2021
    Configuration menu
    Copy the full SHA
    aefd3b5 View commit details
    Browse the repository at this point in the history

Commits on Aug 25, 2021

  1. further rl toolkit code simplifications

    yaqiu committed Aug 25, 2021
    Configuration menu
    Copy the full SHA
    db99ce2 View commit details
    Browse the repository at this point in the history

Commits on Aug 26, 2021

  1. more numpy-based optimization in RL toolkit

    yaqiu committed Aug 26, 2021
    Configuration menu
    Copy the full SHA
    b3a244d View commit details
    Browse the repository at this point in the history

Commits on Aug 27, 2021

  1. moved replay buffer inside policy

    yaqiu committed Aug 27, 2021
    Configuration menu
    Copy the full SHA
    505cf4e View commit details
    Browse the repository at this point in the history

  2. bug fixes

    yaqiu committed Aug 27, 2021
    Configuration menu
    Copy the full SHA
    af1eed6 View commit details
    Browse the repository at this point in the history

Commits on Aug 29, 2021

  1. numpy optimization and associated refactoring

    yaqiu committed Aug 29, 2021
    Configuration menu
    Copy the full SHA
    e924495 View commit details
    Browse the repository at this point in the history

Commits on Aug 31, 2021

  1. extracted shaping logic out of env_sampler

    ysqyang committed Aug 31, 2021
    Configuration menu
    Copy the full SHA
    7c407a4 View commit details
    Browse the repository at this point in the history

  2. fixed bug in CIM shaping and lint issues

    yaqiu committed Aug 31, 2021
    Configuration menu
    Copy the full SHA
    07a051b View commit details
    Browse the repository at this point in the history

Commits on Sep 1, 2021

  1. preliminary implemetation of parallel batch inference

    yaqiu committed Sep 1, 2021
    Configuration menu
    Copy the full SHA
    6a027fa View commit details
    Browse the repository at this point in the history

Commits on Sep 2, 2021

  1. fixed bug in ddpg transition recording

    ysqyang committed Sep 2, 2021
    Configuration menu
    Copy the full SHA
    fde7895 View commit details
    Browse the repository at this point in the history

  2. put get_state, get_env_actions, get_reward back in EnvSampler

    yaqiu committed Sep 2, 2021
    Configuration menu
    Copy the full SHA
    b9010ef View commit details
    Browse the repository at this point in the history

Commits on Sep 5, 2021

  1. simplified exploration and core model interfaces

    yaqiu committed Sep 5, 2021
    Configuration menu
    Copy the full SHA
    aa69409 View commit details
    Browse the repository at this point in the history

Commits on Sep 6, 2021

  1. bug fixes and doc update

    yaqiu committed Sep 6, 2021
    Configuration menu
    Copy the full SHA
    2dbf3c3 View commit details
    Browse the repository at this point in the history

  2. added improve() interface for RLPolicy for single-thread support

    yaqiu committed Sep 6, 2021
    Configuration menu
    Copy the full SHA
    f136e3c View commit details
    Browse the repository at this point in the history

Commits on Sep 7, 2021

  1. fixed simple policy manager bug

    yaqiu committed Sep 7, 2021
    Configuration menu
    Copy the full SHA
    92561f6 View commit details
    Browse the repository at this point in the history

Commits on Sep 11, 2021

  1. updated doc, rst, notebook

    yaqiu committed Sep 11, 2021
    Configuration menu
    Copy the full SHA
    013d0fb View commit details
    Browse the repository at this point in the history

  2. updated notebook

    yaqiu committed Sep 11, 2021
    Configuration menu
    Copy the full SHA
    8f652b4 View commit details
    Browse the repository at this point in the history

  3. fixed lint issues

    yaqiu committed Sep 11, 2021
    Configuration menu
    Copy the full SHA
    8dd708f View commit details
    Browse the repository at this point in the history

Commits on Sep 12, 2021

  1. fixed entropy bugs in ac.py

    yaqiu committed Sep 12, 2021
    Configuration menu
    Copy the full SHA
    971fd04 View commit details
    Browse the repository at this point in the history

  2. reverted to simple policy manager as default

    yaqiu committed Sep 12, 2021
    Configuration menu
    Copy the full SHA
    bf3cadb View commit details
    Browse the repository at this point in the history

Commits on Sep 14, 2021

  1. 1. unified single-thread and distributed mode in learning_loop.py; 2.… 

    … updated api doc for algorithms and rst for rl toolkit
    yaqiu committed Sep 14, 2021
    Configuration menu
    Copy the full SHA
    e89b6db View commit details
    Browse the repository at this point in the history

  2. fixed lint issues and updated rl toolkit images

    yaqiu committed Sep 14, 2021
    Configuration menu
    Copy the full SHA
    3738bd1 View commit details
    Browse the repository at this point in the history

Commits on Sep 15, 2021

  1. removed obsolete images

    ysqyang committed Sep 15, 2021
    Configuration menu
    Copy the full SHA
    69c5a56 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'v0.2_rl_refinement' of github.com:microsoft/maro into v… 

    …0.2_rl_refinement
    ysqyang committed Sep 15, 2021
    Configuration menu
    Copy the full SHA
    372d44c View commit details
    Browse the repository at this point in the history

  3. added back agent2policy for general workflow use

    yaqiu committed Sep 15, 2021
    Configuration menu
    Copy the full SHA
    9030200 View commit details
    Browse the repository at this point in the history

Commits on Sep 16, 2021

  1. V0.2 rl refinement dist (#377) 

    * Support `slice` operation in ExperienceSet

* Support naive distributed policy training by proxy

* Dynamically allocate trainers according to number of experience

* code check

* code check

* code check

* Fix a bug in distributed trianing with no gradient

* Code check

* Move Back-Propagation from trainer to policy_manager and extract trainer-allocation strategy

* 1.call allocate_trainer() at first of update(); 2.refine according to code review

* Code check

* Refine code with new interface

* Update docs of PolicyManger and ExperienceSet

* Add images for rl_toolkit docs

* Update diagram of PolicyManager

* Refine with new interface

* Extract allocation strategy into `allocation_strategy.py`

* add `distributed_learn()` in policies for data-parallel training

* Update doc of RL_toolkit

* Add gradient workers for data-parallel

* Refine code and update docs

* Lint check

* Refine by comments

* Rename `trainer` to `worker`

* Rename `distributed_learn` to `learn_with_data_parallel`

* Refine allocator and remove redundant code in policy_manager

* remove arugments in allocate_by_policy and so on
    @buptchan
    buptchan authored Sep 16, 2021
    Configuration menu
    Copy the full SHA
    f2dd5c0 View commit details
    Browse the repository at this point in the history

Commits on Sep 17, 2021

  1. Merge branch 'v0.2_rl_refinement' of github.com:microsoft/maro into v… 

    …0.2_rl_refinement
    ysqyang committed Sep 17, 2021
    Configuration menu
    Copy the full SHA
    1a70410 View commit details
    Browse the repository at this point in the history

  2. added checkpointing for simple and multi-process policy managers

    ysqyang committed Sep 17, 2021
    Configuration menu
    Copy the full SHA
    5f70b65 View commit details
    Browse the repository at this point in the history

  3. 1. bug fixes in checkpointing; 2. removed version and max_lag in roll… 

    …out manager
    yaqiu committed Sep 17, 2021
    Configuration menu
    Copy the full SHA
    7b76dce View commit details
    Browse the repository at this point in the history

  4. added missing set_state and get_state for CIM policies

    yaqiu committed Sep 17, 2021
    Configuration menu
    Copy the full SHA
    c1d9871 View commit details
    Browse the repository at this point in the history

  5. removed blank line

    yaqiu committed Sep 17, 2021
    Configuration menu
    Copy the full SHA
    fc59379 View commit details
    Browse the repository at this point in the history

Commits on Sep 22, 2021

  1. updated RL workflow README

    yaqiu committed Sep 22, 2021
    Configuration menu
    Copy the full SHA
    f23fdc6 View commit details
    Browse the repository at this point in the history

  2. Integrate data_parallel arguments into worker_allocator (#402)

    @buptchan
    buptchan authored Sep 22, 2021
    Configuration menu
    Copy the full SHA
    78a2cb8 View commit details
    Browse the repository at this point in the history

  3. 1. simplified workflow config; 2. added comments to CIM shaping

    yaqiu committed Sep 22, 2021
    Configuration menu
    Copy the full SHA
    f8f2e6a View commit details
    Browse the repository at this point in the history

  4. lint issue fix

    yaqiu committed Sep 22, 2021
    Configuration menu
    Copy the full SHA
    0b5fcd1 View commit details
    Browse the repository at this point in the history

  5. 1. added algorithm type setting in CIM config; 2. added try-except cl… 

    …ause for initial policy state loading
    yaqiu committed Sep 22, 2021
    Configuration menu
    Copy the full SHA
    190802f View commit details
    Browse the repository at this point in the history

Commits on Sep 24, 2021

  1. 1. moved post_step callback inside env sampler; 2. updated README for… 

    … rl workflows
    yaqiu committed Sep 24, 2021
    Configuration menu
    Copy the full SHA
    6e941a4 View commit details
    Browse the repository at this point in the history

  2. refined READEME for CIM

    yaqiu committed Sep 24, 2021
    Configuration menu
    Copy the full SHA
    1edd4c4 View commit details
    Browse the repository at this point in the history

Commits on Sep 26, 2021

  1. VM scheduling with RL (#375) 

    * added part of vm scheduling RL code

* refined vm env_wrapper code style

* added DQN

* added get_experiences func for ac in vm scheduling

* added post_step callback to env wrapper

* moved Aiming's tracking and plotting logic into callbacks

* added eval env wrapper

* renamed AC config variable name for VM

* vm scheduling RL code finished

* updated README

* fixed various bugs and hard coding for vm_scheduling

* uncommented callbacks for VM scheduling

* Minor revision for better code style

* added part of vm scheduling RL code

* refined vm env_wrapper code style

* vm scheduling RL code finished

* added config.py for vm scheduing

* vm example refactoring

* fixed bugs in vm_scheduling

* removed unwanted files from cim dir

* reverted to simple policy manager as default

* added part of vm scheduling RL code

* refined vm env_wrapper code style

* vm scheduling RL code finished

* added config.py for vm scheduing

* resolved rebase conflicts

* fixed bugs in vm_scheduling

* added get_state and set_state to vm_scheduling policy models

* updated README for vm_scheduling with RL

Co-authored-by: yaqiu <v-yaqiu@microsoft.com>
Co-authored-by: Huoran Li <huoranli@microsoft.com>
    @ysqyang @lihuoran
    3 people authored Sep 26, 2021
    Configuration menu
    Copy the full SHA
    2b4d4eb View commit details
    Browse the repository at this point in the history

  2. SC refinement (#397) 

    * Refine test scripts & pending_order_daily logic

* Refactor code for better code style: complete type hint, correct typos, remove unused items.

Refactor code for better code style: complete type hint, correct typos, remove unused items.

* Polish test_supply_chain.py

* update import format

* Modify vehicle steps logic & remove outdated test case

* Optimize imports

* Optimize imports

* Lint error

* Lint error

* Lint error

* Add SupplyChainAction

* Lint error

Co-authored-by: Jinyu Wang <Wang.Jinyu@microsoft.com>
    @lihuoran
    lihuoran and Jinyu Wang authored Sep 26, 2021
    Configuration menu
    Copy the full SHA
    3a928b9 View commit details
    Browse the repository at this point in the history

Commits on Oct 9, 2021

  1. refined workflow scripts

    yaqiu committed Oct 9, 2021
    Configuration menu
    Copy the full SHA
    ea7fdde View commit details
    Browse the repository at this point in the history

  2. fixed bug in ParallelAgentWrapper

    yaqiu committed Oct 9, 2021
    Configuration menu
    Copy the full SHA
    c1f8faf View commit details
    Browse the repository at this point in the history

Commits on Oct 10, 2021

  1. 1. fixed lint issues; 2. refined main script in workflows

    yaqiu committed Oct 10, 2021
    Configuration menu
    Copy the full SHA
    cf1430a View commit details
    Browse the repository at this point in the history

  2. lint issue fix

    yaqiu committed Oct 10, 2021
    Configuration menu
    Copy the full SHA
    485ffd7 View commit details
    Browse the repository at this point in the history

  3. restored default config for rl example

    yaqiu committed Oct 10, 2021
    Configuration menu
    Copy the full SHA
    4e1d37c View commit details
    Browse the repository at this point in the history

  4. Update rollout.py

    @ysqyang
    ysqyang authored Oct 10, 2021
    Configuration menu
    Copy the full SHA
    5b21e67 View commit details
    Browse the repository at this point in the history

Commits on Oct 11, 2021

  1. refined env var processing in policy manager workflow

    yaqiu committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    868bd53 View commit details
    Browse the repository at this point in the history

Commits on Oct 12, 2021

  1. added hasattr check in agent wrapper

    yaqiu committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    12ffd98 View commit details
    Browse the repository at this point in the history

  2. updated docker_compose_yml.py

    yaqiu committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    c0bae0b View commit details
    Browse the repository at this point in the history

Commits on Oct 13, 2021

  1. Minor refinement

    @lihuoran
    lihuoran committed Oct 13, 2021
    Configuration menu
    Copy the full SHA
    a5ddfd5 View commit details
    Browse the repository at this point in the history

Commits on Oct 14, 2021

  1. Merge branch 'v0.2_rl_refinement' into v0.3

    @lihuoran
    lihuoran committed Oct 14, 2021
    Configuration menu
    Copy the full SHA
    0f2f83e View commit details
    Browse the repository at this point in the history

Commits on Dec 6, 2021

  1. Minor PR. Prepare to merge latest master branch into v0.3 branch. (#412) 

    * Prepare to merge master_mirror

* Lint error

* Minor
    @lihuoran
    lihuoran authored Dec 6, 2021
    Configuration menu
    Copy the full SHA
    6a1179c View commit details
    Browse the repository at this point in the history

Commits on Dec 8, 2021

  1. Merge latest master into v0.3 (#426) 

    * update docker hub init (#367)

* update docker hub init

* replace personal account with maro-team

* update hello files for CIM

* update docker repository name

* update docker file name

* fix bugs in notebook, rectify docs

* fix doc build issue

* remove docs from playground; fix citibike lp example Event issue

* update the exampel for vector env

* update vector env example

* update README due to PR comments

* add link to playground above MARO installation in README

* fix some typos

Co-authored-by: Jinyu Wang <Wang.Jinyu@microsoft.com>

* update package version

* update README for package description

* update image links for pypi package description

* update image links for pypi package description

* change the input topology schema for CIM real data mode (#372)

* change the input topology schema for CIM real data mode

* remove unused importing

* update test config file correspondingly

* add Exception for env test

* add cost factors to cim data dump

* update CimDataCollection field name

* update field name of data collection related code

* update package version

* adjust interface to reflect actual signature (#374)

Co-authored-by: Jeremy Reynolds <jeremr@microsoft.com>

* update dataclasses requirement to setup

* fix: fixing spelling grammarr

* fix: fix typo spelling code commented and data_model.rst

* Fix Geo vis IP address & SQL logic bugs. (#383)

Fix Geo vis IP address & SQL logic bugs (issue [352](#352) and [314](#314)).

* Fix the "Wrong future stop tick predictions" bug (#386)

* Propose my new solution

Refine to the pre-process version

.

* Optimize import

* Fix reset random seed bug (#387)

* update the reset interface of Env and BE

* Try to fix reset routes generation seed issue

* Refine random related logics.

* Minor refinement

* Test check

* Minor

* Remove unused functions so far

* Minor

Co-authored-by: Jinyu Wang <jinywan@microsoft.com>

* update package version

* Add _init_vessel_plans in business_engine.reset (#388)

* update package version

* change the default solver used in Citibike OnlineLP example, from GLPK to CBC (#391)

Co-authored-by: Jinyu Wang <Wang.Jinyu@microsoft.com>

* Refine `event_buffer/` module (#389)

* Core & Business Engine code refinement (#392)

* First version

* Optimize imports

* Add typehint

* Lint check

* Lint check

* add higher python version (#398)

* add higher python version

* update pytorch version

* update torchvision version

Co-authored-by: Jinyu Wang <Wang.Jinyu@microsoft.com>

* CIM scenario refinement (#400)

* Cim scenario refinement (#394)

* CIM refinement

* Fix lint error

* Fix lint error

* Cim test coverage (#395)

* Enrich tests

* Refactor CimDataGenerator

* Refactor CIM parsers

* Minor refinement

* Fix lint error

* Fix lint error

* Fix lint error

* Minor refactor

* Type

* Add two test file folders. Make a slight change to CIM BE.

* Lint error

* Lint error

* Remove unnecessary public interfaces of CIM BE

* Cim disable auto action type detection (#399)

* Haven't been tested

* Modify document

* Add ActionType checking

* Minor

* Lint error

* Action quantity should be a position number

* Modify related docs & notebooks

* Minor

* Change test file name. Prepare to merge into master.

* .

* Minor test patch

* Add `clear()` function to class `SimRandom` (#401)

* Add SimRandom.clear()

* Minor

* Remove commented codes

* Lint error

* update package version

* Minor

* Remove docs/source/examples/multi_agent_dqn_cim.rst

* Update .gitignore

* Update .gitignore

Co-authored-by: Jinyu-W <53509467+Jinyu-W@users.noreply.github.com>
Co-authored-by: Jinyu Wang <Wang.Jinyu@microsoft.com>
Co-authored-by: Jinyu Wang <jinywan@microsoft.com>
Co-authored-by: Jeremy Reynolds <jeremy.reynolds@microsoft.com>
Co-authored-by: Jeremy Reynolds <jeremr@microsoft.com>
Co-authored-by: slowy07 <slowy.arfy@gmail.com>
    @lihuoran @Jinyu-W
    @jreynolds01 @slowy07
    7 people authored Dec 8, 2021
    Configuration menu
    Copy the full SHA
    ff0f706 View commit details
    Browse the repository at this point in the history

Commits on Jan 24, 2022

  1. Change Env.set_seed() logic (#456) 

    * Change Env.set_seed() logic

* Redesign CIM reset logic; fix lint issues;

* Lint

* Seed type assertion
    @lihuoran
    lihuoran authored Jan 24, 2022
    Configuration menu
    Copy the full SHA
    8a25f9e View commit details
    Browse the repository at this point in the history

Commits on Mar 4, 2022

  1. Remove all SC related files (#473)

    @lihuoran
    lihuoran authored Mar 4, 2022
    Configuration menu
    Copy the full SHA
    526627c View commit details
    Browse the repository at this point in the history

Commits on Mar 7, 2022

  1. RL Toolkit V3 (#471) 

    * added daemon=True for multi-process rollout, policy manager and inference

* removed obsolete files

* [REDO][PR#406]V0.2 rl refinement taskq (#408)

* Add a usable task_queue

* Rename some variables

* 1. Add ; 2. Integrate  related files; 3. Remove

* merge `data_parallel` and `num_grad_workers` into `data_parallelism`

* Fix bugs in docker_compose_yml.py and Simple/Multi-process mode.

* Move `grad_worker` into marl/rl/workflows

* 1.Merge data_parallel and num_workers into data_parallelism in config; 2.Assign recently used workers as possible in task_queue.

* Refine code and update docs of `TaskQueue`

* Support priority for tasks in `task_queue`

* Update diagram of policy manager and task queue.

* Add configurable `single_task_limit` and correct docstring about `data_parallelism`

* Fix lint errors in `supply chain`

* RL policy redesign (V2) (#405)

* Drafi v2.0 for V2

* Polish models with more comments

* Polish policies with more comments

* Lint

* Lint

* Add developer doc for models.

* Add developer doc for policies.

* Remove policy manager V2 since it is not used and out-of-date

* Lint

* Lint

* refined messy workflow code

* merged 'scenario_dir' and 'scenario' in rl config

* 1. refined env_sampler and agent_wrapper code; 2. added docstrings for env_sampler methods

* 1. temporarily renamed RLPolicy from polivy_v2 to RLPolicyV2; 2. merged env_sampler and env_sampler_v2

* merged cim and cim_v2

* lint issue fix

* refined logging logic

* lint issue fix

* reversed unwanted changes

* .

.

.

.

ReplayMemory & IndexScheduler

ReplayMemory & IndexScheduler

.

MultiReplayMemory

get_actions_with_logps

EnvSampler on the road

EnvSampler

Minor

* LearnerManager

* Use batch to transfer data & add SHAPE_CHECK_FLAG

* Rename learner to trainer

* Add property for policy._is_exploring

* CIM test scenario for V3. Manual test passed. Next step: run it, make it works.

* env_sampler.py could run

* env_sampler refine on the way

* First runnable version done

* AC could run, but the result is bad. Need to check the logic

* Refine abstract method & shape check error info.

* Docs

* Very detailed compare. Try again.

* AC done

* DQN check done

* Minor

* DDPG, not tested

* Minors

* A rough draft of MAAC

* Cannot use CIM as the multi-agent scenario.

* Minor

* MAAC refinement on the way

* Remove ActionWithAux

* Refine batch & memory

* MAAC example works

* Reproduce-able fix. Policy share between env_sampler and trainer_manager.

* Detail refinement

* Simplify the user configed workflow

* Minor

* Refine example codes

* Minor polishment

* Migrate rollout_manager to V3

* Error on the way

* Redesign torch.device management

* Rl v3 maddpg (#418)

* Add MADDPG trainer

* Fit independent critics and shared critic modes.

* Add a new property: num_policies

* Lint

* Fix a bug in `sum(rewards)`

* Rename `MADDPG` to `DiscreteMADDPG` and fix type hint.

* Rename maddpg in examples.

* Preparation for data parallel (#420)

* Preparation for data parallel

* Minor refinement & lint fix

* Lint

* Lint

* rename atomic_get_batch_grad to get_batch_grad

* Fix a unexpected commit

* distributed maddpg

* Add critic worker

* Minor

* Data parallel related minorities

* Refine code structure for trainers & add more doc strings

* Revert a unwanted change

* Use TrainWorker to do the actual calculations.

* Some minor redesign of the worker's abstraction

* Add set/get_policy_state_dict back

* Refine set/get_policy_state_dict

* Polish policy trainers

move train_batch_size to abs trainer
delete _train_step_impl()
remove _record_impl
remove unused methods
a minor bug fix in maddpg

* Rl v3 data parallel grad worker (#432)

* Fit new `trainer_worker` in `grad_worker` and `task_queue`.

* Add batch dispatch

* Add `tensor_dict` for task submit interface

* Move `_remote_learn` to `AbsTrainWorker`.

* Complement docstring for task queue and trainer.

* Rename train worker to train ops; add placeholder for abstract methods;

* Lint

Co-authored-by: GQ.Chen <v-guanchen@microsoft.com>

* [DRAFT] distributed training pipeline based on RL Toolkit V3 (#450)

* Preparation for data parallel

* Minor refinement & lint fix

* Lint

* Lint

* rename atomic_get_batch_grad to get_batch_grad

* Fix a unexpected commit

* distributed maddpg

* Add critic worker

* Minor

* Data parallel related minorities

* Refine code structure for trainers & add more doc strings

* Revert a unwanted change

* Use TrainWorker to do the actual calculations.

* Some minor redesign of the worker's abstraction

* Add set/get_policy_state_dict back

* Refine set/get_policy_state_dict

* Polish policy trainers

move train_batch_size to abs trainer
delete _train_step_impl()
remove _record_impl
remove unused methods
a minor bug fix in maddpg

* Rl v3 data parallel grad worker (#432)

* Fit new `trainer_worker` in `grad_worker` and `task_queue`.

* Add batch dispatch

* Add `tensor_dict` for task submit interface

* Move `_remote_learn` to `AbsTrainWorker`.

* Complement docstring for task queue and trainer.

* dsitributed training pipeline draft

* added temporary test files for review purposes

* Several code style refinements (#451)

* Polish rl_v3/utils/

* Polish rl_v3/distributed/

* Polish rl_v3/policy_trainer/abs_trainer.py

* fixed merge conflicts

* unified sync and async interfaces

* refactored rl_v3; refinement in progress

* Finish the runnable pipeline under new design

* Remove outdated files; refine class names; optimize imports;

* Lint

* Minor maddpg related refinement

* Lint

Co-authored-by: Default <huo53926@126.com>
Co-authored-by: Huoran Li <huoranli@microsoft.com>
Co-authored-by: GQ.Chen <v-guanchen@microsoft.com>
Co-authored-by: ysqyang <v-yangqi@microsoft.com>

* Miner bug fix

* Coroutine-related bug fix ("get_policy_state") (#452)

* fixed rebase conflicts

* renamed get_policy_func_dict to policy_creator

* deleted unwanted folder

* removed unwanted changes

* resolved PR452 comments

Co-authored-by: ysqyang <v-yangqi@microsoft.com>

* Quick fix

* Redesign experience recording logic (#453)

* Two not important fix

* Temp draft. Prepare to WFH

* Done

* Lint

* Lint

* Calculating advantages / returns (#454)

* V1.0

* Complete DDPG

* Rl v3 hanging issue fix (#455)

* fixed rebase conflicts

* renamed get_policy_func_dict to policy_creator

* unified worker interfaces

* recovered some files

* dist training + cli code move

* fixed bugs

* added retry logic to client

* 1. refactored CIM with various algos; 2. lint

* lint

* added type hint

* removed some logs

* lint

* Make main.py more IDE friendly

* Make main.py more IDE friendly

* Lint

* Final test & format. Ready to merge.

Co-authored-by: ysqyang <v-yangqi@microsoft.com>
Co-authored-by: yaqiu <v-yaqiu@microsoft.com>
Co-authored-by: Huoran Li <huoranli@microsoft.com>

* Rl v3 parallel rollout (#457)

* fixed rebase conflicts

* renamed get_policy_func_dict to policy_creator

* unified worker interfaces

* recovered some files

* dist training + cli code move

* fixed bugs

* added retry logic to client

* 1. refactored CIM with various algos; 2. lint

* lint

* added type hint

* removed some logs

* lint

* Make main.py more IDE friendly

* Make main.py more IDE friendly

* Lint

* load balancing dispatcher

* added parallel rollout

* lint

* Tracker variable type issue; rename to env_sampler_creator;

* Rl v3 parallel rollout follow ups (#458)

* AbsWorker & AbsDispatcher

* Pass env idx to AbsTrainer.record() method, and let the trainer to decide how to record experiences sampled from different worlds.

* Fix policy_creator reuse bug

* Format code

* Merge AbsTrainerManager & SimpleTrainerManager

* AC test passed

* Lint

* Remove AbsTrainer.build() method. Put all initialization operations into __init__

* Redesign AC preprocess batches logic

Co-authored-by: ysqyang <v-yangqi@microsoft.com>
Co-authored-by: yaqiu <v-yaqiu@microsoft.com>
Co-authored-by: Huoran Li <huoranli@microsoft.com>

* MADDPG performance bug fix (#459)

* Fix MARL (MADDPG) terminal recording bug; some other minor refinements;

* Restore Trainer.build() method

* Calculate latest action in the get_actor_grad method in MADDPG.

* Share critic bug fix

* Rl v3 example update (#461)

* updated vm_scheduling example and cim notebook

* fixed bugs in vm_scheduling

* added local train method

* bug fix

* modified async client logic to fix hidden issue

* reverted to default config

* fixed PR comments and some bugs

* removed hardcode

Co-authored-by: ysqyang <v-yangqi@microsoft.com>
Co-authored-by: yaqiu <v-yaqiu@microsoft.com>

* Done (#462)

* Rl v3 load save (#463)

* added load/save feature

* fixed some bugs

* reverted unwanted changes

* lint

* fixed PR comments

Co-authored-by: ysqyang <v-yangqi@microsoft.com>
Co-authored-by: yaqiu <v-yaqiu@microsoft.com>

* RL Toolkit data parallelism revamp & config utils (#464)

* added load/save feature

* fixed some bugs

* reverted unwanted changes

* lint

* fixed PR comments

* 1. fixed data parallelism issue; 2. added config validator; 3. refactored cli local

* 1. fixed rollout exit issue; 2. refined config

* removed config file from example

* fixed lint issues

* fixed lint issues

* added main.py under examples/rl

* fixed lint issues

Co-authored-by: ysqyang <v-yangqi@microsoft.com>
Co-authored-by: yaqiu <v-yaqiu@microsoft.com>

* RL doc string (#465)

* First rough draft

* Minors

* Reformat

* Lint

* Resolve PR comments

* Rl type specific env getter (#466)

* 1. type-sensitive env variable getter; 2. updated READMEs for examples

* fixed bugs

* fixed bugs

* bug fixes

* lint

Co-authored-by: ysqyang <v-yangqi@microsoft.com>
Co-authored-by: yaqiu <v-yaqiu@microsoft.com>

* Example bug fix

* Optimize parser.py

* Resolve PR comments

* Rl config doc (#467)

* 1. type-sensitive env variable getter; 2. updated READMEs for examples

* added detailed doc

* lint

* wording refined

* resolved some PR comments

* resolved more PR comments

* typo fix

Co-authored-by: ysqyang <v-yangqi@microsoft.com>

Co-authored-by: ysqyang <ysqyang@gmail.com>
Co-authored-by: ysqyang <v-yangqi@microsoft.com>
Co-authored-by: yaqiu <v-yaqiu@microsoft.com>

* RL online doc (#469)

* Model, policy, trainer

* RL workflows and env sampler doc in RST (#468)

* First rough draft

* Minors

* Reformat

* Lint

* Resolve PR comments

* 1. type-sensitive env variable getter; 2. updated READMEs for examples

* Rl type specific env getter (#466)

* 1. type-sensitive env variable getter; 2. updated READMEs for examples

* fixed bugs

* fixed bugs

* bug fixes

* lint

Co-authored-by: ysqyang <v-yangqi@microsoft.com>
Co-authored-by: yaqiu <v-yaqiu@microsoft.com>

* Example bug fix

* Optimize parser.py

* Resolve PR comments

* added detailed doc

* lint

* wording refined

* resolved some PR comments

* rewriting rl toolkit rst

* resolved more PR comments

* typo fix

* updated rst

Co-authored-by: Huoran Li <huoranli@microsoft.com>
Co-authored-by: Default <huo53926@126.com>
Co-authored-by: ysqyang <v-yangqi@microsoft.com>
Co-authored-by: yaqiu <v-yaqiu@microsoft.com>

* Finish docs/source/key_components/rl_toolkit.rst

* API doc

* RL online doc image fix (#470)

* resolved some PR comments

* fix

* fixed PR comments

* added numfig=True setting in conf.py for sphinx

Co-authored-by: ysqyang <v-yangqi@microsoft.com>

* Resolve PR comments

* Add example github link

Co-authored-by: ysqyang <ysqyang@gmail.com>
Co-authored-by: ysqyang <v-yangqi@microsoft.com>
Co-authored-by: yaqiu <v-yaqiu@microsoft.com>

* Rl v3 pr comment resolution (#474)

* added load/save feature

* 1. resolved pr comments; 2. reverted maro/cli/k8s

* fixed some bugs

Co-authored-by: ysqyang <v-yangqi@microsoft.com>
Co-authored-by: yaqiu <v-yaqiu@microsoft.com>

Co-authored-by: yaqiu <v-yaqiu@microsoft.com>
Co-authored-by: GQ.Chen <v-guanchen@microsoft.com>
Co-authored-by: ysqyang <ysqyang@gmail.com>
Co-authored-by: ysqyang <v-yangqi@microsoft.com>
    @lihuoran @buptchan @ysqyang
    5 people authored Mar 7, 2022
    Configuration menu
    Copy the full SHA
    696f5b5 View commit details
    Browse the repository at this point in the history

Commits on Mar 9, 2022

  1. RL renaming v2 (#476) 

    * Change all Logger in RL to LoggerV2

* TrainerManager => TrainingManager

* Add Trainer suffix to all algorithms

* Finish docs

* Update interface names

* Minor fix
    @lihuoran
    lihuoran authored Mar 9, 2022
    Configuration menu
    Copy the full SHA
    7b3d78a View commit details
    Browse the repository at this point in the history

Commits on Mar 31, 2022

  1. Cherry pick latest RL (#498) 

    * Cherry pick

* Remove SC related files
    @lihuoran
    lihuoran authored Mar 31, 2022
    Configuration menu
    Copy the full SHA
    00fbcee View commit details
    Browse the repository at this point in the history

Commits on Apr 22, 2022

  1. Cherry pick RL changes from sc_refinement (latest commit: 2a4869) (… 

    …#509)

* Cherry pick RL changes from sc_refinement (2a4869)

* Limit time display precision
    @lihuoran
    lihuoran authored Apr 22, 2022
    Configuration menu
    Copy the full SHA
    0e11ae9 View commit details
    Browse the repository at this point in the history

Commits on Apr 24, 2022

  1. RL incremental refactor (#501) 

    * Refactor rollout logic. Allow multiple sampling in one epoch, so that we can generate more data for training.

AC & PPO for continuous action policy; refine AC & PPO logic.

Cherry pick RL changes from GYM-DDPG

Cherry pick RL changes from GYM-SAC

Minor error in doc string

* Add min_n_sample in template and parser

* Resolve PR comments. Fix a minor issue in SAC.
    @lihuoran
    lihuoran authored Apr 24, 2022
    Configuration menu
    Copy the full SHA
    1219513 View commit details
    Browse the repository at this point in the history

Commits on May 10, 2022

  1. RL component bundle (#513) 

    * CIM passed

* Update workers

* Refine annotations

* VM passed

* Code formatting.

* Minor import loop issue

* Pass batch in PPO again

* Remove Scenario

* Complete docs

* Minor

* Remove segment

* Optimize logic in RLComponentBundle

* Resolve PR comments

* Move 'post methods from RLComponenetBundle to EnvSampler
    @lihuoran
    lihuoran authored May 10, 2022
    Configuration menu
    Copy the full SHA
    333986f View commit details
    Browse the repository at this point in the history

Commits on May 16, 2022

  1. Add method to get mapping of available tick to frame index (#415) 

    * add method to get mapping of available tick to frame index

* fix lint issue

* fix naming issue
    @chaosddp
    chaosddp authored May 16, 2022
    Configuration menu
    Copy the full SHA
    ae83ac0 View commit details
    Browse the repository at this point in the history

Commits on May 18, 2022

  1. Cherry pick from sc_refinement (#527) 

    * Cherry pick from sc_refinement

* Cherry pick from sc_refinement
    @lihuoran
    lihuoran authored May 18, 2022
    Configuration menu
    Copy the full SHA
    10b9c02 View commit details
    Browse the repository at this point in the history

Commits on May 25, 2022

  1. Refine terminal / next_agent_state logic (#531) 

    * Optimize RL toolkit

* Fix bug in terminal/next_state generation

* Rewrite terminal/next_state logic again

* Minor renaming

* Minor bug fix

* Resolve PR comments
    @lihuoran
    lihuoran authored May 25, 2022
    Configuration menu
    Copy the full SHA
    0d132cc View commit details
    Browse the repository at this point in the history

Commits on May 31, 2022

  1. Merge master into v0.3 (#536) 

    * update docker hub init (#367)

* update docker hub init

* replace personal account with maro-team

* update hello files for CIM

* update docker repository name

* update docker file name

* fix bugs in notebook, rectify docs

* fix doc build issue

* remove docs from playground; fix citibike lp example Event issue

* update the exampel for vector env

* update vector env example

* update README due to PR comments

* add link to playground above MARO installation in README

* fix some typos

Co-authored-by: Jinyu Wang <Wang.Jinyu@microsoft.com>

* update package version

* update README for package description

* update image links for pypi package description

* update image links for pypi package description

* change the input topology schema for CIM real data mode (#372)

* change the input topology schema for CIM real data mode

* remove unused importing

* update test config file correspondingly

* add Exception for env test

* add cost factors to cim data dump

* update CimDataCollection field name

* update field name of data collection related code

* update package version

* adjust interface to reflect actual signature (#374)

Co-authored-by: Jeremy Reynolds <jeremr@microsoft.com>

* update dataclasses requirement to setup

* fix: fixing spelling grammarr

* fix: fix typo spelling code commented and data_model.rst

* Fix Geo vis IP address & SQL logic bugs. (#383)

Fix Geo vis IP address & SQL logic bugs (issue [352](#352) and [314](#314)).

* Fix the "Wrong future stop tick predictions" bug (#386)

* Propose my new solution

Refine to the pre-process version

.

* Optimize import

* Fix reset random seed bug (#387)

* update the reset interface of Env and BE

* Try to fix reset routes generation seed issue

* Refine random related logics.

* Minor refinement

* Test check

* Minor

* Remove unused functions so far

* Minor

Co-authored-by: Jinyu Wang <jinywan@microsoft.com>

* update package version

* Add _init_vessel_plans in business_engine.reset (#388)

* update package version

* change the default solver used in Citibike OnlineLP example, from GLPK to CBC (#391)

Co-authored-by: Jinyu Wang <Wang.Jinyu@microsoft.com>

* Refine `event_buffer/` module (#389)

* Core & Business Engine code refinement (#392)

* First version

* Optimize imports

* Add typehint

* Lint check

* Lint check

* add higher python version (#398)

* add higher python version

* update pytorch version

* update torchvision version

Co-authored-by: Jinyu Wang <Wang.Jinyu@microsoft.com>

* CIM scenario refinement (#400)

* Cim scenario refinement (#394)

* CIM refinement

* Fix lint error

* Fix lint error

* Cim test coverage (#395)

* Enrich tests

* Refactor CimDataGenerator

* Refactor CIM parsers

* Minor refinement

* Fix lint error

* Fix lint error

* Fix lint error

* Minor refactor

* Type

* Add two test file folders. Make a slight change to CIM BE.

* Lint error

* Lint error

* Remove unnecessary public interfaces of CIM BE

* Cim disable auto action type detection (#399)

* Haven't been tested

* Modify document

* Add ActionType checking

* Minor

* Lint error

* Action quantity should be a position number

* Modify related docs & notebooks

* Minor

* Change test file name. Prepare to merge into master.

* .

* Minor test patch

* Add `clear()` function to class `SimRandom` (#401)

* Add SimRandom.clear()

* Minor

* Remove commented codes

* Lint error

* update package version

* add branch v0.3 to github workflow

* update github test workflow

* Update requirements.dev.txt (#444)

Added the versions of dependencies and resolve some conflicts occurs when installing. By adding these version number it will tell you the exact.

* Bump ipython from 7.10.1 to 7.16.3 in /notebooks (#460)

Bumps [ipython](https://github.com/ipython/ipython) from 7.10.1 to 7.16.3.
- [Release notes](https://github.com/ipython/ipython/releases)
- [Commits](ipython/ipython@7.10.1...7.16.3)

---
updated-dependencies:
- dependency-name: ipython
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Add & sort requirements.dev.txt

Co-authored-by: Jinyu-W <53509467+Jinyu-W@users.noreply.github.com>
Co-authored-by: Jinyu Wang <Wang.Jinyu@microsoft.com>
Co-authored-by: Jinyu Wang <jinywan@microsoft.com>
Co-authored-by: Jeremy Reynolds <jeremy.reynolds@microsoft.com>
Co-authored-by: Jeremy Reynolds <jeremr@microsoft.com>
Co-authored-by: slowy07 <slowy.arfy@gmail.com>
Co-authored-by: solosilence <abhishekkr23rs@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
    @lihuoran @Jinyu-W
    @jreynolds01 @slowy07 @itsabhishekhere @dependabot
    9 people authored May 31, 2022
    Configuration menu
    Copy the full SHA
    a3dade7 View commit details
    Browse the repository at this point in the history

Commits on Jun 1, 2022

  1. Merge branch 'v0.3' into merge_v0.3_into_master

    @lihuoran
    lihuoran committed Jun 1, 2022
    Configuration menu
    Copy the full SHA
    cdadbc9 View commit details
    Browse the repository at this point in the history

  2. Remove random_config.py

    @lihuoran
    lihuoran committed Jun 1, 2022
    Configuration menu
    Copy the full SHA
    e0209d6 View commit details
    Browse the repository at this point in the history

  3. Remove test_trajectory_utils.py

    @lihuoran
    lihuoran committed Jun 1, 2022
    Configuration menu
    Copy the full SHA
    c135e73 View commit details
    Browse the repository at this point in the history

  4. Pass tests

    @lihuoran
    lihuoran committed Jun 1, 2022
    Configuration menu
    Copy the full SHA
    fb77545 View commit details
    Browse the repository at this point in the history

  5. Update rl docs

    @lihuoran
    lihuoran committed Jun 1, 2022
    Configuration menu
    Copy the full SHA
    c144c3b View commit details
    Browse the repository at this point in the history

  6. Remove python 3.6 in test

    @lihuoran
    lihuoran committed Jun 1, 2022
    Configuration menu
    Copy the full SHA
    0d6ac99 View commit details
    Browse the repository at this point in the history

  7. Merge branch 'master' into merge_v0.3_into_master

    @Jinyu-W
    Jinyu-W authored Jun 1, 2022
    Configuration menu
    Copy the full SHA
    5ce9e4c View commit details
    Browse the repository at this point in the history

  8. Update docs

    @lihuoran
    lihuoran committed Jun 1, 2022
    Configuration menu
    Copy the full SHA
    2e4b436 View commit details
    Browse the repository at this point in the history