Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fast eval option #391

Merged
merged 28 commits into from
Jul 28, 2019
Merged

Fast eval option #391

merged 28 commits into from
Jul 28, 2019

Conversation

kengz
Copy link
Owner

@kengz kengz commented Jul 27, 2019 •
edited
Loading

Faster evaluation alternative

  • introduce TrackReward env wrapper as a simpler way to track total reward. This also works naturally with vec env.
  • retire obsolete custom total_reward tracking logic
  • refactor body.ckpt logic and env logic
  • add backward-compatible meta.rigorous_eval: int spec to use rigorous slow eval, or fast eval by inferring total_reward directly from env

kengz added 18 commits July 25, 2019 09:05
@kengz
remove extra try_scale_rewards from vec_env
67e06ee
@kengz
refactor env base attr setting
8548f49
@kengz
fix spec ref in env base frame attr infer
1d91fb1
@kengz
move total_reward calc to env from body. update from env.step
29df219
@kengz
move update total_reward from util to env.base
6e2823c
@kengz
add info arg to _track_total_reward
ad71c8b
@kengz
update obs wrapper API method
67c91fb
@kengz
reorder wrappers
799f94e
@kengz
add a TrackReward env wrapper for all openai envs
1171e7c
@kengz
update lab env total_reward update for openai and unity
7b77f25
@kengz
return random breakout spec to breakout
3b54df1
@kengz
use nan as default value for total_reward for rigor
8f54917
@kengz
simplify eval ckpt and gen return
809834c
@kengz
update body.total_reward log to source from env
9d1f84d
@kengz
refactor body ckpt
f34ba41
@kengz
reorder env base methods
221c839
@kengz
default eval_frequency and log_frequency to 10k, simplify code
463a4c0
@kengz
add backward compatible meta.rigorous_eval spec var
e353fa6
codeclimate[bot]
codeclimate bot reviewed Jul 27, 2019
View reviewed changes
slm_lab/agent/algorithm/sarsa.py Show resolved Hide resolved
slm_lab/agent/algorithm/actor_critic.py Show resolved Hide resolved
slm_lab/agent/algorithm/dqn.py Show resolved Hide resolved
slm_lab/agent/algorithm/reinforce.py Show resolved Hide resolved
slm_lab/agent/algorithm/sil.py Show resolved Hide resolved
@kengz kengz merged commit ebfb639 into master Jul 28, 2019
@kengz kengz deleted the fast-eval branch July 28, 2019 02:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@codeclimate codeclimate[bot] codeclimate[bot] left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@kengz