Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Vectorized environments #1513

Merged
merged 29 commits into from
Jun 21, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
076c0c2
Initial version of vectorized environments
tristandeleu Jun 1, 2019
9b9bcdf
Raise an exception in the main process if child process raises an exc…
tristandeleu Jun 1, 2019
7e2b853
Add list of exposed functions in vector module
tristandeleu Jun 1, 2019
48b7018
Use deepcopy instead of np.copy
tristandeleu Jun 1, 2019
2fe514d
Add documentation for vector utils
tristandeleu Jun 1, 2019
ad6b83e
Add tests for copy in AsyncVectorEnv
tristandeleu Jun 1, 2019
a1f214d
Add example in documentation for batch_space
tristandeleu Jun 1, 2019
4107861
Add cloudpickle dependency in setup.py
tristandeleu Jun 2, 2019
0184353
Fix __del__ in VectorEnv
tristandeleu Jun 2, 2019
2350b43
Check if all observation spaces are equal in AsyncVectorEnv
tristandeleu Jun 5, 2019
b0f50ef
Check if all observation spaces are equal in SyncVectorEnv
tristandeleu Jun 5, 2019
2d4b314
Fix spaces non equality in SyncVectorEnv for Python 2
tristandeleu Jun 6, 2019
71c1166
Handle None parameter in create_empty_array
tristandeleu Jun 6, 2019
5d53cc5
Fix check_observation_space with spaces equality
tristandeleu Jun 8, 2019
10bfb50
Raise an exception when operations are out of order in AsyncVectorEnv
tristandeleu Jun 8, 2019
c0edbcd
Add version requirement for cloudpickle
tristandeleu Jun 8, 2019
8d504c9
Use a state instead of binary flags in AsyncVectorEnv
tristandeleu Jun 8, 2019
55793cd
Use numpy.zeros when initializing observations in vectorized environm…
tristandeleu Jun 8, 2019
2bdfb4f
Remove poll from public API in AsyncVectorEnv
tristandeleu Jun 8, 2019
7904f20
Remove close_extras from VectorEnv
tristandeleu Jun 8, 2019
6c187dc
Add test between AsyncVectorEnv and SyncVectorEnv
tristandeleu Jun 8, 2019
3d7c3f3
Remove close in check_observation_space
tristandeleu Jun 8, 2019
c904b37
Add documentation for seed and close
tristandeleu Jun 8, 2019
aa3681b
Refactor exceptions for AsyncVectorEnv
tristandeleu Jun 18, 2019
c181a94
Close pipes if the environment raises an error
tristandeleu Jun 13, 2019
81b1bbc
Add tests for out of order operations
tristandeleu Jun 13, 2019
21685ed
Change default argument in create_empty_array to np.zeros
tristandeleu Jun 13, 2019
7db9028
Add get_attr and set_attr methods to VectorEnv
tristandeleu Jun 13, 2019
7a4efe4
Improve consistency in SyncVectorEnv
tristandeleu Jun 14, 2019
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions gym/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,5 +10,6 @@
from gym.spaces import Space
from gym.envs import make, spec, register
from gym import logger
from gym import vector

__all__ = ["Env", "Space", "Wrapper", "make", "spec", "register"]
28 changes: 28 additions & 0 deletions gym/error.py
Original file line number Diff line number Diff line change
Expand Up @@ -137,3 +137,31 @@ class WrapAfterConfigureError(Error):

class RetriesExceededError(Error):
pass

# Vectorized environments errors

class AlreadyPendingCallError(Exception):
"""
Raised when `reset`, or `step` is called asynchronously (e.g. with
`reset_async`, or `step_async` respectively), and `reset_async`, or
`step_async` (respectively) is called again (without a complete call to
`reset_wait`, or `step_wait` respectively).
"""
def __init__(self, message, name):
super(AlreadyPendingCallError, self).__init__(message)
self.name = name

class NoAsyncCallError(Exception):
"""
Raised when an asynchronous `reset`, or `step` is not running, but
`reset_wait`, or `step_wait` (respectively) is called.
"""
def __init__(self, message, name):
super(NoAsyncCallError, self).__init__(message)
self.name = name

class ClosedEnvironmentError(Exception):
"""
Trying to call `reset`, or `step`, while the environment is closed.
"""
pass
47 changes: 47 additions & 0 deletions gym/vector/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
from gym.vector.async_vector_env import AsyncVectorEnv
from gym.vector.sync_vector_env import SyncVectorEnv
from gym.vector.vector_env import VectorEnv

__all__ = ['AsyncVectorEnv', 'SyncVectorEnv', 'VectorEnv', 'make']

def make(id, num_envs=1, asynchronous=True, **kwargs):
"""Create a vectorized environment from multiple copies of an environment,
from its id

Parameters
----------
id : str
The environment ID. This must be a valid ID from the registry.

num_envs : int
Number of copies of the environment. If `1`, then it returns an
unwrapped (i.e. non-vectorized) environment.

asynchronous : bool (default: `True`)
If `True`, wraps the environments in an `AsyncVectorEnv` (which uses
`multiprocessing` to run the environments in parallel). If `False`,
wraps the environments in a `SyncVectorEnv`.

Returns
-------
env : `gym.vector.VectorEnv` instance
The vectorized environment.

Example
-------
>>> import gym
>>> env = gym.vector.make('CartPole-v1', 3)
>>> env.reset()
array([[-0.04456399, 0.04653909, 0.01326909, -0.02099827],
[ 0.03073904, 0.00145001, -0.03088818, -0.03131252],
[ 0.03468829, 0.01500225, 0.01230312, 0.01825218]],
dtype=float32)
"""
from gym.envs import make as make_
def _make_env():
return make_(id, **kwargs)
if num_envs == 1:
return _make_env()
env_fns = [_make_env for _ in range(num_envs)]

return AsyncVectorEnv(env_fns) if asynchronous else SyncVectorEnv(env_fns)
Loading