New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allow custom spaces in VectorEnv #2038
Allow custom spaces in VectorEnv #2038
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like this (even though I don't fully agree with the motivation behind strings as custom spaces)! The only thing I would request is adding a comment (maybe in spaces/space.py) along the lines of:
Space class can be subclassed to create a custom observation / action space, however, keep in mind that most cases should be covered by the existing space classes (Box, Discrete, ...) and container space classes (Tuple, Dict). Moreover, parametrized probability distributions and batching functions are only well-defined for the existing spaces, so some learning algorithms may not work out of the box. TLDR think twice before subclassing
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
looks good to me, thanks!
* Allow custom observation spaces in VectorEnv * Replace np.copy by deepcopy in reset of SyncVectorEnv * Add tests for VectorEnv with custom spaces * Add tests for shared memory and batches of custom spaces * Remove unused import in VectorEnv test * Add warning note in the Space class for custom spaces
AsyncVectorEnv
andSyncVectorEnv
. This will not batch the observations, and will return a tuple of individual observations.