Potential Memory Leak in CarRacing-V0

[SwapnilPande]

I noticed memory usage was increasing while training an RL agent using the CarRacing-V0 environment. Looks similar to #1882.

I was able to isolate the issue to the gym environment by running the following code with a memory profiler:

import gym

env = gym.make('CarRacing-v0')
env.reset()
for i in range(500_000):
    # env.render()
    _, _, done, _ = env.step(env.action_space.sample())
    if done:
        env.reset()

env.close()

The memory profiler returned the following plot.:

When only running env.reset in the loop, I get the following memory growth behavior:

import gym

env = gym.make('CarRacing-v0')
env.reset()
for i in range(500_000):
    env.reset()

env.close()

When only calling reset(), the memory consumption seems to grow slower, but grows nevertheless.

[SwapnilPande]

#2096 Addresses this issue.