Rendering: save animation as files

[mmarti-tsch]

No description provided.

[tinubuerki]

The following code works with gym-envs on a local computer. Hopefully it can be easily transferred to other settings.

import cv2
import os
import matplotlib.pyplot as plt

image_folder = "."
image_format = "png"
video_name = 'episode_video.avi'
frames_per_second = 8

# run episode
state, _ = env.reset()
step = 0

while not done:
	a = select_action(state)
    next_state, reward, done, _ = env.step(a)
    # render env to file
	plt.imsave('{}/frame_{:05d}.{}'.format(image_folder, s, image_format), env.render())

# list of image file names
images = [img for img in os.listdir(image_folder) if img.endswith(image_format)]

# read first image for parameters (shape of the frame)
frame = cv2.imread(os.path.join(image_folder, images[0]))
height, width, layers = frame.shape

# create video from image files
video = cv2.VideoWriter(video_name, 0, frames_per_second, (width, height))

for image in images:
    video.write(cv2.imread(os.path.join(image_folder, image)))
    # delete the image file
    os.remove(os.path.join(image_folder, image))

cv2.destroyAllWindows()
video.release()

[aiAdrian]

What is the env? env.render i dont know the method

[tinubuerki]

What is the env? env.render i dont know the method

render() is a standard method for envs from the gym (and gymnasium) API, see https://www.gymlibrary.dev/api/core/#gym.Env.render

[tinubuerki]

Uups, I just noticed that there is a bug in the while loop. It should look like this:

while not done:
	a = select_action(state)
	next_state, reward, done, _ = env.step(a)
	# render env to file
	plt.imsave('{}/frame_{:05d}.{}'.format(image_folder, step, image_format), env.render())
	step += 1

[aiAdrian]

Would it be possible to test whether the RenderTool and Flatland could benefit from your renderToVideo experiments?
See also: https://github.com/flatland-association/flatland-rl/blob/main/notebooks/simple_animation_demo.ipynb who to access raw rendered image from Flatland's RenderTool.

See as well: https://github.com/RoboEden/flatland-marl/blob/3344327cbf81ef90dd7b26c2ee53838e7aae9e8e/solution/demo.py#L106 - this video rendering is working well : https://github.com/RoboEden/flatland-marl/blob/main/solution/utils/video_writer.py ,

pip install ffmpeg-python==0.2

[tinubuerki]

I added a version 2 of the Colab notebook with a simple extension:

https://github.com/flatland-association/flatland-rl/blob/main/notebooks/simple_animation_demo_2.ipynb

[tinubuerki]

I realized that uploading my notebook failed (it's not my notebook) and I could not manage to upload it so far. It's always just a copy of the original notebook!?! Any ideas what I'm doing wrong?

[tinubuerki]

I once again checked my uploaded notebook (simple_animation_demo_2.ipynb) and found that it is indeed complete and includes my additions. It just does not show the additional cells when browsing the file on github !??!

Maybe someone else can check that too. Can the issue then be closed?