Adding Video tests.

[kolibril13]

I have the following idea of how to test videos in manim:
We generate a video with a very low frame rate, I would suggest 4 frames per second. This is not too low to overlook something that might go wrong and at the same time not too long to slow down the testing process drastically.
For a 5-second test video, this would result in an about 0.1 MB compressed numpy array.
Here is the script how to compress in this case dummy-data:

import numpy as np
data= []
data.append(np.ones((480,854)))
for i in range(0,20):
    data.append((data[-1]))
np.savez_compressed('test', my_video = data)

# and how to reload the data:
loaded_ar = np.load('test.npz')["my_video"]

Then, we make a very similar script to this one:

manim/tests/helpers/graphical_units.py

Line 11 in e67a655

def set_test_scene(scene_object, module_name):

except, that the data is not only created once but multiple times for every 0.25 seconds.
Here is what I could come up right now

from manim import *
from manim import config, file_writer_config
import matplotlib.pyplot as plt
from pathlib import Path

class Example(Scene):
    def construct(self):
        circle = Circle()
        self.play(Animation(circle))

file_writer_config["skip_animations"] = True
file_writer_config["write_to_movie"] = False
file_writer_config["disable_caching"] = True
config["pixel_height"] = 480
config["pixel_width"] = 854
config["frame_rate"] = 2
dir = Path.cwd()
file_writer_config["text_dir"] = Path(dir)/ "text"
file_writer_config["tex_dir"] = Path(dir) / "tex"
scene = Example()
scene.render() ## this line has to be altered! Maybe in a for loop?
data = scene.renderer.get_frame()
plt.imshow(data)
plt.show()

Note that matplotlib is only there for debugging and that this script only produces one frame.
Please leave your thoughts about this, @ManimCommunity/tests

[huguesdevimeux]

This is a very good idea, but I would like to precise a little bit the terminlogy you used.
We actually already have videos tests (see the wiki). What we don't have is graphical unit tests that test several frame, as right now they only test the last frame.

This is where your idea is excellent, it would be to change how graphical unit tests are implemented and to make them compare not only the last frame but a number n.

[kolibril13]

Ohh, that is a great idea!
Soo, should we then add a parameter last_frame -> def set_test_scene(scene_object, module_name, last_frame)
and in case

• last_frame = True -> only the last frame, like it is currently
• last_frame = False -> saves all the individual frames with 4 fps as default

[huguesdevimeux]

Why not, but I think that the main difficulty will be to know how to implement these tests, not how to create the control data :)

[eulertour]

This could be done very elegantly by writing some new renderers, like a SaveFrameDataRenderer and CompareFrameDataRenderer. Since renderer.add_frame() is called on each frame, you could overwrite the methods to either save or compare to stored frame data.

[leotrs]

I had the same idea in the past. Great minds or whatever.

However, do you think this is a task for the renderer or a task for the FileWriter? SaveFrameDataRenderer or SaveFrameDataFileWriter?

[huguesdevimeux]

@leotrs My opinion lends toward SaveFrameDataRender, as we don't want to write anything.

By the way, I think if someone is going to implement this, it's the occasion to refactor a little bit these graphical units tests.

[leotrs]

Ah fair.

[eulertour]

@leotrs The FileWriter is held by the CairoRenderer so that would do the trick, but I don't know if there's a way to pass in a modified FileWriter right now.

[kolibril13]

the time axis projected in one image could be also a good approach for videotests: #849

[leotrs]

I don't think that the functionality proposed in #849 should be used for testing. The problem is that it could only be used to test moving mobjects. If the mobject doesn't move, then each frame will just be put on top of the other, and it will not be possible to know which pixel comes from which frame. I don't see how that is better than the original idea of saving different frames separately.

[jsonvillanueva]

Related to #1035

[behackl]

Animation tests (via rendering and storing multiple frames) exist since the refactor of the testing framework. Closing.