Make livestreaming a thing. #778

NeoPlato · 2020-11-27T10:52:37Z

List of Changes

Change 1

Add a streaming scene that uses its own file writer to override the normal functioning of the original to create a streaming protocol. Then, the original Scene class can use CairoRenderer as a default, until a helper class by the name Stream uses the advantage of renderer initialization for scenes to add a special renderer, that uses a special file writer. Most of the work I've been doing is make this accessible for other scenes. And I can say for sure, totally worth it.

class Stream:
    """This class is really intended for inheritance of the style:

    >>> class Streamer(Stream, Scene): # doctest: +SKIP
    ...     pass
    ...
    >>>

    This order is paramount. This :class:`Stream` class does the switch to
    the specialized renderer, which uses the :class:`StreamFileWriter` to
    handle specialized streaming services. That explains the calls to super,
    which dig through the MRO of a class instead of using just a single
    implementation contained in Scene. Okay, the references in super probably
    point to things in the :class:`Scene` class only, but it's already happened so...

    PS: You can probably already tell using this class on its own will bring you
    errors.

    """

Change 2

Use rtp instead of tcp as the streaming protocol. Mainly for no other reason than it's what I understood and coded in that direction. If you think that means I didn't crack Twitch streaming then you are absolutely right.
I tried tcp and udp already so feel free to tell me how to open a udp stream without blatant packet errors.

Change 3

There are two versions of the livestreaming I made. Both do the same thing. The one under the function livestream is the main version and opens up an interactive session. The other one, stream, just returns the streamer. Easy to use for casual on the fly testing like:

>>> from manim import stream, stuff, other, stuff
>>> manim = stream()
>>> manim.play(...)

Nevertheless, both are available in the main library, depending on whether you need 500 objects in your global variables or not.

Change 4

Any extras were probably made in order to have this working.

Explanation for Changes

I figure that any livestreaming option that was there in the past kind of broke, but the backbone of its capability was continuously carried around like a fossil. So I tried to implement it and it worked.

Testing Status

There's varying amounts of good and bad news here. Let me explain.

Good

Since this Manim can directly influence camera configurations from the command line, this gives you high quality streaming:

python -m manim --livestream

This gives you medium:

python -m manim -qm --livestream

This gives you IMAX:

python -m manim -qk --livestream

Good

Ability to use other base scenes for streaming. Really the crux of my implementation detail.

Concept-Extra-Scenes-Unedited.mp4

Superb depending on utility

Ability to render pre-made scenes with baked setup and/or construct methods.

Concept-Inheritance-Unedited.mp4

You can also use the play_scene function to specify a start and end animation, analogous to the -n keyword from the command line. For example: play_scene(scene, start=0, end=4)

Bad

If I did my job well the ffplay window should be triggered automatically. But if it doesn't for some reason a command along the lines of this may help:

ffplay -x 640 -y 360 -protocol_whitelist "file,rtp,udp" -i "path\to\manim\media\streams\streams.sdp" -reorder_queue_size 0

Bad

ffplay might glitch and show you a lot of red for some reason. Reopening it should fix one of those problems.
For example the avai;able open_client(); get_streamer().wait() sequence should open ffplay again and trigger the opening of the window.
Also there's something about the last frame of the animation not showing, but it's just one frame. You can close the window and run open_client() to restart the window, if this is the beginning of further glitching.

Good

If it works though, expect something like this:

Concept-Basic-Unedited.mp4

Further Comments

From StreamCairoRenderer

"""I wish there was another way to have a renderer that uses the
    file writer created for the purpose. However, I can't do that without
    the original code being aware of the extra implementation, which is probably
    undesirable style.
    """

When using get_streamer, it is technically possible to use multiple classes to inherit more methods. The argument syntax clearly supports it yet its main use is to generates the tuple of the passed scene automatically. Multiple inheritance would be delicate and would only work if the classes after the first merely add more methods than override base ones. For example MovingCameraScene, LinearTransformationScene is a tempting concoction but unstable. For one, naming them in reverse is already an error, courtesy of the assertion in MovingCameraScene. Also, the latter adds attributes used in its methods from __init__, which wouldn't be called thanks to strict initialization minus super() proxies. Then again, multiple inheritance of scenes was always a delicate matter in Manim.

Acknowledgement

I have read the Contributing Guidelines*

*half true
***70 of the commits don't count, I was learning

Please tell me this creates the pull request

This reverts commit 7b22388.

This reverts commit 4c04a9f.

huguesdevimeux · 2021-03-31T07:08:41Z

A general update about this : We've been discussing this feature a little bit in a meeting, and that's a great change, but nevertheless that will be soon outdated by the upcoming openGL renderer.
So, there has been a discussion on whether this should be a plugin or not, and incorporated in the core.

Considering that there is still time before OpenGL is completly stable on manim, I think it's fine to merge it and to migre this to a plugin in the (near?) future. @NeoPlato thoughts ? I personnaly think that making this as a plugin makes way more sense. Manim is flexible enough to allow this, but that's as you wish.

By the way, we are very, very sorry for our inactivity regarding this PR. I'm sure you spent a lot of time in thie work, and we've been really ungrateful regarding this. Sincere apologies.

After @NeoPlato tells their opinion on this, I'll take the responsability to merge this to not make things wait even more. @ManimCommunity/core if you have anything to say about this PR please do it quickly

NeoPlato · 2021-03-31T07:25:31Z

@huguesdevimeux The switch to a plugin shouldn't be too hard (fingers crossed). I'd just have to make the thing into a package of some kind, I think.

As for the use of OpenGL its success is one that the future will tell, so keeping these settings for livestreaming would be great if they don't intermingle too much with Manim itself. Basically, if I wanted to anticipate livestreaming as a plugin I have to redesign everything as a plugin before Manim's roots in other developments grow around it.

I'll try to look into configurations for generating plugins and offer that as the default, as a continuation of this discussion after merging. Perhaps parading an on-the-fly compilation is subjective. That's the only thing I choose to tack onto this project. Any suggestions within the realm of capacity are most welcome

huguesdevimeux · 2021-03-31T08:56:06Z

I see, so let's get this merged ! there is just a merge conflict to fix, and then I'll merge this the tomorrow (01/04) in the case anyone wants to express something about this.

NeoPlato · 2021-04-01T18:50:11Z

Note: if something doesn't work it's possibly some new change in Manim that hasn't accounted for livestreaming settings. I will totally make this a plugin after realizing this truth.

WampyCakes · 2021-04-01T22:59:32Z

@huguesdevimeux are we still planning on merging this today before the release?

huguesdevimeux · 2021-04-02T01:18:02Z

@WampyCakes yea sorry I completely forget, I will merge this tomorrow (that's fine if it's not in the current relapse, that's even better since this is not yet documented)

huguesdevimeux · 2021-04-02T10:26:58Z

Nooo there are merge conflicts

huguesdevimeux · 2021-04-02T10:28:02Z

@NeoPlato Really sorry but here are merge conflicts, could you fix them 😬

That's 100% my fault, I said I would merge this yesterday

behackl · 2021-04-02T10:32:13Z

The main problem for this PR is that #1013 was merged meanwhile, so our CLI interface has migrated from argparse to click. This PR will have to adapt to this new situation as well, maybe @jsonvillanueva can very briefly describe what needs to be done in order to migrate the livestreaming flags?

jsonvillanueva · 2021-04-02T10:50:45Z

Sure. Lemme take a look.

So main_utils.py no longer exists. Any such parameters (options/arguments) you've added there should be moved to the appropriate section under cli/render for which there are 4 current sections: global_options.py, output_options.py, render_options.py and ease_of_access_options.py.

I believe the two options (--livestream and --use-ipython) would best fit under output_options.py but I haven't actually reviewed this code much further than looking at the options. It just sorta feels like any options for this feature would be related to output -- live-stream output. But anyways, just pick a category and add the decorator the list:

I'll review the code more and give places/files to put the code

jsonvillanueva

Not many changes need to be made actually. @ me here or on Discord if you have further questions about the merge-conflicts

jsonvillanueva · 2021-04-02T11:01:48Z

docs/source/tutorials/configuration.rst

-   'partial_movie_dir', 'pixel_height', 'pixel_width', 'plugins', 'png_mode',
+   'partial_movie_dir', 'pixel_height', 'pixel_width', 'plugins', 'png_mode', 
   'preview', 'progress_bar', 'quality', 'right_side', 'save_as_gif', 'save_last_frame',
-   'save_pngs', 'scene_names', 'show_in_file_browser', 'sound', 'tex_dir',
+   'save_pngs', 'scene_names', 'show_in_file_browser', 'sound', 'streaming_dir', 'tex_dir', 
   'tex_template', 'tex_template_file', 'text_dir', 'top', 'transparent',
-   'upto_animation_number', 'use_opengl_renderer', 'use_webgl_renderer', 'verbosity',
-   'video_dir', 'webgl_renderer_path', 'webgl_updater_fps', 'write_all',
+   'upto_animation_number', 'use_opengl_renderer', 'use_webgl_renderer', 'verbosity', 
+   'video_dir', 'webgl_renderer_path', 'webgl_updater_fps', 'write_all', 


As of the click update, this section is no longer being doctested due to popular request, but feel free to update it here anyways.

jsonvillanueva · 2021-04-02T11:02:11Z

docs/source/tutorials/configuration.rst

+     --livestream          Run in streaming mode
+     --use-ipython         Use IPython for streaming


I don't think this section exists after the click merge

jsonvillanueva · 2021-04-02T11:03:05Z

manim/__main__.py

+
+        if args.livestream:
+            livestream(use_ipython=args.use_ipython)
+            return
+


This would need to move to cli/render/commands.py -- just keep the upstream changes for this file entirely

jsonvillanueva · 2021-04-02T11:04:54Z

manim/_config/main_utils.py

+    # Overrides default false streaming so streaming can happen
+    parser.add_argument(
+        "--livestream",
+        action="store_true",
+        help="Run in streaming mode",
+    )
+
+    # Optional use of IPython for streaming
+    parser.add_argument(
+        "--use-ipython",
+        action="store_true",
+        help="Use IPython for streaming",
+    )
+


this is entire file doesn't even exist anymore. these options would need to be moved to one of the four files I mentioned for categories under the render command. Argparse has action="store_true", click does is_flag=True

jsonvillanueva · 2021-04-02T11:05:22Z

manim/_config/utils.py

+        streaming_config = {
+            opt: parser["streaming"].get(opt, fallback="", raw=True)
+            for opt in [
+                "streaming_client",
+                "streaming_protocol",
+                "streaming_ip",
+                "streaming_port",
+            ]
+        }
+        url = parser["streaming"].get("streaming_url", fallback="", raw=True)
+        sdp_name = parser["streaming"].get("sdp_name", fallback="", raw=True)
+        streaming_config["streaming_url"] = url.format(**streaming_config)
+        streaming_config["sdp_path"] = os.path.join(
+            self.get_dir("streaming_dir"), sdp_name.format(**streaming_config)
+        )


This is fine, but the file still needs to resolve conflicts -- keep both changes

NeoPlato · 2021-04-04T08:51:16Z

@jsonvillanueva So far with the newest release of Manim I'm having great success working on the plugin compared to the PR. All attributes are easily accessible and if I figure out the parser, livestream configurations can easily be set in the optional manim.cfg files under a "livestream" key. I'll confirm completion some time next week when it's done.

I wish I'd picked this option from the start!

jsonvillanueva · 2021-04-04T09:56:18Z

I'm having great success working on the plugin compared to the PR.

Is there anything in particular that I could help with in the PR? You're certainly welcome to make this a plugin as well, whichever is easiest I guess -- it's just if you'd rather work on this being a plugin it'd be better to close this PR/ticket. I'm not sure we want to do that though...

NeoPlato · 2021-04-04T16:47:45Z

Is there anything in particular that I could help with in the PR?

Well, yes. I really, really don't want to go through adding livestreaming info into Manim's framework again. There's a manner in which I did it last time that mirrored what was already there, mainly so I didn't have to fully understand what was going on :). However, closing this PR prematurely is indeed irrational so I can try my hand at the plugin and PR simultaneously

NeoPlato · 2021-04-11T11:56:31Z

Update on this:

I have seen OpenGL. I am in tears, my business has been squandered. I finished the plugin though, just shy to enter the hassles of uploading to PyPi and write unit tests.

kilacoda-old · 2021-04-11T12:03:32Z

It'll be useful as long as we still have cairo, so don't worry!

WampyCakes · 2021-04-11T14:26:27Z

Doesn't livestreaming still have useful utility that OpenGL doesn't? Like can't livestreaming be piped into other software? Or am I wrong?

NeoPlato · 2021-04-11T16:09:36Z

@WampyCakes When you say piped into other software, do you mean being able to access the private network with other than ffplay or actually utilizing the video clips? At the moment (I'll update my repo in a minute) I can store streams in temporary directories that get wiped at the start of the next animation which is nice

WampyCakes · 2021-04-11T17:01:57Z

I'm not too familiar with it, but I mean a setup where what manim generates would go into another program rather than just being written to a file. I think when people do livestreams, it's kinda similar? Or like if you wanted to pipe it into a collaboration program in real time during a meeting or presentation. Maybe I'm wrong

NeoPlato · 2021-04-11T17:47:33Z

Or like if you wanted to pipe it into a collaboration program in real time during a meeting or presentation.

That level of customization is neat! I'd need examples but after all the effort that shouldn't be too hard.

NeoPlato · 2021-04-11T17:49:05Z

Anyone want to tell me I write code like a kindergartener? The floor is open. Now to @ some people:

@leotrs
@behackl
@kilacoda

NeoPlato · 2021-04-12T11:53:42Z

Anyone want to tell me I write code like a kindergartener? The floor is open.

It's not the cleanest piece of code out there but it is functional

NeoPlato · 2021-04-17T11:22:52Z

@behackl I've mostly been working on the version of ManimCommunity available on PyPi. My plan is to release that as the first version of livestreaming then work on this later on.

Thanks for the tip! I think I'll have to find a way to redirect from that method in the helper class when I get there.

behackl · 2021-04-30T22:15:05Z

@behackl I've mostly been working on the version of ManimCommunity available on PyPi. My plan is to release that as the first version of livestreaming then work on this later on.

Thanks for the tip! I think I'll have to find a way to redirect from that method in the helper class when I get there.

I should mention that I am really sorry for how this PR was handled from our point of view. Unfortunately, we don't really have the resources to deal with large-ish feature additions efficiently.

Thank you very much for going the extra mile multiple times, and now even moving your code to an extra plugin -- ultimately, I think that it is the correct decision to do that. From what I can tell from the PR in your repo, most (if not all) suggestions and improvements are included.

I particularly want to thank you for contributing to Manim's plugin ecosystem; I'm confident much more will follow in that direction in the coming weeks and months.

I'll close this PR here -- and do hope you will contribute again in the future!

NeoPlato added 30 commits November 26, 2020 17:39

Add files via upload

7725ec2

Add files via upload

a5523e2

Add files via upload

ab07add

Add files via upload

e275d11

Livestreaming option

b4f1fce

Please tell me this creates the pull request

Merge branch 'master' of https://github.com/NeoPlato/manim

8341da8

Add files via upload

1ed3f68

Update __init__.py

fd14d66

Update __init__.py

e8ba6f3

Merge branch 'master' of https://github.com/NeoPlato/manim

4a4be30

Update __init__.py

49e8759

Adding livestreaming support

22bab46

Merge branch 'master' of https://github.com/NeoPlato/manim

2254be0

More livestreaming options

5f77979

More livestreaming

0d08bc7

Update ffi.py

3c1fd9b

Update ffi_pixbuf.py

8d5507a

Delete stream_starter.py

75eeb79

Delete __init__.py

a6d1fa3

Delete __main__.py

44ce2d1

Delete default.cfg

eb7a137

Delete main_utils.py

4deae04

Delete utils.py

995db49

Delete stream_renderer.py

7490898

Delete cairo_renderer.py

6f4ce3b

Delete scene.py

3a67a2f

Delete stream_file_writer.py

4c04a9f

Delete streaming_scene.py

7b22388

Revert "Delete streaming_scene.py"

4f821bd

This reverts commit 7b22388.

Revert "Delete stream_file_writer.py"

3437a19

This reverts commit 4c04a9f.

NeoPlato added 2 commits March 29, 2021 22:48

Missed a spot

eb76955

Update stream_starter.py

515f836

NeoPlato added 2 commits March 31, 2021 12:23

Last of commits

641d68e

Pretty

67f6f14

jsonvillanueva reviewed Apr 2, 2021

View reviewed changes

behackl mentioned this pull request Apr 17, 2021

Add :meth:~.Scene.interactive_embed for OpenGL rendering #1285

Merged

12 tasks

behackl closed this Apr 30, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make livestreaming a thing. #778

Make livestreaming a thing. #778

NeoPlato commented Nov 27, 2020 •

edited

Loading

huguesdevimeux commented Mar 31, 2021 •

edited

Loading

NeoPlato commented Mar 31, 2021 •

edited

Loading

huguesdevimeux commented Mar 31, 2021

NeoPlato commented Apr 1, 2021

WampyCakes commented Apr 1, 2021

huguesdevimeux commented Apr 2, 2021

huguesdevimeux commented Apr 2, 2021

huguesdevimeux commented Apr 2, 2021

behackl commented Apr 2, 2021

jsonvillanueva commented Apr 2, 2021 •

edited

Loading

jsonvillanueva left a comment •

edited

Loading

jsonvillanueva Apr 2, 2021

jsonvillanueva Apr 2, 2021

jsonvillanueva Apr 2, 2021 •

edited

Loading

jsonvillanueva Apr 2, 2021

jsonvillanueva Apr 2, 2021 •

edited

Loading

NeoPlato commented Apr 4, 2021

jsonvillanueva commented Apr 4, 2021

NeoPlato commented Apr 4, 2021

NeoPlato commented Apr 11, 2021

kilacoda-old commented Apr 11, 2021

WampyCakes commented Apr 11, 2021

NeoPlato commented Apr 11, 2021

WampyCakes commented Apr 11, 2021 •

edited

Loading

NeoPlato commented Apr 11, 2021 •

edited

Loading

NeoPlato commented Apr 11, 2021 •

edited

Loading

NeoPlato commented Apr 12, 2021

NeoPlato commented Apr 17, 2021

behackl commented Apr 30, 2021

		--livestream Run in streaming mode
		--use-ipython Use IPython for streaming

Make livestreaming a thing. #778

Make livestreaming a thing. #778

Conversation

NeoPlato commented Nov 27, 2020 • edited Loading

List of Changes

Change 1

Change 2

Change 3

Change 4

Explanation for Changes

Testing Status

Good

Good

Superb depending on utility

Bad

Bad

Good

Further Comments

Acknowledgement

huguesdevimeux commented Mar 31, 2021 • edited Loading

NeoPlato commented Mar 31, 2021 • edited Loading

huguesdevimeux commented Mar 31, 2021

NeoPlato commented Apr 1, 2021

WampyCakes commented Apr 1, 2021

huguesdevimeux commented Apr 2, 2021

huguesdevimeux commented Apr 2, 2021

huguesdevimeux commented Apr 2, 2021

behackl commented Apr 2, 2021

jsonvillanueva commented Apr 2, 2021 • edited Loading

jsonvillanueva left a comment • edited Loading

Choose a reason for hiding this comment

jsonvillanueva Apr 2, 2021

Choose a reason for hiding this comment

jsonvillanueva Apr 2, 2021

Choose a reason for hiding this comment

jsonvillanueva Apr 2, 2021 • edited Loading

Choose a reason for hiding this comment

jsonvillanueva Apr 2, 2021

Choose a reason for hiding this comment

jsonvillanueva Apr 2, 2021 • edited Loading

Choose a reason for hiding this comment

NeoPlato commented Apr 4, 2021

jsonvillanueva commented Apr 4, 2021

NeoPlato commented Apr 4, 2021

NeoPlato commented Apr 11, 2021

kilacoda-old commented Apr 11, 2021

WampyCakes commented Apr 11, 2021

NeoPlato commented Apr 11, 2021

WampyCakes commented Apr 11, 2021 • edited Loading

NeoPlato commented Apr 11, 2021 • edited Loading

NeoPlato commented Apr 11, 2021 • edited Loading

NeoPlato commented Apr 12, 2021

NeoPlato commented Apr 17, 2021

behackl commented Apr 30, 2021

NeoPlato commented Nov 27, 2020 •

edited

Loading

huguesdevimeux commented Mar 31, 2021 •

edited

Loading

NeoPlato commented Mar 31, 2021 •

edited

Loading

jsonvillanueva commented Apr 2, 2021 •

edited

Loading

jsonvillanueva left a comment •

edited

Loading

jsonvillanueva Apr 2, 2021 •

edited

Loading

jsonvillanueva Apr 2, 2021 •

edited

Loading

WampyCakes commented Apr 11, 2021 •

edited

Loading

NeoPlato commented Apr 11, 2021 •

edited

Loading

NeoPlato commented Apr 11, 2021 •

edited

Loading