streaming-tools

Not planned, but…
AI found lots of bugs and issues.
They aren’t fixed or implemented yet (I didn’t even know that).
So… still fixing them and building the Graph System.

Still finishing graph layer to gst.
Frontend not start yet.

Finished implementing the basic function for parseNode.
Still need to update the graph to add more capabilities directly to the node system (such as removing pads from nodes, which is not yet implemented).

The node rewrite is almost complete, with only a few nodes left to migrate to the new type.
Once all tests pass and the migrations are finished, I can start working on the stream (making sure it can push and verify first using the test pipeline built on top of the node system).
After that, I’ll move on to building the frontend.

I am rewriting the node system(which allows frontend nodes to be transformed into backend nodes,and then builds a gst.pipeline to process them),
Which is still incomplete.
My goal is to let you create a pipeline with minimal media processing knowledge, while still being able to control most things and understand the process (just simpler).

Designing Nodes (GStreamer)
Wrap GStreamer components into nodes (like a box) and sync with the frontend.
But with the features GStreamer has, I need to check the differences between the frontend and backend graphs and apply them to the GStreamer pipeline.
Which is difficult, and I haven’t come up with a good idea yet.
If anyone can give me some advice, that would be great.

1. Add a simple frontend using SvelteFlow
   (This is my first time using Svelte)
2. Wrap GStreamer into node-like abstractions
   (This will sync with the frontend)

• Note: This workflow is just to get started with SvelteFlow; it’s not the final result.
• Note: The GStreamer component is still a work in progress (WIP).

Figuring out how to parse video and mux it into the target format (FLV).
Asked AI to understand how GStreamer works and how to build a proper pipeline.
Also tried to start SRT and WHIP listeners.
*WHIP doesn’t work for this use case because it can’t handle multiple audio tracks.

Try to use gst to encode / decode video (from file now)
Use 2 mpv to play video and audio(autoaudio/video won’t work and i have no idea.)
Also platfrom’s chat room fetcher (not complete yet)

There is it’s pipeline:

filesrc → matroskademux ──(video/x-av1)→ av1parse → av1dec → videoconvert → x264enc → queue → appsink ─→ mpv #1 (H.264)
                        └──(audio/x-opus)→ opusdec → queue → audioconvert → wavenc → appsink ─→ mpv #2 (WAV)