Some notes on an idea to make a Video Texture plugin for Ogre3D using GStreamer.

Togra (http://togra.sourceforge.net/) is a Python 3D API that can already feed GStreamer video to OpenGL? textures, and it is worth looking at how they do it, as it is a very similar problem.

According to a (the?) Togra developer, a bin (a chain of GStreamer filters) is passed to Togra, and the video source of a specially named element in the bin is then attached to an internal (to Togra) GStreamer plugin which feeds the resulting video data into the right texture (the plugin is internal to Togra as it needs access to internal Togra state such as the OpenGL context). This seems to be a good option. The bin could be specified as an xml file, which can be created with the gst-editor tool (although I think might not be an editor available for the latest release of gstreamer?), or perhaps using the shorthand gst-launch commandline syntax (which is sufficient for very simple chains).

One of the (planned?) features of Togra is the automatic handling of conversion between the abitrary aspect ratios and sizes of video content, and the texture size limitations of OpenGL (in the most restrictive case this means square power-of-two textures smaller than the current viewing window). (http://togra.sourceforge.net/wiki/index.php/Movie_Textures_Proposal). An early GStreamer ExternalTextureSource would probably quite useful without this. Maybe it would be possible to extract this functionality from Togra at a later stage?

OpenGL (and hence OGRE) can support a number of different pixel formats for textures, and some GStreamer video sources can generate (and convert between) several different formats. Because different formats perform better on different cards, it would probably be useful to be able to control which pixel formats are used for the final texture.

The benefits of a GStreamerTextureSource over the existing Theora video plugin for Ogre3D would be the ability to use any of the video codecs and sources supported by GStreamer. This includes streamed video, flash video, and live video via video4linux. Future codecs and sources would also be automatically supported (although changes in the GStreamer API would still need to be tracked).

It appears that GStreamer is a very powerful but generally misunderstood library for dealing with multimedia content.

– pix - 04 Aug 2006

I'm starting to think that the reason that gstreamer is so underutilised is because it uses GObject (a framework for doing object-orientated programming in C), which makes code that uses GStreamer look quite intimidating.