Managing single buffers (snapshot functionality)

Hello

I am the developer of a video annotation application
( http://liris.cnrs.fr/adv
ene/ ). We are mainly using VLC for the
moment, but I am in the process of writing a gstreamer
wrapper, which
opens a number of great possibilities (see
http://svn.gna.org/viewcvs/adven
e/trunk/lib/advene/player/gstreamer.py?view=auto )

The problem I am facing is the snapshot functionality. The
playbin
plugin offers a 'frame' property which points a gstBuffer
containing the
last decoded frame, in whatever format has been negociated
in the
pipeline. Now, I need this frame in PNG.

I have seen code that sets a video/x-raw-rgb capability, to
be able then
to use the Python Image Library to convert the result to
png. But I find
that it is a pity to have yet another dependency, when
gstreamer has
this capability.

I have been trying to do something like:

pngconverter = gst.parse_launch("freeze name=source !
video/x-raw-rgb,width=160 ! pngenc name=encoder ! fakesink
name=sink")

and then feed pngconverter.get_pad('src') with the
playbin.props.frame,
and get the encoded frame through the handoff signal of
fakesink. No
success so far... So I am asking the list for some advice:
do  you think
I am on the right track, or should I do something else? I
have been
thinking of using a tee to duplicate the video output to
both the
videosink and the conversion pipeline, but I would then need
to find a
way to only activate the converstion pipeline on demand,
since I do not
want to encode every frame.

Any advice would be welcome.

For information, I am also planning of writing a SVG overlay
module (I
have implemented this functionality for VLC, but it looks
like it could
be easy in gstreamer).

Thanks

Olivier


------------------------------------------------------------
-------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the
chance to share your
opinions on IT & business topics through brief surveys -
and earn cash
http://www.techsay.com/default.
php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
gstreamer-devel mailing list
gstreamer-devellists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/gstream
er-devel

On Tue, 2007-01-16 at 15:09 +0100, Olivier Aubert wrote:
> The problem I am facing is the snapshot functionality.
The playbin
> plugin offers a 'frame' property which points a
gstBuffer containing the
> last decoded frame, in whatever format has been
negociated in the
> pipeline. Now, I need this frame in PNG.

Just replying to myself... As I did not get any answer, but
in the end
figured out how to do it, I am posting the solution I use.
It may be of
use to someone else. It defines a conversion pipeline, that
can take a
single gst.Buffer (obtained from the 'frame' property of the
playbin
element) and convert it to png. In addition, some locking is
used to
obtain a synchronous behaviour (the snapshot method returns
the PNG
snapshot).

#! /usr/bin/python

import gst
import time
import gtk
from threading import Condition

player=gst.element_factory_make("playbin",
"player")
p_conv=gst.parse_launch('fakesrc name=src ! queue name=queue
! videoscale ! ffmpegcolorspace ! video/x-raw-rgb,width=160
! pngenc ! fakesink name=sink signal-handoffs=true')
p_conv._lock = Condition()

src=p_conv.get_by_name('src')
queue=p_conv.get_by_name('queue')
sink=p_conv.get_by_name('sink')

# Run the p_conv pipeline
p_conv.set_state(gst.STATE_PLAYING)

def buffer_cb(element, buffer, pad):
    print "Snapshot captured", buffer.timestamp /
gst.MSECOND
    p_conv._lock.acquire()
    p_conv._buffer=buffer
    p_conv._lock.notify()
    p_conv._lock.release()
    return True

sink.connect('handoff', buffer_cb)

def snapshot():
    """Synchronous snapshot method.

    Returns a gst.Buffer containing a scaled-down PNG
version of the
    current frame.
    """
    # Get the current frame
    b=player.props.frame.copy()

    # Lock the snapshot object
    p_conv._lock.acquire()
    p_conv._buffer=None

    # Push it into the conversion pipeline
    queue.get_pad('src').push(b)

    # Wait for the lock to be released
    while p_conv._buffer is None:
        p_conv._lock.wait()

    b=p_conv._buffer.copy()
    p_conv._lock.release()
    return b

player.props.uri='file:///tmp/foo.avi'
player.set_state(gst.STATE_PLAYING)

gtk.main()



------------------------------------------------------------
-------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the
chance to share your
opinions on IT & business topics through brief
surveys-and earn cash
http://www.techsay.com/default.
php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
gstreamer-devel mailing list
gstreamer-devellists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/gstream
er-devel

Olivier Aubert wrote:
> On Tue, 2007-01-16 at 15:09 +0100, Olivier Aubert
wrote:
>> The problem I am facing is the snapshot
functionality. The playbin
>> plugin offers a 'frame' property which points a
gstBuffer containing the
>> last decoded frame, in whatever format has been
negociated in the
>> pipeline. Now, I need this frame in PNG.
> 
> Just replying to myself... As I did not get any answer,
but in the end
> figured out how to do it, I am posting the solution I
use. It may be of
> use to someone else. It defines a conversion pipeline,
that can take a
> single gst.Buffer (obtained from the 'frame' property
of the playbin
> element) and convert it to png. In addition, some
locking is used to
> obtain a synchronous behaviour (the snapshot method
returns the PNG
> snapshot).
> 

that's great news! can you /reliably/ seek a video-file and
snapshot the
same frame?  - i guess id depends on the codec..

It does not seem that gstreamer itself can sync video
playback to an
external time source, but a few lines of python could!
however
pre-buffering snapshots introduces some latency.. let's see
how small it
gets..

robin

------------------------------------------------------------
-------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the
chance to share your
opinions on IT & business topics through brief
surveys-and earn cash
http://www.techsay.com/default.
php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
gstreamer-devel mailing list
gstreamer-devellists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/gstream
er-devel