Roy Samuel wrote:
>
> Thank you for your reply.
> Only a DecodeAndDisplay function is available in our
hardware decoder. I
> would like to know if it would be appropriate to run
DecodeAndDisplay
> function which displays one frame right after
OnTimeSync returns a
> timestamp, so that no frames are lost.Or is there some
mechanism to
> ensure that no frames will be lost?
Please always CC a public mailing list. I may not always be
around
and others, who have more experience in this code, can help
answer
your questions.
Again, I don't know much about your hardware or your system.
You
may not always be able to ensure that no frames are lost, it
is
always possible that you can try to play video that is so
expensive
to decode (HD for example) that your device will *ALWAYS*
drop
frames in order to keep up with the audio time line. I think
that
is the best experience anyway. Seeing video that does not
match
up with the audio doesn't do any good. If I were you I would
take
a look at the rvrenderer code and see how it handles the
display
of frames from the OnTimeSync() calls. I know it has code to
drop
frames that are *late* (behind the audio timeline). I think
you
will have to do something like that. What code is calling
your
DecodeAndDisplay() function? A renderer you wrote yourself?
Perhaps
you should look at our base video renderer code and use
that.
--greg.
> Looking forward to your reply.
>
> Thanks and regards.
>
>
> *Greg Wright <gwright real.com>*
> Sent by: helix-client-dev-bounceshelixcommunity.org
>
> 12/20/2007 01:02 AM
>
>
> To
> Roy Samuel <Roy.Samuellntinfotech.com>
> cc
> helix-client-devlists.helixcommunity.org
> Subject
> Re: [Helix-client-dev] mp4 audio/video sync problem
>
>
>
>
>
>
>
>
> Roy Samuel wrote:
> >
> > Hi Greg, Thanks for your reply. For synchronizing
audio/video, how do I
> > make use of the timestamps? Which frames and how
many frames should I
> > decode after receiving timestamp from
(OnTimeSync)?
>
> Well, I don't know of any hard and fast rule. You
should have enough
> frames decoded so that when an OnTimeSync() call comes
in you can
> respond to it and display that frame for that
timestamp. You may
> get time stamps that are the same as the last one or
timestamps
> that are between to frame times(especially for low
frame rate
> clips). Basically, that time stamp you get from
OnTimeSync() tells
> you where the audio stream is at. That is, what part of
the time
> line the user is actually listening to at that time.
So, you need
> to show the user for the video frame that corresponds
to that moment
> in time.
>
> --greg.
>
>
> >
> > --Roy
> >
> >
> >
> > *Greg Wright <gwrightreal.com>*
> >
> > 12/10/2007 09:12 AM
> >
> >
> > To
> > roy.samuellntinfotech.com
> > cc
> >
> > Subject
> > Re: [Helix-client-dev] mp4
audio/video sync problem
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > Subject: Re: [Helix-client-dev] mp4 audio/video
sync problem
> > Date: Wed, 05 Dec 2007 10:09:16 -0800
> > From: Greg Wright <gwrightreal.com>
> > To: howardaizenveoda.com
> >
> > anish~the~poduval wrote:
> > > Hi Greg,
> > > Thanks for your reply.
> > > Yes that right...I'm only decoding video in
memory and not audio.
> Audio
> > is
> > > being decoded by software itself. The
profile I'm using is
> > > helix-client-all-defines-free. Sorry for
the misdirection with
> mentioning
> > > the profile as helix-client-local-mp3. That
was a typo.
> > > Looking forward to receiving a reply from
you for the queries posted
> > > earlier! 
> >
> > Ok. Reading that DSP document was still good
though since it will
> > give you a good idea of how the timeline is
managed. The thing to
> > remember is that the audio device generates the
master timeline. It
> > will tell the media engine what the time is of
the PCM data the user
> > is hearing right at that point. That time is then
passed into each
> > renderer via OnTimeSync. It is the renderers
responsibility to display
> > the video data associated with that time. So, if
yo have a video
> > renderer and it controls your DSP, then you
should have good A/V
> > sync if you renderer the video frame that
corresponds to that time
> > you get in OnTimeSync(). Now, you can do timeline
smoothing if you
> > want, but basically you need to use that time as
the master time
> > line. It sounds like you were doing that before,
so there must be
> > something going wrong. Perhaps the decoding is
taking too long?
> > Maybe the frames are not making it to the
framebuffer in time?
> >
> > What kind of sync issues do you see, too early,
too late, jitter?
> >
> > --greg.
> >
> >
> >
> >
> > >
> > > Thanks and regards.
> > > -Roy Samuel
> > >
> > > On Tue, 04 Dec 2007 10:18:07 -0800, Greg
Wright <gwrightreal.com>
> wrote:
> > >> anish~the~poduval wrote:
> > >>> Hi Eric & Greg,
> > >>> Thank you for your suggestion
regarding where to start for the
> hardware
> > >>> decoder integration.
> > >>> In my case, I will have to proceed
with the "Video Decode and
> Display"
> > >>> setup, where the decoding and
displaying is done directly from the
> > >> hardware
> > >>> decoder.
> > >>> The DSPDecoderBase class will have
to be used for synchronisation
> > >> purposes.
> > >>
> > >> So, now it sounds like you are only
decoding video in memory, not
> > >> audio? Is audio being decoded in
software like normal? This can make
> > >> a big difference on how you attack the
problem.
> > >>
> > >> Also, please move away from
helix-client-local-mp3 and use
> > >> helix-client-all-defines.
> > >>
> > >>
> > >> --greg.
> > >>
> > >>> As per the guide, I'll have to
create the SampleDSP class. This
> class
> > >> will
> > >>> contain the hardware initialize
function [video_lcd_initialize] and
> > also
> > >>> the decode function
[video_decodeAndDisplay()].
> > >>>
> > >>> This class will have to inherit the
DSPDecoderBase class.
> > >>>
> > >>> I'll also have to create the
wrapper function, which will call the
> > >> init()
> > >>> function of the DSPDecoderBase
class.
> > >>> ***Where should this wrapper
function be present? is this along with
> > the
> > >>> renderer?
> > >>>
> > >>> ***Where exactly do I call the
methods of SampleDSP class, namely,
> > >>> "video_lcd_initialize()"
and "video_decodeAndDisplay()" ? Should
> this
> > be
> > >>> called in the vidrend.cpp (video
renderer)?
> > >>>
> > >>> Also,
> > >>>
> > >>> ***how do I use the
IHXTimelineWatcher, this interface is not
> listed in
> > >> the
> > >>> Interface list of HelixSDK.
> > >>> ***How do I use the methods of this
interface to synchronize
> audio and
> > >>> video?
> > >>>
> > >>> Also,
> > >>>
> > >>> ***How do I obtain the
/datatype/common/dspcodec module? what is the
> > >> target
> > >>> that I should set in the ribosome
build, so that the dspcodec module
> > can
> > >> be
> > >>> built.
> > >>>
> > >>> Thanks and regards,
> > >>>
> > >>> Roy Samuel
> > >>>
> > >>> On Mon, 03 Dec 2007 10:28:13 -0800,
Greg Wright <gwrightreal.com>
> > >> wrote:
> > >>>> Eric Hyche wrote:
> > >>>>> Have you reviewed the
hardware decoder integration guide?
> > >>>>>
> > >>>>> https://client.helixcommunity.org/2004/devdocs/dsp_inte
> > >>>>>
> > >>>>> A/V sync problems when
integrating a hardware
> > >>>>> video decoder usually arise
because the the dummy PCM
> > >>>>> stream returned to helix
does not match up with the
> > >>>>> frames output.
> > >>>>>
> > >>>>> Greg has a lot of
experience with this - he may want
> > >>>>> to comment further...
> > >>>> I would start with that guide.
Also, your profile is a MP3
> > >>>> local playback only, you should
make sure you use a profile
> > >>>> that supports video if the
below is not a typo.
> > >>>>
> > >>>> --greg.
> > >>>>
> > >>>>> Eric
> > >>>>>
> > >>>>>
=============================================
> > >>>>> Eric Hyche (ehychereal.com)
> > >>>>> Technical Lead
> > >>>>> RealNetworks, Inc.
> > >>>>>
> > >>>>>> -----Original
Message-----
> > >>>>>> From:
helix-client-dev-bounceshelixcommunity.org
> > >>>>>>
[mailto:helix-client-dev-bounceshelixcommunity.org] On
> > >>>>>> Behalf Of
anish~the~poduval
> > >>>>>> Sent: Monday, December
03, 2007 1:17 AM
> > >>>>>> To:
helix-client-devlists.helixcommunity.org
> > >>>>>> Subject:
[Helix-client-dev] mp4 audio/video sync problem
> > >>>>>>
> > >>>>>> Hi All,
> > >>>>>> We are working on Helix
Client code with following
> specifications:
> > >>>>>> Branch:
hxclient_3_1_0_atlas
> > >>>>>> Target: splay
> > >>>>>> Profile:
helix-client-local-mp3
> > >>>>>> The Target splay has
been created for arm 9 board with the
> > >>>>>> SYSTEM_ID:
linux-2.2-libc6-armv5te-cross-gcc3.3-ixp4xx-softfloat
> > >>>>>> using Montavista linux
3.4.3 with 1. gcc compiler 3.3.1 2.
> > >>>>>> cross-compiler:
arm_v5t_le-g++ version 3.4.3. 3. Kernel
> > >>>>>> Version: 2.6.10
> > >>>>>>
> > >>>>>> We need to play a .mp4
file with MPEG4 video.we are using h/w
> > >>>>>> engine for
> > >>>>>> decoding and playback.
After we debugged the code we found
> > >>>>>> that the time
> > >>>>>> stamps are being
calculated in CVideoRenderer::OnTimeSync so
> we are
> > >>>>>> decoding and displaying
in that function itself. Is that the
> > >>>>>> right way to
> > >>>>>> do. Are we proceeding
in the right direction??? We are facing AV
> > sync
> > >>>>>> issues after we
integrated. Kindly guide us and suggest us
> > >>>>>> some idea on
> > >>>>>> this regard
> > >>>>>>
> > >>>>>> thanks in advance
> > >>>>>>
> > >>>>>> Anish Poduval
> >
> >
> >
> >
> >
> >
> >
> >
____________________________________________________________
__________
>
>
> _______________________________________________
> Helix-client-dev mailing list
> Helix-client-devhelixcommunity.org
> http://lists.helixcommunity.org/mailman/listinf
o/helix-client-dev
>
>
____________________________________________________________
__________
>
>
>
____________________________________________________________
__________
_______________________________________________
Helix-client-dev mailing list
Helix-client-devhelixcommunity.org
http://lists.helixcommunity.org/mailman/listinf
o/helix-client-dev
|
