Video packetization proposal

----- Original Message ----- 
From: "Mihai Balea" <mihaihates.ms>
To: "Asterisk Developers Mailing List"
<asterisk-devlists.digium.com>
Cc: <asterisk-videolists.digium.com>
Sent: Saturday, June 02, 2007 12:32 AM
Subject: Re: [asterisk-dev] Video packetization proposal


>
> On Jun 1, 2007, at 6:22 PM, Sergio Garcia Murillo
wrote:
>
> > Hi Mihai, just a few comments.
> >
> > IMHO video packetization should be transparent to
IAX (as is in
> > rtp) so I
> > would only
> > allow the data to be only a full video packet (not
transmit
> > partiall video
> > packets or mix
> > several packets in one IAX packet). If a codecs
doesn't support
> > pacektization the
> > specification should be outside of the scope of
this document.
>
> Theora does not support packetization and it is,
unfortunately, the
> only free (as in beer and speech) video codec.  I
believe that if we
> fail to address the issue of packetization, then we
stunt the
> development of open source video solutions.  That being
said, I tried
> to leave video specific information out of the generic
header and
> into the extended portion.  The video specific flags I
describe in
> the document should be interpreted as a suggestion, as
some codecs
> like h264 might not need them.
>

Then, create another document to adress the theora
packetization
independently
of IAX, so we could use it also for rtp.


> >
> > As in rtp, only one bit is really needed to signal
the last packet
> > of a
> > frame. You can detect
> > packets losses and frame changes with secuence
numbers and timestamps.
> Yes, but if one extra bit makes my life as a programmer
easier, I
> would go for that.

Avoiding "if"s are going to make your life as a
programer easier?
And you should put those same to ifs whe dealing rtp>IAX
translation.
Duplicating information on the header I think it's not a
good idea.

>
> > I,P,B framet indication is not needed as it should
the decoder the
> > one who
> > is in charge of that.
> Some application benefit from knowing the type of frame
without
> actually decoding it.  One example would be video
conferencing
> applications, and more specifically app_conference. 
When you want to
> switch the video stream from one person to another, you
want to do it
> on a key frame, otherwise you get garbage.
>

Yes, It will make life easier for app_conference, but again
it will make
very
difficult the rtp>IAX transaltion as you'll have to dig
into the encoded
bitstream
to get that info.

> > The timpestamp conundrum, this is quite a hard
problem.
> > As you have defined in your document the iax field
would be a timer
> > reference or frame duration
> > instead of a timespamp. I've do it that why youll
have a big
> > problem if you
> > want to translate
> > from rtp to IAX. The only way you can do it is
storing all the
> > packets of a
> > frame till you've got
> > the first packet of the next video frame, then
(and only then) you
> > could
> > calculate your "timestamp" field.
> It is not the frame duration, it is the time difference
(in ms)
> between the time of transmission of this frame and the
time of
> transmission of the first frame in the call.

Ok, I understood wrong the header, but then there should be
no problem
converting
from one value to another at all.

>
> > Also, it has another problem, if a frame has more
than one packet,
> > you're
> > going to set the duration on
> > the first one? or in every one?
> My proposal does not allow multiple video frames in one
IAX2 frame.
> RTP packetization for h264 and theora (at least xiph's
proposal) does
> allow for that, but I believe that video frames are
large enough to
> be transported one per IAX frame (or one per multiple
IAX frames).

I wasn't saying to allow multiple video frames in a packet,
just what asking
how
the ts would be handled for the all the packets of the
frame.
But this arise another wuestion, what are you going to do
with h264? I
haven't still
have got much time to study the h263 packetization, but if
seen that it
allows many
NALS to be send in one rtp packet. How would you set the I/P
bit then? Are
you
proposing to split that frame into multiple ones?


Greetings
Sergio

_______________________________________________
--Bandwidth and Colocation provided by Easynews.com --

asterisk-video mailing list
To UNSUBSCRIBE or update options visit:
   http://lists.digium.com/mailman/listinfo/asterisk-video

On Jun 2, 2007, at 5:53 AM, Sergio Garcia Murillo wrote:
> Then, create another document to adress the theora
packetization
> independently
> of IAX, so we could use it also for rtp.

People have already came up with proposals for Theora
packetization  
for RTP. Here's a draft from Xiph:

http://svn.xiph.org/trunk/theora/doc/draft-
ietf-avt-rtp-theora-00.txt

I borrowed liberally from that draft when I wrote my
proposal, but  
there are things in there that I believe are not necessary. 
For  
example they allow multiple video frames per RTP frame.  I
believe  
that it is far more likely to have video frames that are too
big to  
fit in one network frame than to have video frames small
enough to  
warrant coalescing them.

As far as I know that draft hasn't got a lot of circulation
and I'm  
not aware of any implementations.  As such, I don't think it
should  
be considered gospel.

>> Yes, but if one extra bit makes my life as a
programmer easier, I
>> would go for that.
>
> Avoiding "if"s are going to make your life as
a programer easier?
> And you should put those same to ifs whe dealing
rtp>IAX translation.
> Duplicating information on the header I think it's not
a good idea.
Actually it will make my life easier, especially in the case
when we  
have to deal with lost or miss-ordered frames.
Why exactly do you think it is a bad idea? You could easily 

encapsulate the specific video/Theora bits into a RTP frame
making  
the IAX2-RTP translation almost trivial.

>> Some application benefit from knowing the type of
frame without
>> actually decoding it.  One example would be video
conferencing
>> applications, and more specifically app_conference.
 When you want to
>> switch the video stream from one person to another,
you want to do it
>> on a key frame, otherwise you get garbage.
>>
>
> Yes, It will make life easier for app_conference, but
again it will  
> make
> very
> difficult the rtp>IAX transaltion as you'll have to
dig into the  
> encoded
> bitstream
> to get that info.
What I am trying to avoid is digging into the encoded
bitstream in  
application like app_conference.  Again, if we make this
bits part of  
the video/Theora payload, then moving the payload between
IAX and RTP  
stream should be easy.

>>> The timpestamp conundrum, this is quite a hard
problem.
>>> As you have defined in your document the iax
field would be a timer
>>> reference or frame duration
>>> instead of a timespamp. I've do it that why
youll have a big
>>> problem if you
>>> want to translate
>>> from rtp to IAX. The only way you can do it is
storing all the
>>> packets of a
>>> frame till you've got
>>> the first packet of the next video frame, then
(and only then) you
>>> could
>>> calculate your "timestamp" field.
>> It is not the frame duration, it is the time
difference (in ms)
>> between the time of transmission of this frame and
the time of
>> transmission of the first frame in the call.
>
> Ok, I understood wrong the header, but then there
should be no problem
> converting
> from one value to another at all.
Well, there are slight differences in the semantics of the
two values.

> I wasn't saying to allow multiple video frames in a
packet, just  
> what asking
> how
> the ts would be handled for the all the packets of the
frame.
> But this arise another wuestion, what are you going to
do with h264? I
> haven't still
> have got much time to study the h263 packetization, but
if seen  
> that it
> allows many
> NALS to be send in one rtp packet. How would you set
the I/P bit  
> then? Are
> you
> proposing to split that frame into multiple ones?

If the RTP frame comprises one video frame, then there would
be no  
problem in having multiple NALs in the IAX frame as well. 
If the  
NALs mean multiple video frames, then I believe they should
be split  
over into several IAX frames, each one having its own
timestamp.

Mihai
_______________________________________________
--Bandwidth and Colocation provided by Easynews.com --

asterisk-video mailing list
To UNSUBSCRIBE or update options visit:
   http://lists.digium.com/mailman/listinfo/asterisk-video

On 6/2/07, Sergio Garcia Murillo <sergio.garciafontventa.com> wrote:
> > > I,P,B framet indication is not needed as it
should the decoder the
> > > one who is in charge of that.
> > Some application benefit from knowing the type of
frame without
> > actually decoding it.  One example would be video
conferencing
> > applications, and more specifically
app_conference.  When you want to
> > switch the video stream from one person to
another, you want to do it
> > on a key frame, otherwise you get garbage.
>
> Yes, It will make life easier for app_conference, but
again it will make
> very difficult the rtp>IAX transaltion as you'll
have to dig into the encoded
> bitstream to get that info.

I'm no RTP expert, but doesn't the 7-bit payload type (PT)
field in
the RTP header serve the same purpose as the 8-bit payload
type that
Mihai proposes? RFC 3551 describes some well-known mappings
for the
A/V profile.

You seem to be proposing that instead of identifying the
video format
in this header field, the receiver should pick apart the
data payload
to figure out what kind of data is present. This seems
wrought with
peril for two reasons.

First, I don't know of any guarantee that various video
codecs will
have identifiable payloads. Without the payload type header,
we would
be demanding that the data payload have sufficient
information for
both identifying the codec _and_ determining codec
parameters.

Secondly, even if there was a way for receivers to identify
the
payload type, they would then have the burden of knowing the
magic
algorithms for determining the payload types for all the
codecs/formats it might run into. This seems like an undue
burden for
clients.

> > > The timpestamp conundrum, this is quite a
hard problem.
> > > As you have defined in your document the iax
field would be a timer
> > > reference or frame duration instead of a
timespamp. I've do it that why
> > > youll have a big problem if you want to
translate from rtp to IAX. The
> > > only way you can do it is storing all the
packets of a frame till you've
> > > got the first packet of the next video frame,
then (and only then) you
> > > could calculate your "timestamp"
field.
> > It is not the frame duration, it is the time
difference (in ms)
> > between the time of transmission of this frame and
the time of
> > transmission of the first frame in the call.
>
> Ok, I understood wrong the header, but then there
should be no problem
> converting from one value to another at all.

You are right that the obvious conversion would be trivial.
However,
there is a semantic difference between RTP and IAX. The RTP
timestamp
is the presentation time for the media. The IAX timestamp is
the
transmission time. Presentation time and transmission time
may or may
not be related. I really don't know. For asterisk, the
transmission
time is important for dejittering the packet stream. There
seems to be
an implicit assumption that presentation time is "upon
receipt" which
is different than "when the packet says".

Also, the RTP timestamp uses a 90kHz clock. The IAX
timestamp is
measured in milliseconds which is effectively a 1kHz clock.
There is a
rather large difference in precision between these two thus
information would be lost in RTP to IAX mappings and IAX
obviously
does not have sufficient information to match RTP's
precision in the
IAX to RTP mapping case. Does this matter? Seems like it is
worth
consideration.

Pete
_______________________________________________
--Bandwidth and Colocation provided by Easynews.com --

asterisk-video mailing list
To UNSUBSCRIBE or update options visit:
   http://lists.digium.com/mailman/listinfo/asterisk-video