r21189 - trunk/DOCS/tech/nut.txt

Author: michael
Date: Fri Nov 24 13:51:33 2006
New Revision: 21189

Modified:
   trunk/DOCS/tech/nut.txt

Log:
codec_specific_data clarification
if anyone disagrees or has suggestions to improve it then
shout


Modified: trunk/DOCS/tech/nut.txt
============================================================
==================
--- trunk/DOCS/tech/nut.txt	(original)
+++ trunk/DOCS/tech/nut.txt	Fri Nov 24 13:51:33 2006
 -568,6
+568,13 
     the exact format is specified in the codec spec
     for H.264 the NAL units MUST be formatted as in a
bytestream 
     (with 00 00 01 prefixes)
+    codec_specific_data SHOULD contain exactly the
essential global packets
+    needed to decode a stream, more specifically it SHOULD
NOT contain packets
+    which contain only non essential metadata like author,
title, ...
+    it also MUST NOT contain normal packets which cause the
reference decoder
+    to generate any specific decoded samples
+    the encoder name and version, shall be considered
essential as it is very
+    usefull to workaround possible encoder bugs
 
 frame_code
     frame_code is an 8-bit field which exists before every
frame, it can
_______________________________________________
NUT-devel mailing list
NUT-develmplayerhq.hu

http://lists.mplayerhq.hu/mailman/listinfo/nut-devel

On Fri, Nov 24, 2006 at 01:51:34PM +0100, michael wrote:

> if anyone disagrees or has suggestions to improve it
then shout

> +    codec_specific_data SHOULD contain exactly the
essential global packets
> +    needed to decode a stream, more specifically it
SHOULD NOT contain packets
> +    which contain only non essential metadata like
author, title, ...

For codecs with required stream-embedded metadata like ours,
I think 
this is just making work for the muxer. I'd allow such
packets, and
instead say that implementations SHOULD maintain and prefer 
container-level metadata with NUT. The packet should be
there, even
if it's minimal.

 -r
_______________________________________________
NUT-devel mailing list
NUT-develmplayerhq.hu

http://lists.mplayerhq.hu/mailman/listinfo/nut-devel

On Fri, Nov 24, 2006 at 09:05:31AM -0800, Ralph Giles wrote:
> On Fri, Nov 24, 2006 at 01:51:34PM +0100, michael
wrote:
> 
> > if anyone disagrees or has suggestions to improve
it then shout
> 
> > +    codec_specific_data SHOULD contain exactly
the essential global packets
> > +    needed to decode a stream, more specifically
it SHOULD NOT contain packets
> > +    which contain only non essential metadata
like author, title, ...
> 
> For codecs with required stream-embedded metadata like
ours, I think 
> this is just making work for the muxer. I'd allow such
packets, and
> instead say that implementations SHOULD maintain and
prefer 
> container-level metadata with NUT. The packet should be
there, even
> if it's minimal.

The packet can be there if it's required by the spec, but
the metadata
fields should all be blank, and should be completely ignored
by any
player. I would be in favor of a requirement that a
compliant player
MUST NOT present user-oriented metadata from codec bitstream
(just
like how using timestamps from the codec bitstream is
already
forbidden).

Rich

_______________________________________________
NUT-devel mailing list
NUT-develmplayerhq.hu

http://lists.mplayerhq.hu/mailman/listinfo/nut-devel

Hi

On Fri, Nov 24, 2006 at 12:37:31PM -0500, Rich Felker wrote:
> On Fri, Nov 24, 2006 at 09:05:31AM -0800, Ralph Giles
wrote:
> > On Fri, Nov 24, 2006 at 01:51:34PM +0100, michael
wrote:
> > 
> > > if anyone disagrees or has suggestions to
improve it then shout
> > 
> > > +    codec_specific_data SHOULD contain
exactly the essential global packets
> > > +    needed to decode a stream, more
specifically it SHOULD NOT contain packets
> > > +    which contain only non essential
metadata like author, title, ...
> > 
> > For codecs with required stream-embedded metadata
like ours, I think 
> > this is just making work for the muxer. I'd allow
such packets, and
> > instead say that implementations SHOULD maintain
and prefer 
> > container-level metadata with NUT. The packet
should be there, even
> > if it's minimal.
> 
> The packet can be there if it's required by the spec,
but the metadata
> fields should all be blank, and should be completely
ignored by any
> player. I would be in favor of a requirement that a
compliant player
> MUST NOT present user-oriented metadata from codec
bitstream 

hmm, i see 3 possibilities for xiph codecs
1. store the metadata packet as is
2. dont store the metadata packet
3. store a dummy (empty) metadata packet

the metadata has to be parsed and put in a common structure
at some
point in all cases be it the ogg demuxer, vorbis parser, nut
muxer or
whatever otherwise the metadata would be pretty much
unavailable to a nut
player (we cannot require every nut player to be able to
parse codec
specific metadata)

1. would cause the data to be duplicated, if one gets edited
by the user
   we have contradicting data, thats very bad, also the
stream headers
   would be larger then needed, and the stream headers get
repeated ...
2. would need some dummy or correct metadata packet to be
generated somewhere
   at the demuxer side together with the split global to
xiph packets if 
   needed (muxing in ogg or vorbis, ... decoder needing it)
3. would need some dummy metadata packet to be generated
somewhere at the
   muxer side, and then be replaced if needed by the correct
metadata at
   the demuxer side, spliting still is needed for muxing in
ogg or vorbis,
   ... decoders which need it

i dont like 1. at all, 2. and 3. are pretty much the same
if there is an established standard which requires all 3
packets to be
stored then no doubt we should follow that, thats also said
in nut.txt
"the exact format is specified in the codec spec"
but there is no such
thing, RTP says dont store metadata, other containers and
APIs combine
the 3 packets in various ways ...
so if xiph "officially" says store all 3 with the
following format
... in a single packet
then we will certainly do so, if not then i dont know what
would be
best

also in a generic framework its likely that the metadata
packet will
be parsed and the metadata be put into a common structure in
the ogg
demxuer or some parser not the nut muxer, as other muxers
also need
the metadata ...

similarely the reverse of this would be done in the ogg
muxer or some
bitstream filter not the nut demuxer as other demuxers set
metadata and
that should end up in the final ogg stream

[...]

-- 
Michael     GnuPG fingerprint:
9FF2128B147EF6730BADF133611EC787040B0FAB

In the past you could go to a library and read, borrow or
copy any book
Today you'd get arrested for mere telling someone where the
library is
_______________________________________________
NUT-devel mailing list
NUT-develmplayerhq.hu

http://lists.mplayerhq.hu/mailman/listinfo/nut-devel

On Fri, Nov 24, 2006 at 11:35:24PM +0100, Michael
Niedermayer wrote:

> hmm, i see 3 possibilities for xiph codecs
> 1. store the metadata packet as is
> 2. dont store the metadata packet
> 3. store a dummy (empty) metadata packet

I would vote for 1 or 3.

I disagreed with doing 2 in the RTP draft, but didn't insist
because the 
payload implementation already has to be codec-specific. For
general 
container use like this, I think it's important to require
the metadata 
packet where the codec requires it. This is less surprising
for 
implementors.

You do still have to know how the vorbis headers are packed,
but for 
playback there is still a difference between these two
procedures:

# option 1 or 3
for header in global_headers:
  submit_packet_to_codec(header);

# option 2
header_number = 0;
for header in global_headers:
  submit_packet_to_codec(header)
  header_number += 1
  if (header_number == 1):
    header = construct_dummy_metadata_packet()
    submit_packet_to_codec(header)

I'd rather have to write the first one.

This imposes an overhead of (8 bytes + one packet overhead)
on any 
header blob, assuming a minimal vorbis-style comment packet.
I would
leave the encoder id string in the packet, so 

Therefore I'd propose, "The global headers MUST consist
of the normal 
sequence of header packets required for codec
initialization, in the 
order expected by the codec. An implementation MAY strip
metadata and 
other redundant information not necessary for correct
playback from the 
global headers to save space in the bitstream."

 -r
_______________________________________________
NUT-devel mailing list
NUT-develmplayerhq.hu

http://lists.mplayerhq.hu/mailman/listinfo/nut-devel

On Fri, Nov 24, 2006 at 03:49:25PM -0800, Ralph Giles wrote:

> This imposes an overhead of (8 bytes + one packet
overhead) on any 
> header blob, assuming a minimal vorbis-style comment
packet. I would
> leave the encoder id string in the packet, so 

Oops. I forgot the magic, so that should be 7+8 bytes for an
empty 
packet. I would leave the encoder id string in the packet,
which would 
be and extra ~48 bytes in the global header blob.

Speaking of, can NUT info packets be compressed?

 -r
_______________________________________________
NUT-devel mailing list
NUT-develmplayerhq.hu

http://lists.mplayerhq.hu/mailman/listinfo/nut-devel

Hi

On Fri, Nov 24, 2006 at 04:00:37PM -0800, Ralph Giles wrote:
[...]
> 
> Speaking of, can NUT info packets be compressed?

no, there where some disscussions about compressing them by
using a table
of "common" types/values in a header and then
using just numbers instead of
strings but we didnt find a solution which everyone agreed
to

if it just me id add such a table to the first info packet
after the stream
headers ...

[...]
-- 
Michael     GnuPG fingerprint:
9FF2128B147EF6730BADF133611EC787040B0FAB

In the past you could go to a library and read, borrow or
copy any book
Today you'd get arrested for mere telling someone where the
library is
_______________________________________________
NUT-devel mailing list
NUT-develmplayerhq.hu

http://lists.mplayerhq.hu/mailman/listinfo/nut-devel

On Sat, Nov 25, 2006 at 01:17:18AM +0100, Michael
Niedermayer wrote:

> no, there where some disscussions about compressing
them by using a table
> of "common" types/values in a header and then
using just numbers instead of
> strings but we didnt find a solution which everyone
agreed to

What about running all the fields together and compressing
the blob 
with deflate, like the PNG zTXT chunk?

 -r
_______________________________________________
NUT-devel mailing list
NUT-develmplayerhq.hu

http://lists.mplayerhq.hu/mailman/listinfo/nut-devel

Hi

On Fri, Nov 24, 2006 at 03:49:25PM -0800, Ralph Giles wrote:
> On Fri, Nov 24, 2006 at 11:35:24PM +0100, Michael
Niedermayer wrote:
> 
> > hmm, i see 3 possibilities for xiph codecs
> > 1. store the metadata packet as is
> > 2. dont store the metadata packet
> > 3. store a dummy (empty) metadata packet
> 
> I would vote for 1 or 3.
> 
> I disagreed with doing 2 in the RTP draft, but didn't
insist because the 
> payload implementation already has to be
codec-specific. For general 
> container use like this, I think it's important to
require the metadata 
> packet where the codec requires it. This is less
surprising for 
> implementors.
> 
> You do still have to know how the vorbis headers are
packed, but for 
> playback there is still a difference between these two
procedures:
> 
> # option 1 or 3
> for header in global_headers:
>   submit_packet_to_codec(header);
> 
> # option 2
> header_number = 0;
> for header in global_headers:
>   submit_packet_to_codec(header)
>   header_number += 1
>   if (header_number == 1):
>     header = construct_dummy_metadata_packet()
>     submit_packet_to_codec(header)

submit_packet_to_codec(header)
submit_packet_to_codec(dummy_defined_in_a_static_char_array)
submit_packet_to_codec(header)

but in reality the demuxer will likely pass the single
global header
unchanged around, and the vorbis decoder (or wraper) would
then

option 1 or 3
parse_header1();
skip_header2();
parse_header3();

vs.
option 2
parse_header1();
parse_header3();

your method might depening on API end up looking like

demuxer_open(){
    blah blah
    for all streams
        if stream is one of vorbis, theora, ...
            stream->xiph_glob= alloc(3)
            stream->xiph_glob_len= alloc(3)
            stream->xiph_glob_index=0;
            for i in 3
                find len of packet
                stream->xiph_glob_len[i]= len
                stream->xiph_glob[i]= alloc(len)
                read packet into stream->xiph_glob[i]
}

demuxer_get_packet()
    if(stream->xiph_glob_index<3 &&
stream->xiph_glob[stream->xiph_glob_index]){
        packet.dts= dunno has none
        packet.pts= dunno has none
        packet.duration= 0
        packet.flags= ?
        packet.len=
stream->xiph_glob_len[stream->xiph_glob_index]
        memcpy(packet.data,
stream->xiph_glob[stream->xiph_glob_index],
packet.len);
        stream->xiph_glob_index++
       
stream->xiph_glob_len[stream->xiph_glob_index]=0
       
free(stream->xiph_glob[stream->xiph_glob_index])
        return
    }

considering that this would be needed for every non ogg
demuxer i wouldnt 
implement it like that ...


[...]

> Therefore I'd propose, "The global headers MUST
consist of the normal 
> sequence of header packets required for codec
initialization, in the 
> order expected by the codec. An implementation MAY
strip metadata and 
> other redundant information not necessary for correct
playback from the 
> global headers to save space in the bitstream."

i would s/expected by the codec/defined in the codec spec/

otherwise its highly unclear, think about mpeg and its
headers, the spec
is clear what is allowed and what not, specific
implementations might be
more forgiving but still its the mpegvideo spec which we
should follow
in that case


also i would rather say

An implementation MAY/SHOULD/MUST strip metadata and other
redundant
information not necessary for correct playback from the
global headers
as long as no incorrect values are stored and as long as the
striped result
is not less valid per codec spec as before striping

(probably that could be improved too ...)

anyway after thinking about that a little option 3 with the
encoder name
and version in the metadata packet seems like the best
solution ...

[...]
-- 
Michael     GnuPG fingerprint:
9FF2128B147EF6730BADF133611EC787040B0FAB

In the past you could go to a library and read, borrow or
copy any book
Today you'd get arrested for mere telling someone where the
library is
_______________________________________________
NUT-devel mailing list
NUT-develmplayerhq.hu

http://lists.mplayerhq.hu/mailman/listinfo/nut-devel

Hi

On Fri, Nov 24, 2006 at 04:30:02PM -0800, Ralph Giles wrote:
> On Sat, Nov 25, 2006 at 01:17:18AM +0100, Michael
Niedermayer wrote:
> 
> > no, there where some disscussions about
compressing them by using a table
> > of "common" types/values in a header and
then using just numbers instead of
> > strings but we didnt find a solution which
everyone agreed to
> 
> What about running all the fields together and
compressing the blob 
> with deflate, like the PNG zTXT chunk?

dependancy on zlib, and zlib is likely bigger then the
demuxer
furthermore if we compress each info packet seperately then
deflate
will likely not be very efficient, if we compress all info
packets
together then one damaged info packet would cause all info
packets
afterwards to be lost, and this also wont work with
"midstream" info
packets

[...]

-- 
Michael     GnuPG fingerprint:
9FF2128B147EF6730BADF133611EC787040B0FAB

In the past you could go to a library and read, borrow or
copy any book
Today you'd get arrested for mere telling someone where the
library is
_______________________________________________
NUT-devel mailing list
NUT-develmplayerhq.hu

http://lists.mplayerhq.hu/mailman/listinfo/nut-devel