Re: multiple core support

Hmm,

You're actually correct, when you put it that way.  Because
the audio  
blocks are coded independently, you could speed things by
having the  
encoder (or decoder) do a little up-front processing on the
metadata,  
then use the overall settings to divide the file into
sections and  
run the FLAC process in parallel.  However, I think it is
generally  
frowned upon in some circles to create one thread per
processor,  
rather than making the division dependent upon some other
factor that  
is actually independent of the number of processors.  Apart
from that  
caveat, someone could certainly develop an encoder or
decoder which  
uses the FLAC library API to implement a threaded
encoder/decoder -  
well, assuming that the API can write to subsets of a file,
instead  
of requiring that the entire file be written from start to
finish -  
and also assuming that the API can run in multiple threads
and write  
to the same file.

By your method, decoding would be faster, too.  Not
everybody decodes  
just to listen.  Some of us keep original masters in FLAC
format for  
archival purposes, but need the whole thing decoded before
we can mix  
and master.  Anything which speeds up decoding would help
get work  
done faster.

On the flip side, another thing to consider is that not all
encoding  
is done with files that already exist.  If you are recording
live to  
FLAC, you cannot multi-thread the encoder because you don't
have any  
audio besides the current block.

So, decoding for playback and encoding for recording are two
examples  
which cannot be multi-threaded.  But file format conversion
could be  
developed to take advantage of multiple processors.  It
would require  
a lot of tweaks to the library, perhaps.

Note: PC games are multi-threaded because they're already
doing  
several different things at once.  Sometimes the complexity
is  
actually reduced by dedicating a thread to each part of the
game  
logic.  And precisely because there are so many different
things  
going on, multi-threading can speed things up when they
overlap in  
unpredictable ways.  Games would be harder to write in a
single- 
threaded way.  Meanwhile, encoding/decoding FLAC is easier
to write  
in a single-threaded design.

Brian W.


On Sep 6, 2007, at 16:42, Harry Sack wrote:

yes, i totally agree but I'm talking about the ENcoder 
You can perfectly 'cut' the pcm stream 'in half' and let 1
cpu encode
one part , the other 2nd part (or with 4 cpu's in 4 parts,
...),
because it's just a stream one frame after the other,  as
far as i
know (correct me if i'm wrong).
Many pc games synchronise multiple threads on 2 (even 4!)
cores (much
More complicated then reading frames, compressing, writing
compressed
frames like the encoder does), so i think making the encoder
support 2
cpu's is perfectly possible.

of course adding multi-core support to the decoder is silly
because
it's not necessary (any single core cpu is fast enough to
play 1 file
and it would only slow it down)!

2007/9/7, Brian Willoughby <brianwsounds.wa.com>:
> Any software which supports multiple processors must be
multi-
> threaded.  The process of designing multi-threaded code
adds
> complexity to the software, so there must be a good
reason to go
> through all the trouble.  The procedure for decoding a
single file
> could only benefit marginally from being
multi-threaded, because most
> operations would need to wait until the previous
operation was
> completed.  One section of the decoder might be able to
process
> several blocks, but the overall result would still need
to wait until
> all blocks are completed.  Besides, managing dynamic
buffers between
> each step of the decoding process would actually
require the decoder
> to do more, meaning it would slow down to an extent. 
There would
> pretty much be no difference in the total time, whether
single
> processor or multiple processors.
>
> The best way to take advantage of multiple processors
is to decode
> multiple FLAC files at the same time.  This will take
full advantage
> of your system, provided that the disk bandwidth and
memory bandwidth
> can keep up.
>
> Brian W.

_______________________________________________
Flac-dev mailing list
Flac-devxiph.org
http:
//lists.xiph.org/mailman/listinfo/flac-dev

it's really not complicated I think: only api changes to
write on any
position in the file if that's not possible already with
existing
function.

I'm not sure if decoding can have multi-core support: you
need an api
for writing pcm files in different parts then and this is
maybe more
difficult to check if it's valid pcm data since the decoder
can only
check for valid flac streams , not pcm streams i think.

you can make recording also support multi cores: just let
cpu 1 fill a
pcm data buffer while cpu 2 encodes it. In this case you
have a more
reliable encoding with less cpu usage of your system (it's
spread
among 2 cpu's)


2007/9/7, Brian Willoughby <brianwsounds.wa.com>:
> Hmm,
>
> You're actually correct, when you put it that way. 
Because the audio
> blocks are coded independently, you could speed things
by having the
> encoder (or decoder) do a little up-front processing on
the metadata,
> then use the overall settings to divide the file into
sections and
> run the FLAC process in parallel.  However, I think it
is generally
> frowned upon in some circles to create one thread per
processor,
> rather than making the division dependent upon some
other factor that
> is actually independent of the number of processors. 
Apart from that
> caveat, someone could certainly develop an encoder or
decoder which
> uses the FLAC library API to implement a threaded
encoder/decoder -
> well, assuming that the API can write to subsets of a
file, instead
> of requiring that the entire file be written from start
to finish -
> and also assuming that the API can run in multiple
threads and write
> to the same file.
>
> By your method, decoding would be faster, too.  Not
everybody decodes
> just to listen.  Some of us keep original masters in
FLAC format for
> archival purposes, but need the whole thing decoded
before we can mix
> and master.  Anything which speeds up decoding would
help get work
> done faster.
>
> On the flip side, another thing to consider is that not
all encoding
> is done with files that already exist.  If you are
recording live to
> FLAC, you cannot multi-thread the encoder because you
don't have any
> audio besides the current block.
>
> So, decoding for playback and encoding for recording
are two examples
> which cannot be multi-threaded.  But file format
conversion could be
> developed to take advantage of multiple processors.  It
would require
> a lot of tweaks to the library, perhaps.
>
> Note: PC games are multi-threaded because they're
already doing
> several different things at once.  Sometimes the
complexity is
> actually reduced by dedicating a thread to each part of
the game
> logic.  And precisely because there are so many
different things
> going on, multi-threading can speed things up when they
overlap in
> unpredictable ways.  Games would be harder to write in
a single-
> threaded way.  Meanwhile, encoding/decoding FLAC is
easier to write
> in a single-threaded design.
>
> Brian W.
>
>
> On Sep 6, 2007, at 16:42, Harry Sack wrote:
>
> yes, i totally agree but I'm talking about the ENcoder

> You can perfectly 'cut' the pcm stream 'in half' and
let 1 cpu encode
> one part , the other 2nd part (or with 4 cpu's in 4
parts, ...),
> because it's just a stream one frame after the other, 
as far as i
> know (correct me if i'm wrong).
> Many pc games synchronise multiple threads on 2 (even
4!) cores (much
> More complicated then reading frames, compressing,
writing compressed
> frames like the encoder does), so i think making the
encoder support 2
> cpu's is perfectly possible.
>
> of course adding multi-core support to the decoder is
silly because
> it's not necessary (any single core cpu is fast enough
to play 1 file
> and it would only slow it down)!
>
> 2007/9/7, Brian Willoughby <brianwsounds.wa.com>:
> > Any software which supports multiple processors
must be multi-
> > threaded.  The process of designing multi-threaded
code adds
> > complexity to the software, so there must be a
good reason to go
> > through all the trouble.  The procedure for
decoding a single file
> > could only benefit marginally from being
multi-threaded, because most
> > operations would need to wait until the previous
operation was
> > completed.  One section of the decoder might be
able to process
> > several blocks, but the overall result would still
need to wait until
> > all blocks are completed.  Besides, managing
dynamic buffers between
> > each step of the decoding process would actually
require the decoder
> > to do more, meaning it would slow down to an
extent.  There would
> > pretty much be no difference in the total time,
whether single
> > processor or multiple processors.
> >
> > The best way to take advantage of multiple
processors is to decode
> > multiple FLAC files at the same time.  This will
take full advantage
> > of your system, provided that the disk bandwidth
and memory bandwidth
> > can keep up.
> >
> > Brian W.
>
>
_______________________________________________
Flac-dev mailing list
Flac-devxiph.org
http:
//lists.xiph.org/mailman/listinfo/flac-dev