John,
Comments inline below...
> -----Original Message-----
> From: filesystem-dev-bounces helixcommunity.org
> [mailto:filesystem-dev-bounceshelixcommunity.org] On
Behalf
> Of Wood, John
> Sent: Friday, June 16, 2006 4:42 AM
> To: filesystem-devhelixcommunity.org
> Subject: [Filesystem-dev] Using HTTP Filesystem as a
backend
> to Helix Server
>
> Hi,
>
> Apologies in advance if this is not the correct mailing
list to post
> this to.
>
> First just a bit of background:
>
> I'm trying to use the HTTP file system plugin with
Helix. What I'm
> trying to do is have it so that the Helix server uses a
HTTP server as
> the virtual file system when streaming out files.
>
> So for example, I'd use the URL:
>
> rtsp://helixserver/httpfs/http://webserver/myfile.rm
>
> This will then first hit the helix server using the
"httpfs" mount
> point. Then this uses the full "file path"
(which is of course a full
> URL in its own right) to then hit the webserver.
>
> I've had pretty good success with this - I can get
Helix to serve out
> anything from the webserver using both RTSP and MMS
protocols.
>
> However, what I've noticed is that when its doing
this, Helix server
> seems to use a lot of the CPU and a huge amount of
memory while the
> files are being streamed.
>
> (Before I continue I should mention that I have raised
a bug to this
> effect which explains this problem in more detail: Bug
ID 5043 -
> https://bugs.helixcommunity.org/show_bug.cgi?id=5
043&link=0).
>
> So, as I understand it, the HTTP filesystem plugin is
> designed to cache
> downloaded data to prevent lots of unnecessary hits of
the web server.
> However, it looks like it is caching the *full* file
each time.
>
> So, when I try streaming two different 200MB files from
two different
> clients, helix uses up over 400MB of memory.
>
> Also, for much of this time the CPU is running at 100%.
I've trying
> debugging the code, but I can't work out where the CPU
usage is coming
> from (even if I "slow down" the ProcessIdle
loop, the problem still
> remains, so I don't believe it is happening there).
>
> Its worth pointing out that both these unwanted
behaviours are NOT
> evident when I stream files using a different
filesystem plugin, hence
> why I believe the issue is in http-filesystem.
>
> So my questions are:
>
> 1) Is what I'm trying to do outside the scope of the
HTTP filesystem?
> Perhaps it is just not designed to allow streaming of
lots of large
> media files
filesystem/http/full is primarily designed to be a client
plugin.
It caches the file in chunks so that when seeks are done, it
doesn't
have to do another HTTP request to get the data.
I'm not sure right off the top of my head why the CPU usage
would be
so high - probably some interaction with some server
component. You might
try downloading/playing two files on the client and see if
the
CPU usage is as high. I doubt it would be - I've never seen
httpfsys
consume too much CPU.
> 2) I noticed a few preprocessor directives that seemed
to talk about
> caching. Is there any way I can tweak those to limit
the memory usage
> problem? (we can possibly live with the CPU behaviour
if the memory
> behaviour is controlled)
>
It sounds like you want to pretty much turn off caching all
together.
Well, I guess the only cache you would hold in memory would
be the
portion of the file in between the current read point and
the
current download point. I suppose you would not keep chunks
of data
which had already been read by the file format plugin.
There aren't any preprocessor defines to do exactly that,
but I
would think you could modify httpfsys to do that without
much
trouble.
==============================================
Eric Hyche (ehychereal.com)
Technical Lead
Embedded Player and Technologies
RealNetworks, Inc.
> The requirement to be able to use a HTTP server as a
backend
> filesystem
> for streaming is a "must have". So, I'm
willing to try and fix the
> existing filesystem if I could have a bit of guidance
from people.
>
> My alternative option would be to code my own plugin
which
> did something
> similar to filesystem-http but much more simple,
perhaps using a third
> party HTTP client such as libcurl.
>
> Any help would be greatly appreciated.
>
> John Wood
> johnw - at - hp.com
>
> _______________________________________________
> Filesystem-dev mailing list
> Filesystem-devhelixcommunity.org
> http://lists.helixcommunity.org/mailman/listinfo/
filesystem-dev
>
_______________________________________________
Filesystem-dev mailing list
Filesystem-devhelixcommunity.org
http://lists.helixcommunity.org/mailman/listinfo/
filesystem-dev
|