CR-Client: MemCache support for HTTP

Modified by: milkoreal.com
Date: 08:22:07
Project: Atlas

Synopsis: MemCache support for HTTP

Overview:
The change supplies mem-cache functionality to
HTTP file system.

Mem-cache is memory resident MRU driven cache
that automatically trims during download.
The cache is useful to maintain an appropriate
possibly non-contiguous pool of data that is
in use and provide limited, localized random
access around most recently used blocks thus
eliminating need for re-acquisition of the data
from the server.

HTTP file system already supported MRU driven
disk based cache (aka ChunkyRes) which stored
the entire download on the disk.  For clients
utilizing Super-buffer (RecordControl) this
represented double storage of data.  Use
of disk also introduced performance issues on
systems displaying video on primary thread due
to the synchronous disk I/O.  For those reason
use of mem-cache is much preferred.

Already existing HELIX_FEATURE_HTTPFSYS_MEM_GROWTH_LIMIT
feature was overly simplistic to be a general solution
as it turnes the ChunkyRes into a linear cache which
does not work for formats requiring mulitple read
pointers into the file.
HELIX_FEATURE_HTTPFSYS_MEM_GROWTH_LIMIT also suffers from
possibility of having data trapped beyond the range of
data being read casing the buffer to fill and being unable
to
purge.

HELIX_FEATURE_HTTPFSYS_MEM_GROWTH_LIMIT can still be used
when new HTTP mem-cache is enabled.  Mem-cache fixes the
issue
of trapped data as it is purges data in non-liear fassion
when buffer is filled.

The new mem-cache feature can be enabled at compile time to
be used by default via HELIX_FEATURE_HTTP_MEMCACHE.
It can be also enabled/disabled at run-time (regardless of
compile-time setting) via HTTPMemCacheSize preference:
HTTPMemCacheSize = 0
	(mem-cache is disabled, disk cache is used unless
	 configured differently)
HTTPMemCacheSize = <size of cache in bytes>
	<mem-cache is enabled of the indicated size, disk

Mem-cache default size is 4MB and 0.5MB for MIN_HEAP
builds.
Trimming of mem-cache occurs in 25% of total size
incerements
when cache is full and any of the read pointers have less
than their equal share of 25% of total cache size of
contiguous
data ahead of them.

This changes also changes the deafult ChunkyRes settings:
MIN_HEAP build:
	Memory resident data	256KB	->	256KB	(no change)
	Page size		 32kb	->	 32kb	(no change)
Regular build:
	Memory resident data	256KB	->	  4MB
	Page size		 32KB	->	 64KB	

Modified files:
/common/fileio/chunkres.cpp,
common/fileio/pub/chunkres.h
/filesystem/http/httpfsys.cpp
/filesystem/http/httpfsys.h,

Image Size and Heap Use impact (Client -Only):
Small increase.

Platforms and Profiles Affected:
All

Distribution Libraries Affected:
none

Distribution library impact and planned action:
n/a

Platforms and Profiles Build Verified:
system id: macos-gcc4-xcode22,
target: LitePlayer
profile: helix-client-playerx
branch: playerx
Options: nodll

Platforms and Profiles Functionality verified:
OS-X

Branches:
HEAD, hxclient_3_1_0_atlas

_______________________________________________
Helix-client-dev mailing list
Helix-client-devhelixcommunity.org
http://lists.helixcommunity.org/mailman/listinf
o/helix-client-dev

CR-Client: MemCache support for HTTP

Thread: CR-Client: MemCache support for HTTP