tunning server for just 1 page

hello groups,

i will have a server just for serving 1 static page. i think
lighttpd is enough tunned and with enough performance for
this, but i wonder 
about server optimization. some questions are flying in my
head...

- somebody told me about putting the html file VM cache
- what filesystem is better ?
- this config will be the best ?
	server.max-worker = 8
	server.max-fds = 8192
	server.max-connections = 16384
	server.max-keep-alive-requests = 256
	server.event-handler = "linux-sysepoll"
	server.network-backend = "linux-sendfile"
- squid ?
- varnishd ?


any ideas ? 

thank you !

On 2007-05-30 00:45:17 +0200, Skiiiks wrote:
> i will have a server just for serving 1 static page. i
think lighttpd is 
> enough tunned and with enough performance for this, but
i wonder about 
> server optimization. some questions are flying in my
head...
> 
> - somebody told me about putting the html file VM
cache
> - what filesystem is better ?

uhm it really depends how big your content is. for a small
set of files
i would use shmfs (/dev/shm on linux) and as long it isnt
NFS or so you
should be fine.

> - this config will be the best ?
> 	server.max-worker = 8

are you sure this is needed?
how many different static files do you have? how big? if
they are small
enough put them into /dev/shm and using a single worker (aka
leaving out
the setting) should be fast enough

> 	server.max-fds = 8192
> 	server.max-connections = 16384

quoting my default config for the suse rpm:
[[[
##
## As lighttpd is a single-threaded server, its main
resource limit is
## the number of file descriptors, which is set to 1024 by
default (on
## most systems).
##
## If you are running a high-traffic site you might want to
increase this
## limit by setting server.max-fds.
##
## Changing this setting requires root permissions on
startup. see
## server.username/server.groupname.
##
## By default lighttpd would not change the operation system
default.
## But setting it to 2048 is a better default for busy
servers.
##
server.max-fds = 2048
##
## Fine tuning for the request handling
##
## max-connections == max-fds/2 (maybe /3)
## means the other file handles are used for fastcgi/files
##
server.max-connections = 1024
]]]

to explain the /[23] part:
1. fd => network socket
2. fd => file/fcgi socket
3. fd => access log?

you might want to toogle your numbers or increase max-fds
drastically
although max-fds at 16384 and max-connections at 8192 sounds
reasonable.

> 	server.max-keep-alive-requests = 256

if it should only serve one static page this can be tuned
down. to hm
... like 4 or 8 not more.

> 	server.event-handler = "linux-sysepoll"
> 	server.network-backend = "linux-sendfile"

good choice

> - squid ?
> - varnishd ?

why?

    darix

-- 
           openSUSE - SUSE Linux is my linux
               openSUSE is good for you
                   www.opensuse.org

On 5/29/07, Marcus Rueckert <darixweb.de> wrote:
> On 2007-05-30 00:45:17 +0200, Skiiiks wrote:
> > i will have a server just for serving 1 static
page. i think lighttpd is
> > enough tunned and with enough performance for
this, but i wonder about
> > server optimization. some questions are flying in
my head...
> >
> > - somebody told me about putting the html file VM
cache
> > - what filesystem is better ?
>
> uhm it really depends how big your content is. for a
small set of files
> i would use shmfs (/dev/shm on linux) and as long it
isnt NFS or so you
> should be fine.

Indeed.  Is it one file that will totally fit in memory or
is it a big
video file or what?

Basically, if it will all fit into memory then it's pretty
much
irrelevant what filesystem it originally comes from.

> > - this config will be the best ?
> >       server.max-worker = 8
>
> are you sure this is needed?
> how many different static files do you have? how big?
if they are small
> enough put them into /dev/shm and using a single worker
(aka leaving out
> the setting) should be fast enough

Also, how many cores do you have on the hardware.

Of course, how fat and how many network drops will also play
into this.


[...]
> >       server.max-keep-alive-requests = 256
>
> if it should only serve one static page this can be
tuned down. to hm
> ... like 4 or 8 not more.

Um, er, if it's just one static file then there shouldn't be
any need
for keep-alive, right? 

Take care,
John

On 2007-05-30 10:30:40 +0200, Skiiiks wrote:
> Marcus Rueckert wrote:
> >On 2007-05-30 00:45:17 +0200, Skiiiks wrote:
> >  
> >>i will have a server just for serving 1 static
page. i think lighttpd is 
> >>enough tunned and with enough performance for
this, but i wonder about 
> >>server optimization. some questions are flying
in my head...
> >>
> >>- somebody told me about putting the html file
VM cache
> >>- what filesystem is better ?
> >>    
> >
> >uhm it really depends how big your content is. for
a small set of files
> >i would use shmfs (/dev/shm on linux) and as long
it isnt NFS or so you
> >should be fine.
> >  
> 
> But this is what i do not understand (when you say
about putting this in 
> other dir/partition)...
> I have to put JUST the web file in that folder ? or
lighttpd also with 
> access_log ?
> I need the logging module... logs also in that dir ?
> Then I set homedir to that dir ?
> answering to John (other email), it's a 20K static
file.

[[[
mkdir /dev/shm/lighttpd/
cp /path/to/yourfile /dev/shm/lighttpd
]]]

you have to repeat those steps on every reboot.


and in the lighty config
server.document-root="/dev/shm/lighttpd"

that will give you most performance as all operations are
operatios
inside the ram.

logging of course go to disk. but those will be idle.

    darix

-- 
           openSUSE - SUSE Linux is my linux
               openSUSE is good for you
                   www.opensuse.org

On Wed, 30 May 2007 11:14:26 -0400
Thomas Guyot-Sionnest <dermothaei.ca> wrote:

> The main difference between Apache and Lighty is that
on Apache, idle
> keepalive'ed connections are blocking a whole process
(even worse, if
> you use things like mod_php you could imagine it as
having one php
> process per idle connection) while on Lighty, your idle
connections are
> only using one FD in the select/poll/whatever polling
function.

Thanks Thomas,
is this true also of apache in mpm mode?  ( i know, going a
bit OT )
> 
> I believe that the IIS architecture is kinda like
Lighty w/ FastCGI but
> appears to be much slower and flakier.

yes, in IIS , under heavy load, it makes quite some
difference - but i think
it's more about the tcp stack than anything else (all those
threads keeping
sockets open... but dont quote me on that <G> )
B

_________________________
{Beto|Norberto|Numard} Meijome

"Lots of people who complained about us receiving the
MBE received theirs for
heroism in the war -- for killing people. We received ours
for entertaining
other people. I'd say we deserve ours more." John
Lennon

I speak for myself, not my employer. Contents may be hot.
Slippery when wet.
Reading disclaimers makes you go blind. Writing them is
worse. You have been
Warned.

On Thu, 31 May 2007 04:55:09 -0400
Thomas Guyot-Sionnest <dermothaei.ca> wrote:

> > Thanks Thomas,
> > is this true also of apache in mpm mode?  ( i
know, going a bit OT )  
> 
> I guess you're talking about the worker MPM (By default
Apache 2 use the
> prefork MPM on UNIX, and Apache 1.3 used a prefork
model too).

doh! right, worker MPM - sorry.

> 
> Basically each child use multiple threads to process
requests in
> parallel. Since threads have much less overhead that
standalone
> processes this is much more efficient, but IMHO not as
much as using
> Lighty.

thanks

> Also using threads for things like PHP can cause
problems if
> some of the libraries are not thread-safe, though I'm
not sure what's
> the current status on this.

yeah....thread-safe status for everything built into php is
something i havent found yet.

thanks for your time 
B
_________________________
{Beto|Norberto|Numard} Meijome

"We must openly accept all ideologies and systems as 
means of solving humanity's problems. One country, one
nation, one ideology, one system is not sufficient."
   Dalai Lama.

I speak for myself, not my employer. Contents may be hot.
Slippery when wet. Reading disclaimers makes you go blind.
Writing them is worse. You have been Warned.

On 5/30/07, Norberto Meijome <freebsdmeijome.net> wrote:
> On Tue, 29 May 2007 16:54:54 -0700
> "John D. Mitchell" <jdmitchellgmail.com> wrote:
[...]
> > Um, er, if it's just one static file then there
shouldn't be any need
> > for keep-alive, right? 
>
> Indeed - keepalives are bad things for serving heavy
traffic (and completelly
> useless if you only have a few files to serve, or no
"recurring" traffic from
> the same client, eg, a download site with big files).
>
> Actually... I know that is true for IIS and Apache - is
it also true with
> lighty?

Yes and no.  No in the sense that lighty is using various
offloading
and non-blocking IO facilities so that the cost of idle
connections is
minimized. Yes in the sense that you're still taking up
things like
file handles and having to manage them and the open sockets,
etc.

Take care,
John

The limiting factor in a server that's doing this
(regardless of the  
software used) is going to be open connections and by virtue
of that-  
memory.

Each open connection takes up a file descriptor, and some
other  
memory for the socket buffers and such.  So what you want to
do is  
get clients off of the box as soon as possible.

This means- in the webserver, you configure it with a very
low read- 
timeout (to drop non-responsive clients) and you turn off
keep-alive.

In lighttpd this appears to be done with:

server.max-keep-alive-requests = 0

And since you know that the file is only 20k, you can set a
low write- 
timeout just for fun too:

server.max-read-idle = 3
server.max-write-idle = 3

(the write-timeout should come in to play only in very
extreme  
circumstances here)

Next you want to take the disk out of the equation.
Unfortunately the best way to do this can only be known
through testing-
lighttpd's mod_mem_cache is the easiest way to do this- and
if it's  
robust I suspect it will also be the fastest.

Without mod_mem_cache you will have to test with the various
network  
backends (sendfile will likely be slower here because of the
context  
switch), the ramdisk tricks mentioned, and you will
definitely want  
to turn on stat-caching.

To take the disk out of things further you can use
mod_access_log's  
syslog switch and send the logs to a buffering syslogd and
set up a  
large memory area for it to use.
(the benefits of this will also have to be tested)

 From there, it's about setting up your max_conns number so
that you  
don't make the server run out of memory and setting the
max_fd and  
the various network tuning options in your OS.

Before you change the read_timeout settings, note your
memory usage-  
open up 2000 connections to the box, and see how much the
memory  
usage went up.  This should give you a very rough idea of
how much  
memory is being used per connection and will give you some
sort of  
sane number to set it at.

OS tuning is out of the realm of this email, so good luck


--
"Jupiter accepts your offer..."
AIM:IMFDUP



On May 29, 2007, at 3:45 PM, Skiiiks wrote:

> hello groups,
>
> i will have a server just for serving 1 static page. i
think  
> lighttpd is enough tunned and with enough performance
for this, but  
> i wonder about server optimization. some questions are
flying in my  
> head...
>
> - somebody told me about putting the html file VM
cache
> - what filesystem is better ?
> - this config will be the best ?
> 	server.max-worker = 8
> 	server.max-fds = 8192
> 	server.max-connections = 16384
> 	server.max-keep-alive-requests = 256
> 	server.event-handler = "linux-sysepoll"
> 	server.network-backend = "linux-sendfile"
> - squid ?
> - varnishd ?
>
>
> any ideas ? 
>
> thank you !

On Sun, 3 Jun 2007 17:38:23 -0700
Nathan Seven <scosolscosol.org> wrote:

> From: Nathan Seven <scosolscosol.org>
> To: Skiiiks <skiiiksgmail.com>
> Cc: lighttpdip94.kneschke.de
> Subject: Re: tunning server for just 1 page
> Date: Sun, 3 Jun 2007 17:38:23 -0700
> X-Mailer: Apple Mail (2.752.2)
> 
> The limiting factor in a server that's doing this
(regardless of the  
> software used) is going to be open connections and by
virtue of that-  
> memory.

just a thought before I hit the sack, and maybe it is a very
dumb idea - but if
the file is known to be static and that it will not change
that much at all,
would it be faster to bring the file closer to the lighty
core itself? 

Build a module (C? lua? ) with the contents of the file in
it and just stream
that out without ever having to do a 'read' from disk? (
compile the module
with the .gz version of the file and only support all modern
browsers,which
will support compressed content (not sure about
proxies....)

anyway... I hope I'm not charged with disminishing the
overall knowledge of the
world ;)

catcha!
_________________________
{Beto|Norberto|Numard} Meijome

"They redundantly repeated themselves over and over
again incessantly without
end ad infinitum" ibid.

I speak for myself, not my employer. Contents may be hot.
Slippery when wet.
Reading disclaimers makes you go blind. Writing them is
worse. You have been
Warned.

On 2007-06-04 23:16:30 +1000, Norberto Meijome wrote:
> just a thought before I hit the sack, and maybe it is a
very dumb idea - but if
> the file is known to be static and that it will not
change that much at all,
> would it be faster to bring the file closer to the
lighty core itself? 
> 
> Build a module (C? lua? ) with the contents of the file
in it and just stream
> that out without ever having to do a 'read' from disk?
( compile the module
> with the .gz version of the file and only support all
modern browsers,which
> will support compressed content (not sure about
proxies....)
> 
> anyway... I hope I'm not charged with disminishing the
overall knowledge of the
> world ;)

sendfile is fast enough.

-- 
           openSUSE - SUSE Linux is my linux
               openSUSE is good for you
                   www.opensuse.org