non-blocking NIO HTTP transport (server side)

Folks,

I have largely finished the server side components of the
event-driven
HTTP transport and am in the process of putting the final
touches on the
client side components. The code is still a little rough
around the
edges but so far has been holding up quite well for me.
Content chunk
coding, header folding, content footers, proper protocol
version
handling, expect-continue handshake are supported. The HTTP
transport
should be able to handle content of arbitrary length with
(almost)
content memory footprint. There is no intermediate buffering
involved at
all in those cases where no content coding is required.
Request /
response content gets directly read from / written to the
underlying
socket channel. 

Here is a sample of a simple HTTP server demonstrating the
new API in
action

http://svn.apache.org/repos/asf/j
akarta/httpcomponents/httpcore/trunk/module-nio/src/examples
/org/apache/http/nio/examples/NHttpServer.java

My next steps will be to complete the client side components
and to
implement a compatibility layer on top of the even driven
API for those
applications that rely on InputStream / OutputStream
interface to
consume / produce content. Some early feedback would be
_hugely_
appreciated, though.

Cheers

Oleg





------------------------------------------------------------
---------
To unsubscribe, e-mail: httpclient-dev-unsubscribejakarta.apache.org
For additional commands, e-mail: httpclient-dev-helpjakarta.apache.org

Folks,

I have completed the first cut at the non-blocking NIO HTTP
transport.
Both client and server side implementations are (more or
less) feature
complete at this point. 

These are the sample apps demonstrating the new API in
action

Non blocking HTTP server:
http://svn.apache.org/repos/asf/j
akarta/httpcomponents/httpcore/trunk/module-nio/src/examples
/org/apache/http/nio/examples/NHttpServer.java

Non blocking HTTP client:
http://svn.apache.org/repos/asf/j
akarta/httpcomponents/httpcore/trunk/module-nio/src/examples
/org/apache/http/nio/examples/NHttpClient.java

Before I go on hacking I would love to get some initial
feedback. Please
let me know what you think

Next two big things on my to-do list are
* InputStream / OutputStream wrappers
* Default IO reactor refinements. I would like to add an
option enabling
the IO reactor to run multiple I/O event dispatch threads
(mainly for
multiprocessor systems)

Cheers

Oleg 


On Thu, 2006-10-26 at 10:24 +0200, Oleg Kalnichevski wrote:
> Folks,
> 
> I have largely finished the server side components of
the event-driven
> HTTP transport and am in the process of putting the
final touches on the
> client side components. The code is still a little
rough around the
> edges but so far has been holding up quite well for me.
Content chunk
> coding, header folding, content footers, proper
protocol version
> handling, expect-continue handshake are supported. The
HTTP transport
> should be able to handle content of arbitrary length
with (almost)
> content memory footprint. There is no intermediate
buffering involved at
> all in those cases where no content coding is required.
Request /
> response content gets directly read from / written to
the underlying
> socket channel. 
> 
> Here is a sample of a simple HTTP server demonstrating
the new API in
> action
> 
> http://svn.apache.org/repos/asf/j
akarta/httpcomponents/httpcore/trunk/module-nio/src/examples
/org/apache/http/nio/examples/NHttpServer.java
> 
> My next steps will be to complete the client side
components and to
> implement a compatibility layer on top of the even
driven API for those
> applications that rely on InputStream / OutputStream
interface to
> consume / produce content. Some early feedback would be
_hugely_
> appreciated, though.
> 
> Cheers
> 
> Oleg
> 
> 
> 
> 
> 
>
------------------------------------------------------------
---------
> To unsubscribe, e-mail: httpclient-dev-unsubscribejakarta.apache.org
> For additional commands, e-mail:
httpclient-dev-helpjakarta.apache.org
> 
> 


------------------------------------------------------------
---------
To unsubscribe, e-mail: httpclient-dev-unsubscribejakarta.apache.org
For additional commands, e-mail: httpclient-dev-helpjakarta.apache.org

Oleg Kalnichevski wrote:
> I have completed the first cut at the non-blocking NIO
HTTP transport.
> Both client and server side implementations are (more
or less) feature
> complete at this point. 
> These are the sample apps demonstrating the new API in
action 
>  ...
> Before I go on hacking I would love to get some initial
feedback. Please
> let me know what you think

I have only had a quick look, I am currently busy with my
own proxy.

One thing with the example code that looks suspicious to me
is the keep
alive handling both the client and the server contains
something like:
     if (!this.connStrategy.keepAlive(response, context)) {
         conn.close();
     }

Sure it is only a few lines of code, but why not say that
client and
server always close the connection and then handle
persistent
connections on a higher level (by the connection manager or
whatever it
is called).
The current code demands that all users of http client does
the right
thing with regards to keep alive.

One thing for the wish list that could be nice to have:
support for
FileChannel.transferTo and FileChannel.transferFrom. At
least on linux
they become sendfile. If you want to build a fast web server
it is
needed. The api for transferTo/From is a bit different from
read + write
since data is never in user space. Any thoughts?

/robo


------------------------------------------------------------
---------
To unsubscribe, e-mail: httpclient-dev-unsubscribejakarta.apache.org
For additional commands, e-mail: httpclient-dev-helpjakarta.apache.org

Hi Robert,

> One thing with the example code that looks suspicious
to me is the keep
> alive handling both the client and the server contains
something like:
>     if (!this.connStrategy.keepAlive(response,
context)) {
>         conn.close();
>     }
> 
> Sure it is only a few lines of code, but why not say
that client and
> server always close the connection and then handle
persistent
> connections on a higher level (by the connection
manager or whatever it
> is called).
> The current code demands that all users of http client
does the right
> thing with regards to keep alive.

There currently is no HttpClient 4.0. We only have HttpCore.
We'll add automatic keep-alive handling in the HttpClient.
The examples just show that keep-alive is possible with
HttpCore alone.

cheers,
  Roland


------------------------------------------------------------
---------
To unsubscribe, e-mail: httpclient-dev-unsubscribejakarta.apache.org
For additional commands, e-mail: httpclient-dev-helpjakarta.apache.org

On Sun, 2006-10-29 at 17:13 +0100, Robert Olofsson wrote: 
> Oleg Kalnichevski wrote:
> > I have completed the first cut at the non-blocking
NIO HTTP transport.
> > Both client and server side implementations are
(more or less) feature
> > complete at this point. 
> > These are the sample apps demonstrating the new
API in action 
> >  ...
> > Before I go on hacking I would love to get some
initial feedback. Please
> > let me know what you think
> 
> I have only had a quick look, I am currently busy with
my own proxy.
> 
> One thing with the example code that looks suspicious
to me is the keep
> alive handling both the client and the server contains
something like:
>      if (!this.connStrategy.keepAlive(response,
context)) {
>          conn.close();
>      }
> 

Hi Robert

For the client side demo this code does not make any sense
indeed. As
Roland said this is just to show the way the API is meant to
be used.

> Sure it is only a few lines of code, but why not say
that client and
> server always close the connection and then handle
persistent
> connections on a higher level (by the connection
manager or whatever it
> is called).
> The current code demands that all users of http client
does the right
> thing with regards to keep alive.
> 
> One thing for the wish list that could be nice to have:
support for
> FileChannel.transferTo and FileChannel.transferFrom. At
least on linux
> they become sendfile. If you want to build a fast web
server it is
> needed. The api for transferTo/From is a bit different
from read + write
> since data is never in user space. Any thoughts?
> 

This would call for some really hard design decisions. One
can either
(1) expose the underlying socket channel in the public API,
which is
kind of bad IMO or (2) extend ContentDecoder and
ContentEncoder
interfaces with methods that take FileChannel as an input
parameter.

I am not sure this all is worth the trouble, as the raw
throughput is
certainly not one of NIO's strong points. As far as I can
tell classic
blocking IO easily outperforms NIO in terms of raw
throughput by as much
as 2 to 3 times, when the number of concurrent connections
is around
200. I doubt the use of FileChannel#transferTo/From would
dramatically
change the situation.

Oleg

> /robo
> 
> 
>
------------------------------------------------------------
---------
> To unsubscribe, e-mail: httpclient-dev-unsubscribejakarta.apache.org
> For additional commands, e-mail:
httpclient-dev-helpjakarta.apache.org
> 
> 


------------------------------------------------------------
---------
To unsubscribe, e-mail: httpclient-dev-unsubscribejakarta.apache.org
For additional commands, e-mail: httpclient-dev-helpjakarta.apache.org

Oleg Kalnichevski wrote:
>> One thing for the wish list that could be nice to
have: support for
>> FileChannel.transferTo and
FileChannel.transferFrom. At least on linux
>> they become sendfile. If you want to build a fast
web server it is
>> needed. The api for transferTo/From is a bit
different from read + write
>> since data is never in user space. Any thoughts?
> 
> This would call for some really hard design decisions.
One can either
> (1) expose the underlying socket channel in the public
API, which is
> kind of bad IMO or (2) extend ContentDecoder and
ContentEncoder
> interfaces with methods that take FileChannel as an
input parameter.

Yes it would require api changes. You can of course add
layers of api
on top of it to hide some parts of it.

> I am not sure this all is worth the trouble, as the raw
throughput is
> certainly not one of NIO's strong points. As far as I
can tell classic
> blocking IO easily outperforms NIO in terms of raw
throughput by as much
> as 2 to 3 times, when the number of concurrent
connections is around
> 200. I doubt the use of FileChannel#transferTo/From
would dramatically
> change the situation.

I am not sure that that is a well done benchmark. I do not
have any good
benchmarks myself to compare blocking vs selector based io.
I could
perhaps use my proxy to try it, but it has changed quite a
lot since the
change from threaded io.

I could benchmark my proxy with and without
transferTo/transferFom
though, that could be an interesting benchmark and that is
easy to
check.

/robo

------------------------------------------------------------
---------
To unsubscribe, e-mail: httpclient-dev-unsubscribejakarta.apache.org
For additional commands, e-mail: httpclient-dev-helpjakarta.apache.org

Robert Olofsson wrote:
> I could benchmark my proxy with and without
transferTo/transferFom
> though, that could be an interesting benchmark and that
is easy to
> check.

A small and benchmark with my proxy, rabbit, gives me
interesting
numbers. I run this command:
ab2 -n10000 -c100 -X localhost:9666 
http://www.kh
elekore.org/~robo/arcs.png

To enable/disable transfer support I change one line:
"if (mayTransfer ()" into "if (false
&& mayTransfer ()"

With transfer support:
Requests per second:    1478.70 [#/sec] (mean)

Without transfer:
Requests per second:    1095.34 [#/sec] (mean)

This is not a very good benchmark, so I omit the details.
I do think that the speed difference is quite big...

/robo

------------------------------------------------------------
---------
To unsubscribe, e-mail: httpclient-dev-unsubscribejakarta.apache.org
For additional commands, e-mail: httpclient-dev-helpjakarta.apache.org

On Sun, 2006-10-29 at 19:26 +0100, Robert Olofsson wrote:
> Oleg Kalnichevski wrote:
> >> One thing for the wish list that could be nice
to have: support for
> >> FileChannel.transferTo and
FileChannel.transferFrom. At least on linux
> >> they become sendfile. If you want to build a
fast web server it is
> >> needed. The api for transferTo/From is a bit
different from read + write
> >> since data is never in user space. Any
thoughts?
> > 
> > This would call for some really hard design
decisions. One can either
> > (1) expose the underlying socket channel in the
public API, which is
> > kind of bad IMO or (2) extend ContentDecoder and
ContentEncoder
> > interfaces with methods that take FileChannel as
an input parameter.
> 

I'd probably go with the second option and make this
facility available
through an optional interface

interface ContentContentDecoder extends ContentDecoder {

 void writeDirect(FileChannel channel, ...);

}

> Yes it would require api changes. You can of course add
layers of api
> on top of it to hide some parts of it.
> 
> > I am not sure this all is worth the trouble, as
the raw throughput is
> > certainly not one of NIO's strong points. As far
as I can tell classic
> > blocking IO easily outperforms NIO in terms of raw
throughput by as much
> > as 2 to 3 times, when the number of concurrent
connections is around
> > 200. I doubt the use of
FileChannel#transferTo/From would dramatically
> > change the situation.
> 
> I am not sure that that is a well done benchmark. I do
not have any good
> benchmarks myself to compare blocking vs selector based
io. I could
> perhaps use my proxy to try it, but it has changed
quite a lot since the
> change from threaded io.

I'd be happy to be proved wrong, but this is what I have
been seeing so
far. This is also consistent with the experience of Tomcat
developers. 

I use AB (Apache benchmark) to measure performance of HTTP
transports
and as far as I can tell this can be hardly considered an
extravagant
way of performing HTTP benchmarks. 

At some point mainstream JVMs may improve and NIO
performance will match
that of classic IO, but with JVMs widely deployed today
blocking IO
appears clearly ahead in terms of the raw throughput. 

Anyways, I am very open to reconsidering my attitude towards
NIO given
enough empirical data. 

Cheers

Oleg

> 
> I could benchmark my proxy with and without
transferTo/transferFom
> though, that could be an interesting benchmark and that
is easy to
> check.
> 
> /robo
> 
>
------------------------------------------------------------
---------
> To unsubscribe, e-mail: httpclient-dev-unsubscribejakarta.apache.org
> For additional commands, e-mail:
httpclient-dev-helpjakarta.apache.org
> 
> 


------------------------------------------------------------
---------
To unsubscribe, e-mail: httpclient-dev-unsubscribejakarta.apache.org
For additional commands, e-mail: httpclient-dev-helpjakarta.apache.org

On Sun, 2006-10-29 at 19:44 +0100, Robert Olofsson wrote: 
> Robert Olofsson wrote:
> > I could benchmark my proxy with and without
transferTo/transferFom
> > though, that could be an interesting benchmark and
that is easy to
> > check.
> 
> A small and benchmark with my proxy, rabbit, gives me
interesting
> numbers. I run this command:
> ab2 -n10000 -c100 -X localhost:9666 
> http://www.kh
elekore.org/~robo/arcs.png
> 
> To enable/disable transfer support I change one line:
> "if (mayTransfer ()" into "if (false
&& mayTransfer ()"
> 
> With transfer support:
> Requests per second:    1478.70 [#/sec] (mean)
> 
> Without transfer:
> Requests per second:    1095.34 [#/sec] (mean)
> 
> This is not a very good benchmark, so I omit the
details.
> I do think that the speed difference is quite big...
> 
> /robo

Hi Robert

This does look interesting. The difference is significant
indeed. We
should definitely look into possibility of making use of
transferTo/transferForm methods in order to increase
performance of disk
bound operations.

These are the numbers comparing the blocking (threaded) I/O
with
non-blocking based on the latest HttpCore SVN snapshot. The
difference
is 70%, which is quite a lot. When serving larger content
and using
chunk-encoding the difference should get closer to 100%. I
fully admit
the benchmark may not be very representative, but it largely
supports my
observation about raw performance of NIO based transports.  

Oleg

========================================================
Blocking I/O
========================================================
olegdld810:~$ ab2 -n 200000 -c 50 -k http://localhost:808
0/index.html
This is ApacheBench, Version 2.0.41-dev <$Revision: 1.141
$> apache-2.0
Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd,
http://www.zeustech.net/
Copyright (c) 1998-2002 The Apache Software Foundation,
http://www.apache.org/

Benchmarking localhost (be patient)

Server Software:        Jakarta-HttpComponents/1.1
Server Hostname:        localhost
Server Port:            8080

Document Path:          /index.html
Document Length:        12928 bytes

Concurrency Level:      50
Time taken for tests:   54.63056 seconds
Complete requests:      200000
Failed requests:        0
Write errors:           0
Keep-Alive requests:    200000
Total transferred:      -1676135260 bytes
HTML transferred:       -1708940672 bytes
Requests per second:    3699.38 [#/sec] (mean)
Time per request:       13.516 [ms] (mean)
Time per request:       0.270 [ms] (mean, across all
concurrent
requests)
Transfer rate:          -30276.68 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       3
Processing:     0   12  24.2     12    2636
Waiting:        0   11  24.0     11    2633
Total:          0   12  24.2     12    2637

========================================================
Non-blocking I/O
========================================================

olegdld810:~$ ab2 -n 200000 -c 50 -k http://localhost:808
0/index.html
This is ApacheBench, Version 2.0.41-dev <$Revision: 1.141
$> apache-2.0
Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd,
http://www.zeustech.net/
Copyright (c) 1998-2002 The Apache Software Foundation,
http://www.apache.org/

Benchmarking localhost (be patient)

Server Software:        Jakarta-HttpComponents-NIO/1.1
Server Hostname:        localhost
Server Port:            8080

Document Path:          /index.html
Document Length:        12928 bytes

Concurrency Level:      50
Time taken for tests:   77.89444 seconds
Complete requests:      200000
Failed requests:        0
Write errors:           0
Keep-Alive requests:    200000
Total transferred:      -1675636336 bytes
HTML transferred:       -1709238016 bytes
Requests per second:    2594.39 [#/sec] (mean)
Time per request:       19.272 [ms] (mean)
Time per request:       0.385 [ms] (mean, across all
concurrent
requests)
Transfer rate:          -21226.81 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       2
Processing:     1   18   7.8     17     136
Waiting:        0   11   6.3     10      98
Total:          1   18   7.8     17     136





> 
>
------------------------------------------------------------
---------
> To unsubscribe, e-mail: httpclient-dev-unsubscribejakarta.apache.org
> For additional commands, e-mail:
httpclient-dev-helpjakarta.apache.org
> 
> 


------------------------------------------------------------
---------
To unsubscribe, e-mail: httpclient-dev-unsubscribejakarta.apache.org
For additional commands, e-mail: httpclient-dev-helpjakarta.apache.org