Array

This is a patch adding automatic TCP send socket buffer
sizing.  Normally
the socket buffers are static (either derived from global
defaults or set
with setsockopt) and do not adapt to real network
conditions.  Two things
happen: a) your socket buffers are too small and you can't
reach the full
potential of the network between both hosts; b) your socket
buffers are
too big and you waste a lot of kernel memory for data just
sitting around.

With automatic TCP send socket buffers we can start with a
small buffer
and quickly grow it in parallel with the TCP congestion
window to match
real network conditions.

FreeBSD has a default 32K send socket buffer.  This supports
a maximal
transfer rate of only slightly more than 2Mbit/s on a 100ms
RTT trans-
continental link.  Or at 200ms just above 1Mbit/s.  With TCP
send buffer
auto scaling and the default values below it supports
20Mbit/s at 100ms
and 10Mbit/s at 200ms.  That's an improvement of factor 10,
or 1000%.

New sysctl's are:

  net.inet.tcp.sndbuf_auto=1	(enabled)
  net.inet.tcp.sndbuf_inc=8192	(8K, step size)
  net.inet.tcp.sndbuf_max=262144	(256K, growth limit)

The patch is available here:

  http://people.freebsd.org/~andre/tcp_auto_sndbuf-2
0061116.diff

Any testers, especially with busy FTP servers, are very
welcome.

-- 
Andre

_______________________________________________
freebsd-currentfreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-curre
nt
To unsubscribe, send any mail to
"freebsd-current-unsubscribefreebsd.org"

Andre Oppermann wrote:
> This is a patch adding automatic TCP send socket buffer
sizing.  Normally
> the socket buffers are static (either derived from
global defaults or set
> with setsockopt) and do not adapt to real network
conditions.  Two things
> happen: a) your socket buffers are too small and you
can't reach the full
> potential of the network between both hosts; b) your
socket buffers are
> too big and you waste a lot of kernel memory for data
just sitting around.
> 
> With automatic TCP send socket buffers we can start
with a small buffer
> and quickly grow it in parallel with the TCP congestion
window to match
> real network conditions.
> 
> FreeBSD has a default 32K send socket buffer.  This
supports a maximal
> transfer rate of only slightly more than 2Mbit/s on a
100ms RTT trans-
> continental link.  Or at 200ms just above 1Mbit/s. 
With TCP send buffer
> auto scaling and the default values below it supports
20Mbit/s at 100ms
> and 10Mbit/s at 200ms.  That's an improvement of factor
10, or 1000%.
> 
> New sysctl's are:
> 
>  net.inet.tcp.sndbuf_auto=1    (enabled)
>  net.inet.tcp.sndbuf_inc=8192    (8K, step size)
>  net.inet.tcp.sndbuf_max=262144    (256K, growth limit)
> 
> The patch is available here:
> 
>  http://people.freebsd.org/~andre/tcp_auto_sndbuf-2
0061116.diff

The patch was missing the changes to tcp_usrreq.c and
socketvar.h.
I've replaced it with an updated one.  Please fetch again.

-- 
Andre

_______________________________________________
freebsd-currentfreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-curre
nt
To unsubscribe, send any mail to
"freebsd-current-unsubscribefreebsd.org"

> This is a patch adding automatic TCP send socket buffer

> sizing.

<snip>

> The patch is available here:
> 
>   http://people.freebsd.org/~andre/tcp_auto_sndbuf-2
0061116.diff
> 
> Any testers, especially with busy FTP servers, are very
welcome.
> 


Very nice indeed! I've actually been looking for something
like this  I
would very much like to try it out but I need to know if I
can benefit from
it with my setup. My network knowledge on this deep level is
very limited so
I need to ask a few questions that probably sounds stupid...
but here we go:

Would this patch only benefit traffic generated from or
destined to the
FreeBSD box itself or would it also benefit traffic
generated behind it on a
LAN if the FreeBSD box was configured as:

a) a router with NAT

b) a router without NAT

c) a bridge only

Add to this the extra complexity of pf with synproxy and
modulate state. I
simply don't know how (if at all) FreeBSD interacts with or
manipulates
packets going through it under any of these circumstances,
so I have to ask
to learn 



The patch didn't apply cleanly to my 6.1-RELEASE. Since this
patch was
cross-posted to -current I guess it wasn't meant for me. Any
chance you can
provide a patch for 6.1-RELEASE? This is the output:

# patch <tcp_auto_sndbuf-20061116.diff
Hmm...  Looks like a unified diff to me...
The text leading up to this was:
--------------------------
|Index: tcp_output.c
|===========================================================
========
|RCS file: /home/ncvs/src/sys/netinet/tcp_output.c,v
|retrieving revision 1.121
|diff -u -p -r1.121 tcp_output.c
|--- tcp_output.c       22 Oct 2006 11:52:16 -0000     
1.121
|+++ tcp_output.c       16 Nov 2006 18:35:43 -0000
--------------------------
File to patch: /usr/src/sys/netinet/tcp_output.c
Patching file /usr/src/sys/netinet/tcp_output.c using Plan
A...
Hunk #1 succeeded at 49 (offset 1 line).
Hunk #2 failed at 105.
Hunk #3 failed at 395.
2 out of 3 hunks failed--saving rejects to
/usr/src/sys/netinet/tcp_output.c.rej
Done



Lastly, is it enough to rebuild only the kernel after
applying this patch?


Once again, sorry for these stupid questions but this is the
only way for me
to learn and I really would like to have this patch running
on my system.

Kind regards
Morgan

_______________________________________________
freebsd-currentfreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-curre
nt
To unsubscribe, send any mail to
"freebsd-current-unsubscribefreebsd.org"

Andre Oppermann wrote:
> With automatic TCP send socket buffers we can start
with a small buffer
> and quickly grow it in parallel with the TCP congestion
window to match
> real network conditions.
 >
> The patch is available here:
> 
>  http://people.freebsd.org/~andre/tcp_auto_sndbuf-2
0061116.diff
> 
> Any testers, especially with busy FTP servers, are very
welcome.

A RELENG_6 version (for FreeBSD 6.x) of the patch is here:

  http://people.freebsd.org/~andre/tcp_auto
_sndbuf-20061116-RELENG_6.diff

Just apply this patch and recompile your kernel.  It is
activated by default.
Be aware that all socket buffer sizing events get logged to
syslog under
LOG_DEBUG.  This may affect overall system performance and
you may want to
disable logging to disk of this in syslogd.conf.

-- 
Andre
_______________________________________________
freebsd-currentfreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-curre
nt
To unsubscribe, send any mail to
"freebsd-current-unsubscribefreebsd.org"

Andre Oppermann wrote:


> With automatic TCP send socket buffers we can start
with a small buffer
> and quickly grow it in parallel with the TCP congestion
window to match
> real network conditions.
Are you planning to implement something similar for the
receive path?

-- 
Pieter

_______________________________________________
freebsd-currentfreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-curre
nt
To unsubscribe, send any mail to
"freebsd-current-unsubscribefreebsd.org"

Pieter de Boer wrote:
> Andre Oppermann wrote:
> 
> 
>> With automatic TCP send socket buffers we can start
with a small buffer
>> and quickly grow it in parallel with the TCP
congestion window to match
>> real network conditions.
> 
> Are you planning to implement something similar for the
receive path?

Yes, but it's a bit harder.

-- 
Andre
_______________________________________________
freebsd-currentfreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-curre
nt
To unsubscribe, send any mail to
"freebsd-current-unsubscribefreebsd.org"

> A RELENG_6 version (for FreeBSD 6.x) of the patch is
here:
>   
> http://people.freebsd.org/~andre/tcp_auto_sndbuf-2
0061116-RELE
> NG_6.diff
> 
> Just apply this patch and recompile your kernel.  It is

> activated by default.

Downloaded, applied, recompiled, installed and rebooted
without any errors.
I did however add the sysctl variables to my
/etc/sysctl.conf just in case.

> Be aware that all socket buffer sizing events get
logged to 
> syslog under LOG_DEBUG.

Does this mean they will automatically show up in
/var/log/debug.log?

> This may affect overall system 
> performance and you may want to disable logging to disk
of 
> this in syslogd.conf.

I could use some help with the syntax for this... 



And a final question: Will this feature work regardless of
what side
initiates the TCP connection? Or will I have to instruct my
ftp-users to use
active mode only when they transfer files to/from my FreeBSD
box?


Regards
Morgan

_______________________________________________
freebsd-currentfreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-curre
nt
To unsubscribe, send any mail to
"freebsd-current-unsubscribefreebsd.org"

On 16/11/06, Andre Oppermann <andrefreebsd.org> wrote:
> This is a patch adding automatic TCP send socket buffer
sizing.  Normally
> the socket buffers are static (either derived from
global defaults or set
> with setsockopt) and do not adapt to real network
conditions.  Two things
> happen: a) your socket buffers are too small and you
can't reach the full
> potential of the network between both hosts; b) your
socket buffers are
> too big and you waste a lot of kernel memory for data
just sitting around.
>
> With automatic TCP send socket buffers we can start
with a small buffer
> and quickly grow it in parallel with the TCP congestion
window to match
> real network conditions.
>
> FreeBSD has a default 32K send socket buffer.  This
supports a maximal
> transfer rate of only slightly more than 2Mbit/s on a
100ms RTT trans-
> continental link.  Or at 200ms just above 1Mbit/s. 
With TCP send buffer
> auto scaling and the default values below it supports
20Mbit/s at 100ms
> and 10Mbit/s at 200ms.  That's an improvement of factor
10, or 1000%.
>
> New sysctl's are:
>
>  net.inet.tcp.sndbuf_auto=1    (enabled)
>  net.inet.tcp.sndbuf_inc=8192  (8K, step size)
>  net.inet.tcp.sndbuf_max=262144        (256K, growth
limit)
>
> The patch is available here:
>
>  http://people.freebsd.org/~andre/tcp_auto_sndbuf-2
0061116.diff
>
> Any testers, especially with busy FTP servers, are very
welcome.
>
> --
> Andre
>
> _______________________________________________
> freebsd-netfreebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to
"freebsd-net-unsubscribefreebsd.org"
>

Ok I done some intitial tesing and so far looks sweet.

I set default send window to 32120 and started a fxp
transfer to
another which on 64240 was limited to around 40mbit so 32120
would
normally give about 20mbit, and I previously had 128280 for
80mbit,
now with a default of 32120 and your default sndbuf settings
it zoomed
along at around 80mbit.

this from debug.log shows it working.

Nov 22 06:33:33 heaven kernel: tcp_output: inc sockbuf, old
82456, new
90648, sb_cc 80272, snd_wnd 66984, sendwnd 66608
Nov 22 06:33:33 heaven kernel: tcp_output: inc sockbuf, old
90648, new
98840, sb_cc 89664, snd_wnd 72776, sendwnd 69504
Nov 22 06:33:33 heaven kernel: tcp_output: inc sockbuf, old
98840, new
107032, sb_cc 97608, snd_wnd 81464, sendwnd 73848
Nov 22 06:33:33 heaven kernel: tcp_output: inc sockbuf, old
107032,
new 115224, sb_cc 107000, snd_wnd 87256, sendwnd 76744
Nov 22 06:33:33 heaven kernel: tcp_output: inc sockbuf, old
115224,
new 123416, sb_cc 112296, snd_wnd 93048, sendwnd 79640
Nov 22 06:33:33 heaven kernel: tcp_output: inc sockbuf, old
123416,
new 131608, sb_cc 121688, snd_wnd 98840, sendwnd 82536
Nov 22 06:33:33 heaven kernel: tcp_output: inc sockbuf, old
131608,
new 139800, sb_cc 129632, snd_wnd 107528, sendwnd 86880
Nov 22 06:33:34 heaven kernel: tcp_output: inc sockbuf, old
139800,
new 147992, sb_cc 137992, snd_wnd 112944, sendwnd 110048
Nov 22 06:33:34 heaven kernel: tcp_output: inc sockbuf, old
147992,
new 156184, sb_cc 147384, snd_wnd 118736, sendwnd 112944
Nov 22 06:33:34 heaven kernel: tcp_output: inc sockbuf, old
156184,
new 164376, sb_cc 155328, snd_wnd 127424, sendwnd 114261

cant wait for the recv side of this patch.

Chris
_______________________________________________
freebsd-currentfreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-curre
nt
To unsubscribe, send any mail to
"freebsd-current-unsubscribefreebsd.org"

On Fri, 17 Nov 2006 08:02:28 -0600, Andre Oppermann
<andrefreebsd.org>  
wrote:

> Andre Oppermann wrote:
>> With automatic TCP send socket buffers we can start
with a small buffer
>> and quickly grow it in parallel with the TCP
congestion window to match
>> real network conditions.
>  >
>> The patch is available here:
>>   http://people.freebsd.org/~andre/tcp_auto_sndbuf-2
0061116.diff
>>  Any testers, especially with busy FTP servers, are
very welcome.
>
> A RELENG_6 version (for FreeBSD 6.x) of the patch is
here:
>
>   http://people.freebsd.org/~andre/tcp_auto
_sndbuf-20061116-RELENG_6.diff
>
> Just apply this patch and recompile your kernel.  It is
activated by  
> default.
> Be aware that all socket buffer sizing events get
logged to syslog under
> LOG_DEBUG.  This may affect overall system performance
and you may want  
> to
> disable logging to disk of this in syslogd.conf.

Is this relate with net.inet.tcp.sendspace? I reset my
sysctl config back  
to default that was for wine+utorrent, so tested with
wine+utorrent on  
today (Nov 25th) RELENG_6 and works fine so far.

[...]
Nov 25 14:35:22 mezz kernel: tcp_output: inc sockbuf, old
65612, new  
73804, sb_cc 58553, snd_wnd 149760, sendwnd 36719
Nov 25 14:35:27 mezz kernel: tcp_output: inc sockbuf, old
73804, new  
81996, sb_cc 65608, snd_wnd 148896, sendwnd 38283
Nov 25 14:35:27 mezz kernel: tcp_output: inc sockbuf, old
81996, new  
90188, sb_cc 72604, snd_wnd 148896, sendwnd 38283
Nov 25 14:35:29 mezz kernel: tcp_output: inc sockbuf, old
90188, new  
98380, sb_cc 79093, snd_wnd 149760, sendwnd 39988
[...goes on...]

A bit off point, I have to setup this below in sysctl to get
wine+utorrent  
download/upload very fast and ustable without hog the CPU.

kern.ipc.maxsockbuf=2097152
net.inet.tcp.recvspace=262144
net.inet.tcp.sendspace=262144

Cheers,
Mezz


-- 
mezz7cox.net  -  mezzFreeBSD.org
FreeBSD GNOME Team  -  FreeBSD Multimedia Hat (ports, not
src)
http://www.FreeBSD.org/
gnome/  -  gnomeFreeBSD.org
http://wiki.freebs
d.org/multimedia  -  multimediaFreeBSD.org
_______________________________________________
freebsd-currentfreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-curre
nt
To unsubscribe, send any mail to
"freebsd-current-unsubscribefreebsd.org"

On 17/11/06, Andre Oppermann <andrefreebsd.org> wrote:
> Pieter de Boer wrote:
> > Andre Oppermann wrote:
> >
> >
> >> With automatic TCP send socket buffers we can
start with a small buffer
> >> and quickly grow it in parallel with the TCP
congestion window to match
> >> real network conditions.
> >
> > Are you planning to implement something similar
for the receive path?
>
> Yes, but it's a bit harder.
>
> --
> Andre
> _______________________________________________
> freebsd-netfreebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to
"freebsd-net-unsubscribefreebsd.org"
>

Is there progress on this yet? still no problems with the
current patch 

Chris
_______________________________________________
freebsd-currentfreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-curre
nt
To unsubscribe, send any mail to
"freebsd-current-unsubscribefreebsd.org"