Re:

Arnaldo,

Setting tx_qlen=0 did not help, the receiver receives only 1
packet, despite of sender's sending 10 packets. Attached are
the dmesg output of sender & receiver. I modified the
DCCP code on the sender side to print some debug messages
(input/output to/from functions, etc.).

BTW, I am using CCID3.


----- Original Message ----
From: Arnaldo Carvalho de Melo <arnaldo.melogmail.com>
To: Ian McDonald <ian.mcdonaldjandi.co.nz>
Cc: Leandro Melo de Sales <leandroalgmail.com>; burak gorkemli <burakgorkemliyahoo.com>; DCCP Mailing List <dccpvger.kernel.org>
Sent: Thursday, November 30, 2006 12:51:11 PM
Subject: Re:

On 11/29/06, Ian McDonald <ian.mcdonaldjandi.co.nz> wrote:
> Folks,
>
> Another thing to note is that with a limit on tx queue
sizes now you
> will get EAGAIN if the queue is full...

Yes, 4, 5 packets should be, in dccp_sendmsg:

        if (sysctl_dccp_tx_qlen &&
            (sk->sk_write_queue.qlen >=
sysctl_dccp_tx_qlen)) {
                rc = -EAGAIN;
                goto out_release;
        }

Leandro and Burak: could you please try setting
sysctl_dccp_tx_qlen to
zero? doing this:

[rootqemu ~]# cat /proc/sys/net/dccp/default/tx_qlen
5
[rootqemu ~]# echo 0 >
/proc/sys/net/dccp/default/tx_qlen
[rootqemu ~]# cat /proc/sys/net/dccp/default/tx_qlen
0
[rootqemu ~]#

And report the results?

- Arnaldo
-
To unsubscribe from this list: send the line
"unsubscribe dccp" in
the body of a message to majordomovger.kernel.org
More majordomo info at  http://vge
r.kernel.org/majordomo-info.html

On 11/30/06, Burak Gorkemli <burakgmail-dccpyahoo.com> wrote:
> Arnaldo,
>
> Setting tx_qlen=0 did not help, the receiver receives
only 1 packet, despite of sender's sending 10 packets.
Attached are the dmesg output of sender & receiver. I
modified the DCCP code on the sender side to print some
debug messages (input/output to/from functions, etc.).
>
> BTW, I am using CCID3.

Ok, now to the next troubleshooting test: can you please try
using
CCID2? That way we can isolate the problem, is it CCID3
specific?
CCID2? Core? Something else?

This is only a rethorical question, as Gerrit theorised on
another
followup message to yours, this is probably a problem in the
CCID3
code, but just to exercise the troubleshooting process,
could you try
with CCID2?

Thanks,

- Arnaldo
-
To unsubscribe from this list: send the line
"unsubscribe dccp" in
the body of a message to majordomovger.kernel.org
More majordomo info at  http://vge
r.kernel.org/majordomo-info.html

Quoting Burak Gorkemli:
|  Arnaldo,
|  
|  Setting tx_qlen=0 did not help, the receiver receives
only 1 packet, despite of sender's sending 10 packets.
Attached are the dmesg output of sender & receiver. I
modified the DCCP code on the sender side to print some
debug messages (input/output to/from functions, etc.).
|  
|  BTW, I am using CCID3.
|  
Thank you very much for this input. I have been running the
same tests and I think I have found one solution
to this problem already. The nofeedback timer is set
according to the book, i.e. max(4*R, 2*s/X). On LANS,
RTTs are small and hence the nofeedback timer expires
horribly often (every two jiffies). This further halves
the sending rate each time. And since we don't have the
integer sending rate computation fixed, the sending
rate therefore very quickly converges towards zero. 
I can see that in your client log the feedback timer is also
triggered very often, but I don't see the 
messages when they are triggered (is
CONFIG_IP_DCCP_CCID3_DEBUG enabled)?

This has been a recent patch; I am working on a
configuration option to use a higher RTO value (the 1 second
that TCP uses), which will
	a) restore the previous state
	b) make it possible to experiment with different RTO values

I hope to send the patch within the next hour or so, subject
to some testing.
Many thanks
Gerrit

|  
|  ----- Original Message ----
|  From: Arnaldo Carvalho de Melo <arnaldo.melogmail.com>
|  To: Ian McDonald <ian.mcdonaldjandi.co.nz>
|  Cc: Leandro Melo de Sales <leandroalgmail.com>; burak gorkemli <burakgorkemliyahoo.com>; DCCP Mailing List <dccpvger.kernel.org>
|  Sent: Thursday, November 30, 2006 12:51:11 PM
|  Subject: Re:
|  
|  On 11/29/06, Ian McDonald <ian.mcdonaldjandi.co.nz> wrote:
|  > Folks,
|  >
|  > Another thing to note is that with a limit on tx
queue sizes now you
|  > will get EAGAIN if the queue is full...
|  
|  Yes, 4, 5 packets should be, in dccp_sendmsg:
|  
|          if (sysctl_dccp_tx_qlen &&
|              (sk->sk_write_queue.qlen >=
sysctl_dccp_tx_qlen)) {
|                  rc = -EAGAIN;
|                  goto out_release;
|          }
|  
|  Leandro and Burak: could you please try setting
sysctl_dccp_tx_qlen to
|  zero? doing this:
|  
|  [rootqemu ~]# cat /proc/sys/net/dccp/default/tx_qlen
|  5
|  [rootqemu ~]# echo 0 >
/proc/sys/net/dccp/default/tx_qlen
|  [rootqemu ~]# cat /proc/sys/net/dccp/default/tx_qlen
|  0
|  [rootqemu ~]#
|  
|  And report the results?
|  
|  - Arnaldo
|  -
|  To unsubscribe from this list: send the line
"unsubscribe dccp" in
|  the body of a message to majordomovger.kernel.org
|  More majordomo info at  http://vge
r.kernel.org/majordomo-info.html
|  
|  
|  
-
To unsubscribe from this list: send the line
"unsubscribe dccp" in
the body of a message to majordomovger.kernel.org
More majordomo info at  http://vge
r.kernel.org/majordomo-info.html

This is the patch which stops the nofeedback timer from
spinning
every couple of microseconds, triggered by small RTTs. 

However, there are a few more bugs in CCID 3; this is work
in progress.

-----------------> Commit Message
<-----------------------------------
[DCCP]: Use higher RTO default for CCID3 

The TFRC nofeedback timer normally expires after the maximum
of 4
RTTs and twice the current send interval (RFC 3448, 4.3). On
LANs
with a small RTT this can mean a high processing load and
reduced
performance, since then the nofeedback timer is triggered
very
frequently. As a result, the sending rate quickly converges
towards
zero.

This patch provides a configuration option to set the bound
for the
nofeedback timer, using as default the TCP RTO timeout of 1
second.

By setting the configuration option to 0, RFC 3448 behaviour
can
be enforced for the nofeedback timer.

Has been tested to compile and work.

Signed-off-by: Gerrit Renker <gerriterg.abdn.ac.uk>
---
 net/dccp/ccids/Kconfig |   22 ++++++++++++++++++++++
 net/dccp/ccids/ccid3.c |   20 +++++++++++---------
 2 files changed, 33 insertions(+), 9 deletions(-)

--- a/net/dccp/ccids/Kconfig
+++ b/net/dccp/ccids/Kconfig
 -89,4
+89,26  config IP_DCCP_CCID3_DEBUG
 	    parameter to 0 or 1.
 
 	    If in doubt, say N.
+
+config IP_DCCP_CCID3_RTO
+	  int "Use higher (TCP) RTO for nofeedback
timer"
+	  default 1
+	  depends on IP_DCCP_CCID3
+	  ---help---
+	    Use higher (TCP) RTO value for nofeedback expiration.
+
+	    The TFRC nofeedback timer normally expires after the
maximum of 4
+	    RTTs and twice the current send interval (RFC 3448,
4.3). On LANs
+	    with a small RTT this can mean a high processing load
and reduced
+	    performance, since then the nofeedback timer is
triggered very
+	    frequently. As a result, the sending rate quickly
converges towards
+	    zero.
+
+	    This option enables to set a higher upper bound for
the nofeedback
+	    value. Values in units of seconds can be set here.
+
+	    A value of 0 disables this feature by enforcing the
value specified
+	    in RFC 3448. It is recommended to use the default
value of 1 second,
+	    as this is also the suggested TCP RTO default (RFC
2988, 2.4).
+
 endmenu
--- a/net/dccp/ccids/ccid3.c
+++ b/net/dccp/ccids/ccid3.c
 -245,9
+245,10  static void ccid3_hc_tx_no_feedback_time
 		}
 		/*
 		 * Schedule no feedback timer to expire in
-		 * max(4 * R, 2 * s/X)  =  max(4 * R, 2 * t_ipi)
+		 * max(t_RTO, 2 * s/X)  =  max(t_RTO, 2 * t_ipi)
+		 * See comments above regarding the value of t_RTO.
 		 */
-		t_nfb = max(4 * hctx->ccid3hctx_rtt, 2 *
hctx->ccid3hctx_t_ipi);
+		t_nfb = max(hctx->ccid3hctx_t_rto, 2 *
hctx->ccid3hctx_t_ipi);
 		break;
 	case TFRC_SSTATE_NO_SENT:
 		DCCP_BUG("Illegal %s state NO_SENT, sk=%p",
dccp_role(sk), sk);
 -481,7
+482,7  static void ccid3_hc_tx_packet_recv(stru
 		}
 
 		ccid3_pr_debug("%s, sk=%p, New RTT estimate=%uus,
"
-			       "r_sample=%usn", dccp_role(sk), sk,
+			       "r_sample=%uusn", dccp_role(sk), sk,
 			       hctx->ccid3hctx_rtt, r_sample);
 
 		/* Update receive rate */
 -512,16
+513,17  static void ccid3_hc_tx_packet_recv(stru
 		 */
 		sk->sk_write_space(sk);
 
-		/* Update timeout interval. We use the alternative
variant of
-		 * [RFC 3448, 3.1] which sets the upper bound of t_rto to
one
-		 * second, as it is suggested for TCP (see RFC 2988,
2.4). */
+		/* Update timeout interval. A value of
IP_DCCP_CCID3_RTO=0
+		 * lets the nofeedback timer expire as per [RFC 3448,
3.1];
+		 * a value of 1 second corresponds to the RTO suggested
for
+		 * TCP (see RFC 2988, 2.4). */
 		hctx->ccid3hctx_t_rto = max_t(u32, 4 *
hctx->ccid3hctx_rtt,
-					      	   USEC_PER_SEC            );
+					CONFIG_IP_DCCP_CCID3_RTO * USEC_PER_SEC);
 		/*
 		 * Schedule no feedback timer to expire in
-		 * max(4 * R, 2 * s/X)  =  max(4 * R, 2 * t_ipi)
+		 * max(t_RTO, 2 * s/X)  =  max(t_RTO, 2 * t_ipi)
 		 */
-		t_nfb = max(4 * hctx->ccid3hctx_rtt, 2 *
hctx->ccid3hctx_t_ipi);
+		t_nfb = max(hctx->ccid3hctx_t_rto, 2 *
hctx->ccid3hctx_t_ipi);
 			
 		ccid3_pr_debug("%s, sk=%p, Scheduled no feedback
timer to "
 			       "expire in %lu jiffies (%luus)n",
-
To unsubscribe from this list: send the line
"unsubscribe dccp" in
the body of a message to majordomovger.kernel.org
More majordomo info at  http://vge
r.kernel.org/majordomo-info.html

|  > BTW, I am using CCID3.
|  
|  Ok, now to the next troubleshooting test: can you please
try using
|  CCID2? That way we can isolate the problem, is it CCID3
specific?
|  CCID2? Core? Something else?

Yes, please do send all kinds of debug information which you
encounter to the list; 
this makes tracking down the errors much simpler. 

Both CCID 2 and CCID 3 now have debugging options, 

 CONFIG_IP_DCCP_CCID2_DEBUG=y
 CONFIG_IP_DCCP_CCID3_DEBUG=y

but in order to get debugging output, these must be
explicitly set via module parameter. 
This can be automated. 

If you paste the following into a file
/etc/modutils/options,

# options for DCCP
options dccp            dccp_debug=1
options dccp_ccid2      ccid2_debug=1
options dccp_ccid3      ccid3_debug=1
# EOF

and run `update-modules' [alternatively: paste the above at
the end of /etc/modules.conf ]

The same options should also be placed into a file
/etc/modprobe.d/options (if you are running modprobe);
or alternatively into  /etc/modprobe.conf (man 5
modprobe.conf).

This all is important, much helpful output (e.g.
DCCP_BUG(...)) depends on having debugging enabled.

-
To unsubscribe from this list: send the line
"unsubscribe dccp" in
the body of a message to majordomovger.kernel.org
More majordomo info at  http://vge
r.kernel.org/majordomo-info.html

IETF mailing list,

I'm wondering if you'd like to comment on this as it seems
to be a
real implementation issue - using TFRC on local lans (100
Mbits/sec, <
2 ms rtt) seems to cause high CPU load when doing testing
using tools
such as iperf.

Is it worth considering having a minimum no feedback timer
for the
spec as checking things this often is not so good and would
definitely
hinder uptake on local boxes if we were being RFC compliant.

Thanks Gerrit for raising this.

Regards,

Ian
---------- Forwarded message ----------
From: Gerrit Renker <gerriterg.abdn.ac.uk>
Date: Dec 1, 2006 2:18 AM
Subject: [PATCH] [DCCP]: Use higher timeout value for
nofeedback timer
To: Arnaldo Carvalho de Melo <arnaldo.melogmail.com>, Ian McDonald
<ian.mcdonaldjandi.co.nz>
Cc: DCCP Mailing List <dccpvger.kernel.org>,
Leandro Melo de Sales
<leandroalgmail.com>, Burak Gorkemli
<burakgmail-dccpyahoo.com>


This is the patch which stops the nofeedback timer from
spinning
every couple of microseconds, triggered by small RTTs.

However, there are a few more bugs in CCID 3; this is work
in progress.

-----------------> Commit Message
<-----------------------------------
[DCCP]: Use higher RTO default for CCID3

The TFRC nofeedback timer normally expires after the maximum
of 4
RTTs and twice the current send interval (RFC 3448, 4.3). On
LANs
with a small RTT this can mean a high processing load and
reduced
performance, since then the nofeedback timer is triggered
very
frequently. As a result, the sending rate quickly converges
towards
zero.

This patch provides a configuration option to set the bound
for the
nofeedback timer, using as default the TCP RTO timeout of 1
second.

By setting the configuration option to 0, RFC 3448 behaviour
can
be enforced for the nofeedback timer.

Has been tested to compile and work.

Signed-off-by: Gerrit Renker <gerriterg.abdn.ac.uk>
-- 
Web: http://wand.net.nz/~iam4
Blog: http://imcdnzl.blogspot.c
om
WAND Network Research Group
-
To unsubscribe from this list: send the line
"unsubscribe dccp" in
the body of a message to majordomovger.kernel.org
More majordomo info at  http://vge
r.kernel.org/majordomo-info.html

On Nov 30, 2006, at 11:13, Ian McDonald wrote:
> This patch provides a configuration option to set the
bound for the
> nofeedback timer, using as default the TCP RTO timeout
of 1 second.

The default TCP RTO is 3 seconds (RFC 2988). Linux is
apparently not  
conformant to that RFC?

Lars

Quoting Lars Eggert:
|  On Nov 30, 2006, at 11:13, Ian McDonald wrote:
|  > This patch provides a configuration option to set
the bound for the
|  > nofeedback timer, using as default the TCP RTO
timeout of 1 second.
|  
|  The default TCP RTO is 3 seconds (RFC 2988). Linux is
apparently not  
|  conformant to that RFC?
Ian didn't copy the remainder of the mail, which had the
clarifying explanation
that this setting is related to section 2.4 of RFC 2988:
  (2.4) "Whenever RTO is computed, if it is less than 1
second then the
         RTO SHOULD be rounded up to 1 second."

Also [RFC 3448, sec. 1]: 
  "Another possibility would be to set t_RTO = max(4R,
one second), 
   to match the recommended minimum of one second on the RTO
[5]." 
   where [5] = RFC 2988.

Currently, the nofeedback timer is set to 

  t_nfb = max(4 * R, 2*s/X)

When R is very small (e.g. 250 usec), 4 * R is close to the
dimension of one or
two clock ticks.
-
To unsubscribe from this list: send the line
"unsubscribe dccp" in
the body of a message to majordomovger.kernel.org
More majordomo info at  http://vge
r.kernel.org/majordomo-info.html

Let me respond simply on the issue of 3 seconds.

I know the value of 3 seconds has not respected in all TCP
Implementations, but this is never the less the value that
was put by the
IETF - for good reasons. Smaller values of RTO can and do
cause
interoperability problems over some Internet Paths.

This issue is not new, many many years ago one major
manufacturer chose a
smaller initial RTO and caused major upset among people
using satellite
links... fortunately the implementation was soon patched,
but not before
it entered deployment on some web servers, where the problem
persisted.

I suggest a value of 3 Secs is sufficient for the General
internet...

Gorry

> On Nov 30, 2006, at 11:13, Ian McDonald wrote:
>> This patch provides a configuration option to set
the bound for the
>> nofeedback timer, using as default the TCP RTO
timeout of 1 second.
>
> The default TCP RTO is 3 seconds (RFC 2988). Linux is
apparently not
> conformant to that RFC?
>
> Lars
>
>


-
To unsubscribe from this list: send the line
"unsubscribe dccp" in
the body of a message to majordomovger.kernel.org
More majordomo info at  http://vge
r.kernel.org/majordomo-info.html