low tx speed in ccid3 after loss

Hi,

After applying Ian's patches (except for bestpacketnext) I
did some tests with ccid3 and came across a problem. In some
cases when packet loss occurs, transmit rate drops down and
stays there, with almost no increase. Besides, p value is
set to a value bigger than zero after the loss, and does not
change again, although no loss occurs - which may be the
explanation of the unchanging tx rate ???

My test configuration is as follows: I am using two
machines, one of which is the ttcp_acme sender and the other
ttcp_acme receiver. netem configured to control the rate on
both of them such that the link between the machines is
128Kbps with RTT=100 ms. Below is the output of the
ttcp_acme sender.  I modified the code to print  tfrctx_x
and  tfrctv_x_calc also, so colums below are as follows:
"tv_sec.tv_usec tfrctx_x tfrctx_x_calc tfrctx_x_recv
tfrctx_rtt tfrctx_p".

Packet loss occurs only at the beginning, and after then I
see no dropped packets in netem - only requeues.

ttcp-t: buflen=256, nbuf=1000, align=16384/+0, port=5001 
dccp(inet)  -> thinkpad
0.000050 256 0 0 0 0
0.555739 2463 0 0 103924 0
0.615747 2462 0 507 103959 0
0.719787 2462 0 8539 103989 0
0.823764 4920 0 2460 104017 0
0.927793 4920 0 2461 104037 0
0.979774 9840 0 4922 104049 0
1.031858 9840 0 4924 104065 0
1.067841 9840 0 4922 104083 0
1.087751 9844 0 4922 104100 0
1.203755 9844 0 10672 105716 0
1.279759 19688 0 10106 107169 0
1.339794 19688 0 10105 108078 0
1.411944 26946 0 13473 110096 0
1.511800 20361 20361 13330 113510 14492
1.527950 19893 19893 12800 116184 14492
1.567852 12696 12696 12806 118187 27777
1.587862 7595 7595 16010 119993 52631
1.623776 6303 6303 12800 122805 62500
1.643772 4167 4167 12793 124150 90909
1.663772 2648 2648 12793 126161 125000
1.683771 1080 1080 16020 127169 200000
4.859899 81 81 12787 130415 500000
8.060019 83 83 160 127810 500000
11.260141 85 85 79 125473 500000
14.460259 86 86 79 123375 500000
17.660275 87 87 79 121494 500000
20.860398 89 89 79 119808 500000
24.060514 90 90 79 118297 500000
26.548605 91 91 79 116944 500000
29.392712 92 92 102 115731 500000
32.236830 93 93 90 114644 500000
35.080827 93 93 90 113674 500000
37.924931 94 94 90 112805 500000
40.769037 95 95 90 112029 500000
43.613338 95 95 90 111335 500000
46.461251 96 96 90 110715 500000
49.305354 96 96 89 110162 500000
52.149359 97 97 90 109671 500000
54.993464 97 97 90 109237 500000
57.837571 98 98 90 108854 500000
60.681677 98 98 90 108514 500000
63.525782 98 98 90 108213 500000
66.369890 98 98 90 107949 500000
69.213895 99 99 90 107718 500000
72.058002 99 99 90 107515 500000
74.902109 99 99 90 107338 500000
77.750214 99 99 90 107185 500000
80.594322 99 99 89 107053 500000
83.438427 99 99 90 106939 500000
86.282434 100 100 90 106840 500000
88.558516 100 100 90 106759 500000
91.118614 100 100 112 106692 500000
93.678708 100 100 99 106636 500000
96.238804 100 100 99 106591 500000
98.798898 100 100 99 106156 500000
101.358995 101 101 99 105770 500000
. . . . .
. . . . . 
1120.236353 103 103 99 103240 500000
1122.796393 103 103 99 103239 500000
1125.356436 103 103 99 103243 500000  (The transfer was not
complete, I killed the sender.)

However, this problem does not always occur. Below is the
output of another test, under the same conditions.

ttcp-t: buflen=256, nbuf=1000, align=16384/+0, port=5001 
dccp(inet)  -> thinkpad
0.000052 256 0 0 0 0
0.554416 2463 0 0 103925 0
0.618324 2459 0 507 104101 0
0.722335 2459 0 8005 104258 0
0.826360 4918 0 2460 104402 0
0.930354 4918 0 2461 104382 0
0.982371 9836 0 4923 104359 0
1.034341 9836 0 4924 104344 0
1.070354 9836 0 4921 104331 0
1.090338 9846 0 4923 104463 0
1.206352 9846 0 10670 106183 0
1.282348 19692 0 10105 107731 0
1.342440 19692 0 10105 108724 0
1.418357 26942 0 13471 110818 0
1.514405 15952 15952 13333 113101 21276
1.534362 15559 15559 12806 115957 21276
1.550356 9627 9627 12806 118126 40000
1.590358 6824 6824 12800 120879 58823
1.610358 5556 5556 16020 123357 71428
1.630359 4119 4119 12793 125584 90909
1.650359 2618 2618 12812 127589 125000
1.670361 1565 1565 12787 128996 166666
2.074386 615 615 12800 131365 250000
2.494402 628 628 1207 128800 250000
2.902416 639 639 609 126493 250000
3.302459 650 650 627 124417 250000
3.682443 660 660 639 122550 250000
4.062493 669 669 673 120871 250000
4.450471 677 677 673 119360 250000
4.826487 685 685 659 118001 250000
5.198501 692 692 680 116778 250000
5.566516 699 699 688 115679 250000
5.934527 705 705 695 114691 250000
6.294541 710 710 695 113801 250000
6.650554 716 716 711 113001 250000
7.010569 720 720 719 112282 250000
7.362608 724 724 711 111635 250000
7.718594 728 728 727 111054 250000
8.074606 732 732 719 110532 250000
8.418620 735 735 719 110064 250000
8.770632 737 737 744 109642 250000
9.122644 740 740 727 109264 250000
9.462659 742 742 727 108924 250000
9.806675 744 744 752 108618 250000
10.154683 746 746 744 108344 250000
. . . . .
. . . . .
50.063967 788 788 780 102639 250000
50.391980 788 788 780 102645 250000
50.719992 788 788 780 102652 250000
51.048004 788 788 780 102658 250000
51.375939 788 788 780 102664 250000
51.703943 788 788 780 102670 250000
. . . . .
. . . . .
100.277566 780 780 780 103654 250000
100.605581 780 780 780 103662 250000
100.933589 780 780 780 103669 250000
101.265604 780 780 780 103677 250000
101.593620 780 780 771 103685 250000
101.921629 780 780 780 103693 250000
. . . . .
. . . . .
150.107154 772 772 771 104679 250000
150.439168 772 772 771 104687 250000
150.771189 772 772 771 104694 250000
151.103191 772 772 771 104701 250000
151.435206 772 772 771 104707 250000
151.767217 772 772 771 104714 250000
. . . . .
. . . . .
200.040748 765 765 761 105705 250000
200.380760 765 765 761 105712 250000
200.716774 765 765 752 105718 250000
201.052786 765 765 761 105725 250000
201.388800 765 765 761 105731 250000
201.724812 765 765 761 105738 250000
. . . . .
. . . . .
250.214351 787 787 780 102739 250000
250.542364 787 787 780 102746 250000
250.870374 787 787 780 102753 250000
251.198402 787 787 780 102759 250000
251.526401 787 787 780 102766 250000
251.854413 787 787 780 102773 250000
. . . . .
. . . . .
264.326790 785 785 780 103018 250000
264.654817 785 785 780 103026 250000
264.982814 785 785 780 103033 250000
265.310825 785 785 780 103037 250000
ttcp-t: 256000 bytes in 265.31 real seconds = 0.94 KB/sec
+++
ttcp-t: 256000 bytes in 0.00 CPU seconds = 62500.00 KB/cpu
sec
ttcp-t: 1000 I/O calls, msec/call = 271.68, calls/sec = 3.77
ttcp-t: 0.0user 0.0sys 4:25real 0% 0i+0d 0maxrss 0+2pf
1649+1csw
ttcp-t: buffer address 0x8050000

Any comments?


-
To unsubscribe from this list: send the line
"unsubscribe dccp" in
the body of a message to majordomovger.kernel.org
More majordomo info at  http://vge
r.kernel.org/majordomo-info.html

On 8/27/06, Burak Gorkemli <burakgmail-dccpyahoo.com> wrote:
> Hi,
>
> After applying Ian's patches (except for
bestpacketnext) I did some tests with ccid3 and came across
a problem. In some cases when packet loss occurs, transmit
rate drops down and stays there, with almost no increase.
Besides, p value is set to a value bigger than zero after
the loss, and does not change again, although no loss occurs
- which may be the explanation of the unchanging tx rate ???

I haven't seen that behaviour - my p keeps changing.
>
> My test configuration is as follows: I am using two
machines, one of which is the ttcp_acme sender and the other
ttcp_acme receiver. netem configured to control the rate on
both of them such that the link between the machines is
128Kbps with RTT=100 ms. Below is the output of the
ttcp_acme sender.  I modified the code to print  tfrctx_x
and  tfrctv_x_calc also, so colums below are as follows:
"tv_sec.tv_usec tfrctx_x tfrctx_x_calc tfrctx_x_recv
tfrctx_rtt tfrctx_p".
>
Which machine was netem on or did you put it on both? I
usually put it
on a middle box. I haven't rate limited though - I put loss
in
instead. I'll test it like that next week. I've stopped
using ttcp now
and use iperf as ttcp was giving me some strange results
earlier ages
ago - I should relook at it though.

> Packet loss occurs only at the beginning, and after
then I see no dropped packets in netem - only requeues.
>
> ttcp-t: buflen=256, nbuf=1000, align=16384/+0,
port=5001  dccp(inet)  -> thinkpad
> 0.000050 256 0 0 0 0
> 0.555739 2463 0 0 103924 0

Will have a better look at later. I do have a theory though
- the
CCID3 receiving code only informs about loss when there is a
change in
loss. It should stabilise the sending in equilibrium at 128
kbits/sec.
However I am thinking if your buffers for netem aren't big
enough and
you get a loss early on then it will get that loss, bring
the rate
down and then you won't get any more loss so it won't
increase the
rate. I'll have to check whether we are RFC compliant there
- seems
nasty that you can't report the loss dropping - at present
the code
recalculates loss every time there is one and if you only
get one then
you could get in trouble...

Ian
-- 
Ian McDonald
Web: http://wand.net.nz/~iam4
Blog: http://imcdnzl.blogspot.c
om
WAND Network Research Group
Department of Computer Science
University of Waikato
New Zealand
-
To unsubscribe from this list: send the line
"unsubscribe dccp" in
the body of a message to majordomovger.kernel.org
More majordomo info at  http://vge
r.kernel.org/majordomo-info.html

Burak wrote on the Linux DCCP mailing list:
> > After applying Ian's patches (except for
bestpacketnext) I did some tests with ccid3 and came across
a problem. In some cases when packet loss occurs, transmit
rate drops down and stays there, with almost no increase.
Besides, p value is set to a value bigger than zero after
the loss, and does not change again, although no loss occurs
- which may be the explanation of the unchanging tx rate ???
>

I have a question for people. If DCCP detects loss once and
once only
(as I suspect is happening here (Burak - can you confirm
through a
packet trace please?) then I haven't read anything in the
spec of TFRC
or CCID3 which says that you can lower the loss rate later
on when you
don't have a loss. As I read the spec you only recalculate
the loss
rate when you get a loss and thus if you have a loss when
you are
starting you have a real issue.

Am I missing something obvious here? I probably am but I
can't quite
see at the moment - can someone point it out to me. As I
read it the
calculation is based on this part of the TFRC spec:
             If (p > 0)
                 Calculate X_calc using the TCP throughput
equation.
                 X = max(min(X_calc, min_rate), s/t_mbi);
If p doesn't change then X_calc doesn't change and
therefore you can't
go above X_calc

Ian
-- 
Ian McDonald
Web: http://wand.net.nz/~iam4
Blog: http://imcdnzl.blogspot.c
om
WAND Network Research Group
Department of Computer Science
University of Waikato
New Zealand
-
To unsubscribe from this list: send the line
"unsubscribe dccp" in
the body of a message to majordomovger.kernel.org
More majordomo info at  http://vge
r.kernel.org/majordomo-info.html

On Sun, Aug 27, 2006 at 04:33:15PM +1200, Ian McDonald
wrote:
> Burak wrote on the Linux DCCP mailing list:
> >> After applying Ian's patches (except for
bestpacketnext) I did some 
> >tests with ccid3 and came across a problem. In some
cases when packet loss 
> >occurs, transmit rate drops down and stays there,
with almost no increase. 
> >Besides, p value is set to a value bigger than zero
after the loss, and 
> >does not change again, although no loss occurs -
which may be the 
> >explanation of the unchanging tx rate ???
> >
> 
> I have a question for people. If DCCP detects loss once
and once only
> (as I suspect is happening here (Burak - can you
confirm through a
> packet trace please?) then I haven't read anything in
the spec of TFRC
> or CCID3 which says that you can lower the loss rate
later on when you
> don't have a loss. As I read the spec you only
recalculate the loss
> rate when you get a loss and thus if you have a loss
when you are
> starting you have a real issue.

Hi Ian,

I beleive you need to recompute the loss rate on every
received data packet.
While there is no packet loss, the average loss interval
will increase and 
p will decrease.

This is because we take into account the interval between
the lastest loss
event and the current sequence number if it helps us
increase the rate.

RFC 3448:
"When calculating the average loss interval we need to
decide whether
    to include the interval since the most recent packet
loss event.  We
    only do this if it is sufficiently large to increase the
average
    loss interval."

Without this, TFRC will never push the rate up to find newly
added bw, unless
it is in slowstart ofcourse.

Best regards 
-- 
        Programmer
        Edgar E. Iglesias <edgar.iglesiasaxis.com> 46.46.272.1946
-
To unsubscribe from this list: send the line
"unsubscribe dccp" in
the body of a message to majordomovger.kernel.org
More majordomo info at  http://vge
r.kernel.org/majordomo-info.html

On Mon, Aug 28, 2006 at 08:17:49AM +0200, Edgar E. Iglesias
wrote:
> On Sun, Aug 27, 2006 at 04:33:15PM +1200, Ian McDonald
wrote:
> > Burak wrote on the Linux DCCP mailing list:
> > >> After applying Ian's patches (except for
bestpacketnext) I did some 
> > >tests with ccid3 and came across a problem. In
some cases when packet loss 
> > >occurs, transmit rate drops down and stays
there, with almost no increase. 
> > >Besides, p value is set to a value bigger than
zero after the loss, and 
> > >does not change again, although no loss occurs
- which may be the 
> > >explanation of the unchanging tx rate ???
> > >
> > 
> > I have a question for people. If DCCP detects loss
once and once only
> > (as I suspect is happening here (Burak - can you
confirm through a
> > packet trace please?) then I haven't read
anything in the spec of TFRC
> > or CCID3 which says that you can lower the loss
rate later on when you
> > don't have a loss. As I read the spec you only
recalculate the loss
> > rate when you get a loss and thus if you have a
loss when you are
> > starting you have a real issue.
> 
> Hi Ian,
> 
> I beleive you need to recompute the loss rate on every
received data packet.
> While there is no packet loss, the average loss
interval will increase and 
> p will decrease.

Some second thoughts 

Maybe you can get away with recomputing p only when the time
allows us to 
send a new feedback packet.

Best regards
-- 
        Programmer
        Edgar E. Iglesias <edgar.iglesiasaxis.com> 46.46.272.1946
-
To unsubscribe from this list: send the line
"unsubscribe dccp" in
the body of a message to majordomovger.kernel.org
More majordomo info at  http://vge
r.kernel.org/majordomo-info.html

On 8/28/06, Edgar E. Iglesias <edgar.iglesiasaxis.com> wrote:
> Hi Ian,
>
> I beleive you need to recompute the loss rate on every
received data packet.
> While there is no packet loss, the average loss
interval will increase and
> p will decrease.
>
> This is because we take into account the interval
between the lastest loss
> event and the current sequence number if it helps us
increase the rate.
>
> RFC 3448:
> "When calculating the average loss interval we
need to decide whether
>     to include the interval since the most recent
packet loss event.  We
>     only do this if it is sufficiently large to
increase the average
>     loss interval."
>
> Without this, TFRC will never push the rate up to find
newly added bw, unless
> it is in slowstart ofcourse.
>
Thanks for this. I suspected that there would be something
like this
but I just didn't see it!

At some stage I might code this up for the Linux
implementation but
I'm happy for somebody else to attempt (Burak?) as it's
not affecting
me right now.

Ian
-- 
Ian McDonald
Web: http://wand.net.nz/~iam4
Blog: http://imcdnzl.blogspot.c
om
WAND Network Research Group
Department of Computer Science
University of Waikato
New Zealand
-
To unsubscribe from this list: send the line
"unsubscribe dccp" in
the body of a message to majordomovger.kernel.org
More majordomo info at  http://vge
r.kernel.org/majordomo-info.html

Hi Ian,

I am reading the ccid3 code for a while now and it seems to
me that "the interval since the most recent packet
loss event" is not taken into account while
calculating I_mean, as you and Edgar suspect.  The first
interval used for calculating i_tot0 in
dccp_li_hist_calc_i_mean() should point to "the
interval since the most recent packet loss event", but
it does not, I guess. In the morning I will debug this more
and try to fix it, if all goes well. Actually I did some
modifications today but they did not relieve my problems
altogether.

BTW, can you recommend me a tool for debugging the ccid (and
dccp) module? I think there is Ostra but you seem to
discourage its usage in the Ostra wiki page (http://wiki.linux.net.
nz/Ostra). 

--
Burak Gorkemli
Ph.D. Candidate
Koc University
Turkey


----- Original Message ----
From: Ian McDonald <ian.mcdonaldjandi.co.nz>
To: Edgar E. Iglesias <edgar.iglesiasaxis.com>
Cc: Burak Gorkemli <burakgmail-dccpyahoo.com>; dccpvger.kernel.org; DCCP mailing list <dccpietf.org>
Sent: Tuesday, August 29, 2006 2:38:09 AM
Subject: Re: low tx speed in ccid3 after loss

On 8/28/06, Edgar E. Iglesias <edgar.iglesiasaxis.com> wrote:
> Hi Ian,
>
> I beleive you need to recompute the loss rate on every
received data packet.
> While there is no packet loss, the average loss
interval will increase and
> p will decrease.
>
> This is because we take into account the interval
between the lastest loss
> event and the current sequence number if it helps us
increase the rate.
>
> RFC 3448:
> "When calculating the average loss interval we
need to decide whether
>     to include the interval since the most recent
packet loss event.  We
>     only do this if it is sufficiently large to
increase the average
>     loss interval."
>
> Without this, TFRC will never push the rate up to find
newly added bw, unless
> it is in slowstart ofcourse.
>
Thanks for this. I suspected that there would be something
like this
but I just didn't see it!

At some stage I might code this up for the Linux
implementation but
I'm happy for somebody else to attempt (Burak?) as it's
not affecting
me right now.

Ian
-- 
Ian McDonald
Web: http://wand.net.nz/~iam4
Blog: http://imcdnzl.blogspot.c
om
WAND Network Research Group
Department of Computer Science
University of Waikato
New Zealand



-
To unsubscribe from this list: send the line
"unsubscribe dccp" in
the body of a message to majordomovger.kernel.org
More majordomo info at  http://vge
r.kernel.org/majordomo-info.html

On 8/29/06, Burak Gorkemli <burakgmail-dccpyahoo.com> wrote:
> Hi Ian,

Hi Burak,

Just replying on the dccp list only - ietf don't need to
know
implementation details.
>
> I am reading the ccid3 code for a while now and it
seems to me that "the interval since the most recent
packet loss event" is not taken into account while
calculating I_mean, as you and Edgar suspect.  The first
interval used for calculating i_tot0 in
dccp_li_hist_calc_i_mean() should point to "the
interval since the most recent packet loss event", but
it does not, I guess. In the morning I will debug this more
and try to fix it, if all goes well. Actually I did some
modifications today but they did not relieve my problems
altogether.

Correct. I am sure of that since I wrote that code - you are
running
the latest patches I presume? Basically how I would write
the code is
to see if p including the non-loss time is less than p
excluding it
then change p.

>
> BTW, can you recommend me a tool for debugging the ccid
(and dccp) module? I think there is Ostra but you seem to
discourage its usage in the Ostra wiki page (http://wiki.linux.net.
nz/Ostra).

Ostra is a great tool but it is a little bit out of date at
present as
Arnaldo has been very busy.

I used a mixture of printk's spread around the place and
dccp_probe
(which is on my website in the patches).

Ian
-- 
Ian McDonald
Web: http://wand.net.nz/~iam4
Blog: http://imcdnzl.blogspot.c
om
WAND Network Research Group
Department of Computer Science
University of Waikato
New Zealand
-
To unsubscribe from this list: send the line
"unsubscribe dccp" in
the body of a message to majordomovger.kernel.org
More majordomo info at  http://vge
r.kernel.org/majordomo-info.html