QCA: Bug in TLS::Private::update() and perhaps a solution

Hi,

as I told you before, I've written a "secure"
IRC-like chat program with
TLS sockets and certificate-based authentication using the
QCA. For the
sockets, I took the TlsSocket class from
$QCA/examples/tlssocket/ as a
basis and extended it to my needs.

Most of the time, the handshake of client and server
succeeds, but
sometimes it hangs and no TLS-connection is established. So
I enabled
the QCA log messages, and here is the server-side log of a
typicle
successful handshake:
        
        (src/qca_securelayer.cpp:372) tls[]: c->start()
        (src/qca_securelayer.cpp:769) tls[]:
c->resultsReady()
        (src/qca_securelayer.cpp:494) tls[]: update
        (src/qca_securelayer.cpp:575) tls[]: c->update
        (src/qca_securelayer.cpp:769) tls[]:
c->resultsReady()
        (src/qca_securelayer.cpp:1100) tls[]: writeIncoming
124
        (src/qca_securelayer.cpp:494) tls[]: update
        (src/qca_securelayer.cpp:575) tls[]: c->update
        (src/qca_securelayer.cpp:769) tls[]:
c->resultsReady()
        (src/qca_securelayer.cpp:621) tls[]: to_net 1792
        (src/qca_securelayer.cpp:1100) tls[]: writeIncoming
2287
        (src/qca_securelayer.cpp:494) tls[]: update
        (src/qca_securelayer.cpp:575) tls[]: c->update
        (src/qca_securelayer.cpp:769) tls[]:
c->resultsReady()
        (src/qca_securelayer.cpp:621) tls[]: to_net 59
        (src/qca_securelayer.cpp:390) tls[]:
continueAfterStep
        (src/qca_securelayer.cpp:494) tls[]: update
        (src/qca_securelayer.cpp:504) tls[]: ignoring update
while processing actions
        (src/qca_securelayer.cpp:440) tls[]: handshaken
        (src/qca_securelayer.cpp:390) tls[]:
continueAfterStep
        ...

When the handshake hangs, the server-side log looks as
follows:

        (src/qca_securelayer.cpp:372) tls[]: c->start()
        (src/qca_securelayer.cpp:769) tls[]:
c->resultsReady()
        (src/qca_securelayer.cpp:494) tls[]: update
        (src/qca_securelayer.cpp:575) tls[]: c->update
        (src/qca_securelayer.cpp:1100) tls[]: writeIncoming
124
        (src/qca_securelayer.cpp:494) tls[]: update
        (src/qca_securelayer.cpp:512) tls[]: ignoring update
while operation active
        (src/qca_securelayer.cpp:769) tls[]:
c->resultsReady()

That's all, no more messages after that. Notice that
"c->update" is not
directely followed by a "c->resultsReady()",
but new data has been
arriving meanwhile. I can always observe this message
ordering when the
handshake hangs. Obviously, this results in the new data not
being
processed after the TLSContext::resultsReady() signal is
emitted.

I don't fully understand all the mechanisms by now, but I
think when the
call to TLS::Private::update() doesn't lead to data
processing, the
need_update flag must be set to true (see attached patch).
After this
change no more hanging occured, even though the
"critical" order of log
messages still appeard:

        (src/qca_securelayer.cpp:372) tls[]: c->start()
        (src/qca_securelayer.cpp:770) tls[]:
c->resultsReady()
        (src/qca_securelayer.cpp:494) tls[]: update
        (src/qca_securelayer.cpp:576) tls[]: c->update
        (src/qca_securelayer.cpp:1101) tls[]: writeIncoming
124
        (src/qca_securelayer.cpp:494) tls[]: update
        (src/qca_securelayer.cpp:512) tls[]: ignoring update
while operation active
        (src/qca_securelayer.cpp:770) tls[]:
c->resultsReady()
        (src/qca_securelayer.cpp:405) tls[]: need_update
        (src/qca_securelayer.cpp:494) tls[]: update
        (src/qca_securelayer.cpp:576) tls[]: c->update
        (src/qca_securelayer.cpp:770) tls[]:
c->resultsReady()
        (src/qca_securelayer.cpp:622) tls[]: to_net 1792
        (src/qca_securelayer.cpp:1101) tls[]: writeIncoming
2287
        (src/qca_securelayer.cpp:494) tls[]: update
        (src/qca_securelayer.cpp:576) tls[]: c->update
        (src/qca_securelayer.cpp:770) tls[]:
c->resultsReady()
        (src/qca_securelayer.cpp:622) tls[]: to_net 59
        (src/qca_securelayer.cpp:390) tls[]:
continueAfterStep
        ...

I'm not sure, was this the right way to fix it? 

BTW: The code of SASL::Private::update() is nearly
identical, so I
assume that the same problem could occure there as well.
-- 
Regards,

Christian Schneider

_______________________________________________
Delta mailing list
Deltalists.affinix.com
http://lists.affinix.com/listinfo.cgi/delta-affinix.com

On Wednesday 20 February 2008 12:20 pm, Christian Schneider
wrote:
> Most of the time, the handshake of client and server
succeeds, but
> sometimes it hangs and no TLS-connection is
established. So I enabled
> the QCA log messages, and here is the server-side log
of a typicle
> successful handshake:
[snip]

Good catch, and good debugging.

> I don't fully understand all the mechanisms by now, but
I think when the
> call to TLS::Private::update() doesn't lead to data
processing, the
> need_update flag must be set to true (see attached
patch). After this
> change no more hanging occured, even though the
"critical" order of log
> messages still appeard:
[snip]
> I'm not sure, was this the right way to fix it?

It looks like it might be.  Thanks for the patch.

I think this problem could happen for clients as well.  All
it really comes 
down to is using writeIncoming() or write() during an active
update (the 
simplest scenario being if you called both of these
functions at once).  
Perhaps the asynchronous nature of XMPP helps keep this from
being an issue 
for Psi/Kopete, since a newly arriving packet will correct
the state.

This definitely warrants a new QCA base library release. 
Shucks.  I'll put a 
plan together...

> BTW: The code of SASL::Private::update() is nearly
identical, so I
> assume that the same problem could occure there as
well.

Probably.

-Justin
_______________________________________________
Delta mailing list
Deltalists.affinix.com
http://lists.affinix.com/listinfo.cgi/delta-affinix.com