List Info

Thread: Another case of duplicate keys (via dbmail-smtp)




Another case of duplicate keys (via dbmail-smtp)
country flaguser name
Italy
2008-03-04 13:20:57
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I did some further testing on 2.2.9 and was able
to reproduce the duplicate key problem.

The following message fed to dbmail via
cat /tmp/test | dbmail-smtp -d testgufonero.com -r xxxxgufonero.com

produces the duplicate key message

- -------------------------------------------
Return-Path: <xxxxgufonero.com>
Received: by mail.gufonero.com (Postfix, from userid 0)
    id 8DB13AFB94; Tue,  4 Mar 2008 03:39:04 -0500 (EST)
To: testgufonero.com
Subject: Test
Message-Id: <20080304083904.8DB13AFB94mail.gufonero.com>
Date: Tue,  4 Mar 2008 03:39:04 -0500 (EST)
From: xxxxgufonero.com

Simple test
- ----------------------------------------

Instead just changing the message a bit, the following
message
produces NO error when submitted by:

cat /tmp/test | dbmail-smtp -d testgufonero.com -r xxxxxxgufonero.com

- ---------------------------------------
Return-Path: <xxxxxxgufonero.com>
Received: by mail.gufonero.com (Postfix, from userid 0)
    id 8DB13AFB94; Tue,  4 Mar 2008 03:39:04 -0500 (EST)
To: testgufonero.com
Subject: Test
Message-Id: <20080304083904.8DB13AFB94mail.gufonero.com>
Date: Tue,  4 Mar 2008 03:39:04 -0500 (EST)
From: xxxxxxgufonero.com

Simple test
- ------------------------------------

but strangely has inserted TWO entries for Return-Path
into dbmail_headervalue xxxxxxgufonero.com and blank.

see attached xxxx.log for first case and xxxxxx.log
for second case.

John

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.4-svn0 (GNU/Linux)
Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org


iD8DBQFHzaEZd4I3jTtt9EIRAiW/AKDZOKYYrNGjZC8qdkUQUrQf3WofHgCe
ITZm
oEw4RYLuKfTYNMuJe5q7wkY=
=BF6P
-----END PGP SIGNATURE-----

_______________________________________________
DBmail mailing list
DBmaildbmail.org
htt
ps://mailman.fastxs.nl/mailman/listinfo/dbmail

  
  
(patch) Another case of duplicate keys (via dbmail-smtp)
country flaguser name
Netherlands
2008-03-04 15:15:02
I posted a patch that should fix this earlier today. I
havent reproduced
this problem (mysql settings involved here I suspect) so I
can't verify
this patch beyond not causing regressions.

John Fawcett wrote:
> I did some further testing on 2.2.9 and was able
> to reproduce the duplicate key problem.
> 
> The following message fed to dbmail via
> cat /tmp/test | dbmail-smtp -d testgufonero.com -r xxxxgufonero.com
> 
> produces the duplicate key message
> 
> -------------------------------------------
> Return-Path: <xxxxgufonero.com>
> Received: by mail.gufonero.com (Postfix, from userid
0)
>     id 8DB13AFB94; Tue,  4 Mar 2008 03:39:04 -0500
(EST)
> To: testgufonero.com
> Subject: Test
> Message-Id: <20080304083904.8DB13AFB94mail.gufonero.com>
> Date: Tue,  4 Mar 2008 03:39:04 -0500 (EST)
> From: xxxxgufonero.com
> 
> Simple test
> ----------------------------------------
> 
> Instead just changing the message a bit, the following
message
> produces NO error when submitted by:
> 
> cat /tmp/test | dbmail-smtp -d testgufonero.com -r xxxxxxgufonero.com
> 
> ---------------------------------------
> Return-Path: <xxxxxxgufonero.com>
> Received: by mail.gufonero.com (Postfix, from userid
0)
>     id 8DB13AFB94; Tue,  4 Mar 2008 03:39:04 -0500
(EST)
> To: testgufonero.com
> Subject: Test
> Message-Id: <20080304083904.8DB13AFB94mail.gufonero.com>
> Date: Tue,  4 Mar 2008 03:39:04 -0500 (EST)
> From: xxxxxxgufonero.com
> 
> Simple test
> ------------------------------------
> 
> but strangely has inserted TWO entries for Return-Path
> into dbmail_headervalue xxxxxxgufonero.com and blank.
> 
> see attached xxxx.log for first case and xxxxxx.log
> for second case.
> 
> John
> 

------------------------------------------------------------
------------

_______________________________________________
DBmail mailing list
DBmaildbmail.org
htt
ps://mailman.fastxs.nl/mailman/listinfo/dbmail


-- 
 
____________________________________________________________
____
  Paul Stevens                                      paul at
nfg.nl
  NET FACILITIES GROUP                     GPG/PGP:
1024D/11F8CD31
  The Netherlands________________________________http://www.nfg.nl

_______________________________________________
DBmail mailing list
DBmaildbmail.org
htt
ps://mailman.fastxs.nl/mailman/listinfo/dbmail

  
Re: (patch) Another case of duplicate keys (via dbmail-smtp)
country flaguser name
Italy
2008-03-04 16:16:15
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Paul J Stevens wrote:
> I posted a patch that should fix this earlier today. I
havent reproduced
> this problem (mysql settings involved here I suspect)
so I can't verify
> this patch beyond not causing regressions.
> 
I couldn't get the patch to work (it didn't insert all the
headers into dbmail_headervalue but skipped over quite a
few
including the Return-Path).

I ran it with a slight modification:

static gboolean g_str_case_equal(gconstpointer a,
gconstpointer b)
{
        return MATCH((const char *)a,(const char *)b);
}


the patch seems ok, but does not solve the issue. The errors
I
reported are with the above patch already done.

I cannot work it out at the moment why some message are ok
others are not. For some reason I think the return path
with
<> round the email address is not working as it
should.

John
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.4-svn0 (GNU/Linux)
Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org


iD8DBQFHzcovd4I3jTtt9EIRAvmSAKDdwtvJg4ZdMHyOhyB/bCpc/mXBtQCg
yyKG
PqaAXC4F+eO6INRA/AVSx7g=
=Er8m
-----END PGP SIGNATURE-----
_______________________________________________
DBmail mailing list
DBmaildbmail.org
htt
ps://mailman.fastxs.nl/mailman/listinfo/dbmail

Re: (patch) Another case of duplicate keys (via dbmail-smtp)
country flaguser name
Italy
2008-03-05 03:50:38
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

John Fawcett wrote:
> 
> the patch seems ok, but does not solve the issue. The
errors I
> reported are with the above patch already done.
> 
> I cannot work it out at the moment why some message are
ok
> others are not. For some reason I think the return path
with
> <> round the email address is not working as it
should.
> 
> John

ok, I've done some more debugging and have found one part of
the problem:

the values in the GRelation which stores the headers/values
relations
are being corrupted by the call to g_mime_message_set_header
in
dbmail_message_set_header.

I printed them before and after that call and they are
different:
I used this code which is the same as the code which will be
used
later for reading the headers to be inserted.

        GTuples *values;
        unsigned char *raw;
	unsigned i;

        values =
g_relation_select(self->headers,header,0);
        for (i=0; i<values->len;i++) {
                raw = (unsigned char
*)g_tuples_index(values,i,1);
                TRACE(TRACE_DEBUG,"raw value
[%s]",(char*)raw);
        }



in the case of xxxxxx.log (where two headers were inserted,
one correct
and one blank) these are the values before and after the
g_mime_message_set_header call:

before: raw value [<xxxxxxgufonero.com>]
after :raw value [^A]

the [^A] value eventually gets sanitized to ""
which is why a blank
header gets inserted into the dbmail_headervalue table. It
looks like
the call to g_mime_message_set_header is deallocating some
memory
which the GRelation structure points to.

in the case of xxxx.log (where there was a duplicate entry)

before: raw value [<xxxxgufonero.com>]
after :raw value [xxxxgufonero.com]

so it looks as though the call to g_mime_message_set_header
has
overwritten the previous memory with a new value but kept
memory position the same.

The code then goes on to add a new Return-Path header
xxxxgufonero.com
as specified by the command line parameter -r, so that's
why we end up with two Return-Path headers which are the
same.

There is clearly a duplicate header problem to solve, but
this is
compounded by undefined side effects on in memory headers
after the call to g_mime_message_set_header.

There is also an additional problem with
g_mime_message_set_header
which is that it does not add a header if it is missing
but only change the value of an existing header, so I am
getting
message that come in without a return-path being stored
without
a return-path.

These are my packages on opensuse 10.3
gmime-2.2.1-10
glibc-2.4-31.1

We can spend time trying to get this to work, but I'm just
wondering
whether we need to use g_mime_message_set_header at all.
Seeing as
in the end we must have only one Return-Path header in the
delivered
message and we are trying to update an existing header,
maybe it
would be easier to ignore existing return-path headers and
just
add our own: that way we also avoid duplicate Return-Path
headers in the in-memory structure which is the fundamental
issue.

John

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.4-svn0 (GNU/Linux)
Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org


iD8DBQFHzmzud4I3jTtt9EIRAqsZAJ9cUG0v7kEQmdN4KeLJ0NIjDj0DQwCe
OC04
wbTv4QhwM3fkl/3Ccmt4a+Y=
=CTiK
-----END PGP SIGNATURE-----
_______________________________________________
DBmail mailing list
DBmaildbmail.org
htt
ps://mailman.fastxs.nl/mailman/listinfo/dbmail

Re: (patch) Another case of duplicate keys (via dbmail-smtp)
country flaguser name
United States
2008-03-05 04:02:23
John Fawcett wrote:
> 
> We can spend time trying to get this to work, but I'm
just wondering
> whether we need to use g_mime_message_set_header at
all. Seeing as
> in the end we must have only one Return-Path header in
the delivered
> message and we are trying to update an existing header,
maybe it
> would be easier to ignore existing return-path headers
and just
> add our own: that way we also avoid duplicate
Return-Path
> headers in the in-memory structure which is the
fundamental issue.
> 

I don't know much about C if anything, but just to chime in.
As far as I 
understand this code deals with general header handling.
Isn't it perfectly 
legal to have several X-headers with absolutely the same
values in a message? 
What happens then?
_______________________________________________
DBmail mailing list
DBmaildbmail.org
htt
ps://mailman.fastxs.nl/mailman/listinfo/dbmail

Re: (patch) Another case of duplicate keys (via dbmail-smtp)
country flaguser name
Netherlands
2008-03-05 13:18:41
Paul,

> I'll release 2.2.10 real-soon-now (this week) unless
there are any showstop bugs
> in the tracker I've missed.

If you release 2.2.10 could you also do an update on
dbmail-imapd.txt in the man pages.
It was already changed for the 2.3.x devel.

>
> Peter Rabbitson wrote:
>> Hi Paul,
>>
>> I see that the fix made it to the repository. Do
you have an ETA for a
>> 2.10 debian package? The 2.9 version has rolled
into official debian
>> testing and I need to figure out how to proceed
with my upgrade plans
>> (i.e. to mark the packages as held, or just wait a
couple of days if a
>> new release comes soon). Either way is fine, I just
wanted to stay
>> informed 
>>
>> Thanks

Sjoerd

_______________________________________________
DBmail mailing list
DBmaildbmail.org
htt
ps://mailman.fastxs.nl/mailman/listinfo/dbmail

  
[1-6]

about | contact  Other archives ( Real Estate discussion Medical topics )