Re: enormous DB Traffic using imapsync

On Donnerstag, 6. Dezember 2007 Michael Monnerie wrote:
> I was searching for performance issues, and found
something very
> strange. We have postgreSQL on one host, dbmail on
another, so
> connections to/from the DB are on the network. I
installed a new
> dbmail instance, where only I am connected. I use
imapsync to sync
> from a local server account to an account on the new
dbmail instance.
>
> My PC generated 16MB IMAP traffic to the server, and
dbmail on port
> 5432 to the database 600MB (!) so far. It's still far
from being
> finished, and looks to me as if something does not work
correctly. I
> use "iptraf" to monitor the traffic on the
dbmail server directly.
> Any ideas what could cause that lot of DB traffic?

Thanks for all the discussion, I'd like to have this thread
back to it's 
original meaning. I made my test again today, as there's
nothing else 
running, seems everybody else has holidays, just me stupid
hacking 
around.

I make an imapsync to my mailserver, which connects to the
PostgreSQL 
db, and I get these numbers:

mailsrv -> db 15.482.309
db -> mailsrv 648.119.851

client -> mailsrv 1.771.458
mailsrv -> client 15.956.176

So with ~18MB IMAP traffic I produce ~663MB database
traffic? Can 
somebody explain me this?

mfg zmi
-- 
// Michael Monnerie, Ing.BSc    -----      http://it-management.at
// Tel: 0676/846 914 666                     
.network.your.ideas.
// PGP Key:         "curl -s http://zmi.at/zmi.asc | gpg
--import"
// Fingerprint: EA39 8918 EDFF 0A68 ACFB  11B7 BA2D 060F
1C6F E6B0
// Keyserver: www.keyserver.net                   Key-ID:
1C6FE6B0

_______________________________________________
DBmail mailing list
DBmaildbmail.org
htt
ps://mailman.fastxs.nl/mailman/listinfo/dbmail

Michael Monnerie wrote:

> I make an imapsync to my mailserver, which connects to
the PostgreSQL 
> db, and I get these numbers:
> 
> mailsrv -> db 15.482.309
> db -> mailsrv 648.119.851
> 
> client -> mailsrv 1.771.458
> mailsrv -> client 15.956.176
> 
> So with ~18MB IMAP traffic I produce ~663MB database
traffic? Can 
> somebody explain me this?

Interesting figures. Can you produce a base-line? Like say:
traffic
numbers for syncing a single account, containing a single
folder with a
single message of a known size, blah blah. Log the query
patterns
triggered, and the traffic numbers involved. Out of
curiosity. Perhaps
the explanation will jump out if the logs.

One hypothesis a priori: we don't cache as much as we could.
In fact,
dbmail caches *very* little and will at times happily call
the same
query a thousand times. And it doesn't even use prepared
statements (yet).

-- 
 
____________________________________________________________
____
  Paul Stevens                                      paul at
nfg.nl
  NET FACILITIES GROUP                     GPG/PGP:
1024D/11F8CD31
  The Netherlands________________________________http://www.nfg.nl
_______________________________________________
DBmail mailing list
DBmaildbmail.org
htt
ps://mailman.fastxs.nl/mailman/listinfo/dbmail