List Info

Thread: disk suggestions for large-scale maia implementation




disk suggestions for large-scale maia implementation
user name
2006-05-18 17:41:14
David, I am in the same boat as you currently design
infrastructure to implement a large scale Maia operation for
about 3000 residential subscribers. 

My biggest concern at the moment is the database setup and
if I want an exclusive machine to handle database
connections for the two mail relays. And what the
implications of this would be as all the spam assassin rules
are held within the database.
If anyone has any input on this please advise.
I love maia and it's capabilities.
Adam
-----Original Message-----
From: maia-users-bouncesrenaissoft.com
[mailto:maia-users-bouncesrenaissoft.com] On Behalf
Of David Morton
Sent: May 18, 2006 12:45 PM
To: Aaron Bennett
Cc: maia-usersrenaissoft.com
Subject: Re: [Maia-users] disk suggestions for large-scale
maia implementation

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Aaron Bennett wrote:
> Hello,
> 
> I'm planning a reasonably large maia implementation --
about 3000
> users.  I've got two mail relays which will run
amavisd-maia and I'm
> purchasing a small box as a web server -- something
like a dl360, 3.8Gz
> Xeon.

> We accept about 60,000 emails in a day.  That's a
substantial amount of
> transactions.  What I'm trying to decide is between
one of three options...

In my experience, everyone worries too much about the
database.  Let's ask
another question: Has anyone had any database load problems
yet?  

I'm just guessing, but I think any of those solutions will
work fine. I've run
25k to 50k messages through a single server that housed
sql/web/amavisd-maia...
everything.   1GB RAM, 2 software mirrored SATA 10k Raptors.

On a similar note; In my experience a set of SATA WD Raptors
has smoked a
similar Ultra320 setup.  I thought SCSI was supposed to be
the champ here; has
anyone else seen this?

A project I'd like to set up is to make a test feed to pump
as much mail through
a server as possible to make some benchmarks.  Does anyone
have any hardware to
donate? 

- --
David Morton
Maia Mailguard                        - http://www.maiamailguard
.com
Morton Software Design and Consulting - http://www.dgrmm.net
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org


iD8DBQFEbKR/SIxC85HZHLMRApTBAJ9JrcOp828Sd1M1oQfJKNKfovI2XQCf
e05r
suaKiitSazvM1+MD/gEqTSI=
=y1RD
-----END PGP SIGNATURE-----
_______________________________________________
Maia-users mailing list
Maia-usersrenaissoft.com
http://www.renaissoft.com/mailman/listinfo/maia-users

-- 
No virus found in this incoming message.
Checked by AVG Free Edition.
Version: 7.1.392 / Virus Database: 268.6.0/342 - Release
Date: 17/05/2006
 

-- 
No virus found in this outgoing message.
Checked by AVG Free Edition.
Version: 7.1.392 / Virus Database: 268.6.0/342 - Release
Date: 17/05/2006
 
_______________________________________________
Maia-users mailing list
Maia-usersrenaissoft.com
http://www.renaissoft.com/mailman/listinfo/maia-users
disk suggestions for large-scale maia implementation
user name
2006-05-18 18:03:31
We have several standalone mail servers running with
postfix/cyrus/maia. 
For your load I would suggest dual CPU (dual core is fine),
2-4GB RAM, 
and at least 72GB of storage space. More than 144GB is
probably not 
necessary unless you have some huge mailboxes or most of
your users are 
using IMAP/Webmail. A single RAID 5 array with 3-4 drives is
simple and 
should manage fine, get a hot spare if you have the means.

The thing with MySQL is that it caches a lot of information
in RAM, the 
big disk IO hit is actually going to be coming from
postfix/sendmail 
which store all their information in the file system. The
main disk 
usage culprits are going to be mailboxes themselves, log
files (amavis, 
postfix, your pop server) and your maia mail table which
stores 
quarantined/cached emails. You can cut down on mailboxes by
implementing 
user quotas. Log files can be rotated by logrotate, but you
may have to 
setup specific log rotate scripts for services which don't
come with 
them. And lastly you can cut down on the mail that maia
caches in the 
database by lowering your "oversized mail item"
setting (nearly 100% of 
spam and virus are under 500kb) and by clearing old messages
on a weekly 
basis.

I don't have experience storing working data across SAN or
NFS, simply 
because a standalone server has always been more than enough
to handle 
the loads of messages that I've seen (~120,000 msg/day).
While you can 
use a separate db/web server, the load of these functions
hasn't been 
enough (for us) to justify the downsides (overhead, latency,

reliability) of moving them to a separate box from the MTA.

--Blake

logs wrote:
> David, I am in the same boat as you currently design
infrastructure to implement a large scale Maia operation for
about 3000 residential subscribers. 
>
> My biggest concern at the moment is the database setup
and if I want an exclusive machine to handle database
connections for the two mail relays. And what the
implications of this would be as all the spam assassin rules
are held within the database.
> If anyone has any input on this please advise.
> I love maia and it's capabilities.
> Adam
> -----Original Message-----
> From: maia-users-bouncesrenaissoft.com
[mailto:maia-users-bouncesrenaissoft.com] On Behalf
Of David Morton
> Sent: May 18, 2006 12:45 PM
> To: Aaron Bennett
> Cc: maia-usersrenaissoft.com
> Subject: Re: [Maia-users] disk suggestions for
large-scale maia implementation
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Aaron Bennett wrote:
>   
>> Hello,
>>
>> I'm planning a reasonably large maia
implementation -- about 3000
>> users.  I've got two mail relays which will run
amavisd-maia and I'm
>> purchasing a small box as a web server -- something
like a dl360, 3.8Gz
>> Xeon.
>>     
>
>   
>> We accept about 60,000 emails in a day.  That's a
substantial amount of
>> transactions.  What I'm trying to decide is
between one of three options...
>>     
>
> In my experience, everyone worries too much about the
database.  Let's ask
> another question: Has anyone had any database load
problems yet?  
>
> I'm just guessing, but I think any of those solutions
will work fine. I've run
> 25k to 50k messages through a single server that housed
sql/web/amavisd-maia...
> everything.   1GB RAM, 2 software mirrored SATA 10k
Raptors.
>
> On a similar note; In my experience a set of SATA WD
Raptors has smoked a
> similar Ultra320 setup.  I thought SCSI was supposed to
be the champ here; has
> anyone else seen this?
>
> A project I'd like to set up is to make a test feed to
pump as much mail through
> a server as possible to make some benchmarks.  Does
anyone have any hardware to
> donate? 
>
> - --
> David Morton
> Maia Mailguard                        - http://www.maiamailguard
.com
> Morton Software Design and Consulting - http://www.dgrmm.net
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.2 (MingW32)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

>
>
iD8DBQFEbKR/SIxC85HZHLMRApTBAJ9JrcOp828Sd1M1oQfJKNKfovI2XQCf
e05r
> suaKiitSazvM1+MD/gEqTSI=
> =y1RD
> -----END PGP SIGNATURE-----
> _______________________________________________
> Maia-users mailing list
> Maia-usersrenaissoft.com
> http://www.renaissoft.com/mailman/listinfo/maia-users
>
>   

_______________________________________________
Maia-users mailing list
Maia-usersrenaissoft.com
http://www.renaissoft.com/mailman/listinfo/maia-users
disk suggestions for large-scale maia implementation
user name
2006-05-18 18:23:20
logs wrote:

> And what the implications of this would be as all the
spam assassin rules are held within the database.

Maia does not store the SpamAssassin rules themselves in the
database,
only the rule names, descriptions, and scores are stored in
order to
provide lookups for the Mail Viewer.  These are not
consulted at all by
SpamAssassin or amavisd-maia, just by the web GUI, and only
in the Mail
Viewer.

On the other hand, if you opt to store your Bayes and/or AWL
databases
in SQL (I'd recommend it), Maia does provide database
tables for these.
 These tables will be consulted every time an e-mail gets
scanned by
SpamAssassin, but it's no worse than any of the other
database lookups
that amavisd-maia performs during the scanning process.

-- 
Robert LeBlanc <rjlrenaissoft.com>
Renaissoft, Inc.
Maia Mailguard <http://www.maiamail
guard.com/>

_______________________________________________
Maia-users mailing list
Maia-usersrenaissoft.com
http://www.renaissoft.com/mailman/listinfo/maia-users
disk suggestions for large-scale maia implementation
user name
2006-05-18 18:56:37
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

logs wrote:

> My biggest concern at the moment is the database setup
and if I want an exclusive machine to handle database
connections for the two mail relays. And what the
implications of this would be as all the spam assassin rules
are held within the database.
> If anyone has any input on this please advise.

The spamassassin rules are mainly used from the files as
usual with
spamassassin.  The fact that the rules are in the database
only comes into play
for while viewing a message; it's for reporting purposes
only

- --
David Morton
Maia Mailguard                        - http://www.maiamailguard
.com
Morton Software Design and Consulting - http://www.dgrmm.net
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org


iD8DBQFEbMNlSIxC85HZHLMRAqUyAJ9BzI22//jEUC3geKioQ+I98BmZfwCf
dOj9
JX3+D2p3/ihN6QqGYp2uP0o=
=kjkI
-----END PGP SIGNATURE-----
_______________________________________________
Maia-users mailing list
Maia-usersrenaissoft.com
http://www.renaissoft.com/mailman/listinfo/maia-users
[1-4]

about | contact  Other archives ( Real Estate discussion Medical topics )