List Info

Thread: Speed up web access
Speed up web access
country flaguser name
United States		 2007-12-07 10:55:38 

  


  

    

We are running Maia on a Suse 10 Linux Box with Apache 2


 ;


Does anyone have any tips / tricks, to speed up the web access?


 


It takes sometimes 20-30 sec to jump between screens (Login / spam etc)


 


The Box is doing a lot of work filtering the mail, just wondering if anyone has tweaked anything in say apache it's self or somewhere to bring the screens up a little faster?


 


I can live with it, but users think it "Hung" on them. They have no patience :')


 


 








George Armstrong
Assistant Director of Technology
Monroe-Woodbury CSD
Harriman, NY
845-460-6600
 garmstromw.k12.ny.us">garmstromw.k12.ny.us
 george.armstrongmw.k12.ny.us">george.armstrongmw.k12.ny.us





  

  
  
   
     
    
 
        
    

    
    

  

    Re:  Speed up web access
  


  

     country flaguser name
United States		
     2007-12-07 12:29:14
  


  

    
George Armstrong wrote:
> We are running Maia on a Suse 10 Linux Box with Apache
2
>  
> Does anyone have any tips / tricks, to speed up the web
access?
>  
>

Several things your can do ... search for tuning mysql - you
may need to
give it more RAM to work with.   You could separate the web
server to
another box, and also move the mysql database to another box
that is
tuned for database use.

You also might tweak the cache expiry time in the system
admin page, to
make it shorter, which would reduce the size of the email
caches shown.

_______________________________________________
Maia-users mailing list
Maia-usersrenaissoft.com
http://www.renaissoft.com/mailman/listinfo/maia-users



  


  

    
  

  
  
   
     
    

  

     large maia DB
  


  

     country flaguser name
United States		
     2007-12-14 15:16:33
  


  

    
My maia db has grown substantially in the last few days, and
is  
causing sql delays that cause the mail queues to slow down
and the web  
interface to be almost un-usable.
(maia_mail.idb is 11gigs, and maia_mail_recipients.idb is
304megs)

About 23 days ago I rebuilt our sql server (FreeBSD 6.2 2gig
of ram,  
mysql 5.0.45) and started with a clean maia_mail table and
just dumped/ 
imported the user config tables.

I am using InnoDB and have attached my my.cnf at the end.

Looking back in the cron logs for the expire-queue script I
can see  
the times it takes to process the cache is grown
dramatically.

Day one of the new db, nothing should expire because 3
days...  But to  
give you an idea of mail per day...

2007-11-20 03:10:33 Maia: [expire-quarantine-cache]
Suspected Spam  
items               :    118961
2007-11-20 03:10:33 Maia: [expire-quarantine-cache]
Virus/Malware  
items                :       117
2007-11-20 03:10:33 Maia: [expire-quarantine-cache] Items
with Invalid  
Mail Headers    :       221
2007-11-20 03:10:33 Maia: [expire-quarantine-cache] Items
with Banned  
File Attachments :        11
2007-11-20 03:10:33 Maia: [expire-quarantine-cache]
Suspected Non-Spam  
items           :     42552

Day 4, things should start to expire, this is what the size
of the db  
I expect to stay at.  Takes an hour and 45min to run...

2007-11-23 03:00:01 Maia: [expire-quarantine-cache] Expiring
Suspected  
Spam items [S] older than 3 days
2007-11-23 03:35:15 Maia: [expire-quarantine-cache] Expiring
Virus/ 
Malware items [V] older than 3 days
2007-11-23 03:35:28 Maia: [expire-quarantine-cache] Expiring
Invalid  
Mail Header items [B] older than 3 days
2007-11-23 03:35:36 Maia: [expire-quarantine-cache] Expiring
Banned  
File Attachment items [F] older than 3 days
2007-11-23 03:35:36 Maia: [expire-quarantine-cache] Expiring
Suspected  
Non-Spam items [H] older than 3 days
2007-11-23 03:46:27 Maia: [expire-quarantine-cache] Expiring
Orphaned  
items (mail with no recipients) older than one hour
2007-11-23 03:46:42 Maia: [expire-quarantine-cache] Expiring
Stray  
References (recipient references with no mail items)
2007-11-23 04:09:21 Maia: [expire-quarantine-cache]
Recalculating  
statistics
2007-11-23 04:46:24 Maia: [expire-quarantine-cache] Expiring
Tokens  
due to expire now
2007-11-23 04:46:24 Maia: [expire-quarantine-cache] 103068
Suspected  
Spam items expired
2007-11-23 04:46:24 Maia: [expire-quarantine-cache] 102
Virus/Malware  
items expired
2007-11-23 04:46:24 Maia: [expire-quarantine-cache] 176
items with  
Invalid Mail Headers expired
2007-11-23 04:46:24 Maia: [expire-quarantine-cache] 6 items
with  
Banned File Attachments expired
2007-11-23 04:46:24 Maia: [expire-quarantine-cache] 39674
Suspected  
Non-Spam items expired
2007-11-23 04:46:24 Maia: [expire-quarantine-cache] Current
Cache  
After Expiration:
2007-11-23 04:46:24 Maia: [expire-quarantine-cache]
Suspected Spam  
items               :    322492
2007-11-23 04:46:24 Maia: [expire-quarantine-cache]
Virus/Malware  
items                :       344
2007-11-23 04:46:24 Maia: [expire-quarantine-cache] Items
with Invalid  
Mail Headers    :       264
2007-11-23 04:46:24 Maia: [expire-quarantine-cache] Items
with Banned  
File Attachments :        18
2007-11-23 04:46:24 Maia: [expire-quarantine-cache]
Suspected Non-Spam  
items           :     69670


Now forward to Dec 12th, this is the last one to complete. 
After that  
the process has been stopped by me restarting the sql server
to try  
and make things better. Now it is taking 11 hours to run!

2007-12-12 03:00:00 Maia: [expire-quarantine-cache] Expiring
Suspected  
Spam items [S] older than 3 days
2007-12-12 08:55:14 Maia: [expire-quarantine-cache] Expiring
Virus/ 
Malware items [V] older than 3 days
2007-12-12 08:57:33 Maia: [expire-quarantine-cache] Expiring
Invalid  
Mail Header items [B] older than 3 days
2007-12-12 08:57:59 Maia: [expire-quarantine-cache] Expiring
Banned  
File Attachment items [F] older than 3 days
2007-12-12 08:57:59 Maia: [expire-quarantine-cache] Expiring
Suspected  
Non-Spam items [H] older than 3 days
2007-12-12 11:17:25 Maia: [expire-quarantine-cache] Expiring
Orphaned  
items (mail with no recipients) older than one hour
2007-12-12 11:20:46 Maia: [expire-quarantine-cache] Expiring
Stray  
References (recipient references with no mail items)
2007-12-12 15:23:43 Maia: [expire-quarantine-cache]
Recalculating  
statistics
2007-12-12 22:57:32 Maia: [expire-quarantine-cache] Expiring
Tokens  
due to expire now
2007-12-12 22:57:32 Maia: [expire-quarantine-cache] 199703
Suspected  
Spam items expired
2007-12-12 22:57:32 Maia: [expire-quarantine-cache] 415
Virus/Malware  
items expired
2007-12-12 22:57:32 Maia: [expire-quarantine-cache] 60 items
with  
Invalid Mail Headers expired
2007-12-12 22:57:32 Maia: [expire-quarantine-cache] 0 items
with  
Banned File Attachments expired
2007-12-12 22:57:32 Maia: [expire-quarantine-cache] 50607
Suspected  
Non-Spam items expired
2007-12-12 22:57:32 Maia: [expire-quarantine-cache] Current
Cache  
After Expiration:
2007-12-12 22:57:32 Maia: [expire-quarantine-cache]
Suspected Spam  
items               :    758218
2007-12-12 22:57:32 Maia: [expire-quarantine-cache]
Virus/Malware  
items                :       664
2007-12-12 22:57:32 Maia: [expire-quarantine-cache] Items
with Invalid  
Mail Headers    :       222
2007-12-12 22:57:32 Maia: [expire-quarantine-cache] Items
with Banned  
File Attachments :        15
2007-12-12 22:57:32 Maia: [expire-quarantine-cache]
Suspected Non-Spam  
items           :    106368


I am about to just drop the maia_mail table again and start
over, but  
how can I prevent this from happening again?


My cron events to do processing...
0 * * * * /usr/local/maia/scripts/process-quarantine.pl
--learn --report
0 0 * * *
/usr/local/maia/scripts/send-quarantine-reminders.pl
0 3 * * *
/usr/local/maia//scripts/expire-quarantine-cache.pl
#30 8-17 * * *
/usr/local/maia/scripts/send-quarantine-digests.pl

[mysqld]
port            = 3306
socket          = /tmp/mysql.sock
skip-locking
key_buffer = 32M
max_allowed_packet = 5M
table_cache = 512
sort_buffer_size = 6M
read_buffer_size = 2M
read_rnd_buffer_size = 8M
myisam_sort_buffer_size = 2M
thread_cache = 32
thread_cache_size = 40
query_cache_size = 64M
query_cache_limit = 20M
query_cache_type = 1
# Try number of CPU's*2 for thread_concurrency
thread_concurrency = 8
max_connections = 150
server-id       = 1

innodb_data_home_dir = /var/db/mysql/
#innodb_data_file_path =
ibdata1:2000M;ibdata2:10M:autoextend
innodb_log_group_home_dir = /var/db/mysql/
innodb_log_arch_dir = /var/db/mysql/
innodb_buffer_pool_size = 1000M
innodb_additional_mem_pool_size = 20M
#innodb_log_file_size = 250M
innodb_log_buffer_size = 8M
innodb_flush_log_at_trx_commit = 0
innodb_lock_wait_timeout = 50
innodb_status_file=0
innodb_locks_unsafe_for_binlog=1
innodb_thread_concurrency=0
innodb_file_per_table=1
innodb_flush_method=O_DIRECT


MySQL 5.0.45             uptime 0 1:14:16       Fri Dec 14
13:10:41 2007

__ Key
____________________________________________________________
_____
Buffer used   222.00k of  32.00M  %Used:   0.68
   Current       3.90M            %Usage:  12.19
Write hit       1.82%
Read hit       92.22%

__ Questions
___________________________________________________________
Total           3.32M   745.8/s
   DMS           2.17M   486.8/s  %Total:  65.27
   QC Hits     927.26k   208.1/s           27.90
   Com_        152.56k    34.2/s            4.59
   COM_QUIT     74.30k    16.7/s            2.24
   -Unknown        123     0.0/s            0.00
Slow               66     0.0/s            0.00  %DMS:  
0.00
DMS             2.17M   486.8/s           65.27
   SELECT        1.09M   244.7/s           32.81        
50.27
   UPDATE      637.34k   143.0/s           19.18        
29.38
   INSERT      313.40k    70.3/s            9.43        
14.45
   DELETE      128.00k    28.7/s            3.85         
5.90
   REPLACE           0       0/s            0.00         
0.00
Com_          152.56k    34.2/s            4.59
   set_option   73.79k    16.6/s            2.22
   show_variab  36.48k     8.2/s            1.10
   commit       31.80k     7.1/s            0.96

__ SELECT and Sort
_____________________________________________________
Scan           37.08k     8.3/s %SELECT:   3.40
Range         374.73k    84.1/s           34.36
Full join           2     0.0/s            0.00
Range check         0       0/s            0.00
Full rng join       0       0/s            0.00
Sort scan          91     0.0/s
Sort range      3.93k     0.9/s
Sort mrg pass       0       0/s

__ Query Cache
_________________________________________________________
Memory usage   21.61M of  64.00M  %Used:  33.76
Block Fragmnt   8.01%
Hits          927.26k   208.1/s
Inserts       578.71k   129.9/s
Insrt:Prune 578.71k:1   129.9/s
Hit:Insert     1.60:1

__ Table Locks
_________________________________________________________
Waited              1     0.0/s  %Total:   0.00
Immediate       2.14M   479.9/s

__ Tables
____________________________________________________________
__
Open              289 of  512    %Cache:  56.45
Opened            295     0.1/s

__ Connections
_________________________________________________________
Max used          125 of  150      %Max:  83.33
Total          74.31k    16.7/s

__ Created Temp
________________________________________________________
Disk table         22     0.0/s
Table          36.99k     8.3/s
File                5     0.0/s

__ Threads
____________________________________________________________
_
Running            14 of   88
Cached             29 of   40      %Hit:  99.51
Created           366     0.1/s
Slow                0       0/s

__ Aborted
____________________________________________________________
_
Clients             2     0.0/s
Connects            0       0/s

__ Bytes
____________________________________________________________
___
Sent          864.66M  194.0k/s
Received      641.11M  143.9k/s

__ InnoDB Buffer Pool
__________________________________________________
Usage         1000.00 of 1000.00  %Used: 100.00
Read ratio      0.004
Pages
   Free              0            %Total:   0.00
   Data         63.14k                     98.66 %Drty:  
8.02
   Misc            858                      1.34
   Latched          28                      0.04
Reads         161.30M   36.2k/s
   From file   654.42k   146.9/s            0.41
   Ahead Rnd      5581     1.3/s
   Ahead Sql       108     0.0/s
Writes          9.22M    2.1k/s
Flushes       907.58k   203.7/s
Wait Free           0       0/s

__ InnoDB Lock
_________________________________________________________
Waits            9695     2.2/s
Current             0
Time acquiring
   Total       5860583 ms
   Average         604 ms
   Max           14459 ms

__ InnoDB Data, Pages, Rows
____________________________________________
Data
   Reads       706.49k   158.5/s
   Writes      710.45k   159.4/s
   fsync        37.23k     8.4/s
   Pending
     Reads           4
     Writes          0
     fsync           0

Pages
   Created      30.19k     6.8/s
   Read        874.41k   196.2/s
   Written     907.58k   203.7/s

Rows
   Deleted     188.28k    42.3/s
   Inserted    313.39k    70.3/s
   Read         39.17M    8.8k/s
   Updated       1.22M   273.2/s





_______________________________________________
Maia-users mailing list
Maia-usersrenaissoft.com
http://www.renaissoft.com/mailman/listinfo/maia-users



  


  

    
  

  
  
   
     
    






 [1-3]











   
 





about | contact  Other archives ( Real Estate discussion Medical topics )