Hi,
Your 2.0 load average is a clue... I am not a database
person... So I do
not know where to go from here... There seems to be lots of
data 'cached'
in memory, but linux uses cache and buffers to the max....
You might want
to research what the 'cached' memory is used for. Here is a
top from a
running Maia filter on a 2.4 Ghz P4 with 1 GB of memory:
top - 11:52:49 up 63 days, 16:47, 1 user, load average:
0.09, 0.05, 0.05
Tasks: 61 total, 2 running, 59 sleeping, 0 stopped,
0 zombie
Cpu(s): 8.3% us, 1.3% sy, 0.0% ni, 88.7% id, 1.7% wa,
0.0% hi, 0.0%
si
Mem: 1035128k total, 914740k used, 120388k free,
161568k buffers
Swap: 746980k total, 12476k used, 734504k free,
134796k cached
Dave
************************************************************
***********
On Thu, 11 Oct 2007, Rick Zeman wrote:
> free
> total used free shared
buffers
> cached
> Mem: 2074872 2022764 52108 0
0
> 1590628
> -/+ buffers/cache: 432136 1642736
> Swap: 2104432 100 2104332
>
> top shows:
> top - 12:13:36 up 1 day, 23:48, 5 users, load
average: 2.00, 2.01,
> 2.33
> Tasks: 123 total, 1 running, 122 sleeping, 0
stopped, 0 zombie
> Cpu(s): 0.8%us, 0.3%sy, 0.0%ni, 1.8%id, 96.8%wa,
0.0%hi, 0.2%si,
> 0.0%st
> Mem: 2074872k total, 2023072k used, 51800k free,
0k buffers
> Swap: 2104432k total, 100k used, 2104332k free,
1590904k cached
>
> mysqld is the top running processing (sorted by
"k" cpu use), but it's %
> utilization _going by top_ never goes above 5%. But,
looking at the top
> summary, something sure as heck is chewing on it but
not showing up in
> top...and as soon as mysqld is shut down it goes back
to being virtually
> totally idle. Very strange.
>
> Here's what mysql says about itself. I haven't changed
anything except
> max_allowed_packet = 32M from what shipped from SuSE as
the defaults.
>
>
> mysql> show variables;
>
+---------------------------------+-------------------------
---+
> | Variable_name | Value
|
>
+---------------------------------+-------------------------
---+
> | auto_increment_increment | 1
|
> | auto_increment_offset | 1
|
> | automatic_sp_privileges | ON
|
> | back_log | 50
|
> | basedir | /usr/
|
> | binlog_cache_size | 32768
|
> | bulk_insert_buffer_size | 8388608
|
> | character_set_client | latin1
|
> | character_set_connection | latin1
|
> | character_set_database | latin1
|
> | character_set_filesystem | binary
|
> | character_set_results | latin1
|
> | character_set_server | latin1
|
> | character_set_system | utf8
|
> | character_sets_dir |
/usr/share/mysql/charsets/ |
> | collation_connection | latin1_swedish_ci
|
> | collation_database | latin1_swedish_ci
|
> | collation_server | latin1_swedish_ci
|
> | completion_type | 0
|
> | concurrent_insert | 1
|
> | connect_timeout | 5
|
> | datadir | /var/lib/mysql/
|
> | date_format | %Y-%m-%d
|
> | datetime_format | %Y-%m-%d %H:%i:%s
|
> | default_week_format | 0
|
> | delay_key_write | ON
|
> | delayed_insert_limit | 100
|
> | delayed_insert_timeout | 300
|
> | delayed_queue_size | 1000
|
> | div_precision_increment | 4
|
> | engine_condition_pushdown | OFF
|
> | expire_logs_days | 0
|
> | flush | OFF
|
> | flush_time | 0
|
> | ft_boolean_syntax | +
-><()~*:""&| |
> | ft_max_word_len | 84
|
> | ft_min_word_len | 4
|
> | ft_query_expansion_limit | 20
|
> | ft_stopword_file | (built-in)
|
> | group_concat_max_len | 1024
|
> | have_archive | NO
|
> | have_bdb | NO
|
> | have_blackhole_engine | NO
|
> | have_compress | YES
|
> | have_crypt | YES
|
> | have_csv | NO
|
> | have_dynamic_loading | NO
|
> | have_example_engine | NO
|
> | have_federated_engine | NO
|
> | have_geometry | YES
|
> | have_innodb | YES
|
> | have_isam | NO
|
> | have_merge_engine | YES
|
> | have_ndbcluster | NO
|
> | have_openssl | NO
|
> | have_query_cache | YES
|
> | have_raid | NO
|
> | have_rtree_keys | YES
|
> | have_symlink | YES
|
> | init_connect |
|
> | init_file |
|
> | init_slave |
|
> | innodb_additional_mem_pool_size | 1048576
|
> | innodb_autoextend_increment | 8
|
> | innodb_buffer_pool_awe_mem_mb | 0
|
> | innodb_buffer_pool_size | 8388608
|
> | innodb_checksums | ON
|
> | innodb_commit_concurrency | 0
|
> | innodb_concurrency_tickets | 500
|
> | innodb_data_file_path |
ibdata1:10M:autoextend |
> | innodb_data_home_dir |
|
> | innodb_doublewrite | ON
|
> | innodb_fast_shutdown | 1
|
> | innodb_file_io_threads | 4
|
> | innodb_file_per_table | OFF
|
> | innodb_flush_log_at_trx_commit | 1
|
> | innodb_flush_method |
|
> | innodb_force_recovery | 0
|
> | innodb_lock_wait_timeout | 50
|
> | innodb_locks_unsafe_for_binlog | OFF
|
> | innodb_log_arch_dir |
|
> | innodb_log_archive | OFF
|
> | innodb_log_buffer_size | 1048576
|
> | innodb_log_file_size | 5242880
|
> | innodb_log_files_in_group | 2
|
> | innodb_log_group_home_dir | ./
|
> | innodb_max_dirty_pages_pct | 90
|
> | innodb_max_purge_lag | 0
|
> | innodb_mirrored_log_groups | 1
|
> | innodb_open_files | 300
|
> | innodb_support_xa | ON
|
> | innodb_sync_spin_loops | 20
|
> | innodb_table_locks | ON
|
> | innodb_thread_concurrency | 8
|
> | innodb_thread_sleep_delay | 10000
|
> | interactive_timeout | 28800
|
> | join_buffer_size | 131072
|
> | key_buffer_size | 16777216
|
> | key_cache_age_threshold | 300
|
> | key_cache_block_size | 1024
|
> | key_cache_division_limit | 100
|
> | language |
/usr/share/mysql/english/ |
> | large_files_support | ON
|
> | large_page_size | 0
|
> | large_pages | OFF
|
> | lc_time_names | en_US
|
> | license | GPL
|
> | local_infile | ON
|
> | locked_in_memory | OFF
|
> | log | OFF
|
> | log_bin | OFF
|
> | log_bin_trust_function_creators | OFF
|
> | log_error |
|
> | log_queries_not_using_indexes | OFF
|
> | log_slave_updates | OFF
|
> | log_slow_queries | OFF
|
> | log_warnings | 1
|
> | long_query_time | 10
|
> | low_priority_updates | OFF
|
> | lower_case_file_system | OFF
|
> | lower_case_table_names | 0
|
> | max_allowed_packet | 33553408
|
> | max_binlog_cache_size | 4294967295
|
> | max_binlog_size | 1073741824
|
> | max_connect_errors | 10
|
> | max_connections | 100
|
> | max_delayed_threads | 20
|
> | max_error_count | 64
|
> | max_heap_table_size | 16777216
|
> | max_insert_delayed_threads | 20
|
> | max_join_size | 4294967295
|
> | max_length_for_sort_data | 1024
|
> | max_prepared_stmt_count | 16382
|
> | max_relay_log_size | 0
|
> | max_seeks_for_key | 4294967295
|
> | max_sort_length | 1024
|
> | max_sp_recursion_depth | 0
|
> | max_tmp_tables | 32
|
> | max_user_connections | 0
|
> | max_write_lock_count | 4294967295
|
> | multi_range_count | 256
|
> | myisam_data_pointer_size | 6
|
> | myisam_max_sort_file_size | 2147483647
|
> | myisam_recover_options | OFF
|
> | myisam_repair_threads | 1
|
> | myisam_sort_buffer_size | 8388608
|
> | myisam_stats_method | nulls_unequal
|
> | net_buffer_length | 8192
|
> | net_read_timeout | 30
|
> | net_retry_count | 10
|
> | net_write_timeout | 60
|
> | new | OFF
|
> | old_passwords | OFF
|
> | open_files_limit | 1024
|
> | optimizer_prune_level | 1
|
> | optimizer_search_depth | 62
|
> | pid_file |
/var/lib/mysql/mysqld.pid |
> | port | 3306
|
> | preload_buffer_size | 32768
|
> | prepared_stmt_count | 0
|
> | protocol_version | 10
|
> | query_alloc_block_size | 8192
|
> | query_cache_limit | 1048576
|
> | query_cache_min_res_unit | 4096
|
> | query_cache_size | 0
|
> | query_cache_type | ON
|
> | query_cache_wlock_invalidate | OFF
|
> | query_prealloc_size | 8192
|
> | range_alloc_block_size | 2048
|
> | read_buffer_size | 258048
|
> | read_only | OFF
|
> | read_rnd_buffer_size | 520192
|
> | relay_log_purge | ON
|
> | relay_log_space_limit | 0
|
> | rpl_recovery_rank | 0
|
> | secure_auth | OFF
|
> | server_id | 1
|
> | skip_external_locking | ON
|
> | skip_networking | OFF
|
> | skip_show_database | OFF
|
> | slave_compressed_protocol | OFF
|
> | slave_load_tmpdir |
/var/lib/mysql/.tmp/ |
> | slave_net_timeout | 3600
|
> | slave_skip_errors | OFF
|
> | slave_transaction_retries | 10
|
> | slow_launch_time | 2
|
> | socket |
/var/lib/mysql/mysql.sock |
> | sort_buffer_size | 524280
|
> | sql_big_selects | ON
|
> | sql_mode |
|
> | sql_notes | ON
|
> | sql_warnings | OFF
|
> | ssl_ca |
|
> | ssl_capath |
|
> | ssl_cert |
|
> | ssl_cipher |
|
> | ssl_key |
|
> | storage_engine | MyISAM
|
> | sync_binlog | 0
|
> | sync_frm | ON
|
> | system_time_zone | EDT
|
> | table_cache | 64
|
> | table_lock_wait_timeout | 50
|
> | table_type | MyISAM
|
> | thread_cache_size | 0
|
> | thread_stack | 196608
|
> | time_format | %H:%i:%s
|
> | time_zone | SYSTEM
|
> | timed_mutexes | OFF
|
> | tmp_table_size | 33554432
|
> | tmpdir |
/var/lib/mysql/.tmp/ |
> | transaction_alloc_block_size | 8192
|
> | transaction_prealloc_size | 4096
|
> | tx_isolation | REPEATABLE-READ
|
> | updatable_views_with_limit | YES
|
> | version | 5.0.26
|
> | version_comment | SUSE MySQL RPM
|
> | version_compile_machine | i686
|
> | version_compile_os | suse-linux-gnu
|
> | wait_timeout | 28800
|
>
+---------------------------------+-------------------------
---+
> 220 rows in set (0.00 sec)
>
>
>
> >>> David Sims <dpsims dpsims.com> 10/11/2007 11:59 AM >>>
> Hi,
>
> What does 'top' show you while this is running? Also,
please have a
> look
> at ht
tp://www.maiamailguard.com/maia/wiki/MySQLTuning to see
if that
> provides any clues. 'free' will show you both memory
and swap usage....
> I would think that this must be a MySQL tuning issue of
some sort, but I
> do not know what the specific issue might be....
Anything in the MySQL
> log
> file?
>
> Dave
>
************************************************************
********
>
>
> On Thu, 11 Oct 2007, Rick Zeman wrote:
>
> > >>> David Morton <mortondadgrmm.net> 10/11/2007 1:06 AM >>>
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> >
> >
> > On Oct 10, 2007, at 12:12 PM, David Sims wrote:
> >
> > > Hi,
> > >
> > > As I recall, this machine ran for a while
without running
> > > process-quarantine.pl.... I am thinking that
you need to process the
> > > quarantine to remove LOTS of mail.... If it's
offline, then why not
> > > just
> >
> > I'd start by turning down the cache expry time,
and running the
> > expire script first, to narrow down the file.
> >
> > We can also drop and re-add just the tables that
hold the mail, but
> > keep the user settings... if we come to that
point.
> >
> > ________
> >
> > That was a superb idea, David. Changed both
values to 1 and then ran
> the
> > expire script and it dropped over 12,000 messages.
Reran quarantine
> and
> > it's pluging away (and bogging down again after 60
messages). BUT,
> other
> > than a couple of system generated mails, there
should be none newer
> than 2
> > days because the server isn't processing any new
mail (and hasn't for
> over
> > 7 days). If I wasn't confused before I am now!
> >
> > This entire session. Look at the time stamps:
> >
> > ./process-quarantine.pl
> > 2007-10-11 09:39:43 Maia: [process-quarantine-sub]
Warning: --report
> not
> > specified, reporting is disabled.
> > 2007-10-11 09:39:49 Maia: [process-quarantine-sub]
Learned mail item
> > 18115 as spam
> > 2007-10-11 09:39:55 Maia: [process-quarantine-sub]
Learned mail item
> > 18116 as spam
> > 2007-10-11 09:40:00 Maia: [process-quarantine-sub]
Learned mail item
> > 18118 as spam
> > 2007-10-11 09:40:18 Maia: [process-quarantine-sub]
Learned mail item
> > 18122 as spam
> > 2007-10-11 09:40:29 Maia: [process-quarantine-sub]
Learned mail item
> > 18125 as spam
> > 2007-10-11 09:40:29 Maia: [process-quarantine-sub]
5 spam items
> processed
> > (5 learned, 0 reported)
> > 2007-10-11 09:41:20 Maia: [process-quarantine-sub]
Learned mail item
> > 190499 as non-spam
> > 2007-10-11 09:42:26 Maia: [process-quarantine-sub]
Learned mail item
> > 190504 as non-spam
> > 2007-10-11 09:42:35 Maia: [process-quarantine-sub]
Learned mail item
> > 190506 as non-spam
> > 2007-10-11 09:42:55 Maia: [process-quarantine-sub]
Learned mail item
> > 190512 as non-spam
> > 2007-10-11 09:54:56 Maia: [process-quarantine-sub]
Learned mail item
> > 190515 as non-spam
> > 2007-10-11 09:54:56 Maia: [process-quarantine-sub]
5 non-spam items
> > processed (5 learned)
> > 2007-10-11 09:54:56 Maia: [process-quarantine-sub]
Warning: --report
> not
> > specified, reporting is disabled.
> > 2007-10-11 09:55:20 Maia: [process-quarantine-sub]
Learned mail item
> > 18126 as spam
> > 2007-10-11 09:55:53 Maia: [process-quarantine-sub]
Learned mail item
> > 18127 as spam
> > 2007-10-11 09:56:14 Maia: [process-quarantine-sub]
Learned mail item
> > 18128 as spam
> > 2007-10-11 09:56:30 Maia: [process-quarantine-sub]
Learned mail item
> > 18131 as spam
> > 2007-10-11 09:56:51 Maia: [process-quarantine-sub]
Learned mail item
> > 18133 as spam
> > 2007-10-11 09:56:51 Maia: [process-quarantine-sub]
5 spam items
> processed
> > (5 learned, 0 reported)
> > 2007-10-11 09:57:10 Maia: [process-quarantine-sub]
Learned mail item
> > 190517 as non-spam
> > 2007-10-11 09:57:26 Maia: [process-quarantine-sub]
Learned mail item
> > 190524 as non-spam
> > 2007-10-11 09:57:53 Maia: [process-quarantine-sub]
Learned mail item
> > 190534 as non-spam
> > 2007-10-11 09:59:22 Maia: [process-quarantine-sub]
Learned mail item
> > 190536 as non-spam
> > 2007-10-11 10:02:53 Maia: [process-quarantine-sub]
Learned mail item
> > 190543 as non-spam
> > 2007-10-11 10:02:54 Maia: [process-quarantine-sub]
5 non-spam items
> > processed (5 learned)
> > 2007-10-11 10:02:54 Maia: [process-quarantine-sub]
Warning: --report
> not
> > specified, reporting is disabled.
> > 2007-10-11 10:03:05 Maia: [process-quarantine-sub]
Learned mail item
> > 18138 as spam
> > 2007-10-11 10:03:19 Maia: [process-quarantine-sub]
Learned mail item
> > 18139 as spam
> > 2007-10-11 10:03:32 Maia: [process-quarantine-sub]
Learned mail item
> > 18140 as spam
> > 2007-10-11 10:03:50 Maia: [process-quarantine-sub]
Learned mail item
> > 18141 as spam
> > 2007-10-11 10:04:34 Maia: [process-quarantine-sub]
Learned mail item
> > 18142 as spam
> > 2007-10-11 10:04:34 Maia: [process-quarantine-sub]
5 spam items
> processed
> > (5 learned, 0 reported)
> > 2007-10-11 10:05:20 Maia: [process-quarantine-sub]
Learned mail item
> > 190551 as non-spam
> > 2007-10-11 10:06:16 Maia: [process-quarantine-sub]
Learned mail item
> > 190552 as non-spam
> > 2007-10-11 10:08:46 Maia: [process-quarantine-sub]
Learned mail item
> > 190559 as non-spam
> > 2007-10-11 10:10:21 Maia: [process-quarantine-sub]
Learned mail item
> > 190566 as non-spam
> > 2007-10-11 10:11:25 Maia: [process-quarantine-sub]
Learned mail item
> > 190571 as non-spam
> > 2007-10-11 10:11:26 Maia: [process-quarantine-sub]
5 non-spam items
> > processed (5 learned)
> > 2007-10-11 10:11:26 Maia: [process-quarantine-sub]
Warning: --report
> not
> > specified, reporting is disabled.
> > 2007-10-11 10:11:36 Maia: [process-quarantine-sub]
Learned mail item
> > 18144 as spam
> > 2007-10-11 10:11:57 Maia: [process-quarantine-sub]
Learned mail item
> > 18146 as spam
> > 2007-10-11 10:12:13 Maia: [process-quarantine-sub]
Learned mail item
> > 18148 as spam
> > 2007-10-11 10:12:23 Maia: [process-quarantine-sub]
Learned mail item
> > 18149 as spam
> > 2007-10-11 10:12:53 Maia: [process-quarantine-sub]
Learned mail item
> > 18153 as spam
> > 2007-10-11 10:12:53 Maia: [process-quarantine-sub]
5 spam items
> processed
> > (5 learned, 0 reported)
> > 2007-10-11 10:13:47 Maia: [process-quarantine-sub]
Learned mail item
> > 190574 as non-spam
> > 2007-10-11 10:14:09 Maia: [process-quarantine-sub]
Learned mail item
> > 190575 as non-spam
> > 2007-10-11 10:14:09 Maia: [process-quarantine-sub]
Skipping mail item
> > 190577 (too big: 9073816 > 262144)
> > 2007-10-11 10:14:13 Maia: [process-quarantine-sub]
5 non-spam items
> > processed (2 learned)
> > 2007-10-11 10:14:13 Maia: [process-quarantine-sub]
Warning: --report
> not
> > specified, reporting is disabled.
> > 2007-10-11 10:14:28 Maia: [process-quarantine-sub]
Learned mail item
> > 18157 as spam
> > 2007-10-11 10:15:07 Maia: [process-quarantine-sub]
Learned mail item
> > 18162 as spam
> > 2007-10-11 10:15:41 Maia: [process-quarantine-sub]
Learned mail item
> > 18163 as spam
> > 2007-10-11 10:15:47 Maia: [process-quarantine-sub]
Learned mail item
> > 18164 as spam
> > 2007-10-11 10:16:34 Maia: [process-quarantine-sub]
Learned mail item
> > 18165 as spam
> > 2007-10-11 10:16:34 Maia: [process-quarantine-sub]
5 spam items
> processed
> > (5 learned, 0 reported)
> > 2007-10-11 10:25:29 Maia: [process-quarantine-sub]
Learned mail item
> > 190579 as non-spam
> > 2007-10-11 10:25:42 Maia: [process-quarantine-sub]
Learned mail item
> > 190588 as non-spam
> > 2007-10-11 10:25:42 Maia: [process-quarantine-sub]
Skipping mail item
> > 190596 (too big: 3701512 > 262144)
> > 2007-10-11 10:25:46 Maia: [process-quarantine-sub]
Skipping mail item
> > 190597 (too big: 6338040 > 262144)
> > 2007-10-11 10:26:21 Maia: [process-quarantine-sub]
Learned mail item
> > 190606 as non-spam
> > 2007-10-11 10:26:21 Maia: [process-quarantine-sub]
5 non-spam items
> > processed (3 learned)
> > 2007-10-11 10:26:21 Maia: [process-quarantine-sub]
Warning: --report
> not
> > specified, reporting is disabled.
> > 2007-10-11 10:26:48 Maia: [process-quarantine-sub]
Learned mail item
> > 18170 as spam
> > 2007-10-11 10:27:07 Maia: [process-quarantine-sub]
Learned mail item
> > 18171 as spam
> > 2007-10-11 10:27:55 Maia: [process-quarantine-sub]
Learned mail item
> > 18172 as spam
> > 2007-10-11 10:28:23 Maia: [process-quarantine-sub]
Learned mail item
> > 18174 as spam
> > 2007-10-11 10:28:51 Maia: [process-quarantine-sub]
Learned mail item
> > 18175 as spam
> > 2007-10-11 10:28:51 Maia: [process-quarantine-sub]
5 spam items
> processed
> > (5 learned, 0 reported)
> > 2007-10-11 10:29:14 Maia: [process-quarantine-sub]
Learned mail item
> > 190608 as non-spam
> > 2007-10-11 10:29:26 Maia: [process-quarantine-sub]
Learned mail item
> > 190620 as non-spam
> > 2007-10-11 10:29:46 Maia: [process-quarantine-sub]
Learned mail item
> > 190623 as non-spam
> > 2007-10-11 10:30:03 Maia: [process-quarantine-sub]
Learned mail item
> > 190625 as non-spam
> > 2007-10-11 10:30:27 Maia: [process-quarantine-sub]
Learned mail item
> > 190633 as non-spam
> > 2007-10-11 10:30:27 Maia: [process-quarantine-sub]
5 non-spam items
> > processed (5 learned)
> > 2007-10-11 10:30:27 Maia: [process-quarantine-sub]
Warning: --report
> not
> > specified, reporting is disabled.
> > 2007-10-11 10:30:44 Maia: [process-quarantine-sub]
Learned mail item
> > 18176 as spam
> > 2007-10-11 10:31:02 Maia: [process-quarantine-sub]
Learned mail item
> > 18180 as spam
> > 2007-10-11 10:31:39 Maia: [process-quarantine-sub]
Learned mail item
> > 18181 as spam
> > 2007-10-11 10:31:51 Maia: [process-quarantine-sub]
Learned mail item
> > 18184 as spam
> > 2007-10-11 10:32:06 Maia: [process-quarantine-sub]
Learned mail item
> > 18187 as spam
> > 2007-10-11 10:32:07 Maia: [process-quarantine-sub]
5 spam items
> processed
> > (5 learned, 0 reported)
> > 2007-10-11 10:33:18 Maia: [process-quarantine-sub]
Learned mail item
> > 190635 as non-spam
> > 2007-10-11 10:40:21 Maia: [process-quarantine-sub]
Learned mail item
> > 190642 as non-spam
> > 2007-10-11 11:14:18 Maia: [process-quarantine-sub]
Learned mail item
> > 190643 as non-spam
> >
> >
> > --
> > Rick Zeman
> > Manager of Information Technology
> > Melwood Horticultural Training Center
> > 301.599.4574 - HelpDesk
> > 301.599.4560 - MyDesk
> > http://www.melwood.org
> >
> _______________________________________________
> Maia-users mailing list
> Maia-usersrenaissoft.com
> http://www.renaissoft.com/mailman/listinfo/maia-users
>
_______________________________________________
Maia-users mailing list
Maia-usersrenaissoft.com
http://www.renaissoft.com/mailman/listinfo/maia-users
|
