fwd debug...

Dear Colleagues,
Any idea what is going on here...?! CPU is continuously at
100 %. We use a special configuration with R55p on IPSO 4.1.
Reboot didn't help, neither did policy re-installation...

$FWDIR/log/fwd.elg:

[FWD 8234 192768]fw02a[19 Mar 10:54:40] process 10689
terminated(nprocess=2)
[FWD 8234 192768]fw02a[19 Mar 10:54:40] fwauthd: in.aufpd
was executed on fd -1 (nprocess = 3) pid 10692
[FWD 8234 192768]fw02a[19 Mar 10:54:40]
fwasync_connbuf_realloc: reallocating 0 from 0 to 1028
[FWD 8234 1655296]fw02a[19 Mar 10:54:41] signals_handler:
dispatched signal 20 to handler 0x400d2860
[FWD 8234 192768]fw02a[19 Mar 10:54:41] fw_do_fireman:
waitpid return 10691
[FWD 8234 192768]fw02a[19 Mar 10:54:41] can't remove file
in.asessiond.pid
 fwauthd: restarting in.asessiond
[FWD 8234 192768]fw02a[19 Mar 10:54:41] process 10691
terminated(nprocess=2)
[FWD 8234 192768]fw02a[19 Mar 10:54:41] fwauthd:
in.asessiond was executed on fd -1 (nprocess = 3) pid 10693
[FWD 8234 192768]fw02a[19 Mar 10:54:41]
fwasync_connbuf_realloc: reallocating 0 from 0 to 1028
[FWD 8234 1655296]fw02a[19 Mar 10:54:41] signals_handler:
dispatched signal 20 to handler 0x400d2860
[FWD 8234 192768]fw02a[19 Mar 10:54:41] fw_do_fireman:
waitpid return 10690
[FWD 8234 192768]fw02a[19 Mar 10:54:41] can't remove file
mdq.pid
 fwauthd: restarting mdq
[FWD 8234 192768]fw02a[19 Mar 10:54:41] process 10690
terminated(nprocess=2)
[FWD 8234 192768]fw02a[19 Mar 10:54:41] fwauthd: mdq was
executed on fd -1 (nprocess = 3) pid 10694
[FWD 8234 192768]fw02a[19 Mar 10:54:41]
fwasync_connbuf_realloc: reallocating 0 from 0 to 1028
[FWD 8234 1655296]fw02a[19 Mar 10:54:41] signals_handler:
dispatched signal 20 to handler 0x400d2860
[FWD 8234 192768]fw02a[19 Mar 10:54:41] fw_do_fireman:
waitpid return 10692
[FWD 8234 192768]fw02a[19 Mar 10:54:41] can't remove file
in.aufpd.pid
 fwauthd: restarting in.aufpd
[FWD 8234 192768]fw02a[19 Mar 10:54:41] process 10692
terminated(nprocess=2)
[FWD 8234 192768]fw02a[19 Mar 10:54:41] fwauthd: in.aufpd
was executed on fd -1 (nprocess = 3) pid 10695
[FWD 8234 192768]fw02a[19 Mar 10:54:41]
fwasync_connbuf_realloc: reallocating 0 from 0 to 1028
[FWD 8234 1655296]fw02a[19 Mar 10:54:42] signals_handler:
dispatched signal 20 to handler 0x400d2860
[FWD 8234 192768]fw02a[19 Mar 10:54:42] fw_do_fireman:
waitpid return 10694
[FWD 8234 192768]fw02a[19 Mar 10:54:42] can't remove file
mdq.pid
 fwauthd: restarting mdq

-lari-



Scanned by Check Point Total Security Gateway.


=================================================
To set vacation, Out-Of-Office, or away messages,
send an email to LISTSERVamadeus.us.checkpoint.com
in the BODY of the email add:
set fw-1-mailinglist nomail
=================================================
To unsubscribe from this mailing list,
please see the instructions at
http:
//www.checkpoint.com/services/mailing.html
=================================================
If you have any questions on how to change your
subscription options, email
fw-1-ownerts.checkpoint.com
=================================================

Use ps -auxwww to see what process is using all the CPU

Check your snmp, if it is the snmp process delete your snmp
config
reboot then re-apply

JP



-----Original Message-----
From: Mailing list for discussion of Firewall-1
[mailto:FW-1-MAILINGLISTAMADEUS.US.CHECKPOINT.COM]
On Behalf Of Lari
Luoma
Sent: Wednesday, 19 March 2008 10:05 PM
To: FW-1-MAILINGLISTAMADEUS.US.CHECKPOINT.COM
Subject: [FW-1] fwd debug...

Dear Colleagues,
Any idea what is going on here...?! CPU is continuously at
100 %. We use
a special configuration with R55p on IPSO 4.1. Reboot didn't
help,
neither did policy re-installation...

$FWDIR/log/fwd.elg:

[FWD 8234 192768]fw02a[19 Mar 10:54:40] process 10689
terminated(nprocess=2)
[FWD 8234 192768]fw02a[19 Mar 10:54:40] fwauthd: in.aufpd
was executed
on fd -1 (nprocess = 3) pid 10692
[FWD 8234 192768]fw02a[19 Mar 10:54:40]
fwasync_connbuf_realloc:
reallocating 0 from 0 to 1028
[FWD 8234 1655296]fw02a[19 Mar 10:54:41] signals_handler:
dispatched
signal 20 to handler 0x400d2860
[FWD 8234 192768]fw02a[19 Mar 10:54:41] fw_do_fireman:
waitpid return
10691
[FWD 8234 192768]fw02a[19 Mar 10:54:41] can't remove file
in.asessiond.pid
 fwauthd: restarting in.asessiond
[FWD 8234 192768]fw02a[19 Mar 10:54:41] process 10691
terminated(nprocess=2)
[FWD 8234 192768]fw02a[19 Mar 10:54:41] fwauthd:
in.asessiond was
executed on fd -1 (nprocess = 3) pid 10693
[FWD 8234 192768]fw02a[19 Mar 10:54:41]
fwasync_connbuf_realloc:
reallocating 0 from 0 to 1028
[FWD 8234 1655296]fw02a[19 Mar 10:54:41] signals_handler:
dispatched
signal 20 to handler 0x400d2860
[FWD 8234 192768]fw02a[19 Mar 10:54:41] fw_do_fireman:
waitpid return
10690
[FWD 8234 192768]fw02a[19 Mar 10:54:41] can't remove file
mdq.pid
 fwauthd: restarting mdq
[FWD 8234 192768]fw02a[19 Mar 10:54:41] process 10690
terminated(nprocess=2)
[FWD 8234 192768]fw02a[19 Mar 10:54:41] fwauthd: mdq was
executed on fd
-1 (nprocess = 3) pid 10694
[FWD 8234 192768]fw02a[19 Mar 10:54:41]
fwasync_connbuf_realloc:
reallocating 0 from 0 to 1028
[FWD 8234 1655296]fw02a[19 Mar 10:54:41] signals_handler:
dispatched
signal 20 to handler 0x400d2860
[FWD 8234 192768]fw02a[19 Mar 10:54:41] fw_do_fireman:
waitpid return
10692
[FWD 8234 192768]fw02a[19 Mar 10:54:41] can't remove file
in.aufpd.pid
 fwauthd: restarting in.aufpd
[FWD 8234 192768]fw02a[19 Mar 10:54:41] process 10692
terminated(nprocess=2)
[FWD 8234 192768]fw02a[19 Mar 10:54:41] fwauthd: in.aufpd
was executed
on fd -1 (nprocess = 3) pid 10695
[FWD 8234 192768]fw02a[19 Mar 10:54:41]
fwasync_connbuf_realloc:
reallocating 0 from 0 to 1028
[FWD 8234 1655296]fw02a[19 Mar 10:54:42] signals_handler:
dispatched
signal 20 to handler 0x400d2860
[FWD 8234 192768]fw02a[19 Mar 10:54:42] fw_do_fireman:
waitpid return
10694
[FWD 8234 192768]fw02a[19 Mar 10:54:42] can't remove file
mdq.pid
 fwauthd: restarting mdq

-lari-



Scanned by Check Point Total Security Gateway.


=================================================
To set vacation, Out-Of-Office, or away messages,
send an email to LISTSERVamadeus.us.checkpoint.com
in the BODY of the email add:
set fw-1-mailinglist nomail
=================================================
To unsubscribe from this mailing list,
please see the instructions at
http:
//www.checkpoint.com/services/mailing.html
=================================================
If you have any questions on how to change your
subscription options, email
fw-1-ownerts.checkpoint.com
=================================================
############################################################
#########################
Important: This electronic message and attachments (if any)
are confidential
and may be legally privileged. If you are not the intended
recipient do not
copy, disclose or use the contents in any way. Please let us
know by return
e-mail immediately and then destroy this message.
############################################################
#########################

Scanned by Check Point Total Security Gateway.

=================================================
To set vacation, Out-Of-Office, or away messages,
send an email to LISTSERVamadeus.us.checkpoint.com
in the BODY of the email add:
set fw-1-mailinglist nomail
=================================================
To unsubscribe from this mailing list,
please see the instructions at
http:
//www.checkpoint.com/services/mailing.html
=================================================
If you have any questions on how to change your
subscription options, email
fw-1-ownerts.checkpoint.com
=================================================

The point is that ps wont show the system CPU usage, so the
first step is to
determine whether the kernel is hogging the CPU or whether
it's user space
processes by using vmstat. Then if it's user space you
continue with ps.


On Tue, Mar 25, 2008 at 11:03 AM, Jean-Paul Baillon <
jean-paul.baillonsafecom.co.nz> wrote:

> I had the very same case
>
> Although vmstat was helpful, only with ps was I able to
pinpoint the
> rougue process (/bin/snmp)
>
> JP
>
> -----Original Message-----
> From: Mailing list for discussion of Firewall-1
> [mailto:FW-1-MAILINGLISTAMADEUS.US.CHECKPOINT.COM]
On Behalf Of Hugo
> van der Kooij
> Sent: Thursday, 20 March 2008 9:17 AM
> To: FW-1-MAILINGLISTAMADEUS.US.CHECKPOINT.COM
> Subject: Re: [FW-1] fwd debug...
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Jean-Paul Baillon wrote:
> | Use ps -auxwww to see what process is using all the
CPU
> |
> | Check your snmp, if it is the snmp process delete
your snmp config
> | reboot then re-apply
>
> Start with `vmstat 1`. Only if you have a lot of user
space CPU usage
> will you get any information from running ps.
>
> Hugo.
>
> - --
> hvdkooijvanderkooij.org               http://hugo.vanderkooij.
org/
> PGP/GPG <http://hugo.
vanderkooij.org/PGP/GPG>? Use:
> http://hug
o.vanderkooij.org/0x58F19981.asc
>
>        A: Yes.
>        >Q: Are you sure?
>        >>A: Because it reverses the logical flow
of conversation.
>        >>>Q: Why is top posting frowned upon?
>
> Bored? Click on http://spamornot.org/ and
rate those images.
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.7 (GNU/Linux)
>
>
iD8DBQFH4ZDqBvzDRVjxmYERAmmkAJ9MAtNKz3nI3Ba0GvaRWm6l8AwAVwCc
CfSV
> AS7SaR0wZjVnD1JiP1PuNi4=
> =8wWs
> -----END PGP SIGNATURE-----
>
> Scanned by Check Point Total Security Gateway.
>
> =================================================
> To set vacation, Out-Of-Office, or away messages,
> send an email to LISTSERVamadeus.us.checkpoint.com
> in the BODY of the email add:
> set fw-1-mailinglist nomail
> =================================================
> To unsubscribe from this mailing list,
> please see the instructions at
> http:
//www.checkpoint.com/services/mailing.html
> =================================================
> If you have any questions on how to change your
> subscription options, email
> fw-1-ownerts.checkpoint.com
> =================================================
>
>
############################################################
#########################
> Important: This electronic message and attachments (if
any) are
> confidential
> and may be legally privileged. If you are not the
intended recipient do
> not
> copy, disclose or use the contents in any way. Please
let us know by
> return
> e-mail immediately and then destroy this message.
>
>
############################################################
#########################
>
> Scanned by Check Point Total Security Gateway.
>
> =================================================
> To set vacation, Out-Of-Office, or away messages,
> send an email to LISTSERVamadeus.us.checkpoint.com
> in the BODY of the email add:
> set fw-1-mailinglist nomail
> =================================================
> To unsubscribe from this mailing list,
> please see the instructions at
> http:
//www.checkpoint.com/services/mailing.html
> =================================================
> If you have any questions on how to change your
> subscription options, email
> fw-1-ownerts.checkpoint.com
> =================================================
>


Scanned by Check Point Total Security Gateway.

=================================================
To set vacation, Out-Of-Office, or away messages,
send an email to LISTSERVamadeus.us.checkpoint.com
in the BODY of the email add:
set fw-1-mailinglist nomail
=================================================
To unsubscribe from this mailing list,
please see the instructions at
http:
//www.checkpoint.com/services/mailing.html
=================================================
If you have any questions on how to change your
subscription options, email
fw-1-ownerts.checkpoint.com
=================================================

Jubei Trippataka wrote:
> The point is that ps wont show the system CPU usage, so
the first step is to
> determine whether the kernel is hogging the CPU or
whether it's user space
> processes by using vmstat. Then if it's user space you
continue with ps.
> 

also top is a good utility to sort processes by cpu usage
(not ot 
mention a lot more easier to read than vmstat output) and
you can also 
see if the system has to process a lot of interrupts.

sin.

Scanned by Check Point Total Security Gateway.

=================================================
To set vacation, Out-Of-Office, or away messages,
send an email to LISTSERVamadeus.us.checkpoint.com
in the BODY of the email add:
set fw-1-mailinglist nomail
=================================================
To unsubscribe from this mailing list,
please see the instructions at
http:
//www.checkpoint.com/services/mailing.html
=================================================
If you have any questions on how to change your
subscription options, email
fw-1-ownerts.checkpoint.com
=================================================

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

sin wrote:
| Jubei Trippataka wrote:
|> The point is that ps wont show the system CPU usage,
so the first step
|> is to
|> determine whether the kernel is hogging the CPU or
whether it's user
|> space
|> processes by using vmstat. Then if it's user space you
continue with ps.
|>
|
| also top is a good utility to sort processes by cpu usage
(not ot
| mention a lot more easier to read than vmstat output) and
you can also
| see if the system has to process a lot of interrupts.

Only if top is in fact available. Which does not apply at
all to IPSO.
And for Solaris you need to get it elsewhere.

So it is better to learn to use tools available on all
platforms.

Hugo.

- --
hvdkooijvanderkooij.org               http://hugo.vanderkooij.
org/
PGP/GPG? Use: http://hug
o.vanderkooij.org/0x58F19981.asc

	A: Yes.
	>Q: Are you sure?
	>>A: Because it reverses the logical flow of
conversation.
	>>>Q: Why is top posting frowned upon?

Bored? Click on http://spamornot.org/ and
rate those images.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)

iD8DBQFH6TsCBvzDRVjxmYERAqWLAJ9a0Q2e4Q1a+2aUkz2PcoiscHcVbQCg
rSAE
5lygY6kTfQdhEzi0ZKnJtSw=
=J9w4
-----END PGP SIGNATURE-----

Scanned by Check Point Total Security Gateway.

=================================================
To set vacation, Out-Of-Office, or away messages,
send an email to LISTSERVamadeus.us.checkpoint.com
in the BODY of the email add:
set fw-1-mailinglist nomail
=================================================
To unsubscribe from this mailing list,
please see the instructions at
http:
//www.checkpoint.com/services/mailing.html
=================================================
If you have any questions on how to change your
subscription options, email
fw-1-ownerts.checkpoint.com
=================================================

Hugo van der Kooij wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> sin wrote:
> | Jubei Trippataka wrote:
> |> The point is that ps wont show the system CPU
usage, so the first step
> |> is to
> |> determine whether the kernel is hogging the CPU
or whether it's user
> |> space
> |> processes by using vmstat. Then if it's user
space you continue with ps.
> |>
> |
> | also top is a good utility to sort processes by cpu
usage (not ot
> | mention a lot more easier to read than vmstat output)
and you can also
> | see if the system has to process a lot of
interrupts.
> 
> Only if top is in fact available. Which does not apply
at all to IPSO.
> And for Solaris you need to get it elsewhere.
> 
> So it is better to learn to use tools available on all
platforms.
> 
>

I did not see that the OP was using IPSO and I defaulted to
Linux 

Scanned by Check Point Total Security Gateway.

=================================================
To set vacation, Out-Of-Office, or away messages,
send an email to LISTSERVamadeus.us.checkpoint.com
in the BODY of the email add:
set fw-1-mailinglist nomail
=================================================
To unsubscribe from this mailing list,
please see the instructions at
http:
//www.checkpoint.com/services/mailing.html
=================================================
If you have any questions on how to change your
subscription options, email
fw-1-ownerts.checkpoint.com
=================================================