Firewall Log format

Hi,

For the last 5 years logs have been archived from the
Firewall. 
Yesterday, the logs were analysed the first line of each CSV
was read 
and the file was analysed to find the maximum size of each
field. 
According to the results of 10,000 csv logs, there were 9571
record 
formats. Has Checkpoint ever published details of csv output
format?

earliest format and field sizes (prefixed fw)
num;date;time;orig;type;action;alert;i/f_name;i/f_dir;proto;
src;dst;service;s_port;len;rule;xlatesrc;xlatedst;xlatesport
;xlatedport;agent;orig_from;orig_to;from;to;reason;icmp-type
;icmp-code;reason:;srckeyid;dstkeyid;scheme:;methods:;h_len;
ip_vers;message;error 
notification:;IKE Log:;Negotiation
Id:;user;res_action;resource;sys_msgs

fwnum:5;fwdate:9;fwtime:8;fworig:13;fwtype:3;fwaction:6;fwal
ert:8;fwifname:4;fwifdir:8;fwproto:3;fwsrc:13;fwdst:11;fwser
vice:10;
fwsport:5;fwlen:2;fwrule:1;fwxlatesrc:12;fwxlatedst:13;fwxla
tesport:5;fwxlatedport:3;fwagent:13;fworigfrom:86;fworigto:4
4;fwfrom:86;
fwto:44;fwreason:83;fwicmptype:1;fwicmpcode:1;fwreason1:30;f
wsrckeyid:10;fwdstkeyid:10;fwscheme:3;fwmethods:33;fwhlen:2;
fwipvers:1;
fwmessage:42;fwerrornotification:10;fwikelog:38;fwnegotiatio
nid:16;fwuser:9;fwresaction:11;fwresource:30;fwsysmsgs:0


latest format and field sizes (prefixed fw)
num;date;time;orig;type;action;alert;i/f_name;i/f_dir;produc
t;log_sys_message;rule;rule_uid;rule_name;service_id;src;dst
;proto;xlatesrc;xlatedst;NAT_rulenum;NAT_addtnl_rulenum;serv
ice;s_port;xlatedport;xlatesport;scheme:;methods:;peer 
gateway;encryption 
failure:;partner;community;fw_subproduct;vpn_feature_name;IC
MP;ICMP 
Type;ICMP Code;message_info;msg;TCP packet out of 
state;tcp_flags;vpn_user;srckeyid;dstkeyid;IKE:;CookieI;Cook
ieR;msgid;IKE 
notification:;Certificate DN:;IKE 
IDs:;user;reason:;Session:;L2TP:;PPP:;MAC:;OM:;om_method:;as
signed_IP:;machine:;PS;Attack 
Info;attack;DCE-RPC Interface UUID;Total logs;Suppressed
logs;VPN 
internal source IP;start_time;elapsed

fwnum:5;fwdate:9;fwtime:8;fworig:13;fwtype:3;fwaction:6;fwal
ert:5;fwifname:3;fwifdir:7;fwproduct:18;fwlogsysmessage:60;f
wrule:2;
fwruleuid:38;fwrulename:0;fwserviceid:4;fwsrc:12;fwdst:13;fw
proto:3;fwxlatesrc:13;fwxlatedst:13;fwnatrulenum:2;fwnataddt
nlrulenum:1;
fwservice:4;fwsport:4;fwxlatedport:10;fwxlatesport:5;fwschem
e:3;fwmethods:29;fwpeergateway:13;fwencryptionfailure:52;fwp
artner:0;
fwcommunity:10;fwfwsubproduct:5;fwvpnfeaturename:3;fwicmp:12
;fwicmptype:1;fwicmpcode:1;fwmessageinfo:12;fwmsg:225;
fwtcppacketoutofstate:22;fwtcpflags:3;fwvpnuser:0;fwsrckeyid
:10;fwdstkeyid:10;fwike:21;fwcookiei:16;fwcookier:16;fwmsgid
:8;
fwikenotification:0;fwcertificatedn:0;fwikeids:52;fwuser:9;f
wreason:89;fwsession:0;fwl2tp:0;fwppp:0;fwmac:17;fwom:66;fwo
mmethod:8;
fwassignedip:10;fwmachine:0;fwps:42;fwattackinfo:41;fwattack
:29;fwdcerpcinterfaceuuid:36;fwtotallogs:1;fwsuppressedlogs:
1;
fwvpninternalsourceip:10;fwstarttime:18;fwelapsed:10

-- 
Regards

Russell

Email: russell dot aspinwall at flomerics dot co dot uk 
Network and Systems Administrator           Flomerics Ltd
Telephone: 020-8941-8810 x3116              81 Bridge Road
Facsimile: 020-8941-8730                    Hampton Court
                                            Surrey, KT8 9HH
                                            United Kingdom


____________________________________________________________
__________
This email has been scanned by the MessageLabs Email
Security System.
For more information please visit http://www.messagela
bs.com/email 
____________________________________________________________
__________

Flomerics Group plc, Registered Office 81 Bridge Road,
Hampton Court, Surrey, KT8 9HH. Registered No. 2327348. This
e-mail is confidential and intended solely for the use of
the individual to whom it is addressed.  Any views or
opinions presented are solely those of the author and do not
necessarily represent those of Flomerics Group plc or its
subsidiaries. If you are not the intended recipient of this
e-mail you may not copy, use, forward or disclose its
contents to any other person ; please notify our Computer
Service Desk on +44 (0)20 8487 3000 and destroy and delete
the message and attachments from your system.
 
For more information on Flomerics visit our web site at
www.flomerics.com

=================================================
To set vacation, Out-Of-Office, or away messages,
send an email to LISTSERVamadeus.us.checkpoint.com
in the BODY of the email add:
set fw-1-mailinglist nomail
=================================================
To unsubscribe from this mailing list,
please see the instructions at
http:
//www.checkpoint.com/services/mailing.html
=================================================
If you have any questions on how to change your
subscription options, email
fw-1-ownerts.checkpoint.com
=================================================

On Thursday 28 September 2006 09:32, Russell Aspinwall
wrote:
> Hi,
>
> For the last 5 years logs have been archived from the
Firewall.
> Yesterday, the logs were analysed the first line of
each CSV was read
> and the file was analysed to find the maximum size of
each field.
> According to the results of 10,000 csv logs, there were
9571 record
> formats. Has Checkpoint ever published details of csv
output format?

This is just a guess, but since I have been working with
scripts that analyze 
the logs files in CSV format since version 4.1, I have also
noticed that the 
order of the different columns in each record may change
from time to time.

I thing this happens due to the order of the records and
what is being logged. 
If the first record only require field A, B an C, then the
three first 
columns in the CSV file will be these fields. If the second
record also 
require field D, then the 4th column will be field D, and so
on...

As I say... this is only a guess.

But since the name of the columns are in the first line, my
scripts looks at 
it an use it to determin which column contains what...

-- 
Jørn Dahl-Stamnes

=================================================
To set vacation, Out-Of-Office, or away messages,
send an email to LISTSERVamadeus.us.checkpoint.com
in the BODY of the email add:
set fw-1-mailinglist nomail
=================================================
To unsubscribe from this mailing list,
please see the instructions at
http:
//www.checkpoint.com/services/mailing.html
=================================================
If you have any questions on how to change your
subscription options, email
fw-1-ownerts.checkpoint.com
=================================================

Hello,

From the preceding post I can also confirm that the log
format changes and
that the first line contains the name of the columns. It is
also relevant to
know that when you write a script, from versions to versions
new fields will
be added.

For my own development work on firewall log analyzer, I have
written a
module that normalizes the log fields. It is written in
Java. Whenever this
module does find an unknown field it will notify the user.

For each new releases the module has signaled new fields. To
get an overview
of the function

http://www.tla.ch/fla
Iindex.htm

Bye for now,

Christian ALT

Telecom and Logistics Associates
Network Security Company
ISO 27001 Lead Auditor
http://www.tla.ch

-----Original Message-----
From: Mailing list for discussion of Firewall-1
[mailto:FW-1-MAILINGLISTAMADEUS.US.CHECKPOINT.COM]
On Behalf Of Jørn
Dahl-Stamnes
Sent: jeudi, 28. septembre 2006 09:49
To: FW-1-MAILINGLISTAMADEUS.US.CHECKPOINT.COM
Subject: Re: [FW-1] Firewall Log format

On Thursday 28 September 2006 09:32, Russell Aspinwall
wrote:
> Hi,
>
> For the last 5 years logs have been archived from the
Firewall.
> Yesterday, the logs were analysed the first line of
each CSV was read
> and the file was analysed to find the maximum size of
each field.
> According to the results of 10,000 csv logs, there were
9571 record
> formats. Has Checkpoint ever published details of csv
output format?

This is just a guess, but since I have been working with
scripts that
analyze 
the logs files in CSV format since version 4.1, I have also
noticed that the

order of the different columns in each record may change
from time to time.

I thing this happens due to the order of the records and
what is being
logged. 
If the first record only require field A, B an C, then the
three first 
columns in the CSV file will be these fields. If the second
record also 
require field D, then the 4th column will be field D, and so
on...

As I say... this is only a guess.

But since the name of the columns are in the first line, my
scripts looks at

it an use it to determin which column contains what...

-- 
Jørn Dahl-Stamnes

=================================================
To set vacation, Out-Of-Office, or away messages,
send an email to LISTSERVamadeus.us.checkpoint.com
in the BODY of the email add:
set fw-1-mailinglist nomail
=================================================
To unsubscribe from this mailing list,
please see the instructions at
http:
//www.checkpoint.com/services/mailing.html
=================================================
If you have any questions on how to change your
subscription options, email
fw-1-ownerts.checkpoint.com
=================================================

=================================================
To set vacation, Out-Of-Office, or away messages,
send an email to LISTSERVamadeus.us.checkpoint.com
in the BODY of the email add:
set fw-1-mailinglist nomail
=================================================
To unsubscribe from this mailing list,
please see the instructions at
http:
//www.checkpoint.com/services/mailing.html
=================================================
If you have any questions on how to change your
subscription options, email
fw-1-ownerts.checkpoint.com
=================================================