|
List Info
Thread: Log integrity handling on central logsystem
|
|
| Log integrity handling on central
logsystem |

|
2006-08-31 08:00:41 |
|
|
Regarding log signing: is anyone aware of an actual regulatory or legal requirement for log signing?
From: loganalysis-bounces+ericf=windows.microsoft.com lists.shmoo.com on behalf of Christopher L. Petersen Sent: Tue 8/22/2006 6:55 AM To: Patrick Debois; loganalysis lists.shmoo.com Subject: [logs] Re: Log integrity handling on central logsystem
I've always thought ensuring the integrity of the log begins at collection and ends at archival. In defining integrity I would say it means no log entries are altered and the complete log is collected and centralized. As for best practices, I would recommend the following:
- Collect all logs as close to real-time as possible. The longer the source log is on the source host, the more likely it can be altered. - When possible transport logs with a TCP based protocol and encrypt. TCP ensures reliability and combined with encryption, makes altering log entries as they traverse the network significantly more challenging. - If UDP Syslog is the only option and the central repository is across the WAN, try to collect close to the source and then forward via a TCP based protocol. - The central repository should utilize a strong access control mechanism at the operating system, database, and application layer to ensure that log data cannot be altered once written. - Ideally, an archive/backup copy of all centralized logs should be maintained. This provides redundancy in the event the central log database fails or becomes corrupt. - When written to the central repository/archive, additional controls like hashsums or digital signatures can be used to validate integrity. If hashsums and/or signatures are used make sure the proper access controls exist around storing the hashes and signatures to ensure they cannot be modified. This in itself is another line of thought... - If you want to get really serious (and spend some money), write archive copies of the logs to write-once storage.
I'm not sure if/how Splunk addresses the above. Vendors focused on log management (vs SIM) address most if not all of the above.
Chris Petersen, CTO & Founder LogRhythm, Inc. www.LogRhythm.com
> -----Original Message----- > From: loganalysis-bounces+chris=security-conscious.com lists.shmoo.com > [lists.shmoo.com">mailto:loganalysis-bounces+chris=security-conscious.com lists.shmoo.com ] > On Behalf Of Patrick Debois > Sent: Tuesday, August 22, 2006 12:44 AM > To: loganalysis lists.shmoo.com > Subject: [logs] Log integrity handling on central logsystem > > I'm looking for feedback how centralized log solutions handle data > integrity; If you would log directly to a central system, that log is > the only source. So you would miss something to compare against. > > -Would you rely on taking checksums of the logs and storing them on > another system? > -How do you protect yourself from the fact that the central logging is > compromised with a still growing logfile? > Would you consider signing each log line? Signing within a text file is > fairly easy, but what about content stored in a database? > > My customer is currently looking at Splunk. It seems a great way to go > through the logfiles, but I'm not sure that we can fullfill his > dataintegrity requirements with it. But then again it does not stand in > the way of another solution doing it probable. > > Patrick > > > _______________________________________________ > LogAnalysis mailing list > LogAnalysis lists.shmoo.com > http://lists.shmoo.com/mailman/listinfo/loganalysis _______________________________________________ LogAnalysis mailing list LogAnalysis lists.shmoo.com http://lists.shmoo.com/mailman/listinfo/loganalysis
|
| Log integrity handling on central
logsystem |

|
2006-08-31 18:49:46 |
|
| Eric,
There are many regs out their that speak generally about taking steps to prove the integrity of logs. (HIPAA, DCID, NISPOM, FFIEC... the list goes on). Some mention log signing as a technique. Many more focus on security of network transmission (i.e. syslog-ng or similar) and controls over access to the log data repository.
However, real driver for signing is litigation risk, not regulations.
When you present log data as evidence to prosecute intruders, or you are subpoenaed for log evidence, opposing counsel will tend to challenge its integrity on several points:
- can't prove that there weren't additional exculpatory records showing someone other than the defendant had the same opportunity, if the evidence is circumstantial, that were then deleted by the _real_ culprit - can't prove the specific events used as evidence weren't changed
Signing (with a nonce that can be proven to have been protected) both individual log events and groups of consecutive log events is one of the best ways to address this type of challenge.
Furthermore, to the extent that log signing becomes standard practice, a company who does not do it is exposed to civil litigation for not having taken due care, if outsiders, employees, customers, etc. are unable to pursue lawsuits against 3rd parties because of the shakiness of the log evidence.
- Christina On Aug 31, 2006, at 1:00 AM, Eric Fitzgerald wrote: Regarding log signing: is anyone aware of an actual regulatory or legal requirement for log signing? I've always thought ensuring the integrity of the log begins at collection and ends at archival. In defining integrity I would say it means no log entries are altered and the complete log is collected and centralized. As for best practices, I would recommend the following:
- Collect all logs as close to real-time as possible. The longer the source log is on the source host, the more likely it can be altered. - When possible transport logs with a TCP based protocol and encrypt. TCP ensures reliability and combined with encryption, makes altering log entries as they traverse the network significantly more challenging. - If UDP Syslog is the only option and the central repository is across the WAN, try to collect close to the source and then forward via a TCP based protocol. - The central repository should utilize a strong access control mechanism at the operating system, database, and application layer to ensure that log data cannot be altered once written. - Ideally, an archive/backup copy of all centralized logs should be maintained. This provides redundancy in the event the central log database fails or becomes corrupt. - When written to the central repository/archive, additional controls like hashsums or digital signatures can be used to validate integrity. If hashsums and/or signatures are used make sure the proper access controls exist around storing the hashes and signatures to ensure they cannot be modified. This in itself is another line of thought... - If you want to get really serious (and spend some money), write archive copies of the logs to write-once storage.
I'm not sure if/how Splunk addresses the above. Vendors focused on log management (vs SIM) address most if not all of the above.
Chris Petersen, CTO & Founder LogRhythm, Inc. www.LogRhythm.com
> -----Original Message----- > From: loganalysis-bounces+chris=lists.shmoo.com">security-conscious.com lists.shmoo.com > [lists.shmoo.com">mailto:loganalysis-bounces+chris=security-conscious.com lists.shmoo.com ] > On Behalf Of Patrick Debois > Sent: Tuesday, August 22, 2006 12:44 AM > To: lists.shmoo.com">loganalysis lists.shmoo.com > Subject: [logs] Log integrity handling on central logsystem > > I'm looking for feedback how centralized log solutions handle data > integrity; If you would log directly to a central system, that log is > the only source. So you would miss something to compare against. > > -Would you rely on taking checksums of the logs and storing them on > another system? > -How do you protect yourself from the fact that the central logging is > compromised with a still growing logfile? > Would you consider signing each log line? Signing within a text file is > fairly easy, but what about content stored in a database? > > My customer is currently looking at Splunk. It seems a great way to go > through the logfiles, but I'm not sure that we can fullfill his > dataintegrity requirements with it. But then again it does not stand in > the way of another solution doing it probable. > > Patrick > > > _______________________________________________ > LogAnalysis mailing list > lists.shmoo.com">LogAnalysis lists.shmoo.com > http://lists.shmoo.com/mailman/listinfo/loganalysis _______________________________________________ LogAnalysis mailing list lists.shmoo.com">LogAnalysis lists.shmoo.com http://lists.shmoo.com/mailman/listinfo/loganalysis
_______________________________________________ LogAnalysis mailing list |
| Log integrity handling on central
logsystem |

|
2006-08-31 21:09:42 |
> Regarding log signing: is anyone aware of an actual
regulatory or legal
> requirement for log signing?
Isnt't that a bit too specific for most current
regulations? I
personally haven't seen any direct regulatory mandate for
log
signing...
--
Anton Chuvakin, Ph.D., GCIA, GCIH, GCFA
http://www.chuvakin.org
http://chuvakin.blogspot
.com
http://www.securitywar
rior.com
_______________________________________________
LogAnalysis mailing list
LogAnalysis lists.shmoo.com
h
ttp://lists.shmoo.com/mailman/listinfo/loganalysis
|
|
| Log integrity handling on central
logsystem |

|
2006-08-31 19:06:56 |
|
Hi Christina,
The federal rules of evidence actually
point in the other direction, that U.S. Federal courts look at the likelihood
of tampering having actually occurred rather than the possibility of tampering.
http://www.cybercrime.gov/usamarch2001_4.htm
Here's an excerpt from an article by Orin
S. Kerr (a DoJ attorney when the article was written) on Computer Records and
the Federal Rules of Evidence:
"Computer records can
be altered easily, and opposing parties often allege that computer records lack
authenticity because they have been tampered with or changed after they were
created. For example, in United States v.
Whitaker, 127 F.3d 595, 602 (7th Cir. 1997), the government
retrieved computer files from the computer of a narcotics dealer named Frost.
The files from Frost's computer included detailed records of narcotics sales by
three aliases: "Me" (Frost himself, presumably), "Gator"
(the nickname of Frost's co-defendant Whitaker), and "Cruz" (the
nickname of another dealer). After the government permitted Frost to help
retrieve the evidence from his computer and declined to establish a formal
chain of custody for the computer at trial, Whitaker argued that the files
implicating him through his alias were not properly authenticated. Whitaker
argued that "with a few rapid keystrokes, Frost could have easily added
Whitaker's alias, 'Gator' to the printouts in order to finger Whitaker and to
appear more helpful to the government." Id.
at 602."
"The courts have responded with considerable skepticism to such
unsupported claims that computer records have been altered. Absent specific
evidence that tampering occurred, the mere possibility of tampering does not
affect the authenticity of a computer record. See
Whitaker, 127 F.3d at 602 (declining to disturb trial judge's ruling
that computer records were admissible because allegation of tampering was
"almost wild-eyed speculation . . . [without] evidence to support such a
scenario"); United States v. Bonallo,
858 F.2d 1427, 1436 (9th Cir. 1988) ("The fact that it is possible to
alter data contained in a computer is plainly insufficient to establish
untrustworthiness."); United States v. Glasser,
773 F.2d 1553, 1559 (11th Cir. 1985) ("The existence of an air-tight
security system [to prevent tampering] is not, however, a prerequisite to the
admissibility of computer printouts. If such a prerequisite did exist, it would
become virtually impossible to admit computer-generated records; the party
opposing admission would have to show only that a better security system was
feasible."). Id. at 559. This
is consistent with the rule used to establish the authenticity of other
evidence such as narcotics. See United
States v. Allen, 106 F.3d 695, 700 (6th Cir. 1997) ("Merely
raising the possibility of tampering is insufficient to render evidence
inadmissible."). Absent specific evidence of tampering, allegations that
computer records have been altered go to their weight, not their admissibility.
See Bonallo, 858 F.2d at 1436."
I agree that signing (strong evidence
collection & chain of custody procedures) will help dismiss such
accusations more briefly, but I don't believe that signing is necessary for
introduction of logs as evidence.
Cash register receipts, another kind of
electronic evidence, won't have digsig for probably decades yet are readily
admissible providing that receipts are part of normal business process.
Now your last statement regarding best
practices is the real heart of the matter, and I think that's really the most
important concern.
Thank you very much for your detailed
thoughts!
Best regards,
Eric
From: Christina Noren
[mailto:cfrln cfrln.com]
Sent: Thursday, August 31, 2006
11:50 AM
To: Eric Fitzgerald
Cc: loganalysis lists.shmoo.com
Subject: Re: [logs] Re: Log
integrity handling on central logsystem
Eric,
There are many regs out their that speak generally about taking steps
to prove the integrity of logs. (HIPAA, DCID, NISPOM, FFIEC... the list goes
on). Some mention log signing as a technique. Many more focus on security of
network transmission (i.e. syslog-ng or similar) and controls over access to
the log data repository.
However, real driver for signing is litigation risk, not
regulations.
When you present log data as evidence to prosecute intruders, or you
are subpoenaed for log evidence, opposing counsel will tend to challenge its
integrity on several points:
- can't prove that there weren't additional exculpatory records showing
someone other than the defendant had the same opportunity, if the evidence is
circumstantial, that were then deleted by the _real_ culprit
- can't prove the specific events used as evidence weren't changed
Signing (with a nonce that can be proven to have been protected) both
individual log events and groups of consecutive log events is one of the best
ways to address this type of challenge.
Furthermore, to the extent that log signing becomes standard practice,
a company who does not do it is exposed to civil litigation for not having
taken due care, if outsiders, employees, customers, etc. are unable to pursue
lawsuits against 3rd parties because of the shakiness of the log
evidence.
On Aug 31, 2006, at 1:00 AM, Eric Fitzgerald wrote:
Regarding log signing: is anyone aware of
an actual regulatory or legal requirement for log signing?
I've always thought ensuring the integrity of the log
begins at
collection and ends at archival. In defining integrity I would say it
means no log entries are altered and the complete log is collected and
centralized. As for best practices, I would recommend the following:
- Collect all logs as close to real-time as possible. The longer the
source log is on the source host, the more likely it can be altered.
- When possible transport logs with a TCP based protocol and encrypt.
TCP ensures reliability and combined with encryption, makes altering log
entries as they traverse the network significantly more challenging.
- If UDP Syslog is the only option and the central repository is across
the WAN, try to collect close to the source and then forward via a TCP
based protocol.
- The central repository should utilize a strong access control
mechanism at the operating system, database, and application layer to
ensure that log data cannot be altered once written.
- Ideally, an archive/backup copy of all centralized logs should be
maintained. This provides redundancy in the event the central log
database fails or becomes corrupt.
- When written to the central repository/archive, additional controls
like hashsums or digital signatures can be used to validate integrity.
If hashsums and/or signatures are used make sure the proper access
controls exist around storing the hashes and signatures to ensure they
cannot be modified. This in itself is another line of thought...
- If you want to get really serious (and spend some money), write
archive copies of the logs to write-once storage.
I'm not sure if/how Splunk addresses the above. Vendors focused on log
management (vs SIM) address most if not all of the above.
Chris Petersen,
CTO & Founder
LogRhythm, Inc.
www.LogRhythm.com
> -----Original Message-----
> From: loganalysis-bounces+chris=lists.shmoo.com">security-conscious.com lists.shmoo.com
>
[lists.shmoo.com">mailto:loganalysis-bounces+chris=security-conscious.com lists.shmoo.com
]
> On Behalf Of Patrick Debois
> Sent: Tuesday, August 22, 2006 12:44 AM
> To: lists.shmoo.com">loganalysis lists.shmoo.com
> Subject: [logs] Log integrity handling on central logsystem
>
> I'm looking for feedback how centralized log solutions handle data
> integrity; If you would log directly to a central system, that log is
> the only source. So you would miss something to compare against.
>
> -Would you rely on taking checksums of the logs and storing them on
> another system?
> -How do you protect yourself from the fact that the central logging is
> compromised with a still growing logfile?
> Would you consider signing each log line? Signing within a text file
is
> fairly easy, but what about content stored in a database?
>
> My customer is currently looking at Splunk. It seems a great way to go
> through the logfiles, but I'm not sure that we can fullfill his
> dataintegrity requirements with it. But then again it does not stand
in
> the way of another solution doing it probable.
>
> Patrick
>
>
> _______________________________________________
> LogAnalysis mailing list
> lists.shmoo.com">LogAnalysis lists.shmoo.com
> http://lists.shmoo.com/mailman/listinfo/loganalysis
_______________________________________________
LogAnalysis mailing list
lists.shmoo.com">LogAnalysis lists.shmoo.com
http://lists.shmoo.com/mailman/listinfo/loganalysis
_______________________________________________
|
| Log integrity handling on central
logsystem |

|
2006-08-31 18:49:46 |
|
| Eric,
There are many regs out their that speak generally about taking steps to prove the integrity of logs. (HIPAA, DCID, NISPOM, FFIEC... the list goes on). Some mention log signing as a technique. Many more focus on security of network transmission (i.e. syslog-ng or similar) and controls over access to the log data repository.
However, real driver for signing is litigation risk, not regulations.
When you present log data as evidence to prosecute intruders, or you are subpoenaed for log evidence, opposing counsel will tend to challenge its integrity on several points:
- can't prove that there weren't additional exculpatory records showing someone other than the defendant had the same opportunity, if the evidence is circumstantial, that were then deleted by the _real_ culprit - can't prove the specific events used as evidence weren't changed
Signing (with a nonce that can be proven to have been protected) both individual log events and groups of consecutive log events is one of the best ways to address this type of challenge.
Furthermore, to the extent that log signing becomes standard practice, a company who does not do it is exposed to civil litigation for not having taken due care, if outsiders, employees, customers, etc. are unable to pursue lawsuits against 3rd parties because of the shakiness of the log evidence.
- Christina On Aug 31, 2006, at 1:00 AM, Eric Fitzgerald wrote: Regarding log signing: is anyone aware of an actual regulatory or legal requirement for log signing? I've always thought ensuring the integrity of the log begins at collection and ends at archival. In defining integrity I would say it means no log entries are altered and the complete log is collected and centralized. As for best practices, I would recommend the following:
- Collect all logs as close to real-time as possible. The longer the source log is on the source host, the more likely it can be altered. - When possible transport logs with a TCP based protocol and encrypt. TCP ensures reliability and combined with encryption, makes altering log entries as they traverse the network significantly more challenging. - If UDP Syslog is the only option and the central repository is across the WAN, try to collect close to the source and then forward via a TCP based protocol. - The central repository should utilize a strong access control mechanism at the operating system, database, and application layer to ensure that log data cannot be altered once written. - Ideally, an archive/backup copy of all centralized logs should be maintained. This provides redundancy in the event the central log database fails or becomes corrupt. - When written to the central repository/archive, additional controls like hashsums or digital signatures can be used to validate integrity. If hashsums and/or signatures are used make sure the proper access controls exist around storing the hashes and signatures to ensure they cannot be modified. This in itself is another line of thought... - If you want to get really serious (and spend some money), write archive copies of the logs to write-once storage.
I'm not sure if/how Splunk addresses the above. Vendors focused on log management (vs SIM) address most if not all of the above.
Chris Petersen, CTO & Founder LogRhythm, Inc. www.LogRhythm.com
> -----Original Message----- > From: loganalysis-bounces+chris=lists.shmoo.com">security-conscious.com lists.shmoo.com > [lists.shmoo.com">mailto:loganalysis-bounces+chris=security-conscious.com lists.shmoo.com ] > On Behalf Of Patrick Debois > Sent: Tuesday, August 22, 2006 12:44 AM > To: lists.shmoo.com">loganalysis lists.shmoo.com > Subject: [logs] Log integrity handling on central logsystem > > I'm looking for feedback how centralized log solutions handle data > integrity; If you would log directly to a central system, that log is > the only source. So you would miss something to compare against. > > -Would you rely on taking checksums of the logs and storing them on > another system? > -How do you protect yourself from the fact that the central logging is > compromised with a still growing logfile? > Would you consider signing each log line? Signing within a text file is > fairly easy, but what about content stored in a database? > > My customer is currently looking at Splunk. It seems a great way to go > through the logfiles, but I'm not sure that we can fullfill his > dataintegrity requirements with it. But then again it does not stand in > the way of another solution doing it probable. > > Patrick > > > _______________________________________________ > LogAnalysis mailing list > lists.shmoo.com">LogAnalysis lists.shmoo.com > http://lists.shmoo.com/mailman/listinfo/loganalysis _______________________________________________ LogAnalysis mailing list lists.shmoo.com">LogAnalysis lists.shmoo.com http://lists.shmoo.com/mailman/listinfo/loganalysis
_______________________________________________ LogAnalysis mailing list |
| Log integrity handling on central
logsystem |

|
2006-08-31 21:09:42 |
> Regarding log signing: is anyone aware of an actual
regulatory or legal
> requirement for log signing?
Isnt't that a bit too specific for most current
regulations? I
personally haven't seen any direct regulatory mandate for
log
signing...
--
Anton Chuvakin, Ph.D., GCIA, GCIH, GCFA
http://www.chuvakin.org
http://chuvakin.blogspot
.com
http://www.securitywar
rior.com
_______________________________________________
LogAnalysis mailing list
LogAnalysis lists.shmoo.com
h
ttp://lists.shmoo.com/mailman/listinfo/loganalysis
|
|
| Log integrity handling on central
logsystem |

|
2006-08-31 19:06:56 |
|
Hi Christina,
The federal rules of evidence actually
point in the other direction, that U.S. Federal courts look at the likelihood
of tampering having actually occurred rather than the possibility of tampering.
http://www.cybercrime.gov/usamarch2001_4.htm
Here's an excerpt from an article by Orin
S. Kerr (a DoJ attorney when the article was written) on Computer Records and
the Federal Rules of Evidence:
"Computer records can
be altered easily, and opposing parties often allege that computer records lack
authenticity because they have been tampered with or changed after they were
created. For example, in United States v.
Whitaker, 127 F.3d 595, 602 (7th Cir. 1997), the government
retrieved computer files from the computer of a narcotics dealer named Frost.
The files from Frost's computer included detailed records of narcotics sales by
three aliases: "Me" (Frost himself, presumably), "Gator"
(the nickname of Frost's co-defendant Whitaker), and "Cruz" (the
nickname of another dealer). After the government permitted Frost to help
retrieve the evidence from his computer and declined to establish a formal
chain of custody for the computer at trial, Whitaker argued that the files
implicating him through his alias were not properly authenticated. Whitaker
argued that "with a few rapid keystrokes, Frost could have easily added
Whitaker's alias, 'Gator' to the printouts in order to finger Whitaker and to
appear more helpful to the government." Id.
at 602."
"The courts have responded with considerable skepticism to such
unsupported claims that computer records have been altered. Absent specific
evidence that tampering occurred, the mere possibility of tampering does not
affect the authenticity of a computer record. See
Whitaker, 127 F.3d at 602 (declining to disturb trial judge's ruling
that computer records were admissible because allegation of tampering was
"almost wild-eyed speculation . . . [without] evidence to support such a
scenario"); United States v. Bonallo,
858 F.2d 1427, 1436 (9th Cir. 1988) ("The fact that it is possible to
alter data contained in a computer is plainly insufficient to establish
untrustworthiness."); United States v. Glasser,
773 F.2d 1553, 1559 (11th Cir. 1985) ("The existence of an air-tight
security system [to prevent tampering] is not, however, a prerequisite to the
admissibility of computer printouts. If such a prerequisite did exist, it would
become virtually impossible to admit computer-generated records; the party
opposing admission would have to show only that a better security system was
feasible."). Id. at 559. This
is consistent with the rule used to establish the authenticity of other
evidence such as narcotics. See United
States v. Allen, 106 F.3d 695, 700 (6th Cir. 1997) ("Merely
raising the possibility of tampering is insufficient to render evidence
inadmissible."). Absent specific evidence of tampering, allegations that
computer records have been altered go to their weight, not their admissibility.
See Bonallo, 858 F.2d at 1436."
I agree that signing (strong evidence
collection & chain of custody procedures) will help dismiss such
accusations more briefly, but I don't believe that signing is necessary for
introduction of logs as evidence.
Cash register receipts, another kind of
electronic evidence, won't have digsig for probably decades yet are readily
admissible providing that receipts are part of normal business process.
Now your last statement regarding best
practices is the real heart of the matter, and I think that's really the most
important concern.
Thank you very much for your detailed
thoughts!
Best regards,
Eric
From: Christina Noren
[mailto:cfrln cfrln.com]
Sent: Thursday, August 31, 2006
11:50 AM
To: Eric Fitzgerald
Cc: loganalysis lists.shmoo.com
Subject: Re: [logs] Re: Log
integrity handling on central logsystem
Eric,
There are many regs out their that speak generally about taking steps
to prove the integrity of logs. (HIPAA, DCID, NISPOM, FFIEC... the list goes
on). Some mention log signing as a technique. Many more focus on security of
network transmission (i.e. syslog-ng or similar) and controls over access to
the log data repository.
However, real driver for signing is litigation risk, not
regulations.
When you present log data as evidence to prosecute intruders, or you
are subpoenaed for log evidence, opposing counsel will tend to challenge its
integrity on several points:
- can't prove that there weren't additional exculpatory records showing
someone other than the defendant had the same opportunity, if the evidence is
circumstantial, that were then deleted by the _real_ culprit
- can't prove the specific events used as evidence weren't changed
Signing (with a nonce that can be proven to have been protected) both
individual log events and groups of consecutive log events is one of the best
ways to address this type of challenge.
Furthermore, to the extent that log signing becomes standard practice,
a company who does not do it is exposed to civil litigation for not having
taken due care, if outsiders, employees, customers, etc. are unable to pursue
lawsuits against 3rd parties because of the shakiness of the log
evidence.
On Aug 31, 2006, at 1:00 AM, Eric Fitzgerald wrote:
Regarding log signing: is anyone aware of
an actual regulatory or legal requirement for log signing?
I've always thought ensuring the integrity of the log
begins at
collection and ends at archival. In defining integrity I would say it
means no log entries are altered and the complete log is collected and
centralized. As for best practices, I would recommend the following:
- Collect all logs as close to real-time as possible. The longer the
source log is on the source host, the more likely it can be altered.
- When possible transport logs with a TCP based protocol and encrypt.
TCP ensures reliability and combined with encryption, makes altering log
entries as they traverse the network significantly more challenging.
- If UDP Syslog is the only option and the central repository is across
the WAN, try to collect close to the source and then forward via a TCP
based protocol.
- The central repository should utilize a strong access control
mechanism at the operating system, database, and application layer to
ensure that log data cannot be altered once written.
- Ideally, an archive/backup copy of all centralized logs should be
maintained. This provides redundancy in the event the central log
database fails or becomes corrupt.
- When written to the central repository/archive, additional controls
like hashsums or digital signatures can be used to validate integrity.
If hashsums and/or signatures are used make sure the proper access
controls exist around storing the hashes and signatures to ensure they
cannot be modified. This in itself is another line of thought...
- If you want to get really serious (and spend some money), write
archive copies of the logs to write-once storage.
I'm not sure if/how Splunk addresses the above. Vendors focused on log
management (vs SIM) address most if not all of the above.
Chris Petersen,
CTO & Founder
LogRhythm, Inc.
www.LogRhythm.com
> -----Original Message-----
> From: loganalysis-bounces+chris=lists.shmoo.com">security-conscious.com lists.shmoo.com
>
[lists.shmoo.com">mailto:loganalysis-bounces+chris=security-conscious.com lists.shmoo.com
]
> On Behalf Of Patrick Debois
> Sent: Tuesday, August 22, 2006 12:44 AM
> To: lists.shmoo.com">loganalysis lists.shmoo.com
> Subject: [logs] Log integrity handling on central logsystem
>
> I'm looking for feedback how centralized log solutions handle data
> integrity; If you would log directly to a central system, that log is
> the only source. So you would miss something to compare against.
>
> -Would you rely on taking checksums of the logs and storing them on
> another system?
> -How do you protect yourself from the fact that the central logging is
> compromised with a still growing logfile?
> Would you consider signing each log line? Signing within a text file
is
> fairly easy, but what about content stored in a database?
>
> My customer is currently looking at Splunk. It seems a great way to go
> through the logfiles, but I'm not sure that we can fullfill his
> dataintegrity requirements with it. But then again it does not stand
in
> the way of another solution doing it probable.
>
> Patrick
>
>
> _______________________________________________
> LogAnalysis mailing list
> lists.shmoo.com">LogAnalysis lists.shmoo.com
> http://lists.shmoo.com/mailman/listinfo/loganalysis
_______________________________________________
LogAnalysis mailing list
lists.shmoo.com">LogAnalysis lists.shmoo.com
http://lists.shmoo.com/mailman/listinfo/loganalysis
_______________________________________________
|
| Log integrity handling on central
logsystem |

|
2006-08-31 18:49:46 |
|
| Eric,
There are many regs out their that speak generally about taking steps to prove the integrity of logs. (HIPAA, DCID, NISPOM, FFIEC... the list goes on). Some mention log signing as a technique. Many more focus on security of network transmission (i.e. syslog-ng or similar) and controls over access to the log data repository.
However, real driver for signing is litigation risk, not regulations.
When you present log data as evidence to prosecute intruders, or you are subpoenaed for log evidence, opposing counsel will tend to challenge its integrity on several points:
- can't prove that there weren't additional exculpatory records showing someone other than the defendant had the same opportunity, if the evidence is circumstantial, that were then deleted by the _real_ culprit - can't prove the specific events used as evidence weren't changed
Signing (with a nonce that can be proven to have been protected) both individual log events and groups of consecutive log events is one of the best ways to address this type of challenge.
Furthermore, to the extent that log signing becomes standard practice, a company who does not do it is exposed to civil litigation for not having taken due care, if outsiders, employees, customers, etc. are unable to pursue lawsuits against 3rd parties because of the shakiness of the log evidence.
- Christina On Aug 31, 2006, at 1:00 AM, Eric Fitzgerald wrote: Regarding log signing: is anyone aware of an actual regulatory or legal requirement for log signing? I've always thought ensuring the integrity of the log begins at collection and ends at archival. In defining integrity I would say it means no log entries are altered and the complete log is collected and centralized. As for best practices, I would recommend the following:
- Collect all logs as close to real-time as possible. The longer the source log is on the source host, the more likely it can be altered. - When possible transport logs with a TCP based protocol and encrypt. TCP ensures reliability and combined with encryption, makes altering log entries as they traverse the network significantly more challenging. - If UDP Syslog is the only option and the central repository is across the WAN, try to collect close to the source and then forward via a TCP based protocol. - The central repository should utilize a strong access control mechanism at the operating system, database, and application layer to ensure that log data cannot be altered once written. - Ideally, an archive/backup copy of all centralized logs should be maintained. This provides redundancy in the event the central log database fails or becomes corrupt. - When written to the central repository/archive, additional controls like hashsums or digital signatures can be used to validate integrity. If hashsums and/or signatures are used make sure the proper access controls exist around storing the hashes and signatures to ensure they cannot be modified. This in itself is another line of thought... - If you want to get really serious (and spend some money), write archive copies of the logs to write-once storage.
I'm not sure if/how Splunk addresses the above. Vendors focused on log management (vs SIM) address most if not all of the above.
Chris Petersen, CTO & Founder LogRhythm, Inc. www.LogRhythm.com
> -----Original Message----- > From: loganalysis-bounces+chris=lists.shmoo.com">security-conscious.com lists.shmoo.com > [lists.shmoo.com">mailto:loganalysis-bounces+chris=security-conscious.com lists.shmoo.com ] > On Behalf Of Patrick Debois > Sent: Tuesday, August 22, 2006 12:44 AM > To: lists.shmoo.com">loganalysis lists.shmoo.com > Subject: [logs] Log integrity handling on central logsystem > > I'm looking for feedback how centralized log solutions handle data > integrity; If you would log directly to a central system, that log is > the only source. So you would miss something to compare against. > > -Would you rely on taking checksums of the logs and storing them on > another system? > -How do you protect yourself from the fact that the central logging is > compromised with a still growing logfile? > Would you consider signing each log line? Signing within a text file is > fairly easy, but what about content stored in a database? > > My customer is currently looking at Splunk. It seems a great way to go > through the logfiles, but I'm not sure that we can fullfill his > dataintegrity requirements with it. But then again it does not stand in > the way of another solution doing it probable. > > Patrick > > > _______________________________________________ > LogAnalysis mailing list > lists.shmoo.com">LogAnalysis lists.shmoo.com > http://lists.shmoo.com/mailman/listinfo/loganalysis _______________________________________________ LogAnalysis mailing list lists.shmoo.com">LogAnalysis lists.shmoo.com http://lists.shmoo.com/mailman/listinfo/loganalysis
_______________________________________________ LogAnalysis mailing list |
| Log integrity handling on central
logsystem |

|
2006-08-31 21:09:42 |
> Regarding log signing: is anyone aware of an actual
regulatory or legal
> requirement for log signing?
Isnt't that a bit too specific for most current
regulations? I
personally haven't seen any direct regulatory mandate for
log
signing...
--
Anton Chuvakin, Ph.D., GCIA, GCIH, GCFA
http://www.chuvakin.org
http://chuvakin.blogspot
.com
http://www.securitywar
rior.com
_______________________________________________
LogAnalysis mailing list
LogAnalysis lists.shmoo.com
h
ttp://lists.shmoo.com/mailman/listinfo/loganalysis
|
|
| Log integrity handling on central
logsystem |

|
2006-08-31 19:06:56 |
|
Hi Christina,
The federal rules of evidence actually
point in the other direction, that U.S. Federal courts look at the likelihood
of tampering having actually occurred rather than the possibility of tampering.
http://www.cybercrime.gov/usamarch2001_4.htm
Here's an excerpt from an article by Orin
S. Kerr (a DoJ attorney when the article was written) on Computer Records and
the Federal Rules of Evidence:
"Computer records can
be altered easily, and opposing parties often allege that computer records lack
authenticity because they have been tampered with or changed after they were
created. For example, in United States v.
Whitaker, 127 F.3d 595, 602 (7th Cir. 1997), the government
retrieved computer files from the computer of a narcotics dealer named Frost.
The files from Frost's computer included detailed records of narcotics sales by
three aliases: "Me" (Frost himself, presumably), "Gator"
(the nickname of Frost's co-defendant Whitaker), and "Cruz" (the
nickname of another dealer). After the government permitted Frost to help
retrieve the evidence from his computer and declined to establish a formal
chain of custody for the computer at trial, Whitaker argued that the files
implicating him through his alias were not properly authenticated. Whitaker
argued that "with a few rapid keystrokes, Frost could have easily added
Whitaker's alias, 'Gator' to the printouts in order to finger Whitaker and to
appear more helpful to the government." Id.
at 602."
"The courts have responded with considerable skepticism to such
unsupported claims that computer records have been altered. Absent specific
evidence that tampering occurred, the mere possibility of tampering does not
affect the authenticity of a computer record. See
Whitaker, 127 F.3d at 602 (declining to disturb trial judge's ruling
that computer records were admissible because allegation of tampering was
"almost wild-eyed speculation . . . [without] evidence to support such a
scenario"); United States v. Bonallo,
858 F.2d 1427, 1436 (9th Cir. 1988) ("The fact that it is possible to
alter data contained in a computer is plainly insufficient to establish
untrustworthiness."); United States v. Glasser,
773 F.2d 1553, 1559 (11th Cir. 1985) ("The existence of an air-tight
security system [to prevent tampering] is not, however, a prerequisite to the
admissibility of computer printouts. If such a prerequisite did exist, it would
become virtually impossible to admit computer-generated records; the party
opposing admission would have to show only that a better security system was
feasible."). Id. at 559. This
is consistent with the rule used to establish the authenticity of other
evidence such as narcotics. See United
States v. Allen, 106 F.3d 695, 700 (6th Cir. 1997) ("Merely
raising the possibility of tampering is insufficient to render evidence
inadmissible."). Absent specific evidence of tampering, allegations that
computer records have been altered go to their weight, not their admissibility.
See Bonallo, 858 F.2d at 1436."
I agree that signing (strong evidence
collection & chain of custody procedures) will help dismiss such
accusations more briefly, but I don't believe that signing is necessary for
introduction of logs as evidence.
Cash register receipts, another kind of
electronic evidence, won't have digsig for probably decades yet are readily
admissible providing that receipts are part of normal business process.
Now your last statement regarding best
practices is the real heart of the matter, and I think that's really the most
important concern.
Thank you very much for your detailed
thoughts!
Best regards,
Eric
From: Christina Noren
[mailto:cfrln cfrln.com]
Sent: Thursday, August 31, 2006
11:50 AM
To: Eric Fitzgerald
Cc: loganalysis lists.shmoo.com
Subject: Re: [logs] Re: Log
integrity handling on central logsystem
Eric,
There are many regs out their that speak generally about taking steps
to prove the integrity of logs. (HIPAA, DCID, NISPOM, FFIEC... the list goes
on). Some mention log signing as a technique. Many more focus on security of
network transmission (i.e. syslog-ng or similar) and controls over access to
the log data repository.
However, real driver for signing is litigation risk, not
regulations.
When you present log data as evidence to prosecute intruders, or you
are subpoenaed for log evidence, opposing counsel will tend to challenge its
integrity on several points:
- can't prove that there weren't additional exculpatory records showing
someone other than the defendant had the same opportunity, if the evidence is
circumstantial, that were then deleted by the _real_ culprit
- can't prove the specific events used as evidence weren't changed
Signing (with a nonce that can be proven to have been protected) both
individual log events and groups of consecutive log events is one of the best
ways to address this type of challenge.
Furthermore, to the extent that log signing becomes standard practice,
a company who does not do it is exposed to civil litigation for not having
taken due care, if outsiders, employees, customers, etc. are unable to pursue
lawsuits against 3rd parties because of the shakiness of the log
evidence.
On Aug 31, 2006, at 1:00 AM, Eric Fitzgerald wrote:
Regarding log signing: is anyone aware of
an actual regulatory or legal requirement for log signing?
I've always thought ensuring the integrity of the log
begins at
collection and ends at archival. In defining integrity I would say it
means no log entries are altered and the complete log is collected and
centralized. As for best practices, I would recommend the following:
- Collect all logs as close to real-time as possible. The longer the
source log is on the source host, the more likely it can be altered.
- When possible transport logs with a TCP based protocol and encrypt.
TCP ensures reliability and combined with encryption, makes altering log
entries as they traverse the network significantly more challenging.
- If UDP Syslog is the only option and the central repository is across
the WAN, try to collect close to the source and then forward via a TCP
based protocol.
- The central repository should utilize a strong access control
mechanism at the operating system, database, and application layer to
ensure that log data cannot be altered once written.
- Ideally, an archive/backup copy of all centralized logs should be
maintained. This provides redundancy in the event the central log
database fails or becomes corrupt.
- When written to the central repository/archive, additional controls
like hashsums or digital signatures can be used to validate integrity.
If hashsums and/or signatures are used make sure the proper access
controls exist around storing the hashes and signatures to ensure they
cannot be modified. This in itself is another line of thought...
- If you want to get really serious (and spend some money), write
archive copies of the logs to write-once storage.
I'm not sure if/how Splunk addresses the above. Vendors focused on log
management (vs SIM) address most if not all of the above.
Chris Petersen,
CTO & Founder
LogRhythm, Inc.
www.LogRhythm.com
> -----Original Message-----
> From: loganalysis-bounces+chris=lists.shmoo.com">security-conscious.com lists.shmoo.com
>
[lists.shmoo.com">mailto:loganalysis-bounces+chris=security-conscious.com lists.shmoo.com
]
> On Behalf Of Patrick Debois
> Sent: Tuesday, August 22, 2006 12:44 AM
> To: lists.shmoo.com">loganalysis lists.shmoo.com
> Subject: [logs] Log integrity handling on central logsystem
>
> I'm looking for feedback how centralized log solutions handle data
> integrity; If you would log directly to a central system, that log is
> the only source. So you would miss something to compare against.
>
> -Would you rely on taking checksums of the logs and storing them on
> another system?
> -How do you protect yourself from the fact that the central logging is
> compromised with a still growing logfile?
> Would you consider signing each log line? Signing within a text file
is
> fairly easy, but what about content stored in a database?
>
> My customer is currently looking at Splunk. It seems a great way to go
> through the logfiles, but I'm not sure that we can fullfill his
> dataintegrity requirements with it. But then again it does not stand
in
> the way of another solution doing it probable.
>
> Patrick
>
>
> _______________________________________________
> LogAnalysis mailing list
> lists.shmoo.com">LogAnalysis lists.shmoo.com
> http://lists.shmoo.com/mailman/listinfo/loganalysis
_______________________________________________
LogAnalysis mailing list
lists.shmoo.com">LogAnalysis lists.shmoo.com
http://lists.shmoo.com/mailman/listinfo/loganalysis
_______________________________________________
|
|
|