thoughts on ETags and mod_dav

Hi --

   A couple of months ago a short thread started in relation
to the
PRs #16593 and #38034 (which also references #42987) on the
various
problems related to ETags:

http://marc.info/?l=apache-httpd-dev&m=118
831732512678&w=2

http://issues.apache.org/bugzilla/show_bug.cgi?id=16593
http://issues.apache.org/bugzilla/show_bug.cgi?id=38034
http://issues.apache.org/bugzilla/show_bug.cgi?id=42987

   I'm not about to tackle these all immediately, but I
wanted to
send out a couple of proposals and find out what problems
folks
might see with them.


1) Per #38034, it appears that ap_meets_conditions() treats
"*" incorrectly.
The patch attached to the PR duplicates
ap_meets_conditions() for
exclusive use by mod_dav.  The only changes are to add
checks on
resource->exists in two places and to use
resource->hooks->getetag instead
of looking for an ETag header in r->headers_out.

   In the interest of avoiding code duplication, I wonder if
it would
be possible to continue to use ap_meets_conditions() by
using
r->no_local_copy in place of resource->exists, and by
setting/unsetting
that and an ETag header in r->headers_out in mod_dav as
necessary;
e.g., prior to calls to ap_meets_conditions().

   However, it looks to me as though r->no_local_copy is
used in a few
places to bypass the ap_meets_conditions() checks entirely
(i.e., to
never send a 304 response in certain cases).

   Does anyone have comments on what the intended purpose
r->no_local_copy is or was, especially in relation to the
language
of RFC 2616 sections 14.24 and 14.26 ("if '*' is given
and any current
entity exists for that resource ...")?

   Would it be more appropriate and/or required by our
backporting
rules to add another field to request_rec, like
r->exists, to indicate
to ap_meets_conditions() that a resource does not exist?  Or
to use
r->notes or conceivably r->finfo for the same
purpose?

   Is it even possible to backport a change to the action
of
ap_meets_conditions() such that it would start depending on
data
which third-party callers won't be supplying?  Advice is
welcome, please!


2) I haven't looked at #16593 in any detail, but I wonder if
the
change proposed in the patch attached to #38034 where
resource->hooks->getetag is used instead of looking
for an ETag header
in r->headers_out might not be an attempt to deal with
this issue.
If so, then whatever the eventual solution for #38034 turns
out to
be will likely resolve this PR as well.


3) Per #42987, it seems as if there's a subtle difference
between
the requirements for ETags as described in RFC 2616 section
13.3.3
and the way we currently generate them in ap_make_etag().

   When a file's mtime is in the past, ap_make_etag() uses
the file's
inode, length, and/or mtime to create a strong ETag. 
However, when the
mtime is the same as r->request_time, a weak ETag is
generated.  As the
attachment to the PR reveals, though, this isn't really
sufficient when
certain sequences of requests occur within a single second.

   The RFC says that a file's mtime "could" be a
weak validator, because
the file might change more than once in a second.  But I
think the
subtly important issue here is that word "could". 
If I understand it
correctly, the implication is that one has resource for
which one wants
to generate weak ETags only, because one knows the semantic
meaning isn't
changing although the content may be.  In this particular
case, the
modification time *might* be appropriate to use as a weak
validator
(although that seems somewhat unlikely, because non-semantic
changes
that occurred seconds apart would still cause different weak
ETags to be
generated, which probably isn't what you'd want in this
circumstance).

   But, in the cases handled by ap_make_etag(), the content
and/or the
semantic meaning of a resource could always be changing, so
weak ETags
would seem to be, as the PR says, simply never appropriate.

   Alas, though, there is probably no way to quickly
generate a strong
validators like a hash -- which is why we're using inodes,
lengths, and
mtimes in the first place, of course.

   So, a modest proposal, which aims to punt the problem to
the
administrator, and also to resolve the fact that FileETag is
provided
to configure the actions of ap_make_etag(), while
mod_dav_fs's
dav_fs_getetag() actions are fixed.  The highlights would
be:

- refactor ap_make_etag() and dav_fs_getetag() to use a
common
  code base as far as possible

- make dav_fs_getetag() follow the options set by FileETag
(if this
  is a problem, then introduce a DavFileETag directive, but
I'm hoping
  that could be avoided)

- add MD5 and SHA1 as options to FileETag (while obviously
much
  slower, certain applications may require "really
strong" validators);
  for legacy reasons, these would have to be specifically
configured
  and would not be implied by the "All" option

- add a nomtime=none|weak|ignore option to FileETag (in the
manner
  of ProxyPass) which specifies what to do when the MTime
option is
  configured but the file's mtime is the same as
r->request_time;
  the default value is "weak" to retain backward
compatibility

- allow mod_dav_fs to cache ETags in its state directory,
either
  in its properties DBM files or elsewhere, at least when
configured to
  generate "really strong" validators; e.g.,
lookup in cache whether
  file's inode, length, and mtime match cached values; if OK
and mtime is
  not current time then return cached hash value; otherwise
  recalculate hash value and store in cache along with
inode, length,
  and mtime for future comparisons

- change ETag generation logic to respect the new hash
options (MD5, SHA1);
  possibly add support for caching, at least in the mod_dav
case where
  the refactored function is called from dav_fs_getetag()

- change ETag generation logic (from current ap_make_etag()
implementation)
  such that when the MTime option is not specified, no check
against
  r->request_time is done; currently this is always done
and the ETag may
  be made weak even if MTime is not configured

- change ETag generation logic to respect the new
nomtime=none|weak|ignore
  option when MTime is configured; in this case, when the
file's mtime
  equals r->request_time, either generate no ETag (none),
generate a
  weak ETag in the current manner (weak), or generate a
strong ETag (ignore)

   Whew.  Thoughts and criticisms?  Obviously none of this
will get done
in the immediate future, but any discussion will, I hope,
give me
and others some direction on these issues.  Thanks,

Chris.

-- 
GPG Key ID: 366A375B
GPG Key Fingerprint: 485E 5041 17E1 E2BB C263  E4DE C8E3
FA36 366A 375B

Hi,

There is a way we can avoid code duplication of
ap_meets_conditions()
- handle the cases that ap_meets_conditions() does not
handle
separately. For this, we can create a new
dav_meets_conditions() which
calls ap_meets_conditions() and also handles those other
cases. Also,
the problem of ETag header not being set, can be tackled by
calling
the resource->hook->set_headers() hook before calling
dav_meets_conditions().

Attaching a patch containing these changes. The patch is
against
apache v2.2.6. And I've tested it against mod_dav_fs ( w/
DEBUG_GET_HANDLER set to 1) for the following cases:
1. IF-MATCH * with nothing there fails
2. IF-NONE-MATCH * with nothing there succeeds
3. IF-NONE-MATCH * with something there fails
4. IF-MATCH * with something there succeeds
5. IF-MATCH with same etag succeeds
6. IF-MATCH with different etag fails

For test case 5. you've to be careful about the weak etag
bug ( sleep
for a couple of seconds between consecutive requests )

- Paritosh.

On 10/11/07, Chris Darroch <chrisdpearsoncmg.com> wrote:
> Hi --
>
>    A couple of months ago a short thread started in
relation to the
> PRs #16593 and #38034 (which also references #42987) on
the various
> problems related to ETags:
>
> http://marc.info/?l=apache-httpd-dev&m=118
831732512678&w=2
>
> http://issues.apache.org/bugzilla/show_bug.cgi?id=16593
> http://issues.apache.org/bugzilla/show_bug.cgi?id=38034
> http://issues.apache.org/bugzilla/show_bug.cgi?id=42987
>
>    I'm not about to tackle these all immediately, but I
wanted to
> send out a couple of proposals and find out what
problems folks
> might see with them.
>
>
> 1) Per #38034, it appears that ap_meets_conditions()
treats "*" incorrectly.
> The patch attached to the PR duplicates
ap_meets_conditions() for
> exclusive use by mod_dav.  The only changes are to add
checks on
> resource->exists in two places and to use
resource->hooks->getetag instead
> of looking for an ETag header in r->headers_out.
>
>    In the interest of avoiding code duplication, I
wonder if it would
> be possible to continue to use ap_meets_conditions() by
using
> r->no_local_copy in place of resource->exists,
and by setting/unsetting
> that and an ETag header in r->headers_out in mod_dav
as necessary;
> e.g., prior to calls to ap_meets_conditions().
>
>    However, it looks to me as though
r->no_local_copy is used in a few
> places to bypass the ap_meets_conditions() checks
entirely (i.e., to
> never send a 304 response in certain cases).
>
>    Does anyone have comments on what the intended
purpose
> r->no_local_copy is or was, especially in relation
to the language
> of RFC 2616 sections 14.24 and 14.26 ("if '*' is
given and any current
> entity exists for that resource ...")?
>
>    Would it be more appropriate and/or required by our
backporting
> rules to add another field to request_rec, like
r->exists, to indicate
> to ap_meets_conditions() that a resource does not
exist?  Or to use
> r->notes or conceivably r->finfo for the same
purpose?
>
>    Is it even possible to backport a change to the
action of
> ap_meets_conditions() such that it would start
depending on data
> which third-party callers won't be supplying?  Advice
is welcome, please!
>
>
> 2) I haven't looked at #16593 in any detail, but I
wonder if the
> change proposed in the patch attached to #38034 where
> resource->hooks->getetag is used instead of
looking for an ETag header
> in r->headers_out might not be an attempt to deal
with this issue.
> If so, then whatever the eventual solution for #38034
turns out to
> be will likely resolve this PR as well.
>
>
> 3) Per #42987, it seems as if there's a subtle
difference between
> the requirements for ETags as described in RFC 2616
section 13.3.3
> and the way we currently generate them in
ap_make_etag().
>
>    When a file's mtime is in the past, ap_make_etag()
uses the file's
> inode, length, and/or mtime to create a strong ETag. 
However, when the
> mtime is the same as r->request_time, a weak ETag is
generated.  As the
> attachment to the PR reveals, though, this isn't really
sufficient when
> certain sequences of requests occur within a single
second.
>
>    The RFC says that a file's mtime "could"
be a weak validator, because
> the file might change more than once in a second.  But
I think the
> subtly important issue here is that word
"could".  If I understand it
> correctly, the implication is that one has resource for
which one wants
> to generate weak ETags only, because one knows the
semantic meaning isn't
> changing although the content may be.  In this
particular case, the
> modification time *might* be appropriate to use as a
weak validator
> (although that seems somewhat unlikely, because
non-semantic changes
> that occurred seconds apart would still cause different
weak ETags to be
> generated, which probably isn't what you'd want in this
circumstance).
>
>    But, in the cases handled by ap_make_etag(), the
content and/or the
> semantic meaning of a resource could always be
changing, so weak ETags
> would seem to be, as the PR says, simply never
appropriate.
>
>    Alas, though, there is probably no way to quickly
generate a strong
> validators like a hash -- which is why we're using
inodes, lengths, and
> mtimes in the first place, of course.
>
>    So, a modest proposal, which aims to punt the
problem to the
> administrator, and also to resolve the fact that
FileETag is provided
> to configure the actions of ap_make_etag(), while
mod_dav_fs's
> dav_fs_getetag() actions are fixed.  The highlights
would be:
>
> - refactor ap_make_etag() and dav_fs_getetag() to use a
common
>   code base as far as possible
>
> - make dav_fs_getetag() follow the options set by
FileETag (if this
>   is a problem, then introduce a DavFileETag directive,
but I'm hoping
>   that could be avoided)
>
> - add MD5 and SHA1 as options to FileETag (while
obviously much
>   slower, certain applications may require "really
strong" validators);
>   for legacy reasons, these would have to be
specifically configured
>   and would not be implied by the "All"
option
>
> - add a nomtime=none|weak|ignore option to FileETag (in
the manner
>   of ProxyPass) which specifies what to do when the
MTime option is
>   configured but the file's mtime is the same as
r->request_time;
>   the default value is "weak" to retain
backward compatibility
>
> - allow mod_dav_fs to cache ETags in its state
directory, either
>   in its properties DBM files or elsewhere, at least
when configured to
>   generate "really strong" validators; e.g.,
lookup in cache whether
>   file's inode, length, and mtime match cached values;
if OK and mtime is
>   not current time then return cached hash value;
otherwise
>   recalculate hash value and store in cache along with
inode, length,
>   and mtime for future comparisons
>
> - change ETag generation logic to respect the new hash
options (MD5, SHA1);
>   possibly add support for caching, at least in the
mod_dav case where
>   the refactored function is called from
dav_fs_getetag()
>
> - change ETag generation logic (from current
ap_make_etag() implementation)
>   such that when the MTime option is not specified, no
check against
>   r->request_time is done; currently this is always
done and the ETag may
>   be made weak even if MTime is not configured
>
> - change ETag generation logic to respect the new
nomtime=none|weak|ignore
>   option when MTime is configured; in this case, when
the file's mtime
>   equals r->request_time, either generate no ETag
(none), generate a
>   weak ETag in the current manner (weak), or generate a
strong ETag (ignore)
>
>    Whew.  Thoughts and criticisms?  Obviously none of
this will get done
> in the immediate future, but any discussion will, I
hope, give me
> and others some direction on these issues.  Thanks,
>
> Chris.
>
> --
> GPG Key ID: 366A375B
> GPG Key Fingerprint: 485E 5041 17E1 E2BB C263  E4DE
C8E3 FA36 366A 375B
>
>

Hi --

> 1) Per #38034, it appears that ap_meets_conditions()
treats "*" incorrectly.

   More precisely, I should say that ap_meets_conditions()
isn't designed
to support the NULL resources of RFC 2518 (WebDAV).  I'm
certainly no
expert on these issues, so guidance is welcome.

   RFC 2616 section 14.24 (and 14.26 is similar) says,
"If the request
would, without the If-Match header field, result in anything
other than a
2xx or 412 status, then the If-Match header MUST be
ignored."  Thus in
the typical case, if a resource doesn't exist, 404 should be
returned,
so ap_meets_conditions() doesn't need to handle this case at
all.

   RFC 2518 introduces lock-null resources for use with
certain
methods such as LOCK, PUT, and UNLOCK.  (RFC 4918, it seems,
deprecates
lock-null resources in favour of "locked empty
resources".)  For these
particular methods and resources, other language from RFC
2616's
section 14.24 (again, 14.26 is similar) comes into play,
specifically,
"If none of the entity tags match, or if '*' is given
and no current entity
exists, the server MUST NOT perform the requested method,
and MUST return a
412 (Precondition Failed) response."

   Personally, I'd be inclined to try to make
ap_meets_conditions()
handle support these different situations in as generic a
way as
possible (in case other non-mod_dav DAV modules or other
HTTP extensions
need to use the same logic), but that may be difficult to do
without
introducing compatibility problems and/or contorted code.

   Another option would be the one described by Paritosh
Shah:

> For this, we can create a new dav_meets_conditions()
which
> calls ap_meets_conditions() and also handles those
other cases.

   Thoughts, corrections, etc.?

Chris.

-- 
GPG Key ID: 366A375B
GPG Key Fingerprint: 485E 5041 17E1 E2BB C263  E4DE C8E3
FA36 366A 375B

On fre, 2007-10-12 at 00:25 -0400, Chris Darroch wrote:

>    RFC 2616 section 14.24 (and 14.26 is similar) says,
"If the request
> would, without the If-Match header field, result in
anything other than a
> 2xx or 412 status, then the If-Match header MUST be
ignored."  Thus in
> the typical case, if a resource doesn't exist, 404
should be returned,
> so ap_meets_conditions() doesn't need to handle this
case at all.

There is more to HTTP than only GET/HEAD.

If-Match: *
and
If-None-Match: *

is quite relevant only taking 2616 into account

Most notably If-None-Match in combination with PUT, used for
creating a
new resource IFF one do not already exists.

The first examples of PR #38024 is also quite speaking for
itself on
If-Match: *.

Regards
Henrik

Henrik Nordstrom wrote:

> On fre, 2007-10-12 at 00:25 -0400, Chris Darroch
wrote:
> 
>>    RFC 2616 section 14.24 (and 14.26 is similar)
says, "If the request
>> would, without the If-Match header field, result in
anything other than a
>> 2xx or 412 status, then the If-Match header MUST be
ignored."  Thus in
>> the typical case, if a resource doesn't exist, 404
should be returned,
>> so ap_meets_conditions() doesn't need to handle
this case at all.
> 
> There is more to HTTP than only GET/HEAD.
> 
> If-Match: *
> and
> If-None-Match: *
> 
> is quite relevant only taking 2616 into account

   Agreed -- by "in the typical case", I didn't
mean "in all cases
covered by RFC 2616", although I don't think I made
that particularly
clear.

   I just meant that ap_meets_conditions() was likely
written with
the methods that handled by the core handler in mind; that's
why its
logic has remained largely unchanged for many years.  PUT
isn't
supported by the core handler, or most "typical"
content handlers.
WebDAV happens to bring this issue to light, perhaps because
it's
increasingly widely utilized.

   I'm all in favour of trying to make ap_meets_conditions()
as generic
and correct as possible, as I said:

>    Personally, I'd be inclined to try to make
ap_meets_conditions()
> handle support these different situations in as generic
a way as
> possible

   But, to do this it may be necessary to have data
available that
isn't at present.  That's why I was interested in exploring
various
options, such as using r->no_local_copy.  Or, possibly,
we could
add an r->does_not_exist flag which would be 0 by
default; this
could be introduced without compability problems, I think.
Modules which needed to indicate that they were dealing with
a
resource that didn't exist, e.g., lock-null resources in
mod_dav,
could set the flag to 1.

Chris.

-- 
GPG Key ID: 366A375B
GPG Key Fingerprint: 485E 5041 17E1 E2BB C263  E4DE C8E3
FA36 366A 375B

Chris Darroch wrote:
> Hi --
> 
>> 1) Per #38034, it appears that
ap_meets_conditions() treats "*" incorrectly.
> 
>    More precisely, I should say that
ap_meets_conditions() isn't designed
> to support the NULL resources of RFC 2518 (WebDAV). 
I'm certainly no
> expert on these issues, so guidance is welcome.

Clarifying: RFC2518 doesn't invent "NULL
resources", it just assigns a 
term to them.

>    RFC 2616 section 14.24 (and 14.26 is similar) says,
"If the request
> would, without the If-Match header field, result in
anything other than a
> 2xx or 412 status, then the If-Match header MUST be
ignored."  Thus in
> the typical case, if a resource doesn't exist, 404
should be returned,
> so ap_meets_conditions() doesn't need to handle this
case at all.

Unless it is a method that can be applied to a NULL
resource, such as 
PUT, PATCH (?), MKCOL, LOCK...

>    RFC 2518 introduces lock-null resources for use with
certain
> methods such as LOCK, PUT, and UNLOCK.  (RFC 4918, it
seems, deprecates
> lock-null resources in favour of "locked empty
resources".)  For these
> particular methods and resources, other language from
RFC 2616's
> section 14.24 (again, 14.26 is similar) comes into
play, specifically,
> "If none of the entity tags match, or if '*' is
given and no current entity
> exists, the server MUST NOT perform the requested
method, and MUST return a
> 412 (Precondition Failed) response."

That's correct, but see above -- it's not restricted to
lock-null 
resources, and also not to WebDAV methods.

> ...

Best regards, Julian

On Thu, 11 Oct 2007 17:50:23 -0400
Chris Darroch <chrisdpearsoncmg.com> wrote:

> Hi --
> 
>    A couple of months ago a short thread started in
relation to the
> PRs #16593 and #38034 (which also references #42987) on
the various
> problems related to ETags:
> 
> http://marc.info/?l=apache-httpd-dev&m=118
831732512678&w=2
> 
> http://issues.apache.org/bugzilla/show_bug.cgi?id=16593
> http://issues.apache.org/bugzilla/show_bug.cgi?id=38034
> http://issues.apache.org/bugzilla/show_bug.cgi?id=42987
> 
>    I'm not about to tackle these all immediately, but I
wanted to
> send out a couple of proposals and find out what
problems folks
> might see with them.
> 
> 
> 1) Per #38034, it appears that ap_meets_conditions()
treats "*"
> incorrectly. The patch attached to the PR duplicates
> ap_meets_conditions() for exclusive use by mod_dav. 
The only changes
> are to add checks on resource->exists in two places
and to use
> resource->hooks->getetag instead of looking for
an ETag header in
> r->headers_out.
> 
>    In the interest of avoiding code duplication, I
wonder if it would
> be possible to continue to use ap_meets_conditions() by
using
> r->no_local_copy in place of resource->exists,
and by
> setting/unsetting that and an ETag header in
r->headers_out in
> mod_dav as necessary; e.g., prior to calls to
ap_meets_conditions().
> 
>    However, it looks to me as though
r->no_local_copy is used in a few
> places to bypass the ap_meets_conditions() checks
entirely (i.e., to
> never send a 304 response in certain cases).
> 
>    Does anyone have comments on what the intended
purpose
> r->no_local_copy is or was, especially in relation
to the language
> of RFC 2616 sections 14.24 and 14.26 ("if '*' is
given and any current
> entity exists for that resource ...")?
> 
>    Would it be more appropriate and/or required by our
backporting
> rules to add another field to request_rec, like
r->exists, to indicate
> to ap_meets_conditions() that a resource does not
exist?  Or to use
> r->notes or conceivably r->finfo for the same
purpose?

My vote goes to r->notes.

Anything else relies on something with semantic significance
that'll
risk breaking random things.

For the future (e.g. 2.4), we could review other options. 
For example,
abstract out r->finfo to a void* with an API for
inspecting resources
including but not limited to those visible in the
filesystem.

>    Is it even possible to backport a change to the
action of
> ap_meets_conditions() such that it would start
depending on data
> which third-party callers won't be supplying?  Advice
is welcome,
> please!
> 
> 
> 2) I haven't looked at #16593 in any detail, but I
wonder if the
> change proposed in the patch attached to #38034 where
> resource->hooks->getetag is used instead of
looking for an ETag header
> in r->headers_out might not be an attempt to deal
with this issue.
> If so, then whatever the eventual solution for #38034
turns out to
> be will likely resolve this PR as well.

A fix to 38034 that doesn't also fix 16593 would indeed seem
perverse,
unless perhaps there's some strange subtlety lurking.

> 3) Per #42987, it seems as if there's a subtle
difference between
> the requirements for ETags as described in RFC 2616
section 13.3.3
> and the way we currently generate them in
ap_make_etag().
[chop]
>    But, in the cases handled by ap_make_etag(), the
content and/or the
> semantic meaning of a resource could always be
changing, so weak ETags
> would seem to be, as the PR says, simply never
appropriate.

+1

>    So, a modest proposal, which aims to punt the
problem to the
> administrator, and also to resolve the fact that
FileETag is provided
> to configure the actions of ap_make_etag(), while
mod_dav_fs's
> dav_fs_getetag() actions are fixed.  The highlights
would be:

 [chop]

>    Whew.  Thoughts and criticisms?  Obviously none of
this will get
> done in the immediate future, but any discussion will,
I hope, give me
> and others some direction on these issues.  Thanks,

Feel free to ignore my totally half-baked ideas, but mod_dav
is not
the only area where we have issues, with ETags.  There's
also the
whole question of negotiated content to consider, and I'm
still confused
by the problems that still surround PR#39727.  If we're
reworking
ETag logic, we should take the opportunity to deal with
this, too.
Can we incorporate some kind of tokenisation into ETags that
will
express dependence on negotiation?

For example, Henrik Nordstrom suggests ";gzip". 
If we expand that to,
say, ";CE=gzip", then we can start to apply
semantics to decoding the
Etag, and determine equivalence.  So in that case, we could
determine
that the (uncompressed) file does indeed match the ETag. 
Not sure if
there's anything we might do with other negotiated headers,
but maybe
we could leave that open, providing a hook that something
like
mod_negotiation might someday use.

-- 
Nick Kew

Application Development with Apache - the Apache Modules
Book
http://www.apachetutor.or
g/

On tis, 2007-10-16 at 11:33 +0100, Nick Kew wrote:

> For example, Henrik Nordstrom suggests
";gzip".  If we expand that to,
> say, ";CE=gzip", then we can start to apply
semantics to decoding the
> Etag, and determine equivalence.  So in that case, we
could determine
> that the (uncompressed) file does indeed match the
ETag.  Not sure if
> there's anything we might do with other negotiated
headers, but maybe
> we could leave that open, providing a hook that
something like
> mod_negotiation might someday use.

It's not immediately relevant to mod_negotiation, mainly to
filters
doing negotiated transformation of responses where the
original response
has an ETag. mod_deflate is the most apparent and most
frequently used
such filter, but I can think of many other such as dynamic
image
conversion (png->jpg for user-agents not handling png,
lowres for small
devices etc.) etc.

Having some structure on the tokens is helpful, but not
strictly needed
as long as each step in the filter chain can recognise it's
token and
add/remove it as needed.

Add token to ETag when the response goes out, reflecting the
negotiation
process done by the filter. As discussed earlier a token may
need to be
always added to avoid conflicts with other ETag sources
possibly
generating ETag values with the same token format.

remove from If-* when processing requests and conditions
still match for
the step in question. If conditions no longer match the
token then the
condition needs to be rejected.

Having a clear structure may allow you to make efficient
shortcuts in
the processing, but probably only when the resource has
changed making
the original resource ETag different.


Hmm.. looking at mod_negotiate again it may be relevant
there as well
when using the body typemap feature. Today mod_negotiate do
not return
an ETag under such conditions, but using tokens it could,
allowing for
proper cache validation of body typemap negotiated
resources. Using
tokens is in fact not very different from what mod_negotiate
does today
chaining the etags of the typemap and the selected variant.

The token idea is however not relevant to mod_negotiate
multiviews as
the selected variant already has an unique ETag and is
sufficient
identification of the selected variant. No need to
complicate that quite
common case.

Regards
Henrik

Nick Kew wrote:

> My vote goes to r->notes.
> 
> Anything else relies on something with semantic
significance that'll
> risk breaking random things.
> 
> For the future (e.g. 2.4), we could review other
options.  For example,
> abstract out r->finfo to a void* with an API for
inspecting resources
> including but not limited to those visible in the
filesystem.

   Sounds good; maybe an r->notes key named
"NON_EXTANT_RESOURCE" or
"NO_RESOURCE" or some such?

> Feel free to ignore my totally half-baked ideas, but
mod_dav is not
> the only area where we have issues, with ETags. 
There's also the
> whole question of negotiated content to consider, and
I'm still confused
> by the problems that still surround PR#39727.  If we're
reworking
> ETag logic, we should take the opportunity to deal with
this, too.
> Can we incorporate some kind of tokenisation into ETags
that will
> express dependence on negotiation?
> 
> For example, Henrik Nordstrom suggests
";gzip".  If we expand that to,
> say, ";CE=gzip", then we can start to apply
semantics to decoding the
> Etag, and determine equivalence.  So in that case, we
could determine
> that the (uncompressed) file does indeed match the
ETag.  Not sure if
> there's anything we might do with other negotiated
headers, but maybe
> we could leave that open, providing a hook that
something like
> mod_negotiation might someday use.

   I confess I've been following this discussion at an
abstract
level only (at best).  I don't think, though, that it
directly affects
any of the several bugs I'm hoping to tackle, if I
understand the
issue correctly.  But I don't claim to fully understand the
details of
mod_negiotiation.

Chris.

-- 
GPG Key ID: 366A375B
GPG Key Fingerprint: 485E 5041 17E1 E2BB C263  E4DE C8E3
FA36 366A 375B

There seems to be some consensus that we should add a key to
r->notes
as a temporary fix ( at least till 2.4 ). Hence, I'm
attaching a patch
- ap_meets_conditions_fixes.patch, with the appropriate
fixes to
ap_meets_conditions() using "resource-exists" key
for this purpose.
To be sure that we do not break anything, we would want
ap_meets_conditions() to behave as it does now when the key
is not
set. This implies that the value corresponding to the key
needs to
pass two states - one to signify resource exists and another
for
resource does not exist ( using "Y" and
"N" as the corresponding
values here ). I used "resource-exists" as the key
instead of
"NON_EXTANT_RESOURCE" or "NO_RESOURCE"
as suggested by Chris Darroch,
to avoid double negatives.

Also attaching a corresponding patch to mod_dav (
mod_dav_fixes.patch
) which sets "resource-exists" appropriately and
also the Etag header
by calling resource->hooks->set_headers() before
calling
ap_meets_conditions()

Again, the patches are against 2.2.6 and I've tested them
against
mod_dav_fs w/ DEBUG_GET_HANDLER set to 1.

Suggestions ? Comments ?

- Paritosh Shah.


On 10/18/07, Chris Darroch <chrisdpearsoncmg.com> wrote:
> Nick Kew wrote:
>
> > My vote goes to r->notes.
> >
> > Anything else relies on something with semantic
significance that'll
> > risk breaking random things.
> >
> > For the future (e.g. 2.4), we could review other
options.  For example,
> > abstract out r->finfo to a void* with an API
for inspecting resources
> > including but not limited to those visible in the
filesystem.
>
>    Sounds good; maybe an r->notes key named
"NON_EXTANT_RESOURCE" or
> "NO_RESOURCE" or some such?
>
> > Feel free to ignore my totally half-baked ideas,
but mod_dav is not
> > the only area where we have issues, with ETags. 
There's also the
> > whole question of negotiated content to consider,
and I'm still confused
> > by the problems that still surround PR#39727.  If
we're reworking
> > ETag logic, we should take the opportunity to deal
with this, too.
> > Can we incorporate some kind of tokenisation into
ETags that will
> > express dependence on negotiation?
> >
> > For example, Henrik Nordstrom suggests
";gzip".  If we expand that to,
> > say, ";CE=gzip", then we can start to
apply semantics to decoding the
> > Etag, and determine equivalence.  So in that case,
we could determine
> > that the (uncompressed) file does indeed match the
ETag.  Not sure if
> > there's anything we might do with other negotiated
headers, but maybe
> > we could leave that open, providing a hook that
something like
> > mod_negotiation might someday use.
>
>    I confess I've been following this discussion at an
abstract
> level only (at best).  I don't think, though, that it
directly affects
> any of the several bugs I'm hoping to tackle, if I
understand the
> issue correctly.  But I don't claim to fully understand
the details of
> mod_negiotiation.
>
> Chris.
>
> --
> GPG Key ID: 366A375B
> GPG Key Fingerprint: 485E 5041 17E1 E2BB C263  E4DE
C8E3 FA36 366A 375B
>
>