caching questions

I've been running with memcached caching for a couple of
days,
and it works like a charm (450+ hits per second on a
cheap-ass
server isn't all that bad .

however, there are two things that I haven't been able to
sort
out on my own:

- the admin site is cached too, which makes it a bit hard to
use.
it might be a middleware ordering problem, but I haven't
found a
working combination. any ideas?

- I would like to explicitly remove pages from the cache,
based
on the URL. is there some public API for this that I
haven't found ?

(I'm using the "stock" 0.91 release, btw)

cheers /F


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "Django users" group.
To post to this group, send email to django-usersgooglegroups.com
To unsubscribe from this group, send email to
django-users-unsubscribegooglegroups.com
For more options, visit this group at http://gr
oups.google.com/group/django-users
-~----------~----~----~----~------~----~------~--~---

anyone ?

> however, there are two things that I haven't been able
to sort
> out on my own:
>
> - the admin site is cached too, which makes it a bit
hard to use.
> it might be a middleware ordering problem, but I
haven't found a
> working combination. any ideas?
>
> - I would like to explicitly remove pages from the
cache, based
> on the URL. is there some public API for this that I
haven't found ?


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "Django users" group.
To post to this group, send email to django-usersgooglegroups.com
To unsubscribe from this group, send email to
django-users-unsubscribegooglegroups.com
For more options, visit this group at http://gr
oups.google.com/group/django-users
-~----------~----~----~----~------~----~------~--~---

On Mar 1, 2006, at 3:57 PM, fredrikpythonware.com wrote:

>
> anyone ?

I guess guys and gals are busy with PyCon Sprint these
couple days,  
so they couldn't pay much attention on the mailing list.
That's unusual.

Personally I haven't use cache in a production Django site
yet, but I  
will try to share my understandings.

>
>> however, there are two things that I haven't been
able to sort
>> out on my own:
>>
>> - the admin site is cached too, which makes it a
bit hard to use.
>> it might be a middleware ordering problem, but I
haven't found a
>> working combination. any ideas?

How about running the admin site with another process, say
another  
mod_python configuration, separated with the main site? This
way, you  
can have a separate settings for the admin site.

>>
>> - I would like to explicitly remove pages from the
cache, based
>> on the URL. is there some public API for this that
I haven't found ?

There is a section in the Cache API document called
"Controlling  
cache: Using Vary headers" (http://www.djangoprojec
t.com/ 
documentation/cache/#controlling-cache-using-vary-headers).
If you  
didn't check it out, it might be useful for you, although I
didn't  
have chance to use it yet.

Cheers
- Cheng Zhang


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "Django users" group.
To post to this group, send email to django-usersgooglegroups.com
To unsubscribe from this group, send email to
django-users-unsubscribegooglegroups.com
For more options, visit this group at http://gr
oups.google.com/group/django-users
-~----------~----~----~----~------~----~------~--~---

Hi Fredrik,

On Wed, 2006-03-01 at 07:57 +0000, fredrikpythonware.com wrote:
> anyone ?
> 
> > however, there are two things that I haven't been
able to sort
> > out on my own:
> >
> > - the admin site is cached too, which makes it a
bit hard to use.
> > it might be a middleware ordering problem, but I
haven't found a
> > working combination. any ideas?

Not going to touch this one; I'm not sure what is going on.

> > - I would like to explicitly remove pages from the
cache, based
> > on the URL. is there some public API for this that
I haven't found ?

Since you can control caching on a per-view-function basis,
is it
possible to set things up so that the URLs you do not wish
to cache are
served via a different view than the others (even if the
view is just a
wrapper around the common stuff)? If you can organise this,
then using
the cache-control decorator can set the caching mechanism
to "private",
which should achieve what you want. This is documented in
docs/cache.txt.

However, maybe I am not understanding what you are asking,
because you
are saying "remove pages from the cache"
(referring to the Django
implementation?), whereas I am answering "how not to
cache pages on an
HTTP level" (referring to the result going out to the
user). I am
worried that your question is precise enough that I am not
really
answering what is being asked, so feel free to post more
clues if you
think that will help.

Regards,
Malcolm


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "Django users" group.
To post to this group, send email to django-usersgooglegroups.com
To unsubscribe from this group, send email to
django-users-unsubscribegooglegroups.com
For more options, visit this group at http://gr
oups.google.com/group/django-users
-~----------~----~----~----~------~----~------~--~---

On 2/28/06, fredrikpythonware.com <fredrikpythonware.com> wrote:
> - the admin site is cached too, which makes it a bit
hard to use.
> it might be a middleware ordering problem, but I
haven't found a
> working combination. any ideas?

Ah, you must be using the per-site cache, right? I'm
actually not sure
how the per-site cache would interact with the admin site --
the POSTs
would work, but the GETs would probably be cached. Is that
what's
happening?

The best solution to this is to use per-view caches rather
than the
per-site cache; that way, you have more fine-grained control
over
which types of pages get cached.

> - I would like to explicitly remove pages from the
cache, based
> on the URL. is there some public API for this that I
haven't found ?

It's possible to interpret this question two ways, so I'll
answer both:

* If you're wanting to physically delete a page from the
cache, use
the low-level cache API:

from django.core.cache import cache
cache.set('views.decorators.cache.cache_page../some/url/.d4
1d8cd98f00b204e9800998ecf8427e',
None)

Note that the cache key contains "/some/url/",
which is the URL that
you're wanting to remove from the cache, and the string
"d41d8cd98f00b204e9800998ecf8427e", which is a
hash of "empty"
headers. That comes from django.utils.cache.get_cache_key(),
which
looks at all the headers that should be considered with
respect to
creating the cache key.
"d41d8cd98f00b204e9800998ecf8427e" is the hash
for empty headers. Does this make sense? We should probably
have a
simple helper function that clears the cache for a given
URL, assuming
no headers.

* If you want to prevent a particular view from being cached
(when
using site-wide cache middleware), there currently isn't a
way to do
that. The tricky problem there is that it would cause a
slight
performance decrease: Currently the cache middleware just
looks at
each incoming URL, checks the cache for the URL with that
key, and
returns the cached page if the key is found. If we were to
allow
certain views to be left out of the cache, the cache
middleware would
have to do URL dispatching to find out what the associated
view is,
then check the view to see whether it should use the cache,
then
finally check the cache. The extra overhead of URL
dispatching isn't a
huge deal, but it's still kind of counter to caching.
Thoughts on this
are welcome!

Adrian

--
Adrian Holovaty
holovaty.com | djangoproject.com

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "Django users" group.
To post to this group, send email to django-usersgooglegroups.com
To unsubscribe from this group, send email to
django-users-unsubscribegooglegroups.com
For more options, visit this group at http://gr
oups.google.com/group/django-users
-~----------~----~----~----~------~----~------~--~---

Adrian Holovaty wrote:

> > - the admin site is cached too, which makes it a
bit hard to use.
> > it might be a middleware ordering problem, but I
haven't found a
> > working combination. any ideas?
>
> Ah, you must be using the per-site cache, right? I'm
actually not sure
> how the per-site cache would interact with the admin
site -- the POSTs
> would work, but the GETs would probably be cached. Is
that what's
> happening?

yup.  the caching docs didn't say anything about this, so I
happily
assumed that you'd thought of this already 

> The best solution to this is to use per-view caches
rather than the
> per-site cache; that way, you have more fine-grained
control over
> which types of pages get cached.

to what extent does all the caching header magic
(Last-Modified,
Expires, etc) work when I use cache decorators?  do I have
to deal
with that myself, or is a

   cache_page(...)
   def view(...)

exactly equivalent to a plain view plus a full-site cache?

> > - I would like to explicitly remove pages from the
cache, based
> > on the URL. is there some public API for this that
I haven't found ?
>
> It's possible to interpret this question two ways, so
I'll answer both:

I accidentally sent a clarification to directly to Malcolm
(I'm not
entirely google-groups compatible, it seems, and
googlegroups
don't like me either, as can be seen from the ratings my
posts
usually get), and he's sorted out the basics for me
already.
(thanks, Malcolm!)

anyway,

> * If you're wanting to physically delete a page from
the cache, use
> the low-level cache API:

this is what I meant to ask.

> from django.core.cache import cache
>
cache.set('views.decorators.cache.cache_page../some/url/.d4
1d8cd98f00b204e9800998ecf8427e',
> None)
>
> Note that the cache key contains
"/some/url/", which is the URL that
> you're wanting to remove from the cache, and the
string
> "d41d8cd98f00b204e9800998ecf8427e", which
is a hash of "empty"
> headers. That comes from
django.utils.cache.get_cache_key(), which
> looks at all the headers that should be considered with
respect to
> creating the cache key.
"d41d8cd98f00b204e9800998ecf8427e" is the hash
> for empty headers. Does this make sense?

absolutely.

>>> md5.md5("").hexdigest()
'd41d8cd98f00b204e9800998ecf8427e'

> We should probably have a simple helper function that
clears the cache for
> a given URL, assuming no headers.

that would be the function I was looking for 

a "manage purgecache" command might also be
somewhat useful,
at least for people like me who likes to hack on a semi-live
server 

> * If you want to prevent a particular view from being
cached (when
> using site-wide cache middleware), there currently
isn't a way to do
> that. The tricky problem there is that it would cause a
slight
> performance decrease: Currently the cache middleware
just looks at
> each incoming URL, checks the cache for the URL with
that key, and
> returns the cached page if the key is found. If we were
to allow
> certain views to be left out of the cache, the cache
middleware would
> have to do URL dispatching to find out what the
associated view is,
> then check the view to see whether it should use the
cache, then
> finally check the cache. The extra overhead of URL
dispatching isn't a
> huge deal, but it's still kind of counter to caching.
Thoughts on this
> are welcome!

in the application servers I've built, the caching system
generally
uses
the following approach:

   1. look up incoming requests in the cache
   2. if not found, do a full request processing
   3. if the resulting response is flagged as uncachable,
don't update
the
      cache

in other words, cachability is a property of the response,
not the
view.

maybe the existing cache decorators can be used to implement
step 3
today ?

(the docs are not clear enough on the relation between the
cache
system, the caching middleware, and the cache decorators for
me
to be able to figure this out on my own...  do I have to
read the
source ? 

</F>


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "Django users" group.
To post to this group, send email to django-usersgooglegroups.com
To unsubscribe from this group, send email to
django-users-unsubscribegooglegroups.com
For more options, visit this group at http://gr
oups.google.com/group/django-users
-~----------~----~----~----~------~----~------~--~---

Have you seen this Fredrik?

http
://djangoutils.python-hosting.com/wiki/Utils


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "Django users" group.
To post to this group, send email to django-usersgooglegroups.com
To unsubscribe from this group, send email to
django-users-unsubscribegooglegroups.com
For more options, visit this group at http://gr
oups.google.com/group/django-users
-~----------~----~----~----~------~----~------~--~---