Django app initialisation

I've got a situation where one of my django apps needs to
perform some
initialisation that includes processing the models from all
the
installed apps -- and it needs its own models to have been
initialised
also.  Because of this, I can't do my initialization when
the app's
__init__ module is loaded, because it may be too soon for
other models
to have been loaded, and in any case, its own models have
not been
loaded.

The solution I came up with was to hook the request_started
signal, do
the initialisation then, and unhook the signal if
initialisation
completed successfully.  A corollary of this is that the
initialisation will only happen when running a server, and
not (for
example) when running a shell, or tests, or otherwise using
the django
framework.  The code I'm using for this is below; I define
an
initialise() function in my app's __init__ module and then
call
prepare_app(initialise).

Is there a better way to perform application-specific
initialisation,
after all apps and models have been loaded, but before
other
processing begins?

Andrew.



from django.dispatch import dispatcher
from django.core.signals import request_started

class InitialisationHook(object):
    """Self-disconnecting signal
hook."""
    def __init__(self, init_func, dispatcher_args):
        self.init_func = init_func
        self.dispatcher_args = dispatcher_args

    def __call__(self):
        try:
            self.init_func()
        except:
            raise
        else:
            # Since initialisation succeeded, don't listen
for more
requests
            dispatcher.disconnect(self,
**self.dispatcher_args)

def prepare_app(init_func):
    """
    Initialise the current app by calling ``init_func``
immediately
before
    the first request. If an exception is raised by
init_func, it will
be
    propagated through to the handler (usually causing a 500
error),
and
    initialisation will be attempted on the next request.
    """

    # Connect the hook, ensuring that the dispatcher keeps a
reference
to it
    dispatcher_args = dict(signal=request_started,
weak=False)
    dispatcher.connect(InitialisationHook(init_func,
dispatcher_args),
**dispatcher_args)


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "Django users" group.
To post to this group, send email to django-usersgooglegroups.com
To unsubscribe from this group, send email to
django-users-unsubscribegooglegroups.com
For more options, visit this group at htt
p://groups.google.com/group/django-users?hl=en
-~----------~----~----~----~------~----~------~--~---

On Wed, 2007-04-04 at 10:10 +0000, Andrew Durdin wrote:
> I've got a situation where one of my django apps needs
to perform some
> initialisation that includes processing the models from
all the
> installed apps -- and it needs its own models to have
been initialised
> also.  Because of this, I can't do my initialization
when the app's
> __init__ module is loaded, because it may be too soon
for other models
> to have been loaded, and in any case, its own models
have not been
> loaded.
> 
> The solution I came up with was to hook the
request_started signal, do
> the initialisation then, and unhook the signal if
initialisation
> completed successfully.  A corollary of this is that
the
> initialisation will only happen when running a server,
and not (for
> example) when running a shell, or tests, or otherwise
using the django
> framework.  The code I'm using for this is below; I
define an
> initialise() function in my app's __init__ module and
then call
> prepare_app(initialise).
> 
> Is there a better way to perform application-specific
initialisation,
> after all apps and models have been loaded, but before
other
> processing begins?

I cannot think of any reliable way to solve this. It's a
reasonably hard
problem to solve correctly, apparently. I say this because
we currently
are *not* solving it in django/db/models/loading.py when we
try to
initialise a cache of all apps and models. The difficulty is
nested
imports and telling when a module has been fully imported.
Our current
solution works most of the time, but that just makes the
occasional
problem case that much more intrusive.

[Having mentioned loading.py, I would caution against trying
to hook
your processing into the initialisation of the cache. When I
get a
chance I'm going to declare the current approach we have
there a loser
and teach the cache how to handle "partially
initialised" state so that
it tries to fill itself in a bit more if there is a miss.]

Regards,
Malcolm



--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "Django users" group.
To post to this group, send email to django-usersgooglegroups.com
To unsubscribe from this group, send email to
django-users-unsubscribegooglegroups.com
For more options, visit this group at htt
p://groups.google.com/group/django-users?hl=en
-~----------~----~----~----~------~----~------~--~---

On Apr 4, 12:26 pm, Malcolm Tredinnick <malc...pointy-stick.com>
wrote:
>
> I cannot think of any reliable way to solve this. It's
a reasonably hard
> problem to solve correctly, apparently. I say this
because we currently
> are *not* solving it in django/db/models/loading.py
when we try to
> initialise a cache of all apps and models. The
difficulty is nested
> imports and telling when a module has been fully
imported. Our current
> solution works most of the time, but that just makes
the occasional
> problem case that much more intrusive.

As it turns out, my "solution" described above
fails when deployed and
not using runserver -- as the signal from the first request
is sent
before my app has had a chance to hook it.

But back to the problem -- would it not be possible for
django to
import all installed apps (the __init__.py, and subsequently
(if
present) models.py) during its own setup, and make sure
they're cached
(e.g. in sys.modules)*?  Any app could then do independent
initialisation in its __init__.py module.   Django could
then send out
a signal to indicate that it's finished loading apps &
models, and
anything interested could respond to that (having hooked it
in
__init__.py).  Could this be a workable approach, do you
think?

[*] Using __init__.py is not at present a good solution, as
it can be
executed more than once during django's startup by various
__import__() calls.

Andrew


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "Django users" group.
To post to this group, send email to django-usersgooglegroups.com
To unsubscribe from this group, send email to
django-users-unsubscribegooglegroups.com
For more options, visit this group at htt
p://groups.google.com/group/django-users?hl=en
-~----------~----~----~----~------~----~------~--~---

On Tue, 2007-04-17 at 08:48 -0700, Andrew Durdin wrote:
> On Apr 4, 12:26 pm, Malcolm Tredinnick <malc...pointy-stick.com>
> wrote:
> >
> > I cannot think of any reliable way to solve this.
It's a reasonably hard
> > problem to solve correctly, apparently. I say this
because we currently
> > are *not* solving it in
django/db/models/loading.py when we try to
> > initialise a cache of all apps and models. The
difficulty is nested
> > imports and telling when a module has been fully
imported. Our current
> > solution works most of the time, but that just
makes the occasional
> > problem case that much more intrusive.
> 
> As it turns out, my "solution" described
above fails when deployed and
> not using runserver -- as the signal from the first
request is sent
> before my app has had a chance to hook it.
> 
> But back to the problem -- would it not be possible for
django to
> import all installed apps (the __init__.py, and
subsequently (if
> present) models.py) during its own setup, and make sure
they're cached
> (e.g. in sys.modules)*?  Any app could then do
independent
> initialisation in its __init__.py module.   Django
could then send out
> a signal to indicate that it's finished loading apps
& models, and
> anything interested could respond to that (having
hooked it in
> __init__.py).  Could this be a workable approach, do
you think?
> 
> [*] Using __init__.py is not at present a good
solution, as it can be
> executed more than once during django's startup by
various
> __import__() calls.

The problem is that "import all apps" is not a
simple operation and has
lots of unexpected (at least until you get used to expecting
the
unexpected) side-effects. Nested imports are the main cause.


The problem is that if application A imports something from
application
B and application B is also in the list of INSTALLED_APPS
(which it
would have to be in almost all cases), whilst the import of
B is first
being done, the sys.modules entry for A will be empty (it
will have a
key of "A" and a value of None) so that multiple
imports don't happen.
So anything run when B is imported cannot assume the all the
other apps
have been fully imported. Cyclic import sequences and even
bushy import
hierarchies are lots of fun (read "really
painful") to handle. I'm
currently convinced there are some cases which aren't
possible to
completely untangle in loading.py, but that may be because
I'm not
bright enough.

However, don't let my negative experiences scare you off.
Write a
solution and then we can borrow it for loading.py.

Regards,
Malcolm


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "Django users" group.
To post to this group, send email to django-usersgooglegroups.com
To unsubscribe from this group, send email to
django-users-unsubscribegooglegroups.com
For more options, visit this group at htt
p://groups.google.com/group/django-users?hl=en
-~----------~----~----~----~------~----~------~--~---