Testing/Fixtures/Production/TestRunner

Good day,

I'm running into some confusion on the intended way to
architect tests/
users/etc. I've been playing with the latest SVN snapshot.

I've currently got my test cases setup around:

 1) Setup Custom Users, Groups,Permissions that are used for
all test
cases
 2) Change out my authentication middleware (I use CAS for
production
but local accounts work just fine for   unittesting)
 3) Add testcases that aren't related to
INSTALLED_APPLICATIONS
 4) unittest.TestCase()s

I've been adding most data manually for testing but would
like to move
towards using fixtures. Seems like a good job for
django.test.TestCase

Issues I've encountered:

 1)  initial_data has bitten me not realizing it was synced
in
production each time.  I think that initial_data should only
get
inserted if you have an empty new database.  Perhaps things
synced
each time should be called "constant_data".  The
fixtures
documentation in testing does say it gets synched each time
but also
says its purpose is to populate the initial empty new
database.

2) Setting up users/groups/users manually in the runner gets
blown
away by the TestCase architecture.  Works as documented. 
However, I'm
not sure where I should put things that should happen
post-each syncdb
for testing only.  Should there be a testrunner function
that gets
called post each sync or should I create a single TestCase
derived
from django.test.TestCase that does the
"pre-setup"?

3) DateTime serialization won't work if the date is before
1901 - I
have a bug/real life data set where I get some bogus dates
out of
another system.   This is really a python problem.


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "Django users" group.
To post to this group, send email to django-usersgooglegroups.com
To unsubscribe from this group, send email to
django-users-unsubscribegooglegroups.com
For more options, visit this group at htt
p://groups.google.com/group/django-users?hl=en
-~----------~----~----~----~------~----~------~--~---

On 8/9/07, Chris Green <greencmgmail.com> wrote:
>
>  1)  initial_data has bitten me not realizing it was
synced in
> production each time.  I think that initial_data should
only get
> inserted if you have an empty new database.  Perhaps
things synced
> each time should be called "constant_data". 
The fixtures
> documentation in testing does say it gets synched each
time but also
> says its purpose is to populate the initial empty new
database.

There is an argument to be made that initial_data for a
model should
only be loaded once - when the model is added to the
database. I can't
say I've been bitten by this problem myself, but I'm happy
to
entertain discussion/patches that address the issue.

> 2) Setting up users/groups/users manually in the runner
gets blown
> away by the TestCase architecture.  Works as
documented.  However, I'm
> not sure where I should put things that should happen
post-each syncdb
> for testing only.  Should there be a testrunner
function that gets
> called post each sync or should I create a single
TestCase derived
> from django.test.TestCase that does the
"pre-setup"?

I'm not sure I follow. TestCase goes out of its way to
avoid
interfering with setUp/tearDown methods, and in a TestCase,
there is a
complete flush after each test, so any setup would need to
be rerun on
a per-test basis, not a per-sync basis. Can you provide an
example of
the problem?

> 3) DateTime serialization won't work if the date is
before 1901 - I
> have a bug/real life data set where I get some bogus
dates out of
> another system.   This is really a python problem.

Again, if you can provide an example, it would help.
However, if the
problem is with Python itself, there's not a great deal we
can do.

Yours,
Russ Magee %-)

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "Django users" group.
To post to this group, send email to django-usersgooglegroups.com
To unsubscribe from this group, send email to
django-users-unsubscribegooglegroups.com
For more options, visit this group at htt
p://groups.google.com/group/django-users?hl=en
-~----------~----~----~----~------~----~------~--~---

On Aug 9, 7:55 am, "Russell Keith-Magee"
<freakboy3...gmail.com>
wrote:
> On 8/9/07, Chris Green <gree...gmail.com> wrote:

> >  1)  initial_data has bitten me not realizing it
was synced in
> > production each time.

> There is an argument to be made that initial_data for a
model should
> only be loaded once - when the model is added to the
database. I can't
> say I've been bitten by this problem myself, but I'm
happy to
> entertain discussion/patches that address the issue.

I'll look into creating such a patch. The test case was
having some
"basic templates" to choose from but then allowing
users to edit those
templates to meet their own prefs.  The syncdb to add
additional apps
killed the modified templates.


> > 2) Setting up users/groups/users manually in the
runner gets blown
> > away by the TestCase architecture.  Works as
documented.  However, I'm
> > not sure where I should put things that should
happen post-each syncdb
> > for testing only.  Should there be a testrunner
function that gets
> > called post each sync or should I create a single
TestCase derived
> > from django.test.TestCase that does the
"pre-setup"?
>
> I'm not sure I follow. TestCase goes out of its way to
avoid
> interfering with setUp/tearDown methods, and in a
TestCase, there is a
> complete flush after each test, so any setup would need
to be rerun on
> a per-test basis, not a per-sync basis. Can you provide
an example of
> the problem?

Sure: My runner simplified

The suggestion is I need to move setup_auth() to my test
methods.
That does seem to be the right way to do it so I can have
different
perms across each item.

def permission_finder(s):
    """ Given a string in the format
app_label.codename, return the
exact permission """
    parts = s.split('.')
    if len(parts) != 2:
        raise UnknownPermission, "Unable to parse
permission string
%s"%  s

    m,c = parts

    model_perms =
Permission.objects.filter(content_type__app_label__exact=m)

    if not model_perms:
        raise UnknownPermission, "No such model
permissions: %s" % m

    perms = model_perms.filter(codename__exact=c)

    if not perms:
        raise UnknownPermission, "No permissions found
for %s.%s" %
(m,c)

    return perms

def setup_auth():
         # Helpdesk Perms
         helpdesk_perms =
permission_finder('helpdesk.view_logs')

         # Level 2 Perms
         l2_perms = (helpdesk_perms,
 
permission_finder('helpdesk.reset_passwords'))

        l1group = Group(name='Level1 Support')
        l1group.save()

        l2group = Group(name='Level2 Support')
        l2group.save()

        for each in l2_perms:
            assert(each)
            for p in each:
                print "adding %s to %s" % (p,g1)
                g1.permissions.add(p)

        u1 = User.objects.create_user('level1user', 'l1example.com',
'password')
        u1.save()
        u1.groups.add(g1)

def run_tests(...):
    setup_middleware() # get rid of cas for Unit Testing
    setup_test_environment()
    [ Build the test suite ]
    old_name = settings.DATABASE_NAME
    create_test_db(verbosity,autoclobber=True)
    setup_auth()
    unittest.TextTestRunner(verbosity=verbosity).run(suite)
    destroy_test_db(old_name, verbosity)
    teardown_test_environment()





>
> > 3) DateTime serialization won't work if the date
is before 1901 - I
> > have a bug/real life data set where I get some
bogus dates out of
> > another system.   This is really a python
problem.
>
> Again, if you can provide an example, it would help.
However, if the
> problem is with Python itself, there's not a great deal
we can do.
>

class DateModel(models.Model):
         time = models.DateTime()

dm = DateModel(time=datetime.datetime(1,1,1))

Since strptime() breaks with dates before 1901, it gets
serialized out
with dumpdata but fails on a loaddata()


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "Django users" group.
To post to this group, send email to django-usersgooglegroups.com
To unsubscribe from this group, send email to
django-users-unsubscribegooglegroups.com
For more options, visit this group at htt
p://groups.google.com/group/django-users?hl=en
-~----------~----~----~----~------~----~------~--~---

On 8/11/07, Chris Green <greencmgmail.com> wrote:
>
> On Aug 9, 7:55 am, "Russell Keith-Magee"
<freakboy3...gmail.com>
> wrote:
> > On 8/9/07, Chris Green <gree...gmail.com> wrote:

> > > 2) Setting up users/groups/users manually in
the runner gets blown
> > > away by the TestCase architecture.  Works as
documented.  However, I'm
> > > not sure where I should put things that
should happen post-each syncdb
> > > for testing only.  Should there be a
testrunner function that gets
> > > called post each sync or should I create a
single TestCase derived
> > > from django.test.TestCase that does the
"pre-setup"?
> >
> > I'm not sure I follow. TestCase goes out of its
way to avoid
> > interfering with setUp/tearDown methods, and in a
TestCase, there is a
> > complete flush after each test, so any setup would
need to be rerun on
> > a per-test basis, not a per-sync basis. Can you
provide an example of
> > the problem?
>
> Sure: My runner simplified

Ah - I didn't catch on that you were using a customized test
runner.
Understood now.

As for how to fix this problem, I have two suggestions.

1) Are you sure you can't just fix this with a static
fixture? I can
see that you are using python code to generate the custom
permissions,
but are the resulting permissions actually dynamic? Couldn't
you just
build them one, capture them, and add them as an 'test_auth'
fixture
that you specify each time?

2) Alternatively, have you considered using a syncdb
'post_sync'
trigger? When Django loads an application, it looks for
management.py,
and loads it if it is available. This provides an entry
point to
listen for the 'post_sync' signal.

This is how Django installs its permissions to begin with.
In
management.py, you install a listener on the post_sync
message that is
emitted as part of the synchronization process. This signal
is
re-emitted as part of a flush (which is what happens at the
start of
each test). Look at django.contrib.auth.management for an
example of
how to set this up.

> Since strptime() breaks with dates before 1901, it gets
serialized out
> with dumpdata but fails on a loaddata()

Ok - this looks to be a python problem (although I'm seeing
the issue
with strftime, not strptime). It's worth logging as a ticket
so that
it isn't forgotten.

Although this is a python problem, it is a python problem
born of a
desire to solve Y2K issues by supporting '04' as 1904. Since
we are
only using dates on the backend and in serialization, we can
impose
more rigorous datetime formatting constraints, so we might
be able to
solve this problem by using an alternate strftime/strptime
implementation that requires 4 digit years. If you want to
try your
hand at this, feel free.

Yours,
Russ Magee %-)

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "Django users" group.
To post to this group, send email to django-usersgooglegroups.com
To unsubscribe from this group, send email to
django-users-unsubscribegooglegroups.com
For more options, visit this group at htt
p://groups.google.com/group/django-users?hl=en
-~----------~----~----~----~------~----~------~--~---

[ Sorry for long delay,  django is one of my "also
projects" ]

On Aug 11, 11:15 pm, "Russell Keith-Magee"
<freakboy3...gmail.com>
wrote:
> As for how to fix this problem, I have two
suggestions.
>
> 1) Are you sure you can't just fix this with a static
fixture?

Yes, that should work just fine.  The reason I added them
dynamically
is when I was rebuilding the models and playing around a
lot, it
seemed easier to add them by name.

> 2) Alternatively, have you considered using a syncdb
'post_sync'
> trigger? When Django loads an application, it looks for
management.py,
> and loads it if it is available. This provides an entry
point to
> listen for the 'post_sync' signal.

That's cute.  I just learned of that framework the other day
in the
django master slides

> > Since strptime() breaks with dates before 1901, it
gets serialized out
> > with dumpdata but fails on a loaddata()
>
> Ok - this looks to be a python problem (although I'm
seeing the issue
> with strftime, not strptime). It's worth logging as a
ticket so that
> it isn't forgotten.
>
> Although this is a python problem, it is a python
problem born of a
> desire to solve Y2K issues by supporting '04' as 1904.
Since we are
> only using dates on the backend and in serialization,
we can impose
> more rigorous datetime formatting constraints, so we
might be able to
> solve this problem by using an alternate
strftime/strptime
> implementation that requires 4 digit years. If you want
to try your
> hand at this, feel free.


I'll enter a ticket for that one;  I can probably pull
something
together as it bites me often enough on a system I have to
interoperate with.


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "Django users" group.
To post to this group, send email to django-usersgooglegroups.com
To unsubscribe from this group, send email to
django-users-unsubscribegooglegroups.com
For more options, visit this group at htt
p://groups.google.com/group/django-users?hl=en
-~----------~----~----~----~------~----~------~--~---