|
List Info
Thread: New functional tests and test cleanup
|
|
| New functional tests and test cleanup |
|
2007-01-23 18:04:11 |
|
| Nobody liked the old tests, they were cumbersome to write and
difficult to debug.
I wrote a new simple dav client, and added cosmo stuff for a simple
cosmoclient and that's what we wrote the migration testing stuff with.
Last week and this week I ported all the tests that were still
relevant from the old tools over to this one (all the mkcalendar and
reports tests). I also wrote some cmp tests that we've been seriously
lacking.
From my point of view the new toolset has the advantage of being
easier for me to write tests with and easier to debug by anyone. The
downside is that the tests aren't that readable to someone who isn't
fairly familiar with python -- but I'll be writing a wiki page with
notes on how use the tools and debug tests.
The new test tools use py.test for running the tests debugging,
filtering, etc.
We still have the HTTPTest dir in the cosmo source tree, even though
that tool, and all of those tests, are primarily maintained in the
tools repo, so I've removed it.
Since this the new tools and the migration script both use
cosmoclient, I put the migration script in with the new tests for
easier maintenance (since the filename does not start with "test"
py.test won't try to execute it along with the other tests.
I called the new test directory "protocol" since the focus of the
tools is on protocol testing in cosmo. /cosmo/src/test/functional/
protocol.
Since we're feature complete I didn't want to commit anything into
the cosmo source tree until all this had been brought to the list.
|
 Approximate file size 802546 bytes |
| Re: New functional tests and test
cleanup |
|
2007-01-23 16:21:11 |
|
On Tue, 2007-01-23 at 16:04 -0800, Mikeal Rogers wrote:
> Nobody liked the old tests, they were cumbersome to write and
> difficult to debug.
very true, I always cursed the testing suite for not being so modular -
which did not allow linking the test cases. At the basic I would atleast
like to create user objects/accounts for all the new test cases using a
single line.
+
+
+SERVER_URL = 'http://qacosmo.osafoundation.org:8080'
+ADMIN_USER = 'root'
+ADMIN_PASS = 'cosmo'
+
it would be great if we could move this to a common configuration
file..I know it has already been implemented, but a script to change all
the test suite scripts should be possible (instead of manually moving
this).
+TEST_USER = 'test_user_report%s' % str(random.random()).replace('.',
'')
+TEST_PASS = 'test_pass'
+TEST_FIRST_NAME = 'Test'
+TEST_LAST_NAME = 'User'
+TEST_EMAIL = TEST_USER+' osafoundation.org'
again it would be great to have just a simple way of creating a user
instead of declaring these many variables. (just one variable shud auto
populate the other variables)
> From my point of view the new toolset has the advantage of being
> easier for me to write tests with and easier to debug by anyone. The
> downside is that the tests aren't that readable to someone who isn't
> fairly familiar with python -- but I'll be writing a wiki page with
overall the new suite seems to be really nice  . to understand, we
just need an object oriented inclination! yes a wiki page will be
helpful..
_______________________________________________
cosmo-dev mailing list
cosmo-devlists.osafoundation.org
http://lists.osafoundation.org/mailman/listinfo/cosmo-dev
|
| Re: New functional tests and test
cleanup |
|
2007-01-23 16:21:11 |
|
| On Tue, 2007-01-23 at 16:04 -0800, Mikeal Rogers wrote:
> Nobody liked the old tests, they were cumbersome to write and
> difficult to debug.
very true, I always cursed the testing suite for not being so modular -
which did not allow linking the test cases. At the basic I would atleast
like to create user objects/accounts for all the new test cases using a
single line.
+
+
+SERVER_URL = 'http://qacosmo.osafoundation.org:8080'
+ADMIN_USER = 'root'
+ADMIN_PASS = 'cosmo'
+
it would be great if we could move this to a common configuration
file..I know it has already been implemented, but a script to change all
the test suite scripts should be possible (instead of manually moving
this).
+TEST_USER = 'test_user_report%s' % str(random.random()).replace('.',
'')
+TEST_PASS = 'test_pass'
+TEST_FIRST_NAME = 'Test'
+TEST_LAST_NAME = 'User'
+TEST_EMAIL = TEST_USER+'osafoundation.org'
again it would be great to have just a simple way of creating a user
instead of declaring these many variables. (just one variable shud auto
populate the other variables)
> From my point of view the new toolset has the advantage of being
> easier for me to write tests with and easier to debug by anyone. The
> downside is that the tests aren't that readable to someone who isn't
> fairly familiar with python -- but I'll be writing a wiki page with
overall the new suite seems to be really nice . to understand, we
just need an object oriented inclination! yes a wiki page will be
helpful..
_______________________________________________
cosmo-dev mailing list
cosmo-devlists.osafoundation.org
http://lists.osafoundation.org/mailman/listinfo/cosmo-dev
|
| Re: New functional tests and test
cleanup |
|
2007-01-25 12:33:59 |
> At the basic I would atleast
> like to create user objects/accounts for all the new
test cases
> using a
> single line.
>
> +
> +
> +SERVER_URL = 'http://qacosmo.
osafoundation.org:8080'
> +ADMIN_USER = 'root'
> +ADMIN_PASS = 'cosmo'
> +
> it would be great if we could move this to a common
configuration
> file..I know it has already been implemented, but a
script to
> change all
> the test suite scripts should be possible (instead of
manually moving
> this).
Eventually I'll write a little script that can override
module
defaults for every module, so if you wanted to change the
url or
adminuser/adminpass you could do that more easily. This will
become
more of an issue once we expand the tests, right now all the
old
modules have been moved in to only 3 test modules and 1 new
one is
there for CMP.
>
> +TEST_USER = 'test_user_report%s' %
str(random.random()).replace('.',
> '')
> +TEST_PASS = 'test_pass'
> +TEST_FIRST_NAME = 'Test'
> +TEST_LAST_NAME = 'User'
> +TEST_EMAIL = TEST_USER+'osafoundation.org'
>
> again it would be great to have just a simple way of
creating a user
> instead of declaring these many variables. (just one
variable shud
> auto
> populate the other variables)
Those declarations aren't for an existing user, they are for
the name
and options of the user that will be created at the
beginning of the
test module (and removed at the end), and each test module
creates a
different user. Since I use a fairly large random number at
the end
of each test user it's doubtful that we will ever hit a
conflict with
an existing user, so I don't know of a situation where you
would need
to change the defaults.
-Mikeal
_______________________________________________
cosmo-dev mailing list
cosmo-devlists.osafoundation.org
http://lists.osafoundation.org/mailman/listinfo/cosmo-d
ev
|
|
| Re: New functional tests and test
cleanup |
|
2007-01-25 12:33:59 |
|
| > At the basic I would atleast
> like to create user objects/accounts for all the new test cases
> using a
> single line.
>
> +
> +
> +SERVER_URL = 'http://qacosmo.osafoundation.org:8080'
> +ADMIN_USER = 'root'
> +ADMIN_PASS = 'cosmo'
> +
> it would be great if we could move this to a common configuration
> file..I know it has already been implemented, but a script to
> change all
> the test suite scripts should be possible (instead of manually moving
> this).
Eventually I'll write a little script that can override module
defaults for every module, so if you wanted to change the url or
adminuser/adminpass you could do that more easily. This will become
more of an issue once we expand the tests, right now all the old
modules have been moved in to only 3 test modules and 1 new one is
there for CMP.
>
> +TEST_USER = 'test_user_report%s' % str(random.random()).replace('.',
> '')
> +TEST_PASS = 'test_pass'
> +TEST_FIRST_NAME = 'Test'
> +TEST_LAST_NAME = 'User'
> +TEST_EMAIL = TEST_USER+'osafoundation.org'
>
> again it would be great to have just a simple way of creating a user
> instead of declaring these many variables. (just one variable shud
> auto
> populate the other variables)
Those declarations aren't for an existing user, they are for the name
and options of the user that will be created at the beginning of the
test module (and removed at the end), and each test module creates a
different user. Since I use a fairly large random number at the end
of each test user it's doubtful that we will ever hit a conflict with
an existing user, so I don't know of a situation where you would need
to change the defaults.
-Mikeal
_______________________________________________
cosmo-dev mailing list
cosmo-devlists.osafoundation.org
http://lists.osafoundation.org/mailman/listinfo/cosmo-dev
|
| Re: New functional tests and test
cleanup |
|
2007-01-25 19:10:50 |
|
| On Thu, 2007-01-25 at 10:33 -0800, Mikeal Rogers wrote:
>Those declarations aren't for an existing user, they are for the name
>and options of the user that will be created at the beginning of the
Yeah I know. My concern was to reduce the number of lines required to
write a new test case.
So instead of defining this in every test case:
TEST_USER = 'test_user_report%s' % str(random.random()).replace('.',
'')
TEST_PASS = 'test_pass'
TEST_FIRST_NAME = 'Test'
TEST_LAST_NAME = 'User'
TEST_EMAIL = TEST_USER+'osafoundation.org'
it would be nice to have a single declaration like
TEST_USER = 'test_user_acl%s' % str(random.random()).replace('.','') or
not even have that declaration since you use random numbers.
I am not sure on how this is handled but the following declaration in
test_mkcalendar.py confuses me.
TEST_USER = 'test_user_report%s' % str(random.random()).replace('.', '')
TEST_PASS = 'test_pass'
TEST_FIRST_NAME = 'Test'
TEST_LAST_NAME = 'User'
TEST_EMAIL = TEST_USER+'osafoundation.org'
CALENDAR = 'calendar'
FILES_DIR =
os.path.dirname(os.path.abspath(sys.modules[__name__].__file__))+'/files/'
I see that TEST_PASS is never used anywhere in test_mkcalendar.py -
unless it is used some other way (global variable, etc).
_______________________________________________
cosmo-dev mailing list
cosmo-devlists.osafoundation.org
http://lists.osafoundation.org/mailman/listinfo/cosmo-dev
|
| Re: New functional tests and test
cleanup |
|
2007-01-25 19:10:50 |
On Thu, 2007-01-25 at 10:33 -0800, Mikeal Rogers wrote:
>Those declarations aren't for an existing user, they are
for the name
>and options of the user that will be created at the
beginning of the
Yeah I know. My concern was to reduce the number of lines
required to
write a new test case.
So instead of defining this in every test case:
TEST_USER = 'test_user_report%s' %
str(random.random()).replace('.',
'')
TEST_PASS = 'test_pass'
TEST_FIRST_NAME = 'Test'
TEST_LAST_NAME = 'User'
TEST_EMAIL = TEST_USER+'osafoundation.org'
it would be nice to have a single declaration like
TEST_USER = 'test_user_acl%s' %
str(random.random()).replace('.','') or
not even have that declaration since you use random
numbers.
I am not sure on how this is handled but the following
declaration in
test_mkcalendar.py confuses me.
TEST_USER = 'test_user_report%s' %
str(random.random()).replace('.', '')
TEST_PASS = 'test_pass'
TEST_FIRST_NAME = 'Test'
TEST_LAST_NAME = 'User'
TEST_EMAIL = TEST_USER+'osafoundation.org'
CALENDAR = 'calendar'
FILES_DIR =
os.path.dirname(os.path.abspath(sys.modules[__name__].__file
__))+'/files/'
I see that TEST_PASS is never used anywhere in
test_mkcalendar.py -
unless it is used some other way (global variable, etc).
_______________________________________________
cosmo-dev mailing list
cosmo-devlists.osafoundation.org
http://lists.osafoundation.org/mailman/listinfo/cosmo-d
ev
|
|
| Re: New functional tests and test
cleanup |
|
2007-01-26 18:06:07 |
This was really good feedback, it made me realize that I
could get
rid of all the boiler plate.
Committed revision 3498.
Now once you understand how the tool suite works you can
write tests
with basically zero boilerplate.
Thanks,
-Mikeal
> Yeah I know. My concern was to reduce the number of
lines required to
> write a new test case.
>
> So instead of defining this in every test case:
> TEST_USER = 'test_user_report%s' %
str(random.random()).replace('.',
> '')
> TEST_PASS = 'test_pass'
> TEST_FIRST_NAME = 'Test'
> TEST_LAST_NAME = 'User'
> TEST_EMAIL = TEST_USER+'osafoundation.org'
>
> it would be nice to have a single declaration like
> TEST_USER = 'test_user_acl%s' %
str(random.random()).replace
> ('.','') or
> not even have that declaration since you use random
numbers.
>
> I am not sure on how this is handled but the following
declaration in
> test_mkcalendar.py confuses me.
>
> TEST_USER = 'test_user_report%s' %
str(random.random()).replace
> ('.', '')
> TEST_PASS = 'test_pass'
> TEST_FIRST_NAME = 'Test'
> TEST_LAST_NAME = 'User'
> TEST_EMAIL = TEST_USER+'osafoundation.org'
> CALENDAR = 'calendar'
> FILES_DIR =
>
os.path.dirname(os.path.abspath(sys.modules[__name__].__file
__))+'/
> files/'
>
> I see that TEST_PASS is never used anywhere in
test_mkcalendar.py -
> unless it is used some other way (global variable,
etc).
>
_______________________________________________
cosmo-dev mailing list
cosmo-devlists.osafoundation.org
http://lists.osafoundation.org/mailman/listinfo/cosmo-d
ev
|
|
| Re: New functional tests and test
cleanup |
|
2007-01-26 18:06:07 |
This was really good feedback, it made me realize that I
could get
rid of all the boiler plate.
Committed revision 3498.
Now once you understand how the tool suite works you can
write tests
with basically zero boilerplate.
Thanks,
-Mikeal
> Yeah I know. My concern was to reduce the number of
lines required to
> write a new test case.
>
> So instead of defining this in every test case:
> TEST_USER = 'test_user_report%s' %
str(random.random()).replace('.',
> '')
> TEST_PASS = 'test_pass'
> TEST_FIRST_NAME = 'Test'
> TEST_LAST_NAME = 'User'
> TEST_EMAIL = TEST_USER+'osafoundation.org'
>
> it would be nice to have a single declaration like
> TEST_USER = 'test_user_acl%s' %
str(random.random()).replace
> ('.','') or
> not even have that declaration since you use random
numbers.
>
> I am not sure on how this is handled but the following
declaration in
> test_mkcalendar.py confuses me.
>
> TEST_USER = 'test_user_report%s' %
str(random.random()).replace
> ('.', '')
> TEST_PASS = 'test_pass'
> TEST_FIRST_NAME = 'Test'
> TEST_LAST_NAME = 'User'
> TEST_EMAIL = TEST_USER+'osafoundation.org'
> CALENDAR = 'calendar'
> FILES_DIR =
>
os.path.dirname(os.path.abspath(sys.modules[__name__].__file
__))+'/
> files/'
>
> I see that TEST_PASS is never used anywhere in
test_mkcalendar.py -
> unless it is used some other way (global variable,
etc).
>
_______________________________________________
cosmo-dev mailing list
cosmo-devlists.osafoundation.org
http://lists.osafoundation.org/mailman/listinfo/cosmo-d
ev
|
|
[1-9]
|
|