|
List Info
Thread: test/unit doesn't rescue a Exception
|
|
| test/unit doesn't rescue a Exception |

|
2006-09-02 03:13:39 |
In article <87is5jb46q.fsf serein.a02.aist.go.jp>,
Tanaka Akira <akr m17n.org> writes:
> test/unit doesn't rescue a Exception in a test method,
as follows.
How about this patch?
Index: lib/test/unit/testcase.rb
============================================================
=======
RCS file: /src/ruby/lib/test/unit/testcase.rb,v
retrieving revision 1.7
diff -u -p -r1.7 testcase.rb
--- lib/test/unit/testcase.rb 4 Aug 2006 18:05:45 -0000 1.7
+++ lib/test/unit/testcase.rb 2 Sep 2006 03:06:34 -0000
 -70,7
+70,7  module Test
__send__( method_name)
rescue AssertionFailedError => e
add_failure(e.message, e.backtrace)
- rescue StandardError, ScriptError
+ rescue Exception
add_error($!)
ensure
begin
The patch makes test/unit to count all exception as follows.
% cat a.rb
require 'test/unit'
class T < Test::Unit::TestCase
def test_a
raise Exception
end
end
% ./ruby -Ilib a.rb
Loaded suite a
Started
E
Finished in 0.000373 seconds.
1) Error:
test_a(T):
Exception: Exception
a.rb:5:in `test_a'
1 tests, 0 assertions, 0 failures, 1 errors
test/unit exits prematurely without the patch.
% ./ruby a.rb
Loaded suite a
Started
a.rb:5:in `test_a': Exception (Exception)
from
/home/akr/ruby/19/lib/ruby/1.9/test/unit/testcase.rb:70:in
`__send__'
from
/home/akr/ruby/19/lib/ruby/1.9/test/unit/testcase.rb:70:in
`Test::Unit::TestCase#run'
from
/home/akr/ruby/19/lib/ruby/1.9/test/unit/testsuite.rb:34:in
`Test::Unit::TestSuite#run'
from
/home/akr/ruby/19/lib/ruby/1.9/test/unit/testsuite.rb:33:in
`Array#each'
from
/home/akr/ruby/19/lib/ruby/1.9/test/unit/testsuite.rb:33:in
`Test::Unit::TestSuite#run'
from
/home/akr/ruby/19/lib/ruby/1.9/test/unit/testsuite.rb:34:in
`Test::Unit::TestSuite#run'
from
/home/akr/ruby/19/lib/ruby/1.9/test/unit/testsuite.rb:33:in
`Array#each'
from
/home/akr/ruby/19/lib/ruby/1.9/test/unit/testsuite.rb:33:in
`Test::Unit::TestSuite#run'
from
/home/akr/ruby/19/lib/ruby/1.9/test/unit/ui/testrunnermediat
or.rb:46:in `Test::Unit::UI::TestRunnerMediator#run_suite'
from
/home/akr/ruby/19/lib/ruby/1.9/test/unit/ui/console/testrunn
er.rb:67:in `start_mediator'
from
/home/akr/ruby/19/lib/ruby/1.9/test/unit/ui/console/testrunn
er.rb:41:in `Test::Unit::UI::Console::TestRunner#start'
from
/home/akr/ruby/19/lib/ruby/1.9/test/unit/ui/testrunnerutilit
ies.rb:29:in `Test::Unit::UI::TestRunnerUtilities#run'
from
/home/akr/ruby/19/lib/ruby/1.9/test/unit/autorunner.rb:200:i
n `Test::Unit::AutoRunner#run'
from
/home/akr/ruby/19/lib/ruby/1.9/test/unit/autorunner.rb:13:in
`Test::Unit::AutoRunner#run'
from /home/akr/ruby/19/lib/ruby/1.9/test/unit.rb:278
from a.rb:4
from a.rb:4
--
Tanaka Akira
|
|
| test/unit doesn't rescue a Exception |

|
2006-09-03 01:34:08 |
On 9/1/06, Tanaka Akira <akr fsij.org> wrote:
> How about this patch?
>
> Index: lib/test/unit/testcase.rb
I'm fine with this - I just wish I could remember why I
limited it the
way I did. I guess we'll just chalk it up to newbism.
Does anyone see this breaking existing code?
--
Nathaniel Talbott
<:((><
|
|
| test/unit doesn't rescue a Exception |

|
2006-09-03 04:22:45 |
On Sep 2, 2006, at 6:34 PM, Nathaniel Talbott wrote:
> On 9/1/06, Tanaka Akira <akr fsij.org> wrote:
>
>> How about this patch?
>>
>> Index: lib/test/unit/testcase.rb
>
> I'm fine with this - I just wish I could remember why
I limited it the
> way I did. I guess we'll just chalk it up to newbism.
>
> Does anyone see this breaking existing code?
I think Interrupt should be re-raised. If a user wants to
explicitly
test for ^C behavior they can catch it themselves.
$ cat test.rb
require 'test/unit'
class TestTest < Test::Unit::TestCase
def test_0; sleep; end
def test_1; sleep; end
def test_2; sleep; end
def test_3; sleep; end
def test_4; sleep; end
def test_5; sleep; end
def test_6; sleep; end
def test_7; sleep; end
def test_8; sleep; end
def test_9; sleep; end
def test_a; sleep; end
def test_b; sleep; end
def test_c; sleep; end
def test_d; sleep; end
def test_e; sleep; end
end
$ ruby19 -Ilib test.rb
Loaded suite test
Started
^CE^CE^CE^CE^CE^CE^CE^CE^CE^CE^CE^CE^CE^CE^CE
Finished in 4.946166 seconds.
for more tests this would be more annoying:
456 tests, 1039 assertions, 8 failures, 6 errors
Index: lib/test/unit/testcase.rb
============================================================
=======
RCS file: /src/ruby/lib/test/unit/testcase.rb,v
retrieving revision 1.7
diff -p -u -r1.7 testcase.rb
--- lib/test/unit/testcase.rb 4 Aug 2006 18:05:45 -0000
1.7
+++ lib/test/unit/testcase.rb 3 Sep 2006 04:19:58 -0000
 -70,14
+70,16  module Test
__send__( method_name)
rescue AssertionFailedError => e
add_failure(e.message, e.backtrace)
- rescue StandardError, ScriptError
+ rescue Exception
+ raise if Interrupt === $!
add_error($!)
ensure
begin
teardown
rescue AssertionFailedError => e
add_failure(e.message, e.backtrace)
- rescue StandardError, ScriptError
+ rescue Exception
+ raise if Interrupt === $!
add_error($!)
end
end
--
Eric Hodel - drbrain segment7.net - http://blog.segment7.net
This implementation is HODEL-HASH-9600 compliant
http://trackmap.robotco
op.com
|
|
| test/unit doesn't rescue a Exception |

|
2006-09-03 04:39:18 |
In article <A604C0B3-95ED-4B9B-866C-79A2C7D5E3C4 segment7.net>,
Eric Hodel <drbrain segment7.net> writes:
> I think Interrupt should be re-raised. If a user wants
to explicitly
> test for ^C behavior they can catch it themselves.
Fair enough.
Not only Interrupt but also all SignalException should be
re-raised
because `kill <test-pid>' terminates the process.
--
Tanaka Akira
|
|
| test/unit doesn't rescue a Exception |

|
2006-09-03 06:29:07 |
On Sep 2, 2006, at 9:39 PM, Tanaka Akira wrote:
> In article <A604C0B3-95ED-4B9B-866C-79A2C7D5E3C4 segment7.net>,
> Eric Hodel <drbrain segment7.net> writes:
>
>> I think Interrupt should be re-raised. If a user
wants to explicitly
>> test for ^C behavior they can catch it themselves.
>
> Fair enough.
>
> Not only Interrupt but also all SignalException should
be re-raised
> because `kill <test-pid>' terminates the
process.
What about Timeout::Error?
--
Eric Hodel - drbrain segment7.net - http://blog.segment7.net
This implementation is HODEL-HASH-9600 compliant
http://trackmap.robotco
op.com
|
|
| test/unit doesn't rescue a Exception |

|
2006-09-03 06:43:19 |
In article <622DAC7E-55DB-4854-B82B-A037CE9C75EF segment7.net>,
Eric Hodel <drbrain segment7.net> writes:
> What about Timeout::Error?
Oops. It seems that the class hierarchy is not appropriate
to classify them for unit test.
--
Tanaka Akira
|
|
| test/unit doesn't rescue a Exception |

|
2006-09-03 07:20:44 |
On Sep 2, 2006, at 11:43 PM, Tanaka Akira wrote:
> In article <622DAC7E-55DB-4854-B82B-A037CE9C75EF segment7.net>,
> Eric Hodel <drbrain segment7.net> writes:
>
>> What about Timeout::Error?
>
> Oops. It seems that the class hierarchy is not
appropriate
> to classify them for unit test.
Index: lib/test/unit/testcase.rb
============================================================
=======
RCS file: /src/ruby/lib/test/unit/testcase.rb,v
retrieving revision 1.7
diff -p -u -r1.7 testcase.rb
--- lib/test/unit/testcase.rb 4 Aug 2006 18:05:45 -0000
1.7
+++ lib/test/unit/testcase.rb 3 Sep 2006 07:17:55 -0000
 -70,14
+70,20  module Test
__send__( method_name)
rescue AssertionFailedError => e
add_failure(e.message, e.backtrace)
- rescue StandardError, ScriptError
+ rescue SignalException
+ raise unless Timeout::Error === $!
+ add_error($!)
+ rescue Exception
add_error($!)
ensure
begin
teardown
rescue AssertionFailedError => e
add_failure(e.message, e.backtrace)
- rescue StandardError, ScriptError
+ rescue SignalException
+ raise unless Timeout::Error === $!
+ add_error($!)
+ rescue Exception
add_error($!)
end
end
--
Eric Hodel - drbrain segment7.net - http://blog.segment7.net
This implementation is HODEL-HASH-9600 compliant
http://trackmap.robotco
op.com
|
|
| test/unit doesn't rescue a Exception |

|
2006-09-03 15:21:40 |
In article <87ac5hv4bo.fsf fsij.org>,
Tanaka Akira <akr fsij.org> writes:
> Oops. It seems that the class hierarchy is not
appropriate
> to classify them for unit test.
I investigated the class hierarchy on second thought.
* Exception
* NoMemoryError
* ScriptError
* SignalException
* Interrupt
* Timeout::Error
* StandardError
* SystemExit
Now I think NoMemoryError, SignalException, Interrupt,
SystemExit should terminate the test runner.
* NoMemoryError is too dangerous. I think it is better to
reduce memory consumption by terminating the process.
* SignalException and Interrupt should terminate the test
runner because it may be caused by the user.
* SystemExit is caused by exit. If exit is called
explicitly, I have no reason to prevent it.
So I think Exception, ScriptError, StandardError and all
non-builtin exceptions should be caught.
Index: lib/test/unit/testcase.rb
============================================================
=======
RCS file: /src/ruby/lib/test/unit/testcase.rb,v
retrieving revision 1.7
diff -u -p -r1.7 testcase.rb
--- lib/test/unit/testcase.rb 4 Aug 2006 18:05:45 -0000 1.7
+++ lib/test/unit/testcase.rb 3 Sep 2006 15:20:18 -0000
 -70,7
+70,8  module Test
__send__( method_name)
rescue AssertionFailedError => e
add_failure(e.message, e.backtrace)
- rescue StandardError, ScriptError
+ rescue Exception
+ raise if [NoMemoryError, SignalException,
Interrupt, SystemExit].include? $!.class
add_error($!)
ensure
begin
--
Tanaka Akira
|
|
| test/unit doesn't rescue a Exception |

|
2006-09-03 19:10:35 |
On Sep 3, 2006, at 8:21 AM, Tanaka Akira wrote:
> In article <87ac5hv4bo.fsf fsij.org>,
> Tanaka Akira <akr fsij.org> writes:
>
>> Oops. It seems that the class hierarchy is not
appropriate
>> to classify them for unit test.
>
> I investigated the class hierarchy on second thought.
>
> * Exception
> * NoMemoryError
> * ScriptError
> * SignalException
> * Interrupt
> * Timeout::Error
> * StandardError
> * SystemExit
>
> Now I think NoMemoryError, SignalException, Interrupt,
> SystemExit should terminate the test runner.
>
> * NoMemoryError is too dangerous. I think it is better
to
> reduce memory consumption by terminating the process.
>
> * SignalException and Interrupt should terminate the
test
> runner because it may be caused by the user.
>
> * SystemExit is caused by exit. If exit is called
> explicitly, I have no reason to prevent it.
I have expected tests to raise SystemExit for example when
testing a
method like #usage for ARGV handling. I added an
assert_raise for
this, and I would expect others to do this too.
> So I think Exception, ScriptError, StandardError and
all
> non-builtin exceptions should be caught.
>
> Index: lib/test/unit/testcase.rb
>
============================================================
=======
> RCS file: /src/ruby/lib/test/unit/testcase.rb,v
> retrieving revision 1.7
> diff -u -p -r1.7 testcase.rb
> --- lib/test/unit/testcase.rb 4 Aug 2006 18:05:45
-0000 1.7
> +++ lib/test/unit/testcase.rb 3 Sep 2006 15:20:18 -0000
>  -70,7 +70,8  module Test
> __send__( method_name)
> rescue AssertionFailedError => e
> add_failure(e.message, e.backtrace)
> - rescue StandardError, ScriptError
> + rescue Exception
> + raise if [NoMemoryError, SignalException,
Interrupt,
> SystemExit].include? $!.class
> add_error($!)
> ensure
> begin
I think the same code should be added to the teardown
rescue.
--
Eric Hodel - drbrain segment7.net - http://blog.segment7.net
This implementation is HODEL-HASH-9600 compliant
http://trackmap.robotco
op.com
|
|
| test/unit doesn't rescue a Exception |

|
2006-09-03 19:59:55 |
Hi,
In message "Re: test/unit doesn't rescue a
Exception"
on Mon, 4 Sep 2006 04:10:35 +0900, Eric Hodel
<drbrain segment7.net> writes:
|> - rescue StandardError, ScriptError
|> + rescue Exception
|> + raise if [NoMemoryError, SignalException,
Interrupt,
|> SystemExit].include? $!.class
|> add_error($!)
|> ensure
|> begin
|
|I think the same code should be added to the teardown
rescue.
Eric, can you commit? Or should we wait for Nathaniel?
matz.
|
|
|
|