Warnings in tests

There's a thread in perl-qa at the moment in which I've
argued that test
suites which ignore warnings being emitted by the code
they're testing
are broken, and that any unexpected warnings emitted during
testing
should be considered test failures.

As an experiment, I want to see if I can find a way of
promoting other
peoples' uncaught warnings to be fatal.  The obvious way is
to insert
this:
  $SIG = sub { die("Caught a warning, making
it fataln", _) }

into the environment in which their tests run.  Trouble is,
I want this
to happen when I use CPAN.pm to (try to) install their
stuff.  CPAN.pm
calls make(1) to run 'make test', which in turn calls 'perl
blahblah'
all in seperate processes, which internally use fork() and
exec() so I
can't just set that __WARN__ handler when I run the CPAN
shell.  It
needs to be inserted by the perl process that is exec()ed to
run the
tests.  Can anyone think of a way I can make it do that,
without editing
any code in the random modules I'm installing?

The only way I can think of is to replace my perl executable
with a
script that does ...

  #!/bin/sh
  /path/to/real/perl -MThingy::To::Make::Warnings::Fatal
"$"

but that's icky.  Can anyone think of a better way?

-- 
David Cantrell | A machine for turning tea into grumpiness

  engineer: n. one who, regardless of how much effort he
puts in
    to a job, will never satisfy either the suits or the
scientists

On 15/03/07, David Cantrell <davidcantrell.org.uk> wrote:
> The only way I can think of is to replace my perl
executable with a
> script that does ...
>
>   #!/bin/sh
>   /path/to/real/perl
-MThingy::To::Make::Warnings::Fatal "$"
>
> but that's icky.  Can anyone think of a better way?

use PERL5OPT ? (man perlrun)

Rafael Garcia-Suarez wrote:

> On 15/03/07, David Cantrell <davidcantrell.org.uk> wrote:
> > The only way I can think of is to replace my perl
executable with a
> > script that does ...
> >
> >   #!/bin/sh
> >   /path/to/real/perl
-MThingy::To::Make::Warnings::Fatal "$"
> >
> > but that's icky.  Can anyone think of a better
way?
>
> use PERL5OPT ? (man perlrun)

Problem is that it will be ignored when taint mode is
enabled, which is
often the case for tests (and is in my opinion more
important than one
or warnings).

--
Sébastien Aperghis-Tramoni

Close the world, txEn eht nepO.

On 15 Mar 2007, at 11:05, David Cantrell wrote:
[snip]
> The only way I can think of is to replace my perl
executable with a
> script that does ...
>
>   #!/bin/sh
>   /path/to/real/perl
-MThingy::To::Make::Warnings::Fatal "$"
>
> but that's icky.  Can anyone think of a better way?

setenv PERL5OPT -MThingy::To::Make::Warnings::Fatal

?

Adrian

On Thu, Mar 15, 2007 at 12:31:53PM +0100, Sébastien
Aperghis-Tramoni wrote:
> Rafael Garcia-Suarez wrote:
> 
> > On 15/03/07, David Cantrell <davidcantrell.org.uk> wrote:
> > > The only way I can think of is to replace my
perl executable with a
> > > script that does ...
> > >
> > >   #!/bin/sh
> > >   /path/to/real/perl
-MThingy::To::Make::Warnings::Fatal "$"
> > >
> > > but that's icky.  Can anyone think of a
better way?
> >
> > use PERL5OPT ? (man perlrun)
> 
> Problem is that it will be ignored when taint mode is
enabled, which is
> often the case for tests (and is in my opinion more
important than one
> or warnings).

I've not received the original mail yet, but assuming it
follows on from
the perl-qa thread, perhaps setting HARNESS_PERL_SWITCHES is
the answer.
Or, possibly better, adding to it.

-- 
Paul Johnson - paulpjcj.net
http://www.pjcj.net

On 15 Mar 2007, at 11:05, David Cantrell wrote:
> but that's icky.  Can anyone think of a better way?

For Test::Harness you can use HARNESS_PERL_SWITCHES to add -

MMy::UpgradeWarnings to the command line switches that are
passed to  
Perl. Untried but I think that should work.

TAP::Harness currently has a way defining a wrapper for
tests but the  
specifics of that mechanism are changing even as I type
this. Suffice  
it to say that it'll be possible with TAP::Harness too.

-- 
Andy Armstrong, hexten.net

On Thu, Mar 15, 2007 at 12:15:30PM +0100, Rafael
Garcia-Suarez wrote:

> use PERL5OPT ? (man perlrun)

I love you man.

-- 
David Cantrell | A machine for turning tea into grumpiness

    Liberté! Actualité! Une tasse de thé!