On 4/17/07, Steve Hay <steve.hay uk.radan.com> wrote:
> Jerry D. Hedden wrote:
> > Jerry D. Hedden wrote:
> >> Also, should the test be skipped/TODOed under
Win32 until
> >> the problem is fixed?
> >
> > Yves wrote::
> >> We should todo it, although maybe only under
smoke testing.
> >>
> >> Use an environment variable. Something like
> >> WIN32_SMOKE_SKIP_THREADS or something. People
who need it
> >> can set it up in a wrapper script.
> >
> > Patch attached. If $ENV{'WIN32_SMOKE'} is true,
then the
> > offending tests will be set TODO.
>
> I'm not sure that it makes sense to only mark a test as
TODO when run
> under Test-Smoke.
>
> Surely any test that is known to fail should be marked
as TODO
> regardless of how the test is being run.
My feeling is that we should assertively test as much as we
can, and
since the test is known not to fail from the command line on
Win32 I
dont think it should be TODO in that case.
If it starts failing then its not a matter of
"same-old, same-old" but
an indication things have gotten worse and we/I really
should know
about it.
OTOH, sending in FAIL every time for smoking because of a
well known
issue IMO deracts from the value of having smokes at all.
Maybe smoke itself should have some facilities to handle
this. Maybe a
status line of "SUCCESS (with expected failures)"
or something like
that.
Anyway, if the ENV var being checked is less smoke specific
in name
then it doesnt matter. Ill just set up my smoke batch files
to set the
var whatever it is.
Yves
--
perl -Mre=debug -e "/just|another|perl|hacker/"
|