Archive::Extract test failures on Solaris

With a current blead-perl snapshot, I'm getting the
following errors in 
make test on Solaris 8:

Failed Test                               Stat Wstat Total
Fail  List of Failed
------------------------------------------------------------
-------------------
../lib/Archive/Extract/t/01_Archive-Extra    9  2304   885  
 9  267-268 271
                                                            
    278-279 282
                                                            
    699-700 703

Running the test individually, I see:

$ cd t
$ ./perl -I../lib harness -v
../lib/Archive/Extract/t/01_Archive-Extract.t

#   Failed test 'Found correct output file 'y/z''
#   at ../lib/Archive/Extract/t/01_Archive-Extract.t line
339.
#          got: 'tar: blocksize = 0'
#     expected: 'y/z'

#   Failed test 'Output file
'/home/doughera/src/perl/perl-andy/lib/Archive/Extract/t/out
/y/z' exists'
#   at ../lib/Archive/Extract/t/01_Archive-Extract.t line
342.

#   Failed test 'Extract dir is expected
'/home/doughera/src/perl/perl-andy/lib/Archive/Extract/t/out
/y''
#   at ../lib/Archive/Extract/t/01_Archive-Extract.t line
348.
#          got:
'/home/doughera/src/perl/perl-andy/lib/Archive/Extract/t/out
'
#     expected:
'/home/doughera/src/perl/perl-andy/lib/Archive/Extract/t/out
/y'

#   Failed test 'Found correct output file 'y/z''
#   at ../lib/Archive/Extract/t/01_Archive-Extract.t line
339.
#          got: 'tar: blocksize = 0'
#     expected: 'y/z'

#   Failed test 'Output file
'/home/doughera/src/perl/perl-andy/lib/Archive/Extract/t/out
/y/z' exists'
#   at ../lib/Archive/Extract/t/01_Archive-Extract.t line
342.

#   Failed test 'Extract dir is expected
'/home/doughera/src/perl/perl-andy/lib/Archive/Extract/t/out
/y''
#   at ../lib/Archive/Extract/t/01_Archive-Extract.t line
348.
#          got:
'/home/doughera/src/perl/perl-andy/lib/Archive/Extract/t/out
'
#     expected:
'/home/doughera/src/perl/perl-andy/lib/Archive/Extract/t/out
/y'

#   Failed test 'Found correct output file 'y/z''
#   at ../lib/Archive/Extract/t/01_Archive-Extract.t line
339.
#          got: 'tar: blocksize = 0'
#     expected: 'y/z'

#   Failed test 'Output file
'/home/doughera/src/perl/perl-andy/lib/Archive/Extract/t/out
/y/z' exists'
#   at ../lib/Archive/Extract/t/01_Archive-Extract.t line
342.

#   Failed test 'Extract dir is expected
'/home/doughera/src/perl/perl-andy/lib/Archive/Extract/t/out
/y''
#   at ../lib/Archive/Extract/t/01_Archive-Extract.t line
348.
#          got:
'/home/doughera/src/perl/perl-andy/lib/Archive/Extract/t/out
'
#     expected:
'/home/doughera/src/perl/perl-andy/lib/Archive/Extract/t/out
/y'
# Looks like you failed 9 tests of 885.
Failed 1/1 test programs. 9/885 subtests failed.

Note all the 'tar: blocksize = 0' messages.

Does anybody have any idea how to get the test to tell me in
more detail
what it's trying to do (so I can figure out what the problem
is?

Thanks,


-- 
    Andy Dougherty		dougheralafayette.edu

On Sep 13, 2007, at 11:43 PM, Andy Dougherty wrote:

> With a current blead-perl snapshot, I'm getting the
following  
> errors in
> make test on Solaris 8:
>
> Failed Test                               Stat Wstat
Total Fail   
> List of Failed
>
------------------------------------------------------------
---------- 
> ---------
> ../lib/Archive/Extract/t/01_Archive-Extra    9  2304  
885    9   
> 267-268 271
>                                                        
          
> 278-279 282
>                                                        
          
> 699-700 703
>
> Running the test individually, I see:
>
> $ cd t
> $ ./perl -I../lib harness -v
../lib/Archive/Extract/t/01_Archive- 
> Extract.t

[...]

> Does anybody have any idea how to get the test to tell
me in more  
> detail
> what it's trying to do (so I can figure out what the
problem is?

I'm sure the author does 

run it as follows:

  $ ./perl -I../lib harness -v
../lib/Archive/Extract/t/01_Archive- 
Extract.t 1

The extra '1' enables debug info... what probably happens is
that  
your /bin/tar
doesn't provide output the way we expect it on Solaris.

Let me know if i can help more.

Cheers,

--

	Jos Boumans

	How do I prove I am not crazy to people who are?

Hi,

On 14 Sep 2007, at 13:49, Andy Dougherty wrote:
> I think I finally tracked down the problem:  This
version of bzip2  
> won't
> uncompress a file unless it ends in .bz2.  Two possible
solutions are
> to rename the file or read it from stdin.

attached is the patch for Archive::Extract 0.24. It
addresses the  
above issue,
as well as the VMS issues that John M. reported.

For completeness sake, here's the CHANGES entries:

Changes for 0.24    Thu Sep 20 15:05:55 2007
============================================
* Older versions of /bin/bunzip2 only support archives
   with a .bz2 suffix. Attempt to detect this with a new
   method ->have_old_bunzip2, and skip tests
accordingly.
   Also document this in the CAVEATS section of the docs.
* Apply core change 31904: Clean up test files left
   behind (on Win32, at least).

Changes for 0.23_01 Sun Sep  9 12:39:08 2007
============================================
* Address Update ticket #27991 (Archive::Extract and
   very large tar.gz files). Add a CAVEATS entry explianing
   how to deal with large files & memory consumption
* VMS shell out & test fixes by John Malmberg

--
Jos I. Boumans                 RIPE Network Coordination
Center
Database Group Manager         Singel 258 Amsterdam NL
http://www.ripe.net       
    +31 20 535 4444

On 20/09/2007, Jos I. Boumans <kanedwim.org> wrote:
>
> attached is the patch for Archive::Extract 0.24. It
addresses the
> above issue,
> as well as the VMS issues that John M. reported.

Thanks, applied as #31928.

On Thu, 20 Sep 2007, Rafael Garcia-Suarez wrote:

> On 20/09/2007, Jos I. Boumans <kanedwim.org> wrote:
> >
> > attached is the patch for Archive::Extract 0.24.
It addresses the
> > above issue,
> > as well as the VMS issues that John M. reported.
> 
> Thanks, applied as #31928.

I should note that it doesn't actually work, however.  The
problem seems 
to be that after this command is run:

     scalar run( command => [$self->bin_bunzip2,
'--version'],
             verbose => 0,
             buffer  => $buffer
        );

$buffer is actually empty.  (Hence all the tests afterwards
don't work
as expected.) This may be a bug of some sort in IPC::Cmd. 
I'm looking
into it.

Keep this patch for now.

-- 
    Andy Dougherty		dougheralafayette.edu

On Thu, 20 Sep 2007, Jos I. Boumans wrote:

> 
> On Sep 20, 2007, at 4:49 PM, Andy Dougherty wrote:
> 
> > On Thu, 20 Sep 2007, Rafael Garcia-Suarez wrote:
> > 
> > > On 20/09/2007, Jos I. Boumans <kanedwim.org> wrote:
> > > > 
> > > > attached is the patch for
Archive::Extract 0.24. It addresses the
> > > > above issue,
> > > > as well as the VMS issues that John M.
reported.
> > > 
> > > Thanks, applied as #31928.
> > 
> > I should note that it doesn't actually work,
however.  The problem seems
> > to be that after this command is run:
> > 
> >     scalar run( command =>
[$self->bin_bunzip2, '--version'],
> >             verbose => 0,
> >             buffer  => $buffer
> >        );
> > 
> > $buffer is actually empty.  (Hence all the tests
afterwards don't work
> > as expected.) This may be a bug of some sort in
IPC::Cmd.  I'm looking
> > into it.
> > 
> > Keep this patch for now.
> 
> If that's true, you either have a very crazy bunzip2,
or IPC::Open3
> doesn't work... We've tested it with pjcj's Solaris 8,
and there it
> did what we expected...

I'm suspecting IPC::Open3.  Note that I'm testing on a very
clean machine 
as part of the 5.10.0-to-be distribution where there is no
IPC::Run 
available and where the default 'perl' on the system is
5.005_03.  (I have 
many different versions available elsewhere on this machine,
but for 
testing purposes, I tend to keep them very isolated.)

I agree this code ought to work.  It's using IPC::Cmd as
documented, and 
IPC::Cmd is reporting that it can_capture_buffer (though it
later changes 
its mind and reports that it can't capture_buffer).

I'll try to reduce it to a smaller test case, but it'll take
some time.

-- 
    Andy Dougherty		dougheralafayette.edu

On Sep 20, 2007, at 4:49 PM, Andy Dougherty wrote:

> On Thu, 20 Sep 2007, Rafael Garcia-Suarez wrote:
>
>> On 20/09/2007, Jos I. Boumans <kanedwim.org> wrote:
>>>
>>> attached is the patch for Archive::Extract
0.24. It addresses the
>>> above issue,
>>> as well as the VMS issues that John M.
reported.
>>
>> Thanks, applied as #31928.
>
> I should note that it doesn't actually work, however. 
The problem  
> seems
> to be that after this command is run:
>
>      scalar run( command => [$self->bin_bunzip2,
'--version'],
>              verbose => 0,
>              buffer  => $buffer
>         );
>
> $buffer is actually empty.  (Hence all the tests
afterwards don't work
> as expected.) This may be a bug of some sort in
IPC::Cmd.  I'm looking
> into it.
>
> Keep this patch for now.

If that's true, you either have a very crazy bunzip2, or
IPC::Open3
doesn't work... We've tested it with pjcj's Solaris 8, and
there it
did what we expected...

--

Jos Boumans

	"If superman is so smart, why does he wear underpants
over his 	
         trousers?"

	CPANPLUS	http://cpanplus.sf.net

On Thu, 20 Sep 2007, Andy Dougherty wrote:

> On Thu, 20 Sep 2007, Jos I. Boumans wrote:
> 
> > 
> > On Sep 20, 2007, at 4:49 PM, Andy Dougherty
wrote:
> > 
> > > On Thu, 20 Sep 2007, Rafael Garcia-Suarez
wrote:
> > > 
> > > > On 20/09/2007, Jos I. Boumans
<kanedwim.org> wrote:
> > > > > 
> > > > > attached is the patch for
Archive::Extract 0.24. It addresses the
> > > > > above issue,
> > > > > as well as the VMS issues that John
M. reported.
> > > > 
> > > > Thanks, applied as #31928.
> > > 
> > > I should note that it doesn't actually work,
however.  The problem seems
> > > to be that after this command is run:
> > > 
> > >     scalar run( command =>
[$self->bin_bunzip2, '--version'],
> > >             verbose => 0,
> > >             buffer  => $buffer
> > >        );
> > > 
> > > $buffer is actually empty.  (Hence all the
tests afterwards don't work
> > > as expected.) This may be a bug of some sort
in IPC::Cmd.  I'm looking
> > > into it.
> > > 
> > > Keep this patch for now.
> > 
> > If that's true, you either have a very crazy
bunzip2, or IPC::Open3
> > doesn't work... We've tested it with pjcj's
Solaris 8, and there it
> > did what we expected...
> 
> I'm suspecting IPC::Open3.  Note that I'm testing on a
very clean machine 
> as part of the 5.10.0-to-be distribution where there is
no IPC::Run 
> available and where the default 'perl' on the system is
5.005_03.  (I have 
> many different versions available elsewhere on this
machine, but for 
> testing purposes, I tend to keep them very isolated.)
> 
> I agree this code ought to work.  It's using IPC::Cmd
as documented, and 
> IPC::Cmd is reporting that it can_capture_buffer
(though it later changes 
> its mind and reports that it can't capture_buffer).
> 
> I'll try to reduce it to a smaller test case, but it'll
take some time.

It's a bug in IPC somewhere. I've included a smaller test
case below.
If you have a command that prints only to stderr, then
IPC::Cmd run()
doesn't reliably capture the stderr output.  (It gets it
intermittently
-- about 50% of the time in my trials.)  If the command
prints to stdout
as well, then both the stderr and the stdout seem to be
captured 100%
of the time.

I suspect a buffering/flushing of STDERR issue.  (Solaris
doesn't
autoflush upon forking.)  In this test program, if I make
the message
short (e.g. 61 characters in the example below) then this
test passes
every time under Linux, but fails about 50% of the time for
me on
Solaris 8.  If I make the message longer than about 13,000
characters or
so, it succeeds every time on Solaris 8 and Debian Linux,
but fails 
on Linux (Fedora Core 7) (the buffer usually only has 12,288
characters).

Very strange!

As a first step, a test case printing just to stderr
probably ought to be
added to the IPC::Cmd test suite.

#!./perl
BEGIN { INC = './lib'; }
use strict;

use IPC::Cmd  qw[run can_run];

die "Can't use IPC::Open3" unless
IPC::Cmd->can_use_ipc_open3(1);
die "Can't capture buffer" unless
IPC::Cmd->can_capture_buffer;

my ($rtn, $err, $full_buf, $stdout_buff, $stderr_buff);
my $buffer;

my $msg = "A" x 60 . "n";
my $cmd = "perl -e 'print STDERR
"$msg";'";

($rtn, $err, $full_buf, $stdout_buff, $stderr_buff) = 
    run( command => $cmd, verbose => 0, buffer =>
$buffer);

if ($buffer eq $msg) {
    print "Okn" ;
}
else {
    print "Not ok.n";
    print "length($msg) = ", length($msg), 
          " length($buffer) = ", length($buffer),
"n";
}

-- 
    Andy Dougherty		dougheralafayette.edu