IPC::Cmd can lose stderr

# New Ticket Created by  Andy Dougherty 
# Please include the string:  [perl #46293]
# in the subject line of all future correspondence about
this issue. 
# <URL: h
ttp://rt.perl.org/rt3/Ticket/Display.html?id=46293 >


I'm trying to track down a problem where IPC::Cmd::run($cmd)
loses output 
when $cmd prints to stderr and doesn't print anything to
stdout.

I've traced the problem down to the following bit of code in

IPC::Cmd.pm.  In the sub _open3_run, there is the following
loop:


    ### add an epxlicit break statement
    ### code courtesy of theorbtwo from #bond.pm
    OUTER: while ( my ready = $selector->can_read ) {

        for my $h ( ready ) {
            my $buf;
            
            ### $len is the amount of bytes read
            my $len = sysread( $h, $buf, 4096 );    # try to
read 4096 bytes
            
            ### see perldoc -f sysread: it returns undef on
error,
            ### so bail out.
            if( not defined $len ) {
                warn(loc("Error reading from process:
%1", $!));
                last OUTER;
            }
            
            ### check for $len. it may be 0, at which point
we're
            ### done reading, so don't try to process it.
            ### if we would print anyway, we'd provide bogus
information
            $_out_handler->( "$buf" ) if $len
&& $h == $kidout;
            $_err_handler->( "$buf" ) if $len
&& $h == $kiderror;
            
            ### child process is done printing.
            last OUTER if $h == $kidout and $len == 0
        }
    }

    waitpid $pid, 0; # wait for it to die


Note how the loop exits (with "last") whenever the
stdout processing
($kidout) is complete, whether or not stderr processing is
complete.
In the case where ready = (kidout, kiderror) and $kidout is
empty, this
completely skips reading from $kiderror.  Yet from the
comment, this
appears to be deliberate!  I don't know why.  Does anyone?

My attempts to fix it to wait for both stdout and stderr to
be complete 
have led to failures in
lib/CPANPLUS/t/20_CPLANPLUS-Dist-MM.t (which
does its own, different, STDERR redirection).  I haven't yet
figured out
whether my fix the CPANPLUS test is in error.)

Anyway, does anyone have any suggestions?

-- 
    Andy Dougherty		dougheralafayette.edu

Andy Dougherty (via RT) <perlbug-followupperl.org> wrote:
:I'm trying to track down a problem where
IPC::Cmd::run($cmd) loses output 
:when $cmd prints to stderr and doesn't print anything to
stdout.
:
:I've traced the problem down to the following bit of code
in 
:IPC::Cmd.pm.  In the sub _open3_run, there is the
following
:loop:
[...]
:Note how the loop exits (with "last") whenever
the stdout processing
:($kidout) is complete, whether or not stderr processing is
complete.
:In the case where ready = (kidout, kiderror) and $kidout is
empty, this
:completely skips reading from $kiderror.  Yet from the
comment, this
:appears to be deliberate!  I don't know why.  Does anyone?

Looks like a thinko to me. But you could ask theorbtwo, I
think he is
still active at least on perlmonks.

:My attempts to fix it to wait for both stdout and stderr to
be complete 
:have led to failures in
lib/CPANPLUS/t/20_CPLANPLUS-Dist-MM.t (which
:does its own, different, STDERR redirection).  I haven't
yet figured out
:whether my fix the CPANPLUS test is in error.)

What was your fix? I think the easiest approach (in the
spirit of the
existing code) is to replace the C<last OUTER> with a
block that loops
to read any remaining data from STDERR before breaking out.

Hugo

On Wed, 10 Oct 2007, hvcrypt.org wrote:

> Andy Dougherty (via RT) <perlbug-followupperl.org> wrote:
> :I'm trying to track down a problem where
IPC::Cmd::run($cmd) loses output 
> :when $cmd prints to stderr and doesn't print anything
to stdout.
> :
> :I've traced the problem down to the following bit of
code in 
> :IPC::Cmd.pm.  In the sub _open3_run, there is the
following
> :loop:
> [...]
> :Note how the loop exits (with "last")
whenever the stdout processing
> :($kidout) is complete, whether or not stderr
processing is complete.
> :In the case where ready = (kidout, kiderror)
and $kidout is empty, this
> :completely skips reading from $kiderror.  Yet from the
comment, this
> :appears to be deliberate!  I don't know why.  Does
anyone?
> 
> Looks like a thinko to me. But you could ask theorbtwo,
I think he is
> still active at least on perlmonks.

But then I'd have to figure out how to use perlmonks myself.
 I imagine I 
probably have an old account somewhere . . . .
> 
> What was your fix? I think the easiest approach (in the
spirit of the
> existing code) is to replace the C<last OUTER>
with a block that loops
> to read any remaining data from STDERR before breaking
out.

I've attached my fix.  Essentially, the problem turns out to
be that 
stderr and stdout can arrive in any order.  One can be
completed before 
the other is even available.  Hence we just have to wait for
both of them.

--- perl-current/lib/IPC/Cmd.pm	2006-11-28
11:44:04.000000000 -0500
+++ perl-andy/lib/IPC/Cmd.pm	2007-10-11 08:19:44.000000000
-0400
 -437,6
+437,8 
 
     ### add an epxlicit break statement
     ### code courtesy of theorbtwo from #bond.pm
+    my $stdout_done = 0;
+    my $stderr_done = 0;
     OUTER: while ( my ready = $selector->can_read ) {
 
         for my $h ( ready ) {
 -457,9
+459,12 
             ### if we would print anyway, we'd provide
bogus information
             $_out_handler->( "$buf" ) if $len
&& $h == $kidout;
             $_err_handler->( "$buf" ) if $len
&& $h == $kiderror;
-            
-            ### child process is done printing.
-            last OUTER if $h == $kidout and $len == 0
+
+            ### Wait till child process is done printing to
both
+            ### stdout and stderr.
+            $stdout_done = 1 if $h == $kidout   and $len ==
0;
+            $stderr_done = 1 if $h == $kiderror and $len ==
0;
+            last OUTER if ($stdout_done &&
$stderr_done);
         }
     }
 
-- 
    Andy Dougherty		dougheralafayette.edu