Hi Ara,
I'll take a look. In the meantime, I've cc'd the
win32utils-devel
mailing list to see if anyone has any ideas/suggestions.
Regards,
Dan
> -----Original Message-----
> From: ara.t.howard noaa.gov [mailto:ara.t.howardnoaa.gov]
> Sent: Wednesday, November 01, 2006 12:36 PM
> To: Berger, Daniel
> Subject: fatal flaw in popen4 on windows? [WAS] Re:
> Nonblocking IO read (fwd)
>
>
>
> hi dan-
>
> forgot to cc you on this... hopefully you'll have some
suggestions?
>
> regards.
>
> -a
> --
> my religion is very simple. my religion is kindness.
-- the
> dalai lama
>
> ---------- Forwarded message ----------
> Date: Wed, 1 Nov 2006 12:19:12 -0700 (MST)
> From: ara.t.howardnoaa.gov
> To: ruby-talk ML <ruby-talkruby-lang.org>
> Subject: fatal flaw in popen4 on windows? [WAS] Re:
> Nonblocking IO read
>
>
> On Thu, 2 Nov 2006, Robert Klemme wrote:
>
> > Tom Pollard wrote:
> >> Anyway, you would only need nonblocking IO if
you wanted
> to read bits
> >> of the
> >> stderr stream before the command exited, but
that doesn't
> sound like what
> >> you're want.
> >
> > Actually this is not correct: if there is a lot
written to
> stderr then
> > you need to read that concurrently. If you do not
do that then the
> > process will block on some stderr write operation
that fills up the
> > pipe and you get a deadlock because your code
waits for process
> > termination.
>
> not only is that true but, afaik, it's why popen4
cannot even
> work on windows! this program will eventually hang on
either
> windows or unix
>
>
> harp:~ > cat a.rb
> require 'rubygems'
> require 'popen4'
>
> n = (ARGV.shift || 4242).to_i
> ruby = ARGV.shift || 'ruby'
>
> system "ruby -e 42" or abort "ruby
not in your path!"
>
> STDOUT.sync = STDERR.sync = true
>
> program = <<-program
> #.times do
> t = Time.now.to_f
> STDOUT.puts t
> STDERR.puts t
> end
> program
>
>
> POpen4.popen4(ruby) do |stdout, stderr, stdin,
pid|
> STDOUT.puts pid
>
> stdin.puts program
> stdin.close
>
> Thread.new{ stdout.each{|line| STDOUT.puts line}
}
> #
> # uncomment and it won't hang!!!
> #
> #Thread.new{ stderr.each{|line| STDERR.puts
line} }
> end
>
> puts 'done'
>
>
>
> however, on windows it will always hang - even if the
line
> above is uncommented. this is because if one popen4s a
> process it's __essential__, as robert correctly points
out,
> to continually consume any stdout or stderr produced -
> otherwise the program will eventually get stuck in
EPIPE and
> you'll be waiting for this stuck program.
>
> and here's the rub: you cannot reliably consume both
stdout
> and stderr under windows using threads or, afaik,
nonblocking
> io. perhaps the new nonblock_* methods could help with
this?
> it'd no doubt be a major reworking...
>
> *** i'm really hoping someone will chime in here and
prove me
> wrong ***
>
> for reference i'm including the code for my open4 lib's
spawn
> method: which illustrates the logical concept of what
must be
> done to avoid a subprocess blocked writing to it's
parent's pipes...
>
>
> #
> # for complete code see
> # http://codeforpeople.com/lib/ruby/open4/open4-0.
9.1/lib/open4.rb
> # http://rubyforge.org/frs/?group_id=1024&release_i
d=7556
> #
>
> def spawn arg, *argv #--{{{
> argv.unshift(arg)
> opts = ((argv.size > 1 and Hash === argv.last)
? argv.pop : {})
> argv.flatten!
> cmd = argv.join(' ')
>
> getopt = getopts opts
>
> ignore_exit_failure = getopt[
'ignore_exit_failure',
> getopt['quiet', false]
> ]
> ignore_exec_failure = getopt[
'ignore_exec_failure',
> !getopt['raise', true]
> ]
> exitstatus = getopt[ %w( exitstatus exit_status
status ) ]
> stdin = getopt[ %w( stdin in i 0 ) << 0 ]
> stdout = getopt[ %w( stdout out o 1 ) << 1 ]
> stderr = getopt[ %w( stderr err e 2 ) << 2 ]
> pid = getopt[ 'pid' ]
> timeout = getopt[ %w( timeout spawn_timeout ) ]
> stdin_timeout = getopt[ %w( stdin_timeout ) ]
> stdout_timeout = getopt[ %w( stdout_timeout
io_timeout ) ]
> stderr_timeout = getopt[ %w( stderr_timeout ) ]
> status = getopt[ %w( status ) ]
> cwd = getopt[ %w( cwd dir ), Dir.pwd ]
>
> exitstatus =
> case exitstatus
> when TrueClass, FalseClass
> ignore_exit_failure = true if exitstatus
> [0]
> else
> [*(exitstatus || 0)].map{|i| Integer i}
> end
>
> stdin ||= '' if stdin_timeout
> stdout ||= '' if stdout_timeout
> stderr ||= '' if stderr_timeout
>
> started = false
>
> status =
> begin
> Dir.chdir(cwd) do
> Timeout::timeout(timeout) do
> popen4(*argv) do |c, i, o, e|
> started = true
>
> %w( replace pid= << push update
).each do |msg|
> break(pid.send(msg, c)) if
pid.respond_to? msg
> end
>
> #
> # this is the critical bit!!!
> #
>
> te = ThreadEnsemble.new c
>
> te.add_thread(i, stdin) do |i, stdin|
> relay stdin, i, stdin_timeout
> i.close rescue nil
> end
>
> te.add_thread(o, stdout) do |o, stdout|
> relay o, stdout, stdout_timeout
> end
>
> te.add_thread(e, stderr) do |o, stderr|
> relay e, stderr, stderr_timeout
> end
>
> te.run
> end
> end
> end
> rescue
> raise unless(not started and
ignore_exec_failure)
> end
>
> raise SpawnError.new(cmd, status) unless
> (ignore_exit_failure or (status.nil? and
> ignore_exec_failure) or
> exitstatus.include?(status.exitstatus))
>
> status
> #--}}}
> end
>
>
> kind regards.
>
> -a
> --
> my religion is very simple. my religion is kindness.
-- the
> dalai lama
>
> -a
> --
> my religion is very simple. my religion is kindness.
-- the
> dalai lama
>
This communication is the property of Qwest and may contain
confidential or
privileged information. Unauthorized use of this
communication is strictly
prohibited and may be unlawful. If you have received this
communication
in error, please immediately notify the sender by reply
e-mail and destroy
all copies of the communication and any attachments.
_______________________________________________
win32utils-devel mailing list
win32utils-develrubyforge.org
http://rubyforge.org/mailman/listinfo/win32utils-devel
|