RE: using make to run binaries

Hi Dave,

> > We have got a problem using 'make' to start
binaries: we use 'make' not
> > only for compilation but also for testing our
compiled binaries on input
> > data like this:
> > # make output.data
> > to create a line like this:
> > # /path/to/binary -i input.data [more args] >
output.data
> > 
> > The problem is that output.data differs from the
output obtained by the
> > very same command line when run manually. So the
binary is the same, the
> > command line args are the same, too. What could be
the problem? I also
> > already tried environment variables, but they are
the same, too.
> 
> 1.  Are you /completely/ sure there isn't some other
version of the program in
> your PATH that might be getting in the way of one of
your tests but not the
> other?

The program is called always with the absolute path like in
the example
above.

> 2.  How did you check the environment vars were the
same? 
I compared the environment vars at the start of the program.
I modified
the program to output the current vars to cerr. Apart from
three vars
set by make (concerning the make level) there was no
difference compared to manual launch.

> Are you certain you're using the same shell as make
uses? 
Dunno, how can I check this?

> Does the problem occur even at the command-line if you
prefix your command with 'sh -c'?
Positive, problem remains.

> 3.  In what way do the output files differ?  Is it
something the program is
> supposed to do?
We do predictions on a set of chemical compounds stored in
the input
file. The XML output reports those predictions. The output
is then
further processed by perl scripts. The difference is that
the XML output
created by manual launch contains more chemical information.
We use a
chemical library (openbabel) which is loaded as a shared
library by our
program so it could be that two different libraries are
used. However,
this can be ruled out as we have only one single version of
the library
on our host. Also, the problem could be reproduced on
another host.

> 4.  (I'm staking my 50 qatloos on this one  Are you
in a different working
> directory when you try the command-line test as
compared to when you run it
> under make?  Are there different input.data files in
those different
> directories?  Are you looking by accident at the wrong
output.data file?
Sorry, but this is also not an option  We had
this now literally
dozens of times, checking it over and over again and it
differs
*always*.

Greets, Andreas


_______________________________________________
Help-make mailing list
Help-makegnu.org
http:
//lists.gnu.org/mailman/listinfo/help-make

On 17 September 2007 15:22, andreas wrote:

>>> We have got a problem using 'make' to start
binaries: we use 'make' not
>>> only for compilation but also for testing our
compiled binaries on input
>>> data like this: # make output.data
>>> to create a line like this:
>>> # /path/to/binary -i input.data [more args]
> output.data
>>> 
>>> The problem is that output.data differs from
the output obtained by the
>>> very same command line when run manually.

> The program is called always with the absolute path
like in the example
> above.

  OK, just checking that you did so in your manual
commandline tests also.

>> 2.  How did you check the environment vars were the
same?
> I compared the environment vars at the start of the
program. I modified
> the program to output the current vars to cerr. Apart
from three vars
> set by make (concerning the make level) there was no
difference compared to
> manual launch. 

  Right, that seems conclusive.

>> Are you certain you're using the same shell as make
uses? Dunno, how can I
>> check this? 

  Make will invoke your program using 'sh -c', unless you
have set SHELL in
your environment, in which case it uses that.  So it depends
on whether you
have SHELL defined, and whether 'sh' works out to be the
same shell you use
for command-line testing.  (e.g. sh is bash on most linux
systems, if you're
using ksh or zsh for testing it may well behave
differently).

>> 3.  In what way do the output files differ?  Is it
something the program is
>> supposed to do?
> We do predictions on a set of chemical compounds stored
in the input
> file. The XML output reports those predictions. The
output is then
> further processed by perl scripts. The difference is
that the XML output
> created by manual launch contains more chemical
information. We use a
> chemical library (openbabel) which is loaded as a
shared library by our
> program so it could be that two different libraries are
used. However,
> this can be ruled out as we have only one single
version of the library
> on our host. Also, the problem could be reproduced on
another host.
> 
>> 4.  (I'm staking my 50 qatloos on this one  Are you
in a different
>> working 
>> directory when you try the command-line test as
compared to when you run it
>> under make?  Are there different input.data files
in those different
>> directories?  Are you looking by accident at the
wrong output.data file?
> Sorry, but this is also not an option  We had
this now literally
> dozens of times, checking it over and over again and it
differs
> *always*.

  Ok, forgive me for asking, I just wanted to be really sure
that all the
possibilities for simple errors were checked and excluded. 
Unfortunately,
they are, which only leaves us with the complicated errors.

  The thing is, make doesn't do anything particularly
bizarre when it launches
an executable.  It takes one shortcut: if it's a simple
command-line, with no
redirection or pipes or other complex shell features, it
just fork-and-execs
the new application directly.  However, you've got an output
redirection
there, which triggers make to do things "the slow
way"; that is, it builds an
actual command line (by prefixing "sh -c" to your
commands from the make rule)
and invokes it with a call to system().

  So, if you're really sure you're invoking the same
executable in both cases,
and that the environment is the same in both cases, and the
working directory,
and the command-line options, and the input files, you have
a very tricky
problem to solve.  The situation smells like undefined
behaviour to me; if
you're using an uninitialised variable or in some other way
depending on
random memory contents, that could trigger the different
behaviour in your
program.  Your description of the problem is a bit vague,
but by "more
chemical information", I'm guessing the library is
either returning more
records, or more data fields in each record.  You might yet
be able to find
out what's going on by putting strace-alike wrappers around
the calls to the
library functions, see if you can verify that the program is
driving the
library in the same way with the same calls to the same
functions passing the
same arguments in each case.


    cheers,
      DaveK
-- 
Can't think of a witty .sigline today....



_______________________________________________
Help-make mailing list
Help-makegnu.org
http:
//lists.gnu.org/mailman/listinfo/help-make

Dave Korn wrote:
>   So, if you're really sure you're invoking the same
executable in both cases,
> and that the environment is the same in both cases, and
the working directory,
> and the command-line options, and the input files, you
have a very tricky
> problem to solve.  The situation smells like undefined
behaviour to me; if
> you're using an uninitialised variable or in some other
way depending on
> random memory contents, that could trigger the
different behaviour in your
> program.  Your description of the problem is a bit
vague, but by "more
> chemical information", I'm guessing the library is
either returning more
> records, or more data fields in each record.  You might
yet be able to find
> out what's going on by putting strace-alike wrappers
around the calls to the
> library functions, see if you can verify that the
program is driving the
> library in the same way with the same calls to the same
functions passing the
> same arguments in each case.
>
>   
This technique has never failed me: take the command

	/path/to/binary -i input.data [more args] > output.data

and change it to

	echo /path/to/binary -i input.data [more args]; $(SHELL)
-i

Then run make. This will of course print the command you
plan to run and leave you in an interactive shell with the
same exact environment[*], cwd, etc in which the command
would have run. Now you can run the command via
cut-and-paste and debug it with all the tools at your
disposal. At the very least this should take make out of the
equation; at this point you should have two windows running
two interactive shells, producing different output and
neither making direct use of make. It's hard to believe the
problem couldn't be chased down in this environment, though
you might have to go as far as strace or equivalent.

[*] Some shells will read an additional config file when run
in interactive mode. So depending on your shell and rc file
setup, it's possible for the environment here to differ from
that of /bin/sh -c "command" which is not
interactive. However, you say you've already accounted for
this concern with previous tests.

-David Boyce



_______________________________________________
Help-make mailing list
Help-makegnu.org
http:
//lists.gnu.org/mailman/listinfo/help-make

On Mon, 2007-09-17 at 19:16 +0100, Dave Korn wrote:
> Make will invoke your program using 'sh -c', unless you
have set SHELL
> in your environment, in which case it uses that.

This is not true; at least not for any POSIX operating
system.  Not sure
about Windows ports.

GNU make _always_ uses '/bin/sh -c' regardless of the value
of SHELL in
the environment, UNLESS you explicitly set the make variable
SHELL from
within your makefile.  SHELL is a special case variable that
is handled
differently from all other environment variables.

To take the user's $SHELL would be a disaster, since so many
people
continue to use that drain-bamaged horror known as C shell
.

-- 
------------------------------------------------------------
-------------------
 Paul D. Smith <psmithgnu.org>          Find
some GNU make tips at:
 http://www.gnu.org        
             http://make.mad-scientis
t.us
 "Please remain calm...I may be mad, but I am a
professional." --Mad Scientist


_______________________________________________
Help-make mailing list
Help-makegnu.org
http:
//lists.gnu.org/mailman/listinfo/help-make

Hi everybody,

 > This technique has never failed me: take the command
 >
 >     /path/to/binary -i input.data [more args] >
output.data
 >
 > and change it to
 >
 >     echo /path/to/binary -i input.data [more args];
$(SHELL) -i
 >
 > Then run make.

Now this is a *very* clever idea. I am going to pursue that.
Meanwhile I 
have tried to set $SHELL=/bin/bash in the Makefile, since
for manual 
launch I also use bash. Indeed, make then really uses bash
to run the 
above command line. However, this does not affect the
problem in any way.

 > The situation smells like undefined behaviour to me;
if you're using
 > an uninitialised variable or in some other way
depending on random
 > memory contents, that could trigger the different
behaviour in your
 > program. Your description of the problem is a bit
vague, but by
 > "more chemical information", I'm guessing
the library is either
 > returning more records, or more data fields in each
record.

The output consists mainly of chemical compounds. Every
compound has a 
score attached (a floating point number). When run with
make, lower 
scores are attached and thus more chunks fall below a
predefined 
threshold (0.3) which results in less output. The score is a
measure of 
similarity between the compounds and one specific compound,
called the 
query compound. For similarity computation the already
mentioned 
libraries are accessed. It really seems that is has
something to do with 
program internal computation and/or the way libraries are
accessed.

The manual results seem to make much more sense, so in the
meantime I 
have decided to refer to them as "correct".

Regards, Andreas
-- 
http://www.maunz.de

            If you are passed on the right, you're in the
wrong lane.


_______________________________________________
Help-make mailing list
Help-makegnu.org
http:
//lists.gnu.org/mailman/listinfo/help-make

On 18 September 2007 08:26, Andreas Maunz wrote:

>  > The situation smells like undefined behaviour to
me; if you're using
>  > an uninitialised variable or in some other way
depending on random
>  > memory contents, that could trigger the different
behaviour in your
>  > program. Your description of the problem is a bit
vague, but by
>  > "more chemical information", I'm
guessing the library is either
>  > returning more records, or more data fields in
each record.
> 
> The output consists mainly of chemical compounds. Every
compound has a
> score attached (a floating point number). When run with
make, lower
> scores are attached and thus more chunks fall below a
predefined
> threshold (0.3) which results in less output. The score
is a measure of
> similarity between the compounds and one specific
compound, called the
> query compound. For similarity computation the already
mentioned
> libraries are accessed. It really seems that is has
something to do with
> program internal computation and/or the way libraries
are accessed.
> 
> The manual results seem to make much more sense, so in
the meantime I
> have decided to refer to them as "correct".

  Here's a thought: does it still happen if you compile your
program with -O0,
i.e. no optimisation?  


    cheers,
      DaveK
-- 
Can't think of a witty .sigline today....



_______________________________________________
Help-make mailing list
Help-makegnu.org
http:
//lists.gnu.org/mailman/listinfo/help-make