some (trivial) observations on multi-arch builds

  if you'll indulge me, a few simple observations on
multi-arch builds
that i just want to clarify so i'm sure i understand it
thoroughly.

  as i read it, the whole point of designing a multi-arch
build is to
take an "original" makefile (that was written
with the assumption that
it would be invoked in the source directory) and extend it
so it can
be invoked elsewhere (the build directory).  as paul
explains it here:

http://
make.paulandlesley.org/multi-arch.html

this is done by, for the most part, just adding a
"wrapper" around the
original makefile.  as i read it, the goal is to have to
hack that
original makefile as little as possible.

  now, with the use of vpath, this works just fine as long
as your
references to things in the source directory come in the way
of rule
dependencies, so vpath will happily do the work in tracking
those
objects down for you.  however, if you can't represent
something as a
rule dependency, you still might have to explicitly refer to
the
source directory.

  for example, let's say the source directory has a local
"include/"
directory and gcc was originally invoked with
"-I./include".  that
will work fine if you're running in the source directory
itself but
will fail badly elsewhere.  in a case like that, it seems
clear that
your rule commands might need to incorporate a reference to
$
because vpath won't help you with things like compiler
options.

  as a simpler example, let's say i just wanted to collect
a list of
all of the source files in the directory.  originally, i
might have
just written:

  SRCS := $(wildcard *.c)

now, however, i have to be careful to write something like:

  SRCS := $(wildcard $/*.c)

none of this is really deep, it's just stuff that i don't
think was
explained emphatically enough.  in addition to knowing what
using
"vpath" can do, i think it's just as important
to emphasize what it
*can't* do so people know how they still have to adjust
their
makefiles so they can be invoked remotely.

  thoughts?

rday


_______________________________________________
Help-make mailing list
Help-makegnu.org
http:
//lists.gnu.org/mailman/listinfo/help-make

Robert P. J. Day wrote:

>   for example, let's say the source directory has a
local "include/"
> directory and gcc was originally invoked with
"-I./include".  that
> will work fine if you're running in the source
directory itself but
> will fail badly elsewhere.  in a case like that, it
seems clear that
> your rule commands might need to incorporate a
reference to $
> because vpath won't help you with things like compiler
options.

That's correct, AFAIK.  At least, that's what I do.

>   as a simpler example, let's say i just wanted to
collect a list of
> all of the source files in the directory.  originally,
i might have
> just written:
> 
>   SRCS := $(wildcard *.c)
> 
> now, however, i have to be careful to write something
like:
> 
>   SRCS := $(wildcard $/*.c)

Also true, AFAIK.

I have a follow-up to your question about multi-arch and
recursive
make.

In my project, there are a number of source directories that
contain
the code for various modules.  Some of these modules are
program
drivers (i.e. the "main" routines) while others
are program-specific
libraries and still others are shared components referenced
by two
or more other modules.

Due to the shared component pieces, I've been reluctant to
use
recursive make.  Building in parallel is commonplace here
and I
don't want to create race conditions between separate
recursive
invocations of make for the shared components.

I read "Recursive Make Considered Harmful" some
time ago but
confess to having forgotten the details of when it is and is
not
a problem.

To be on the safe side, rather than using recursive make, I
simply
have the top-level Makefile include .mk files from each
top-level
subdirectory which in turn may include .mk files to specify
rules
for building components in deeper directories.  In effect, I
get
one huge Makefile out that contains every single rule and
dependency
to build everything.

This seems to be working rather well so far.  The only
tricky part
has to do with specifying sources.  Because I don't
actually
recurse into deeper source directories, I do this:

  SRCS := $(subst $(SRCDIR)/,,$(wildcard
$(SRCDIR/$(module_dir)/*.c))

Then I only need to define VPATH once and $(SRCDIR) doesn't
get
"hard-coded" into the dependency lists.

An alternative would be to use target-specific definitions
of
VPATH and add $(module_dir)/ to the subst match string.

Any thoughts on this setup?  I am being overly paranoid
concerning
recursive make?

                         -Dave


_______________________________________________
Help-make mailing list
Help-makegnu.org
http:
//lists.gnu.org/mailman/listinfo/help-make

On 2006-5-12 20:08 UTC, David Greene wrote:
> 
> In my project, there are a number of source directories
that contain
> the code for various modules.  Some of these modules
are program
> drivers (i.e. the "main" routines) while
others are program-specific
> libraries and still others are shared components
referenced by two
> or more other modules.
> 
> Due to the shared component pieces, I've been
reluctant to use
> recursive make.  Building in parallel is commonplace
here and I
> don't want to create race conditions between separate
recursive
> invocations of make for the shared components.
> 
> I read "Recursive Make Considered Harmful"
some time ago but
> confess to having forgotten the details of when it is
and is not
> a problem.

I think the main thesis of the paper is that if you have one
standalone submakefile in each of several source
directories,
which doesn't know what other submakefiles are doing, then
it's
impossible for make to manage interdependencies among them.

IOW: make can do the right thing iff it has the complete
DAG.

It's not make that's broken. And it's not recursion
that's
broken, per se: finding the string '$(MAKE)' in a makefile
doesn't prove that it's problematic.

> To be on the safe side, rather than using recursive
make, I simply
> have the top-level Makefile include .mk files from each
top-level
> subdirectory which in turn may include .mk files to
specify rules
> for building components in deeper directories.  In
effect, I get
> one huge Makefile out that contains every single rule
and dependency
> to build everything.

Then it sounds like your single makefile includes the
complete
DAG; yet I believe writing a single makefile doesn't
guarantee
that you can't have the "harmful" problem....

> This seems to be working rather well so far.  The only
tricky part
> has to do with specifying sources.  Because I don't
actually
> recurse into deeper source directories, I do this:
> 
>   SRCS := $(subst $(SRCDIR)/,,$(wildcard
$(SRCDIR/$(module_dir)/*.c))

Does $(module_dir) change during a build--say, because of a
$(foreach) somewhere? I think we can create the
"harmful"
situation with a single makefile:

# Name of this makefile.
self := $(lastword $(MAKEFILE_LIST))

.PHONY: all
all:
	$(MAKE) -f (self) target0 module_dir=dir0
	$(MAKE) -f (self) target1 module_dir=dir1

If target0 depends on target1, then aren't we building
shards of a fractured DAG?

> Then I only need to define VPATH once and $(SRCDIR)
doesn't get
> "hard-coded" into the dependency lists.
> 
> An alternative would be to use target-specific
definitions of
> VPATH and add $(module_dir)/ to the subst match string.
>
> Any thoughts on this setup?  I am being overly paranoid
concerning
> recursive make?

Why does it seem good to simplify VPATH by moving complexity
elsewhere? Would it be possible to write an all-inclusive
vpath
  all_source_directories := srcdir0 srcdir1 srcdir2...
  vpath %.c $(all_source_directories)
  vpath %.h $(all_source_directories)
instead? The only problem I can see is that files in
different
source subdirectories might share the same names; but
wouldn't
it seem better to avoid that anyway?

Then you could just write, say,

  target0_objects := obj0a.o obj0b.o
  target1_objects := obj1a.o obj1b.o

  target0$(EXEEXT): $(target0_objects)
  target1$(SHREXT): $(target1_objects)

and express any order dependencies separately. What
wouldn't
this take care of?


_______________________________________________
Help-make mailing list
Help-makegnu.org
http:
//lists.gnu.org/mailman/listinfo/help-make

David Greene wrote:
> ...
> In my project, there are a number of source directories
that contain
> the code for various modules.  Some of these modules
are program
> drivers (i.e. the "main" routines) while
others are program-specific
> libraries and still others are shared components
referenced by two
> or more other modules.
> 
> Due to the shared component pieces, I've been
reluctant to use
> recursive make.  Building in parallel is commonplace
here and I
> don't want to create race conditions between separate
recursive
> invocations of make for the shared components.

I have never found any mention of it in the manual, but I
found I can 
represent dependencies between 'directories':
If I declare my subdirectories as phony targets, with the
"$(MAKE) -C" 
command as one of the commands to be executed for these
targets, and 
then specify dependencies 'as usual':
   dira/ dirb/ dirc/ : dird/
this perfectly orders the recursion, and hasn't presented
any challenge 
for parallel build yet. Works for Solaris (tested with
2.5-2.7), Linux 
and Cygwin.
Is this a coincidence, or rather a feature?
At least it solved some of my problems 

> I read "Recursive Make Considered Harmful"
some time ago but
> confess to having forgotten the details of when it is
and is not
> a problem.

For my taste, there were too many arguments in the sense of
"people 
won't do it right, and if you don't do it right it won't
work."
Which is correct, only my conclusion is not 'not to do
it', as Peter 
Miller recommended (and used one big makefile instead), but
to automate 
most everything that can be done wrong (for instance,
automatic 
dependency generation, among others).
(Ref: http://aegis.
sourceforge.net/auug97.pdf)

And yes, if you accept that source files have arbitrary
dependencies 
from any file in any subdirectory of any module (main.c
being dependent 
on ../bee/parse.h, in Miller's sample), then the
dependencies need to 
reflect this. But this is more a matter of physical
decomposition of 
modules, not an issue for make, IMHO.

> To be on the safe side, rather than using recursive
make, I simply
> have the top-level Makefile include .mk files from each
top-level
> subdirectory which in turn may include .mk files to
specify rules
> for building components in deeper directories.  In
effect, I get
> one huge Makefile out that contains every single rule
and dependency
> to build everything.
> 
> This seems to be working rather well so far.  The only
tricky part
> has to do with specifying sources.  Because I don't
actually
> recurse into deeper source directories, I do this:
> 
>   SRCS := $(subst $(SRCDIR)/,,$(wildcard
$(SRCDIR/$(module_dir)/*.c))

As long as your structure's depth doesn't increase, it
will work. If 
someone wants to create a new level of submodules, you'll
have work to do.

> Then I only need to define VPATH once and $(SRCDIR)
doesn't get
> "hard-coded" into the dependency lists.

I do not see the advantage of that, assuming the dependency
lists are 
generated automatically.

> An alternative would be to use target-specific
definitions of
> VPATH and add $(module_dir)/ to the subst match string.

target-specific? Can you give an example?
If I'm not mistaken, such variables are only
available/defined within 
the target's command script, so defining VPATH as
target-specific will 
not influence the search for sources...

> Any thoughts on this setup?  I am being overly paranoid
concerning
> recursive make?

Well, I've been using make-recursion successfully since
approx. 1998, 
and no more problems than without recursion.
And having only two-liner makefiles for the developers to
handle 
certainly helps avoiding problems 

Johan
-- 
JB Enterprises - Johan Bezem         Tel:   +49 172 5463210
Software Architect - Project Manager Fax:   +49 172 50
5463210
Realtime / Embedded Consultant       Email: j.bezemcomputer.org
Design - Development - Test - QA     Web:   http://www.bezem.de


_______________________________________________
Help-make mailing list
Help-makegnu.org
http:
//lists.gnu.org/mailman/listinfo/help-make

Greg Chicares wrote:

> Why does it seem good to simplify VPATH by moving
complexity
> elsewhere? Would it be possible to write an
all-inclusive vpath
>   all_source_directories := srcdir0 srcdir1 srcdir2...
>   vpath %.c $(all_source_directories)
>   vpath %.h $(all_source_directories)
> instead? The only problem I can see is that files in
different
> source subdirectories might share the same names; but
wouldn't
> it seem better to avoid that anyway?

Yes, files with the same name is exactly the probblem.

I don't have control over how all the files are named, as
the
modules are developed by separate groups.  My goal is to
make
the build system as flexible as possible, so that
development
groups don't have to worry about details like this.

                        -Dave


_______________________________________________
Help-make mailing list
Help-makegnu.org
http:
//lists.gnu.org/mailman/listinfo/help-make