Shorter build paths.

Those suffering from long build paths, I.E. all,

The long paths BBv2 produces has been a rather contentious
subject in 
the past. Everyone hates them, but no one has come up with a
way to 
reasonably make them shorter. Or as Volodya points out, most
people want 
them shorter but have no idea how 

Some context, this came about when I ran into the problem of
features 
with invalid filename chars in them end up in the build
path. In 
particular I'm in the middle of doing more changes to
doxygen.jam and it 
uses features of the form "doxygen:_x_". So I
added a 
"doxygen:processor" feature and immediately ran
into directory creation 
errors. Next I made the feature "implicit" which
removed the 
objectionable part but scrambled the path again as implicit
features are 
place first in the build path.  So here's my latest idea...

First one would describe path shorthands for subsets of a
property set. 
We already do something similar in the conditionals. For
example:

* Shortening the above doxygen path:

   feature.path doxproc : <doxygen:processor>doxproc ;
   feature.path doxslt : <doxygen:processor>xsltproc ;

* Shortening what is a regular longer build combination:

   feature.path static-mt :
     <link>static <threading>multi ;
   feature.path dynamic-mt :
     <link>dynamic <threading>multi ;
   feature.path vc8 :
     <toolset>msvc-8.0 ;

Which might produce the following paths:

   bin/libs/filesystem/build/vc8/debug/static-mt
   bin/libs/filesystem/build/vc8/debug/dynamic-mt
   bin/libs/filesystem/build/vc8/release/static-mt
   bin/libs/filesystem/build/vc8/release/dynamic-mt

Instead of the current:

  
bin/libs/filesystem/build/msvc-8.0/debug/link-static/threadi
ng-multi
   bin/libs/filesystem/build/msvc-8.0/debug/threading-multi
  
bin/libs/filesystem/build/msvc-8.0/release/link-static/threa
ding-multi
  
bin/libs/filesystem/build/msvc-8.0/release/threading-multi

The general form would be:

   feature /dir-name/ : /requirements/ ;

And of course users would be able to produce a single path
for a 
complicated build:

   feature.path my-build :
     <link>dynamic <threading>multi
<toolset>msvc-8.0 #...etc.
     ;

The path generation algorithm would, given the build
properties, go 
something like:

1. start with an empty build path
2. for each registered feature path:
    a. if the feature path requirements match a subset of
the build 
properties, remove those properties and add the path to the
build path
3. trim the redundant properties
4. split the properties into incidental and non-incidental
5. translate the incidental properties onto the front of the
build path
6. translate the non-incidental properties onto the back of
the build path
7. generate the final build path from the components

The effect is that the path is compressed based on the
defined shorthand 
paths plus any additional specific build properties. The one
requirement 
  would be that the feature paths would need to be unique.
But that is 
something easily checked and an error generated. The overall
form of the 
build paths would then be:

   bin /incidental parts/ ... /shorthands/ ...
/non-incidental parts/

Thoughts?


-- 
-- Grafik - Don't Assume Anything
-- Redshift Software, Inc. - http://redshift-software
.com
-- rrivera/acm.org - grafik/redshift-software.com
-- 102708583/icq - grafikrobot/aim - grafikrobot/yahoo
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build

On Sunday 17 December 2006 02:59, Rene Rivera wrote:

> So here's my latest idea...  

> * Shortening what is a regular longer build
combination:
> 
>    feature.path static-mt :
>      <link>static <threading>multi ;
>    feature.path dynamic-mt :
>      <link>dynamic <threading>multi ;
>    feature.path vc8 :
>      <toolset>msvc-8.0 ;
> 
> Which might produce the following paths:
> 
>    bin/libs/filesystem/build/vc8/debug/static-mt
>    bin/libs/filesystem/build/vc8/debug/dynamic-mt
>    bin/libs/filesystem/build/vc8/release/static-mt
>    bin/libs/filesystem/build/vc8/release/dynamic-mt

I think this approach is similar to what we do with
'composite' features. For example,
	
	<variant>release

is actually a shorthand for

	<optimization>full ..............

and when generating target paths we notice that
<variant>release includes <optimization>full,
so <optimization>full is not added to the target
paths.

So, I think your approach should generally work.

> The path generation algorithm would, given the build
properties, go 
> something like:
> 
> 1. start with an empty build path
> 2. for each registered feature path:
>     a. if the feature path requirements match a subset
of the build 
> properties, remove those properties and add the path to
the build path
> 3. trim the redundant properties

What properties are "redundant"?

> 4. split the properties into incidental and
non-incidental
> 5. translate the incidental properties onto the front
of the build path
> 6. translate the non-incidental properties onto the
back of the build path

Ehm, I don't think we add incidental properties to the path
at all, at the moment.
I think that's right, but in either case, this is
independent from shortening of build paths.

> 7. generate the final build path from the components
> 
> The effect is that the path is compressed based on the
defined shorthand 
> paths plus any additional specific build properties.
The one requirement 
>   would be that the feature paths would need to be
unique. But that is 
> something easily checked and an error generated. 

So basically, if user maps

	<variant>release <threading>multi       to   
release-mt

then if he tries to map

	<variant>release <link>static
<threading>multi       to    release-static-mt

he'll get the error? We might need to act better, but such a
restriction would be fine for
the first version.

The only concern I have is with several projects used
together. If one Jamroot specifies
some set of rules for path shortening, I think those rules
should not affect targets
under other Jamroots -- because those Jamroots might have
their own rules set.

So, I suggest that those path-shortening rules be defined in
Jamfile and apply only
to that Jamfile and children thereof. What do you think
about this?

Thanks,
Volodya
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build

Vladimir Prus wrote:
> On Sunday 17 December 2006 02:59, Rene Rivera wrote:
> 
>> The path generation algorithm would, given the
build properties, go 
>> something like:
>>
>> 1. start with an empty build path
>> 2. for each registered feature path:
>>     a. if the feature path requirements match a
subset of the build 
>> properties, remove those properties and add the
path to the build path
>> 3. trim the redundant properties
> 
> What properties are "redundant"?

I'm not sure precisely what they are... I was just using the
same term 
you have on the comments of property.as-path which says
"trim redundancy".

>> 4. split the properties into incidental and
non-incidental
>> 5. translate the incidental properties onto the
front of the build path
>> 6. translate the non-incidental properties onto the
back of the build path
> 
> Ehm, I don't think we add incidental properties to the
path at all, at the moment.

I meant "implicit" vs. "non-implicit".
Sorry for the confusion.

> I think that's right, but in either case, this is
independent from shortening of build paths.
> 
>> 7. generate the final build path from the
components
>>
>> The effect is that the path is compressed based on
the defined shorthand 
>> paths plus any additional specific build
properties. The one requirement 
>>   would be that the feature paths would need to be
unique. But that is 
>> something easily checked and an error generated. 
> 
> So basically, if user maps
> 
> 	<variant>release <threading>multi       to
   release-mt
> 
> then if he tries to map
> 
> 	<variant>release <link>static
<threading>multi       to    release-static-mt
> 
> he'll get the error? We might need to act better, but
such a restriction would be fine for
> the first version.

No I meant that if they map:

   <variant>release <threading>multi -->
release-mt
   <variant>release <link>static
<threading>multi --> release-mt

It would error as the resolution ends up being ambiguous.
Your example 
above we can handle by preferring larger replacements first.
Where 
larger means the longer list of requirements.

> The only concern I have is with several projects used
together. If one Jamroot specifies
> some set of rules for path shortening, I think those
rules should not affect targets
> under other Jamroots -- because those Jamroots might
have their own rules set.
> 
> So, I suggest that those path-shortening rules be
defined in Jamfile and apply only
> to that Jamfile and children thereof. What do you think
about this?

I see your point but I'm trying to see what we loose by
*only* having 
that in jamfiles. I thought that their would be some benefit
from having 
some standard shorthands, i.e. defined in tools/builtin.jam
for example, 
that would provide the benefit of shorter paths for everyone
out of the box.


-- 
-- Grafik - Don't Assume Anything
-- Redshift Software, Inc. - http://redshift-software
.com
-- rrivera/acm.org - grafik/redshift-software.com
-- 102708583/icq - grafikrobot/aim - grafikrobot/yahoo
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build