BBv2 intolerably slow in Boost tree.

Vladimir Prus <ghostcs.msu.su> writes:

> On Wednesday 26 July 2006 18:15, David Abrahams wrote:
>> Simple things like rebuilding the
"html" target in libs/parameter/doc
>> or rebuilding the "sfinae" target in
libs/parameter/test produce an
>> intolerably long delay for me before any build
actions start.
>
> For a start, is delay the same on the second invocation
as it is on
> the first?

To my perception, yes.

> Also, can you try the attached patch to top-level
Jamfile.v2?

With the patch, it flies!

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build

On Wednesday 26 July 2006 19:42, David Abrahams wrote:
> Vladimir Prus <ghostcs.msu.su> writes:
> > On Wednesday 26 July 2006 18:15, David Abrahams
wrote:
> >> Simple things like rebuilding the
"html" target in libs/parameter/doc
> >> or rebuilding the "sfinae" target
in libs/parameter/test produce an
> >> intolerably long delay for me before any build
actions start.
> >
> > For a start, is delay the same on the second
invocation as it is on
> > the first?
>
> To my perception, yes.

In my case, on cold run it takes ~30 seconds, and then ~2.
Probably because 
there's 6G of memory here, so all disk accesses are cached.

>
> > Also, can you try the attached patch to top-level
Jamfile.v2?
>
> With the patch, it flies!

What's performance of V1? I can replace the globbing code
with that we have 
there, as a quick solution.

I also planed to make globbing lazy -- and don't it it
unless we're installing 
-- but maybe that's not important if V1 code works fine.

- Volodya



-- 
Vladimir Prus
http://vladimir_pru
s.blogspot.com
Boost.Build V2: http://boost.org/boost-
build2
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build

Vladimir Prus wrote:
> What's performance of V1? I can replace the globbing
code with that we have 
> there, as a quick solution.

Just running the globing of headers, on Win2K under MSYS run
a few times 
to cache the filesys info:

grafikKLUDGE /c/DevRoots/Boost/boost
$ date ; bjam > /dev/null ; date ; bjam --v2 >
/dev/null ; date
Wed Jul 26 12:01:04 CDT 2006
Wed Jul 26 12:01:08 CDT 2006
Wed Jul 26 12:01:34 CDT 2006

So thats 4s for BBv1, and 26s for BBv2. Also I just noticed
that the 
BBv2 code still misses some headers it should be copying
that don't have 
  file extensions.


-- 
-- Grafik - Don't Assume Anything
-- Redshift Software, Inc. - http://redshift-software
.com
-- rrivera/acm.org - grafik/redshift-software.com
-- 102708583/icq - grafikrobot/aim - grafikrobot/yahoo
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build

On Wednesday 26 July 2006 21:05, Rene Rivera wrote:
> Vladimir Prus wrote:
> > What's performance of V1? I can replace the
globbing code with that we
> > have there, as a quick solution.
>
> Just running the globing of headers, on Win2K under
MSYS run a few times
> to cache the filesys info:
>
> grafikKLUDGE /c/DevRoots/Boost/boost
> $ date ; bjam > /dev/null ; date ; bjam --v2 >
/dev/null ; date
> Wed Jul 26 12:01:04 CDT 2006
> Wed Jul 26 12:01:08 CDT 2006
> Wed Jul 26 12:01:34 CDT 2006
>
> So thats 4s for BBv1, and 26s for BBv2. Also I just
noticed that the
> BBv2 code still misses some headers it should be
copying that don't have
>   file extensions.

It sounds that borrowing V1 logic would be a good idea.
I'll try to see where 
this performance difference comes from, though, as the code
should be doing
almost the same.

- Volodya


-- 
Vladimir Prus
http://vladimir_pru
s.blogspot.com
Boost.Build V2: http://boost.org/boost-
build2
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build

Vladimir Prus <ghostcs.msu.su> writes:

> On Wednesday 26 July 2006 19:42, David Abrahams wrote:
>> Vladimir Prus <ghostcs.msu.su> writes:
>> > On Wednesday 26 July 2006 18:15, David
Abrahams wrote:
>> >> Simple things like rebuilding the
"html" target in libs/parameter/doc
>> >> or rebuilding the "sfinae"
target in libs/parameter/test produce an
>> >> intolerably long delay for me before any
build actions start.
>> >
>> > For a start, is delay the same on the second
invocation as it is on
>> > the first?
>>
>> To my perception, yes.
>
> In my case, on cold run it takes ~30 seconds, and then
~2. Probably because 
> there's 6G of memory here, so all disk accesses are
cached.

I have 2G; shouldn't that be enough?

>> > Also, can you try the attached patch to
top-level Jamfile.v2?
>>
>> With the patch, it flies!
>
> What's performance of V1? I can replace the globbing
code with that we have 
> there, as a quick solution.

v1 is much faster than v2, even with the patch.  Building
starts
essentially instantaneously.

> I also planed to make globbing lazy -- and don't it it
unless we're installing 
> -- but maybe that's not important if V1 code works
fine.

I think you have slowdowns elsewhere.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build

On Wednesday 26 July 2006 21:18, David Abrahams wrote:
> Vladimir Prus <ghostcs.msu.su> writes:
> > On Wednesday 26 July 2006 19:42, David Abrahams
wrote:
> >> Vladimir Prus <ghostcs.msu.su> writes:
> >> > On Wednesday 26 July 2006 18:15, David
Abrahams wrote:
> >> >> Simple things like rebuilding the
"html" target in libs/parameter/doc
> >> >> or rebuilding the
"sfinae" target in libs/parameter/test produce
an
> >> >> intolerably long delay for me before
any build actions start.
> >> >
> >> > For a start, is delay the same on the
second invocation as it is on
> >> > the first?
> >>
> >> To my perception, yes.
> >
> > In my case, on cold run it takes ~30 seconds, and
then ~2. Probably
> > because there's 6G of memory here, so all disk
accesses are cached.
>
> I have 2G; shouldn't that be enough?

I don't know.

> >> > Also, can you try the attached patch to
top-level Jamfile.v2?
> >>
> >> With the patch, it flies!
> >
> > What's performance of V1? I can replace the
globbing code with that we
> > have there, as a quick solution.
>
> v1 is much faster than v2, even with the patch.  

Then there's something wrong in globbing code. 

> Building starts 
> essentially instantaneously.


>
> > I also planed to make globbing lazy -- and don't
it it unless we're
> > installing -- but maybe that's not important if
V1 code works fine.
>
> I think you have slowdowns elsewhere.

Well, one thing at a time.

- Volodya

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build

Vladimir Prus <ghostcs.msu.su> writes:

> On Wednesday 26 July 2006 21:18, David Abrahams wrote:
>> Vladimir Prus <ghostcs.msu.su> writes:
>> > On Wednesday 26 July 2006 19:42, David
Abrahams wrote:
>> >> Vladimir Prus <ghostcs.msu.su> writes:
>> >> > On Wednesday 26 July 2006 18:15,
David Abrahams wrote:
>> >> >> Simple things like rebuilding the
"html" target in libs/parameter/doc
>> >> >> or rebuilding the
"sfinae" target in libs/parameter/test produce
an
>> >> >> intolerably long delay for me
before any build actions start.
>> >> >
>> >> > For a start, is delay the same on the
second invocation as it is on
>> >> > the first?
>> >>
>> >> To my perception, yes.
>> >
>> > In my case, on cold run it takes ~30 seconds,
and then ~2. Probably
>> > because there's 6G of memory here, so all
disk accesses are cached.
>>
>> I have 2G; shouldn't that be enough?
>
> I don't know.
>
>> >> > Also, can you try the attached patch
to top-level Jamfile.v2?
>> >>
>> >> With the patch, it flies!
>> >
>> > What's performance of V1? I can replace the
globbing code with that we
>> > have there, as a quick solution.
>>
>> v1 is much faster than v2, even with the patch.  
>
> Then there's something wrong in globbing code. 

What makes you think all the time is going into globbing?

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build

On Wednesday 26 July 2006 23:20, David Abrahams wrote:

> >> >> > Also, can you try the attached
patch to top-level Jamfile.v2?
> >> >>
> >> >> With the patch, it flies!
> >> >
> >> > What's performance of V1? I can replace
the globbing code with that we
> >> > have there, as a quick solution.
> >>
> >> v1 is much faster than v2, even with the
patch.
> >
> > Then there's something wrong in globbing code.
>
> What makes you think all the time is going into
globbing?

You reported that my patch improves performance
considerably. The only thing 
the patch does is removing globbing code. 

- Volodya

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build

Vladimir Prus <ghostcs.msu.su> writes:

> On Wednesday 26 July 2006 23:20, David Abrahams wrote:
>
>> >> >> > Also, can you try the
attached patch to top-level Jamfile.v2?
>> >> >>
>> >> >> With the patch, it flies!
>> >> >
>> >> > What's performance of V1? I can
replace the globbing code with that we
>> >> > have there, as a quick solution.
>> >>
>> >> v1 is much faster than v2, even with the
patch.
>> >
>> > Then there's something wrong in globbing
code.
>>
>> What makes you think all the time is going into
globbing?
>
> You reported that my patch improves performance
considerably. The only thing 
> the patch does is removing globbing code. 

Yes, but even with the globbing removed, v2 is still
considerably
slower than v1.  That suggests to me that there is a culprit
other
than globbing.  What am I missing?

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build

On Thursday 27 July 2006 18:10, David Abrahams wrote:
> Vladimir Prus <ghostcs.msu.su> writes:
> > On Wednesday 26 July 2006 23:20, David Abrahams
wrote:
> >> >> >> > Also, can you try the
attached patch to top-level Jamfile.v2?
> >> >> >>
> >> >> >> With the patch, it flies!
> >> >> >
> >> >> > What's performance of V1? I can
replace the globbing code with that
> >> >> > we have there, as a quick
solution.
> >> >>
> >> >> v1 is much faster than v2, even with
the patch.
> >> >
> >> > Then there's something wrong in globbing
code.
> >>
> >> What makes you think all the time is going
into globbing?
> >
> > You reported that my patch improves performance
considerably. The only
> > thing the patch does is removing globbing code.
>
> Yes, but even with the globbing removed, v2 is still
considerably
> slower than v1.  That suggests to me that there is a
culprit other
> than globbing.  What am I missing?

Hmm, let's go back: I said:

> >> > Then there's something wrong in globbing
code.

to which you replied:

> >> What makes you think all the time is going
into globbing?

I did not say that *all* of V2 runtime goes into globbing,
and that by 
removing globbing we can make V2 run in 0 seconds all
projects on any kind.
But if removing code that consists only of globbing makes
big difference in 
performance, it means that globbing code is somehow wrong.
What am I missing?

- Volodya





_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build