testing, profiling and bjam

For sometime I've wanted to subject the serialization
library to execution 
time profiing.  In the past I have spent significant time in
program runtime 
optimization and have found such profiling indispensible.

So I built the serialization tests with

bjam --dump_tests --toolset=gcc-3.3 variant=profile ...

Hoping against hope that I would endup with a profile output
in each test 
directory.

Well, my hopes weren't realized as I'm aware there there
hasn't been much 
interest in this area.

However, good news was that the executables were built with
the proper 
options and when run generated the required gmon.out file.

Bad news is the following:

a) the gmon.out file is created in directory from which bjam
is run.  When 
running a series of tests, each gmon.out file gets
overwritten by its 
successor test.

b) I could re-run the executable in each test directory to
create a new 
gmon.out - but the executable is removed after the test to
save space.

Sooooooooo

I find I can save the executable with a bjam 
switch --preserve-test_targets - which is ok

Then I can craft a simple script which will run each test in
its directory 
and then run gprof to prepare the table.

If I decide I want to spend the effort, I can make these
tables easy to 
browse.

So I'm satisfied.

But I wonder is there some bjam magic which can be helpful
with this 
process?  bjam gurus are encouraged to respond.

Robert Ramey



_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build

Robert Ramey wrote:

> For sometime I've wanted to subject the serialization
library to execution
> time profiing.  In the past I have spent significant
time in program
> runtime optimization and have found such profiling
indispensible.
> 
> So I built the serialization tests with
> 
> bjam --dump_tests --toolset=gcc-3.3 variant=profile
...
> 
> Hoping against hope that I would endup with a profile
output in each test
> directory.
> 
> Well, my hopes weren't realized as I'm aware there
there hasn't been much
> interest in this area.
> 
> However, good news was that the executables were built
with the proper
> options and when run generated the required gmon.out
file.
> 
> Bad news is the following:

The bad news is that gmon.out is useless. You won't get
anyway accurate
information for any C++ library, and you'll end up with 
vector<something>::operator[] on top, or some such
function.

> But I wonder is there some bjam magic which can be
helpful with this
> process?  

The right solution is to run binaries under callgrind, and
store
the output. This is not supported -- you're the first one to
ask.
Note also that profiling the testsuite is not particularly
good idea,
as testsuite are not likely to represent real-world data.

- Volodya


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build

Vladimir Prus wrote:
> Robert Ramey wrote:
>
> The bad news is that gmon.out is useless. You won't get
anyway
> accurate
> information for any C++ library, and you'll end up
with
> vector<something>::operator[] on top, or some
such function.

Hmmm - looking at the gprof output from the serialization
library
tests confirms my view that it's a lot more useful than what
I had
before - which was nothing.

>> But I wonder is there some bjam magic which can be
helpful with this
>> process?
>
> The right solution is to run binaries under callgrind,
and store
> the output. This is not supported -- you're the first
one to ask.
> Note also that profiling the testsuite is not
particularly good idea,
> as testsuite are not likely to represent real-world
data.

It seemed to me that this would/should be a trivial tweak
in
in some Jamfile.  I knew that the test executabes are
removed after
being run so I suspect that moving the gmon.out file to
the target directory after the test is run would be a
trivial
extention - and it would give me a tiny project to around
which to learn a little about bjam - maybe enough to fix
the things in my Jamfile.v2.

After about 4 hours perusing the documentation, jam files,
running little tests I just gave up.  Its too hard.  Perhaps
it
needs a disclaimer that it's only going to be productive
for
the smartest developers.

Since I still needed the gmon.out files in their respective
directories I made a small shell script to execute each
executable and mv the gmon.out to the correct directory.
Took me about 15 minutes to write test and execute.

Also another thing intrigues me. By default the tests
remove
the executables to save space.  I can understand this.  But
why just the executables and not the other files such as
the
object files.  Just curious.

Robert Ramey



_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build