Ant build.xml targets

Emil, I've moved this discussion from the users list
because it makes more sense to developers.

>I'd like to straighten something out. The cc-buildloop
target is not
>supposed to be run by users. Not even when initializing
the sandbox. I
>guess that the confusion is coming from that fact that
this has long
>been the only target that was doing anything meaningful
- running the
>automated tests. However, this is something that users
won't normally
>need to do. The _only_ target that needs to be run in
order to
>initialize the sandbox is the bundles target. This would
clean, and
>compile, create bundle jars. In other words:
>
>  rebuild = clean + compile
>  bundles = rebuild + create all jars
>  cc-buildloop = bundles + test
>  run = runs the project
>
>I agree that this does not really make sense, not to
mention that it is
>far from being intuitive. I therefore propose the
following configuration:
>
>  clean = remains as is
>  comile = remains as is
>  make = compile + bundles
>  rebuild = clean + make
>  cc-buildloop = rebuild + test
>  run  = remains as is.
>
>This way we'll end up having a configuration which
would only require
>
>  ant make run
>
>for users dealing with a new sandbox and would keep the
ant cc-buildloop
>target for developers.
>
>Does this make sense?

It makes some sense to me, but not enough to leave the
matter alone. I've just done another review of the
dependencies, following your recent changes, and I still
have some concerns. Can we discuss them some 
more?

1. Generally speaking, we should have two kinds of build...
with and without clean. clean should take you back to a
sandbox state that corresponds to a newly checked-out
source. I am not certain this is the case, 
but only because I haven't spent the time analysing it
carefully. 

2. I see there is a (new) dependent for clean, called
clean-bundle-repositories. It is also a dependent of make,
which leaves me puzzled about its purpose. I always expect a
target called "make" to be an 
incremental build, i.e. WITHOUT a clean, so only changed
resources are rebuilt. Is the purpose of
clean-bundle-repositiories consistent with this philosophy?
(If yes, perhaps it should be called 
refresh-bundle-repositories or something similar?)

3. "ant make run" contains redundant
information. You should only need to specify your
"ultimate destination" and leave ant to work out
the dependency tree. Thus "ant run" OUGHT to do
what you want. However, 
the dependents of run are confusing... compile and
resource... even though THEY are also dependents of make! IF
run doesn't actually depend on make, perhaps we should
change it?

4. "ant rebuild" depends on the clean and make,
which looks fine to me.

5. "ant cc-buildloop" depends on rebuild and
test, which looks fine to me.

6. My main concern is the bundles target. I apologise for
not having done my OSGI homework, but perhaps you could help
me understand? The run target is dependent on compile and
package, but NOT on 
clean-bundle-repositories... this implies that I can change
any of the source modules or files and safely do an
incremental build WITHOUT touching the bundles.

I'm confused. I can't get my head around the
with-or-without-clean behaviour. I also think this is where
our observers have been getting lost. IF you checkout a new
sandbox, it should have the same appearance 
(practically speaking) as when you've run the clean target
against a sandbox with arbitrary contents.

If you run an incremental target (run, make, etc) against a
virgin sandbox, it should behave exactly the same as if you
run one of the targets dependent on clean (cc-buildloop,
rebuild, etc).

In the past, this has not been the case with
sip-communicator. Hence our recommendation to run
cc-buildloop to initialise a new sandbox - because
previously it was the only target that was dependent on
bundles 
(although I can see our incremental targets now depend on it
too).

I have always presumed (perhaps wrongly) that bundles was a
resource-hungry target, because it re-jars so many files. I
had presumed these jars seldom changed and so the overhead
of re-checking during an 
incremental build was worth avoiding (to an intelligent
developer who was prepared to take the risk). We must have
clean and incremental targets that newbies can run safely,
whether they are changing our source 
or not. On the other hand, is there any point having
"fast path" build targets for developers who
accept the risk of skipping some or all of bundles?

Once we have these matters clarified, it should be trivial
to update the usage target so that all types of users can be
reminded of what they need to run.
Regards,

Brian


------------------------------------------------------------
---------
To unsubscribe, e-mail: dev-unsubscribesip-communicator.dev.java.net
For additional commands, e-mail: dev-helpsip-communicator.dev.java.net

Hello Brian,

Brian Burch wrote:
> 1. Generally speaking, we should have two kinds of
build... with and
> without clean. clean should take you back to a sandbox
state that
> corresponds to a newly checked-out source. I am not
certain this is
> the case, but only because I haven't spent the time
analysing it
> carefully.

Indeed, this is the case.

> 2. I see there is a (new) dependent for clean, called
> clean-bundle-repositories. It is also a dependent of
make, which
> leaves me puzzled about its purpose. I always expect a
target called
> "make" to be an incremental build, i.e.
WITHOUT a clean, so only
> changed resources are rebuilt. Is the purpose of
> clean-bundle-repositiories consistent with this
philosophy? (If yes,
> perhaps it should be called refresh-bundle-repositories
or something
> similar?)

Whenever OSCAR runs it would create bundle repositories
where it would
copy all bundles as well as some system information.
Unfortunately the
version we're using will only get new bundles if they are
not already in
the repository. Hence a make won't have any sense unless
you clean the
repositories and you let OSCAR reinstall the new bundle
versions.

> 3. "ant make run" contains redundant
information. You should only
> need to specify your "ultimate destination"
and leave ant to work out
> the dependency tree. Thus "ant run" OUGHT
to do what you want.

I am afraid this is not really possible with ant and OSGI
(or at least I
wouldn't know how to do it). In order for the
"run" target to detect
whether a make is needed, it would not only have to verify
whether
sources have been changed since the last compilation but
also determine 
whether and which bundle jars would need to be repackaged.

> However, the dependents of run are confusing... compile
and
> resource... even though THEY are also dependents of
make! IF run
> doesn't actually depend on make, perhaps we should
change it?

Right and done. Compile and resource were the simply because
I've
forgotten to to remove them. Thanks for noticing.

> 6. My main concern is the bundles target. I apologise
for not having
> done my OSGI homework, but perhaps you could help me
understand? The
> run target is dependent on compile and package, but NOT
on 
> clean-bundle-repositories... this implies that I can
change any of
> the source modules or files and safely do an
incremental build
> WITHOUT touching the bundles.

As I mentioned above - compile and package were not there on
purpose.
The run target is indeed supposed to be an independent
target that won't
do any changes to the project and that is meant for
developers, users,
or using developers  to be able
to do consecutive runs of the project
without needing to wait through the compilation, jar
building and
deployment.

I agree that the run target would fail mightily and leave
the user in
the dark with some weird error message in case the project
is not built.
We should think about a way of introduing an apropriate
error message in
this case.

> I'm confused. I can't get my head around the
with-or-without-clean
> behaviour. I also think this is where our observers
have been getting
> lost. IF you checkout a new sandbox, it should have the
same
> appearance (practically speaking) as when you've run
the clean target
> against a sandbox with arbitrary contents.

And this is exactly what currently happens upon execution of
`ant clean`. (again, practically speaking)

> If you run an incremental target (run, make, etc)
against a virgin
> sandbox, it should behave exactly the same as if you
run one of the
> targets dependent on clean (cc-buildloop, rebuild,
etc).

I did not quite catch that. cc-buildloop is supposed to
behave the same
way as clean, make, run, replacing the run by test, and
that's what it does

> I have always presumed (perhaps wrongly) that bundles
was a
> resource-hungry target, because it re-jars so many
files.

True.

> I had presumed these jars seldom changed and so the
overhead of
> re-checking during an incremental build was worth
avoiding (to an
> intelligent developer who was prepared to take the
risk). 

I guess you meant to say that rarely more than a few of them
would 
change in a single build (there is always at least one
bundle that needs 
to be changed as a result of compiling the project as there
is no code 
in the SIP Communicator that is executed outside of a
bundle). The 
trouble is that I don't really know if it is possible to
detect (using 
ant) which are the bundles that changed and only rejar
those, leaving 
the rest untouched.

> We must have clean and incremental targets that newbies
can run safely,
> whether they are changing our source or not. On the
other hand, is
> there any point having "fast path" build
targets for developers who
> accept the risk of skipping some or all of bundles?

I agree and if you find a way of making it more efficient
than it 
currently is (e.g. by improving detection of changes in the
jars) I'd be 
really very interested.

> Once we have these matters clarified, it should be
trivial to update
> the usage target so that all types of users can be
reminded of what
> they need to run. Regards,

Hope this clarifies matters at least a bit.

Emil

------------------------------------------------------------
---------
To unsubscribe, e-mail: dev-unsubscribesip-communicator.dev.java.net
For additional commands, e-mail: dev-helpsip-communicator.dev.java.net

Thanks very much, Emil. It was very reassuring to see from
your comments that I hadn't overlooked anything obvious. It
is good to know that we have the same overall philosophy
about the build.

>> However, the dependents of run are confusing...
compile and
>> resource... even though THEY are also dependents of
make! IF run
>> doesn't actually depend on make, perhaps we should
change it?
>
>Right and done. Compile and resource were the simply
because I've
>forgotten to to remove them. Thanks for noticing.
>

I was a bit shocked by your change to the run target. It no
longer has ANY dependents! Isn't that too extreme? Isn't
that too counter-intuitive for newbies? It means the run
target has "no insight" into changes made to 
the source base... an observer could download a source
change from CVS and still execute the "old"
version.

My suggestion was much more orthodox - simply let run be
dependent on make, which is an incremental build. (make
depends on {compile + package + clean-bundle-repositories},
which have their own 
dependencies).

Do you want to change the build for the run dependency, or
shall I do it?



By the way, I feel much happier looking at the build after
your explanation of the oscar issues, so I'll try to
something at a later time to improve the efficiency of the
bundles target. Meanwhile, we ought to concentrate 
on a set of external targets that are intuitively obvious,
perfectly safe and self-consistent. I think this objective
is nearly achieved. Efficiency can come later, when we are
confident of the foundations.


Regards,

Brian


------------------------------------------------------------
---------
To unsubscribe, e-mail: dev-unsubscribesip-communicator.dev.java.net
For additional commands, e-mail: dev-helpsip-communicator.dev.java.net

Hi again Brian,

Brian Burch wrote:
> I was a bit shocked by your change to the run target.
It no longer
> has ANY dependents! Isn't that too extreme? Isn't
that too
> counter-intuitive for newbies? It means the run target
has "no
> insight" into changes made to the source base...
an observer could
> download a source change from CVS and still execute the
"old"
> version.

Well, I understand what you mean but the way I see it,
having the 
possibility to do a light run (one that would not inherently
rebuild all 
your project) is more important than requiring the user to
do a make 
before the run. I don't think this is very counter
intuitive since most 
users accustomed to building projects from source are used
to the 
concept of having to "make" before running and
to the fact that running 
the project at any given moment would run the latest
binaries without 
taking into account any changes in the source that have
occurred since 
the last build. I even think that some people (me included)
count on 
such behavior.

Most IDEs would let you specify an option and choose whether
or not the 
project is to be built before every time it is run. We could
easily add 
the same thing in the build.xml. If other people on the list
agree that 
a run should also build by default and that the opposite
would confuse 
new users, then I'll agree to setting this option to true
by default 
(and keep it false for my sandbox  ).

Cheers
Emil

------------------------------------------------------------
---------
To unsubscribe, e-mail: dev-unsubscribesip-communicator.dev.java.net
For additional commands, e-mail: dev-helpsip-communicator.dev.java.net