Managing plugin releases

Tim Williams wrote:
> On 7/26/06, Ross Gardler <rgardlerapache.org> wrote:
> 
>> Tim Williams (JIRA) wrote:
>> > Discussed here to:
>> > http://marc.theaimsgroup.com/?t=115257903800001&a
mp;r=1&w=4
>> >
>> > I still agree with what I proposed in that
mail.  I actually like 
>> Ross' suggestion too but only if we move towards
better 
>> packaging/managing/releasing of plugins.
>>
>> What do you think is needed?
> 
> 
> o) Independent/PMC managed releases

I think that all the code infrastructure for this is in
place, all that 
is required is a release process and a user document
describing it.

> o) Improved versioning - externally identifiable.  

What do you mean "externally identifiable", is
it not sufficient i the 
current form i.e. "pluginName-0.1",
"pluginName-0.2"

> So that users might
> be able to uninstall v0.3 of a plugin and reinstall 0.2
of a plugin.

Version 0.3 and 0.2 can exist side by side in a single
Forrest 
installation, users just need to specify which version they
want and 
that's it.

There is currently no way of uninstalling an existing
plugin. This might 
be a good idea to save disk space but there is no other
reason that I 
can think of where this would be necessary.

If we want an uninstall then its just an ant task that is
required.

> I'm talking about manually not automagically.

Why manually?

Even if there is a good use case it can already be done
manually using 
the existing ant targets. It's not documented that way as
I've never 
considered a user needing to do this.

> o) Combined with the above, improved manually
downloading of plugins.

OK the plugins index page needs download links to the actual
files 
rather than the download directory and it should have
instructions on 
where to save the files.

We should also consider distributing the plugins via the
mirrors, but 
whilst they are all only a few Kb this is not a major issue.

What else needs to be improved?

However, why would anyone need to download them manually?
Cyriaque has 
solved the proxy problems as far as I can tell, what else
would force 
manual download?

Ross

On 7/26/06, Ross Gardler <rgardlerapache.org> wrote:
> Tim Williams wrote:
> > On 7/26/06, Ross Gardler <rgardlerapache.org> wrote:
> >
> >> Tim Williams (JIRA) wrote:
> >> > Discussed here to:
> >> > http://marc.theaimsgroup.com/?t=115257903800001&a
mp;r=1&w=4
> >> >
> >> > I still agree with what I proposed in
that mail.  I actually like
> >> Ross' suggestion too but only if we move
towards better
> >> packaging/managing/releasing of plugins.
> >>
> >> What do you think is needed?
> >
> >
> > o) Independent/PMC managed releases
>
> I think that all the code infrastructure for this is in
place, all that
> is required is a release process and a user document
describing it.

Ok.

> > o) Improved versioning - externally identifiable.
>
> What do you mean "externally identifiable",
is it not sufficient i the
> current form i.e. "pluginName-0.1",
"pluginName-0.2"

Shucks, I could be showing my ignorance on this plugin
versioning
stuff, but when I go to:
http://forrest.apa
che.org/plugins/
I see only plugin names, not versions.  I would expect to
see
org.apache.forrest.plugin.input.excel.v01.zip
org.apache.forrest.plugin.input.excel.v02.zip
org.apache.forrest.plugin.input.excel.v03.zip
or some such.

> > So that users might
> > be able to uninstall v0.3 of a plugin and
reinstall 0.2 of a plugin.
>
> Version 0.3 and 0.2 can exist side by side in a single
Forrest
> installation, users just need to specify which version
they want and
> that's it.
>
> There is currently no way of uninstalling an existing
plugin. This might
> be a good idea to save disk space but there is no other
reason that I
> can think of where this would be necessary.

True, I guess they just change their forrest properties.

> If we want an uninstall then its just an ant task that
is required.
>
> > I'm talking about manually not automagically.
>
> Why manually?
>
> Even if there is a good use case it can already be done
manually using
> the existing ant targets. It's not documented that way
as I've never
> considered a user needing to do this.

There are very large intranets in the world today that are
not
connected to the internet.  I don't think we can assume an
internet
connection.  What's needed is the ability to burn plugins
to CD and
use the "sneaker-net" to move it to the running
forrest application.

The documentation is needed sure, but the point was more
that
externally identifiable versioning should exist.

> > o) Combined with the above, improved manually
downloading of plugins.
>
> OK the plugins index page needs download links to the
actual files
> rather than the download directory and it should have
instructions on
> where to save the files.

Right and, I think, a list of links of the different
versions
including "Latest" an "Previous".

> We should also consider distributing the plugins via
the mirrors, but
> whilst they are all only a few Kb this is not a major
issue.
>
> What else needs to be improved?

Probably a better plugin page in general.  It's currently
an
overwhelming laundry list of plugins.  Maybe we should
consider only
listing the plugin names with one-line descriptions then
deferring the
rest of the content to the plugin docs?

> However, why would anyone need to download them
manually? Cyriaque has
> solved the proxy problems as far as I can tell, what
else would force
> manual download?

As I say above, there are some folks that are physically
disconnected
from the internet and the "automagic" stuff
doesn't work.

--tim

Tim Williams wrote:
> On 7/26/06, Ross Gardler <rgardlerapache.org> wrote:
> 
>> Tim Williams wrote:
>> > On 7/26/06, Ross Gardler <rgardlerapache.org> wrote:
>> >
>> >> Tim Williams (JIRA) wrote:
>> >> > Discussed here to:
>> >> > http://marc.theaimsgroup.com/?t=115257903800001&a
mp;r=1&w=4
>> >> >
>> >> > I still agree with what I proposed in
that mail.  I actually like
>> >> Ross' suggestion too but only if we move
towards better
>> >> packaging/managing/releasing of plugins.
>> >>
>> >> What do you think is needed?

...

>> > o) Improved versioning - externally
identifiable.
>>
>> What do you mean "externally
identifiable", is it not sufficient i the
>> current form i.e. "pluginName-0.1",
"pluginName-0.2"
> 
> 
> Shucks, I could be showing my ignorance on this plugin
versioning
> stuff, but when I go to:
> http://forrest.apa
che.org/plugins/
> I see only plugin names, not versions.  I would expect
to see
> org.apache.forrest.plugin.input.excel.v01.zip
> org.apache.forrest.plugin.input.excel.v02.zip
> org.apache.forrest.plugin.input.excel.v03.zip
> or some such.

Take a look at http://forrest
.apache.org/plugins/0.7/

see the version numbers?

It works like this:

no version number means the version is not an official
release it is an 
"in development" release. This is what is used
if no version number is 
placed in the forest.properties file.

with a version number it means it is an official release
(not that we 
have a process for official releases yet). Such releases are
stable and 
will never change. This is what is used if a version number
is expressed 
in forrest.properties

This is "documented" at [1] (docs certainly need
improvement)

Doing "ant deploy" on a plugin will deploy the
*in development* version 
this can be done much more frequently than a release as it
does not 
require the same attention to detail as a full release.

Doing "ant release" on a plugin will deploy a
*release* version, 
complete with version number.

David and I discussed this a great deal in an early Forrest
Friday. We 
started some documentation for the release process [2]

Does this satisfy your needs?

>> > I'm talking about manually not automagically.
>>
>> Why manually?
>>
>> Even if there is a good use case it can already be
done manually using
>> the existing ant targets. It's not documented that
way as I've never
>> considered a user needing to do this.
> 
> 
> There are very large intranets in the world today that
are not
> connected to the internet.  I don't think we can
assume an internet
> connection.  What's needed is the ability to burn
plugins to CD and
> use the "sneaker-net" to move it to the
running forrest application.

That has always been supported. Just create a plugin
repository on the 
Intranet and point Forrest at it.

>> > o) Combined with the above, improved manually
downloading of plugins.
>>
>> OK the plugins index page needs download links to
the actual files
>> rather than the download directory and it should
have instructions on
>> where to save the files.
> 
> 
> Right and, I think, a list of links of the different
versions
> including "Latest" an
"Previous".

...

>> What else needs to be improved?
> 
> 
> Probably a better plugin page in general.  It's
currently an
> overwhelming laundry list of plugins.  Maybe we should
consider only
> listing the plugin names with one-line descriptions
then deferring the
> rest of the content to the plugin docs?

The page is currently generated by the sitemap.xmap:

<map:match
pattern="pluginDocs/plugins_(.*)/index(|\.source).xml
" 
type="regexp">
           <map:aggregate
element="pluginList">
             <map:part
src="{lm:plugin.descriptor.forrest}"/>
             <map:part
src="{lm:plugin.descriptor.whiteboard}"/>
           </map:aggregate>
           <map:transform
src="{lm:transform.plugins.xdoc}"/>
           <map:serialize type="xml"/>
         </map:match>

(not saying you should do it, I'm putting a useful
reference here for 
anyone who has the urge - I'll link this thread to the
issue in a few 
minutes)

>> However, why would anyone need to download them
manually? Cyriaque has
>> solved the proxy problems as far as I can tell,
what else would force
>> manual download?
> 
> 
> As I say above, there are some folks that are
physically disconnected
> from the internet and the "automagic" stuff
doesn't work.

I beg to differ. See my comment above.

If there is one central Forrest installation for the whole
Intranet then 
just drop the required plugin zips into the install
directory.

If there are multiple installations of Forrest on the
Intranet then 
create a local repository and use that. Synchronise it
occasionally with 
the live one and bingo all your installations will
automagically update 
themselves.

How people get the zips is not our concern, somewhere along
the line 
someone must have access to the Internet in order to get
Forrest in the 
first plave. They can therefore get the zips using wget or a
mnaul 
download process.

Of course, if you have an idea about an even easier
process...

Ross

[1] 
http://forrest.apache.org/pluginDocs/
plugins_0_80/usingPlugins.html#version

[2] 
http://svn.apache.org/viewvc/f
orrest/trunk/plugins/RELEASE_PROCESS.txt?view=markup

On 7/26/06, Ross Gardler <rgardlerapache.org> wrote:
> Tim Williams wrote:
> > On 7/26/06, Ross Gardler <rgardlerapache.org> wrote:
> >
> >> Tim Williams wrote:
> >> > On 7/26/06, Ross Gardler <rgardlerapache.org> wrote:
> >> >
> >> >> Tim Williams (JIRA) wrote:
> >> >> > Discussed here to:
> >> >> > http://marc.theaimsgroup.com/?t=115257903800001&a
mp;r=1&w=4
> >> >> >
> >> >> > I still agree with what I
proposed in that mail.  I actually like
> >> >> Ross' suggestion too but only if we
move towards better
> >> >> packaging/managing/releasing of
plugins.
> >> >>
> >> >> What do you think is needed?
>
> ...
>
> >> > o) Improved versioning - externally
identifiable.
> >>
> >> What do you mean "externally
identifiable", is it not sufficient i the
> >> current form i.e.
"pluginName-0.1", "pluginName-0.2"
> >
> >
> > Shucks, I could be showing my ignorance on this
plugin versioning
> > stuff, but when I go to:
> > http://forrest.apa
che.org/plugins/
> > I see only plugin names, not versions.  I would
expect to see
> > org.apache.forrest.plugin.input.excel.v01.zip
> > org.apache.forrest.plugin.input.excel.v02.zip
> > org.apache.forrest.plugin.input.excel.v03.zip
> > or some such.
>
> Take a look at http://forrest
.apache.org/plugins/0.7/
>
> see the version numbers?
>
> It works like this:
>
> no version number means the version is not an official
release it is an
> "in development" release. This is what is
used if no version number is
> placed in the forest.properties file.
>
> with a version number it means it is an official
release (not that we
> have a process for official releases yet). Such
releases are stable and
> will never change. This is what is used if a version
number is expressed
> in forrest.properties
>
> This is "documented" at [1] (docs certainly
need improvement)
>
> Doing "ant deploy" on a plugin will deploy
the *in development* version
> this can be done much more frequently than a release as
it does not
> require the same attention to detail as a full release.
>
> Doing "ant release" on a plugin will deploy
a *release* version,
> complete with version number.
>
> David and I discussed this a great deal in an early
Forrest Friday. We
> started some documentation for the release process [2]
>
> Does this satisfy your needs?
>
> >> > I'm talking about manually not
automagically.
> >>
> >> Why manually?
> >>
> >> Even if there is a good use case it can
already be done manually using
> >> the existing ant targets. It's not documented
that way as I've never
> >> considered a user needing to do this.
> >
> >
> > There are very large intranets in the world today
that are not
> > connected to the internet.  I don't think we can
assume an internet
> > connection.  What's needed is the ability to burn
plugins to CD and
> > use the "sneaker-net" to move it to
the running forrest application.
>
> That has always been supported. Just create a plugin
repository on the
> Intranet and point Forrest at it.
>
> >> > o) Combined with the above, improved
manually downloading of plugins.
> >>
> >> OK the plugins index page needs download links
to the actual files
> >> rather than the download directory and it
should have instructions on
> >> where to save the files.
> >
> >
> > Right and, I think, a list of links of the
different versions
> > including "Latest" an
"Previous".
>
> ...
>
> >> What else needs to be improved?
> >
> >
> > Probably a better plugin page in general.  It's
currently an
> > overwhelming laundry list of plugins.  Maybe we
should consider only
> > listing the plugin names with one-line
descriptions then deferring the
> > rest of the content to the plugin docs?
>
> The page is currently generated by the sitemap.xmap:
>
> <map:match
pattern="pluginDocs/plugins_(.*)/index(|\.source).xml
"
> type="regexp">
>            <map:aggregate
element="pluginList">
>              <map:part
src="{lm:plugin.descriptor.forrest}"/>
>              <map:part
src="{lm:plugin.descriptor.whiteboard}"/>
>            </map:aggregate>
>            <map:transform
src="{lm:transform.plugins.xdoc}"/>
>            <map:serialize
type="xml"/>
>          </map:match>
>
> (not saying you should do it, I'm putting a useful
reference here for
> anyone who has the urge - I'll link this thread to the
issue in a few
> minutes)
>
> >> However, why would anyone need to download
them manually? Cyriaque has
> >> solved the proxy problems as far as I can
tell, what else would force
> >> manual download?
> >
> >
> > As I say above, there are some folks that are
physically disconnected
> > from the internet and the "automagic"
stuff doesn't work.
>
> I beg to differ. See my comment above.
>
> If there is one central Forrest installation for the
whole Intranet then
> just drop the required plugin zips into the install
directory.
>
> If there are multiple installations of Forrest on the
Intranet then
> create a local repository and use that. Synchronise it
occasionally with
> the live one and bingo all your installations will
automagically update
> themselves.
>
> How people get the zips is not our concern, somewhere
along the line
> someone must have access to the Internet in order to
get Forrest in the
> first plave. They can therefore get the zips using wget
or a mnaul
> download process.
>
> Of course, if you have an idea about an even easier
process...
>
> Ross
>
> [1]
> http://forrest.apache.org/pluginDocs/
plugins_0_80/usingPlugins.html#version
>
> [2]
> http://svn.apache.org/viewvc/f
orrest/trunk/plugins/RELEASE_PROCESS.txt?view=markup
>

I'm satisfied with this for this release.  I think the
download links
and plugin docs need enhancements prior to a future release
though to
remove some of the sublety of this stuff.  I've [wrongly]
been viewing
each plugin's documentation as more a demo-area for the
plugin rather
than a self-contained "micro-project" site.  I
think download links,
versions, authors, etc. should be moved there instead of the
main
plugin index page.

--tim

Ross Gardler wrote:
> Tim Williams wrote:
> >Ross Gardler wrote:
> >>Tim Williams (JIRA) wrote:
> >>> Discussed here to:
> >>> http://marc.theaimsgroup.com/?t=115257903800001&a
mp;r=1&w=4
> >>>
> >>> I still agree with what I proposed in that
mail.  I actually like 
> >>Ross' suggestion too but only if we move
towards better 
> >>packaging/managing/releasing of plugins.
> >>
> >>What do you think is needed?
> >
> >o) Independent/PMC managed releases
> 
> I think that all the code infrastructure for this is in
place, all that 
> is required is a release process and a user document
describing it.

The PMC needs to vote on each release. Part of the
new release process documentation i suppose.

> >o) Improved versioning - externally identifiable.  
> 
> What do you mean "externally identifiable",
is it not sufficient i the 
> current form i.e. "pluginName-0.1",
"pluginName-0.2"
> 
> >So that users might
> >be able to uninstall v0.3 of a plugin and reinstall
0.2 of a plugin.
> 
> Version 0.3 and 0.2 can exist side by side in a single
Forrest 
> installation, users just need to specify which version
they want and 
> that's it.
> 
> There is currently no way of uninstalling an existing
plugin. This might 
> be a good idea to save disk space but there is no other
reason that I 
> can think of where this would be necessary.
> 
> If we want an uninstall then its just an ant task that
is required.
> 
> >I'm talking about manually not automagically.
> 
> Why manually?
> 
> Even if there is a good use case it can already be done
manually using 
> the existing ant targets. It's not documented that way
as I've never 
> considered a user needing to do this.
> 
> >o) Combined with the above, improved manually
downloading of plugins.
> 
> OK the plugins index page needs download links to the
actual files 
> rather than the download directory and it should have
instructions on 
> where to save the files.
> 
> We should also consider distributing the plugins via
the mirrors, but 
> whilst they are all only a few Kb this is not a major
issue.

Yes we need to do that. Perhaps after the upcoming 0.8
release.
Needs md5 and signatures too.

Most of them are between 30 and 50 Kb. The outstanding ones
are Dispatcher (156 Kb) and Gallery (2.7 Mb).

-David

Tim Williams wrote:
> On 7/26/06, Ross Gardler <rgardlerapache.org> wrote:
> 

...

> I've [wrongly] been viewing
> each plugin's documentation as more a demo-area for
the plugin rather
> than a self-contained "micro-project" site.
 I think download links,
> versions, authors, etc. should be moved there instead
of the main
> plugin index page.

That makes good sense to me too, as long as we make it so
that they are 
auto generated from the build.xml file rather than manually
maintained 
(i.e. how it is already done in the plugins index page).

Ross