testing / qa process

> Karanbir Singh wrote as to:
>> The strategy to release testable rpms to
dev.centos.org

On Fri, 13 Oct 2006, Rex Dieter wrote:
> Instead of blocking on (lack-of) feedback, I'd suggest 
> considering something like:
> 1.  Put pkgs in "testing"
> 2.  If no bugs reported after X days/weeks, move out of

> testing
>
> At least this way nothing gets perpetually stalled in
testing.

Yikes.  To torture the truism, 'An absence of evidence is
not 
evidence of an absence' of problems.

Not to put too fine a point on it, but how is automatic 
promotion out of 'testing' into a chain _desireable_ in an 
enterprise oriented operating environment?

Clearly some so called 'admin's' will clearly implicitly
trust 
anything (ie., look at the constant traffic into mailing
lists 
for distributions where 'yum' is an available updater with 
horrific collections of random archives enabled).  Why take 
the reputational risk here?

It may be proper for Red Hat's Fedora, as it has evolved
(the 
firestorms I see regularly erupt on fedora-devel make me
doubt 
this, but ... those participating there without an redhat.com 
available to them are all volunteers), but not here. 
Putting 
aside stability or security issues, something as simple as 
added support load makes me want to avoid anything with an 
'official' CentOS addon status.  The 'Enemies of Carlotta' 
missed conflict thread I saw today reaffirms my doubt that 
auto-promotion works based on _assumed_ safety.

My solution, as to my archive of packagings, is simple --
Very 
general SRPM's exist, and a person who cannot solve a build 
environment and BuildRequires, (which is documented at my 
site, along with several other sites which I have
contributed 
to over the years) is probably not going to use my
packagings.

When I get a report, I address it.  I do not undertake to 
warrant to any anonymous FTP user, any ongoing (nor even 
present) security, functionality, or other pedigree to the 
packagings.  Indeed, I have marked certain unsafe ones as I 
have re-encountered them.  This makes the maintenance load 
manageable.

I have worked on outlines thinking through some of the
issues, 
on building a trustable, and 'vetted' submitted package 
infrastructure a couple of times.  All of the plans fall
apart 
on the relatively low reward for testing compared with the 
rather high and ongoing load of doing it 'right' and safely.

Before the divergence of cAos and CentOS we were discussing 
these matters:
 	http:
//www.herrold.com/caos/QA-requirements.txt
and earlier, before Red Hat's takeover of fedora.us, I had 
posted this into fedora.us's former mailing lists (the
former 
mailing list host: videl.ics.hawaii.edu no longer responds)
:
 	http://www.owlriver.com/projects/packaging/fedora-flow
.txt
[That latter document provoked Warren for what I considered 
irrational reasons.]

In part CentOS works because it has limited itself to being
a
relatively strict rebuild effort.

-- Russ Herrold
_______________________________________________
CentOS-devel mailing list
CentOS-develcentos.org
http://lists.centos.org/mailman/listinfo/centos-devel

On Thu, 2006-10-19 at 18:30 -0400, R P Herrold wrote:
> > Karanbir Singh wrote as to:
> >> The strategy to release testable rpms to
dev.centos.org
> 
> On Fri, 13 Oct 2006, Rex Dieter wrote:
> > Instead of blocking on (lack-of) feedback, I'd
suggest 
> > considering something like:
> > 1.  Put pkgs in "testing"
> > 2.  If no bugs reported after X days/weeks, move
out of 
> > testing
> >
> > At least this way nothing gets perpetually stalled
in testing.
> 
> Yikes.  To torture the truism, 'An absence of evidence
is not 
> evidence of an absence' of problems.
> 
> Not to put too fine a point on it, but how is automatic

> promotion out of 'testing' into a chain _desireable_ in
an 
> enterprise oriented operating environment?
> 
> Clearly some so called 'admin's' will clearly
implicitly trust 
> anything (ie., look at the constant traffic into
mailing lists 
> for distributions where 'yum' is an available updater
with 
> horrific collections of random archives enabled).  Why
take 
> the reputational risk here?
> 
> It may be proper for Red Hat's Fedora, as it has
evolved (the 
> firestorms I see regularly erupt on fedora-devel make
me doubt 
> this, but ... those participating there without an redhat.com 
> available to them are all volunteers), but not here. 
Putting 
> aside stability or security issues, something as simple
as 
> added support load makes me want to avoid anything with
an 
> 'official' CentOS addon status.  The 'Enemies of
Carlotta' 
> missed conflict thread I saw today reaffirms my doubt
that 
> auto-promotion works based on _assumed_ safety.
> 
> My solution, as to my archive of packagings, is simple
-- Very 
> general SRPM's exist, and a person who cannot solve a
build 
> environment and BuildRequires, (which is documented at
my 
> site, along with several other sites which I have
contributed 
> to over the years) is probably not going to use my
packagings.
> 
> When I get a report, I address it.  I do not undertake
to 
> warrant to any anonymous FTP user, any ongoing (nor
even 
> present) security, functionality, or other pedigree to
the 
> packagings.  Indeed, I have marked certain unsafe ones
as I 
> have re-encountered them.  This makes the maintenance
load 
> manageable.
> 
> I have worked on outlines thinking through some of the
issues, 
> on building a trustable, and 'vetted' submitted package

> infrastructure a couple of times.  All of the plans
fall apart 
> on the relatively low reward for testing compared with
the 
> rather high and ongoing load of doing it 'right' and
safely.
> 
> Before the divergence of cAos and CentOS we were
discussing 
> these matters:
>  	http:
//www.herrold.com/caos/QA-requirements.txt
> and earlier, before Red Hat's takeover of fedora.us, I
had 
> posted this into fedora.us's former mailing lists (the
former 
> mailing list host: videl.ics.hawaii.edu no longer
responds) :
>  	http://www.owlriver.com/projects/packaging/fedora-flow
.txt
> [That latter document provoked Warren for what I
considered 
> irrational reasons.]
> 
> In part CentOS works because it has limited itself to
being a
> relatively strict rebuild effort.
> 

Russ,

All that is true ... and for the [base] and [updates] repos
I would 100%
agree with you.

(As in ... we would not put anything in there that is not
100% vetted,
tested, etc.)

However, I disagree concerning the extras and centosplus
repos.

These are NOT core packages, there are warnings all over the
place
stating that, and they are one of the things that
distinguishes CentOS
from the other rebuild projects.  They are "OPT
IN" items that have been
tested by enterprise users and CentOS Admins and they are
important to
CentOS users ... at least the ones I know 

I understand that people can hire consultants or admins who
can build
these things themselves, however I think that the CentOS
project is
greatly enhanced as a community project when we provide
things like:

Horde, the cenotsplus kernel, maybe the Web Application
Stack, etc.

We have had NO ISSUES to date with bad quality items in any
of the
repos ... and I think everyone understands that the optional
CentOS
repos are a cut above most others where EL3/4 compatibility
and
enterprise readiness is concerned.

That is what we are trying to continue to produce.

We certainly do not want to do that to the detriment of the
base and
updates repo ... nor will anything in the testing repos be
put in either
base or updates.

Thanks,
Johnny Hughes
_______________________________________________
CentOS-devel mailing list
CentOS-develcentos.org
http://lists.centos.org/mailman/listinfo/centos-devel