FTBFS and piuparts failures lists for feisty

Hi,

I rebuilt all source packages in feisty, and ran piuparts on
all binary
packages.

The list of failures for both tests are available on
http
://people.debian.org/~lucas/logs/2007/02/01/
(too bad there's no people.u.c for MOTUs...)

Numbers:
========
rebuilt 12201 packages, 664 failed to build
tested 20464 using piuparts, 848 tests failed

Various details:
================
* all tests were done in i386 chroot on amd64 systems. This
is known to
cause problems with sablevm, so some build failures of java
packages
might be caused by that.

* builds were run as root. This makes a few packages fail to
build
(aegis, subversion, ...)

* internet was not accessible from the nodes. This makes a
few packages
fail to build, and makes a few more fail to install.

* I rebuilt and tested using piuparts packages from main,
restricted and
universe.

* Piuparts tests generate quite a lot of false positives.
There's a list
of known FP on [1], but the list was generated using debian
etch, so
packages not in etch won't be on that list. Common sources
of FP are:
  + the package need a configured DBMS (mysql, pgsql) to
install
  + the package reads info directly from stdin

* piuparts can detect a lot of "minor" problems
(like dangling symlinks
after package removal). Here, "failed" means that
the package failed to
install.

* the tags (e.g HEADER_NO_SUCH_FILE/NO_SUCH_FILE/GCC_ERROR)
are
regexp-based indications of which kind of failure to expect
(see
parse-logs-* in [2] for details). The script is quite dump
and might be
wrong, of course.

* Many of those bugs were fixed in Debian Etch, so that's
probably the
first place to look for a fix (29/10217 packages failed to
build, and
that's mostly false positives)

* Don't hesitate to ping me if you cannot reproduce a
specific failure.
Also, ping me when you think I should run those tests again
(I can run
them overnight).

[1] h
ttp://svn.debian.org/wsvn/collab-qa/debcluster/scripts/piupa
rts/piuparts.untestable?op=file&rev=0&sc=0
[2] http://svn.debian.org/wsvn/collab-qa/log-analys
is/?rev=0&sc=0

Computing resources were provided by the Grid'5000 project.
Grid'5000 is
an highly reconfigurable experimental Grid platform
gathering 9 sites
and featuring a total of 5000 CPUs. It serves as a testbed
for research
in Grid Computing. See https://www.grid5000.fr/

-- 
| Lucas Nussbaum
| lucaslucas-nussbaum.net   http://www.lucas-nussb
aum.net/ |
| jabber: lucasnussbaum.fr             GPG:
1024D/023B3F4F |

-- 
ubuntu-devel mailing list
ubuntu-devellists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Thanks very much for doing this!  Would it be possible to
sort the results
by component, to separate main and universe?  That way, the
core and MOTU
teams could have clear todo lists.

On Fri, Feb 02, 2007 at 09:34:50AM +0100, Lucas Nussbaum
wrote:
> Hi,
> 
> I rebuilt all source packages in feisty, and ran
piuparts on all binary
> packages.
> 
> The list of failures for both tests are available on
> http
://people.debian.org/~lucas/logs/2007/02/01/
> (too bad there's no people.u.c for MOTUs...)
> 
> Numbers:
> ========
> rebuilt 12201 packages, 664 failed to build
> tested 20464 using piuparts, 848 tests failed
> 
> Various details:
> ================
> * all tests were done in i386 chroot on amd64 systems.
This is known to
> cause problems with sablevm, so some build failures of
java packages
> might be caused by that.
> 
> * builds were run as root. This makes a few packages
fail to build
> (aegis, subversion, ...)
> 
> * internet was not accessible from the nodes. This
makes a few packages
> fail to build, and makes a few more fail to install.
> 
> * I rebuilt and tested using piuparts packages from
main, restricted and
> universe.
> 
> * Piuparts tests generate quite a lot of false
positives. There's a list
> of known FP on [1], but the list was generated using
debian etch, so
> packages not in etch won't be on that list. Common
sources of FP are:
>   + the package need a configured DBMS (mysql, pgsql)
to install
>   + the package reads info directly from stdin
> 
> * piuparts can detect a lot of "minor"
problems (like dangling symlinks
> after package removal). Here, "failed" means
that the package failed to
> install.
> 
> * the tags (e.g
HEADER_NO_SUCH_FILE/NO_SUCH_FILE/GCC_ERROR) are
> regexp-based indications of which kind of failure to
expect (see
> parse-logs-* in [2] for details). The script is quite
dump and might be
> wrong, of course.
> 
> * Many of those bugs were fixed in Debian Etch, so
that's probably the
> first place to look for a fix (29/10217 packages failed
to build, and
> that's mostly false positives)
> 
> * Don't hesitate to ping me if you cannot reproduce a
specific failure.
> Also, ping me when you think I should run those tests
again (I can run
> them overnight).
> 
> [1] h
ttp://svn.debian.org/wsvn/collab-qa/debcluster/scripts/piupa
rts/piuparts.untestable?op=file&rev=0&sc=0
> [2] http://svn.debian.org/wsvn/collab-qa/log-analys
is/?rev=0&sc=0
> 
> Computing resources were provided by the Grid'5000
project. Grid'5000 is
> an highly reconfigurable experimental Grid platform
gathering 9 sites
> and featuring a total of 5000 CPUs. It serves as a
testbed for research
> in Grid Computing. See https://www.grid5000.fr/

> -- 
> | Lucas Nussbaum
> | lucaslucas-nussbaum.net   http://www.lucas-nussb
aum.net/ |
> | jabber: lucasnussbaum.fr             GPG:
1024D/023B3F4F |
> 
> -- 
> ubuntu-devel mailing list
> ubuntu-devellists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel


-- 
 - mdz

-- 
ubuntu-devel mailing list
ubuntu-devellists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Hi Guys,

first of all, thanks Lucas for your work.

Lucas, can you fix this sbuild/buildd. There are around 149
packages with 
failed status on your list due to an error in the
sbuild/builld environment.

e.g. aqsis...all log files in your rebuild directory with 1k
are showing this.

Regards,

sh


On Friday 02 February 2007 12:51:15 Matt Zimmerman wrote:
> Thanks very much for doing this!  Would it be possible
to sort the results
> by component, to separate main and universe?  That way,
the core and MOTU
> teams could have clear todo lists.
>
> On Fri, Feb 02, 2007 at 09:34:50AM +0100, Lucas
Nussbaum wrote:
> > Hi,
> >
> > I rebuilt all source packages in feisty, and ran
piuparts on all binary
> > packages.
> >
> > The list of failures for both tests are available
on
> > http
://people.debian.org/~lucas/logs/2007/02/01/
> > (too bad there's no people.u.c for MOTUs...)
> >
> > Numbers:
> > ========
> > rebuilt 12201 packages, 664 failed to build
> > tested 20464 using piuparts, 848 tests failed
> >
> > Various details:
> > ================
> > * all tests were done in i386 chroot on amd64
systems. This is known to
> > cause problems with sablevm, so some build
failures of java packages
> > might be caused by that.
> >
> > * builds were run as root. This makes a few
packages fail to build
> > (aegis, subversion, ...)
> >
> > * internet was not accessible from the nodes. This
makes a few packages
> > fail to build, and makes a few more fail to
install.
> >
> > * I rebuilt and tested using piuparts packages
from main, restricted and
> > universe.
> >
> > * Piuparts tests generate quite a lot of false
positives. There's a list
> > of known FP on [1], but the list was generated
using debian etch, so
> > packages not in etch won't be on that list. Common
sources of FP are:
> >   + the package need a configured DBMS (mysql,
pgsql) to install
> >   + the package reads info directly from stdin
> >
> > * piuparts can detect a lot of "minor"
problems (like dangling symlinks
> > after package removal). Here, "failed"
means that the package failed to
> > install.
> >
> > * the tags (e.g
HEADER_NO_SUCH_FILE/NO_SUCH_FILE/GCC_ERROR) are
> > regexp-based indications of which kind of failure
to expect (see
> > parse-logs-* in [2] for details). The script is
quite dump and might be
> > wrong, of course.
> >
> > * Many of those bugs were fixed in Debian Etch, so
that's probably the
> > first place to look for a fix (29/10217 packages
failed to build, and
> > that's mostly false positives)
> >
> > * Don't hesitate to ping me if you cannot
reproduce a specific failure.
> > Also, ping me when you think I should run those
tests again (I can run
> > them overnight).
> >
> > [1]
> > http://svn.debian.org/wsvn/collab-qa/de
bcluster/scripts/piuparts/piuparts
> >.untestable?op=file&rev=0&sc=0 [2]
> > http://svn.debian.org/wsvn/collab-qa/log-analys
is/?rev=0&sc=0
> >
> > Computing resources were provided by the Grid'5000
project. Grid'5000 is
> > an highly reconfigurable experimental Grid
platform gathering 9 sites
> > and featuring a total of 5000 CPUs. It serves as a
testbed for research
> > in Grid Computing. See https://www.grid5000.fr/

> > --
> >
> > | Lucas Nussbaum
> > | lucaslucas-nussbaum.net   http://www.lucas-nussb
aum.net/ |
> > | jabber: lucasnussbaum.fr            
GPG: 1024D/023B3F4F |
> >
> > --
> > ubuntu-devel mailing list
> > ubuntu-devellists.ubuntu.com
> > Modify settings or unsubscribe at:
> > https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

>
> --
>  - mdz



-- 
ubuntu-devel mailing list
ubuntu-devellists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

On 02/02/07 at 03:51 -0800, Matt Zimmerman wrote:
> Thanks very much for doing this!  Would it be possible
to sort the results
> by component, to separate main and universe?  That way,
the core and MOTU
> teams could have clear todo lists.

Done for the rebuild logs (see
ubuntu-rebuild-failed-{main,restricted,universe}). For
piuparts, I would
need a list of binary packages in main, restricted, universe
and
multiverse. Is there an easy way to generate them ?

On 02/02/07 at 13:54 +0100, Stephan Hermann wrote:
> Lucas, can you fix this sbuild/buildd. There are around
149 packages
> with failed status on your list due to an error in the
sbuild/builld
> environment.
> 
> e.g. aqsis...all log files in your rebuild directory
with 1k are
> showing this.

Ah true, I missed that. I made an unrelated change to
/etc/schroot/schroot.conf during the rebuilds. Well, there's
still a lot
of other failures to analyze (and piuparts is not affected
by this).
I'll make sure those packages get tested when I run this
again in a few
weeks.
-- 
| Lucas Nussbaum
| lucaslucas-nussbaum.net   http://www.lucas-nussb
aum.net/ |
| jabber: lucasnussbaum.fr             GPG:
1024D/023B3F4F |

-- 
ubuntu-devel mailing list
ubuntu-devellists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

On 02/02/07 at 22:33 +0100, Dennis Kaarsemaker wrote:
> On vr, 2007-02-02 at 21:58 +0100, Lucas Nussbaum
wrote:
> > For piuparts, I would
> > need a list of binary packages in main,
restricted, universe and
> > multiverse. Is there an easy way to generate them
? 
> 
> awk '/^Package:/{print $2}' 
>
/var/lib/apt/lists/ch.archive.ubuntu.com_ubuntu_dists_edgy_m
ain_binary-i386_Packages
> 
> Etcetera...

Yeah, actually I should wake up before sending mails. See
ubuntu-piuparts-failed-* .
-- 
| Lucas Nussbaum
| lucaslucas-nussbaum.net   http://www.lucas-nussb
aum.net/ |
| jabber: lucasnussbaum.fr             GPG:
1024D/023B3F4F |

-- 
ubuntu-devel mailing list
ubuntu-devellists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

On Fri, Feb 02, 2007 at 09:58:52PM +0100, Lucas Nussbaum
wrote:
> On 02/02/07 at 03:51 -0800, Matt Zimmerman wrote:
> > Thanks very much for doing this!  Would it be
possible to sort the results
> > by component, to separate main and universe?  That
way, the core and MOTU
> > teams could have clear todo lists.
> 
> Done for the rebuild logs (see
> ubuntu-rebuild-failed-{main,restricted,universe}). For
piuparts, I would
> need a list of binary packages in main, restricted,
universe and
> multiverse. Is there an easy way to generate them ?

The Packages files are already organized by component.  Are
you getting your
package lists from some other source?

-- 
 - mdz

-- 
ubuntu-devel mailing list
ubuntu-devellists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

On vr, 2007-02-02 at 21:58 +0100, Lucas Nussbaum wrote:
> For piuparts, I would
> need a list of binary packages in main, restricted,
universe and
> multiverse. Is there an easy way to generate them ? 

awk '/^Package:/{print $2}' 
/var/lib/apt/lists/ch.archive.ubuntu.com_ubuntu_dists_edgy_m
ain_binary-i386_Packages

Etcetera...
-- 
Dennis K.

Time is an illusion, lunchtime doubly so.

-- 
ubuntu-devel mailing list
ubuntu-devellists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel