[Piuparts-commits] [SCM] piatti.debian.org configuration files annotated tag, 0.42, created. 0.42
Holger Levsen
holger at layer-acht.org
Fri Dec 23 10:44:13 UTC 2011
The annotated tag, 0.42 has been created
at 92d39e6d870fc055daf2df7ddc919618005a2453 (tag)
tagging 148332ef45997ba3aaf38cc2c9a28441083bad49 (commit)
tagged by Holger Levsen
on Fri Dec 23 11:02:37 2011 +0100
- Shortlog ------------------------------------------------------------
0.42 release
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
iQIVAwUATvRRvQkauFYGmqocAQjH5A/+POitav4r/iv730eqShN+GCt8m4BlvJwU
lROlW7K99KWUEgTeIddw+Qjo7pMw1xNVTvt8YdnHyG3Qy0YRyBQHiq4qFBkVyh11
dYPpc7qJWe9m1fAVTPbxbsD7NVY+kbrDWbsw8eL20zgMB8u2zfHric2v9AlpLinT
3SC86zM2AyJvRNih+P43D+j4ZhRcaAAQZdLcAc5CSGTiWxKpKTnNUiZGadQrhwWG
aO5xmmhoiQFTBP8aw3mO6D+nbROZO2iOc+LIYeZV6q63mSwdJBaMb0L6XlFXA2m6
uGphY6ZnbOKIUX19YOKx65eHJoR3bIpFKLGV2LpW/nIDLDVm+dUR1W5PEQPsUVB/
cOr/Is/WyWUpOnqm4S86UGejYCVJxti20DUaTojlYylH3g3Z4S3S/X/oRnC5vrKa
FFVWMY/V32LZsKSL1iP4hn9Zou6KnYSMF/ylUSyCPZPjZqmTuu2bCfeiNyW87gj4
2DvYmgm3UpE6GgqKiRayl5GGuIuA5xPHAFowC0oEakgDGnEETM7nIeabk8dk/2YV
5NE1oguGIZARPgbqbHJTkmCLQiboD6Ooo/EWJriGdeB+WezxwfbxRKZq4QTipoDA
gQxaVjsfsVBPJXOVQm2obu5hg/oqRkY98n8xDs+dnVzKoHWEUc39vSb/xydIZQrs
OzEWz0qFE3Q=
=8LmW
-----END PGP SIGNATURE-----
Andreas Beckmann (31):
detect some more piuparts issues
get circular dependencies from semistable
update known_circular_depends
add circular deps involving virtual packages
update known_circular_depends with manually added entries
update lists of circular deps
update known_circular_depends in the config files
this is still a bash script
new failure analysis: dependency_error
set chroot root password in pre_test_root_password
update regexp for missing tarball errors
use user:group syntax with chown
create backup dir
find log files with relative paths
add new cronscript to run piuparts-analyze
reschedule 25 fail logs older than 30 days
don't hardcode the section name
create output dir
don't try to match sections with wildcards
fix grep call with extended regexp
set --no-eatmydata for lenny and squeeze
generic config file parsing
use new style config parsing
count repeated archive issues
configurable delay for rescheduling untestable packages
reschedule_oldest: improve configurability
parse logfiles once for known errors
only archive *.log
allow customizing the tempdir
incrementally archive old logs
add a cron script for the daily processing
Holger Levsen (50):
fix scripts for dash as default shell. Thanks to Scott Schaefer for the patch. Closes 640649
s#svn#git# and some config updates
update script to use git instead of svn
fix some bugs to make it actually work
support more branches
add python2.7 to PYTHONPATH
start slave with right PYTHONPATH
fix order
Re-create base.tgz's every week now, as they will only be replaced if the recreation was successful.
update piuparts.conf for piatti too
Merge commit 'bf179ba6f9d87f7fd70e43824245f6af0869f4e6' into develop
Merge commit 'bf179ba6f9d87f7fd70e43824245f6af0869f4e6' into piatti
Merge branch 'develop' into piatti
send mails to piuparts-reports at lists.alioth.debian.org instead to holger+luk
announce new piuparts-reports mailinglist
share my templates for filing bugs
Merge branch 'feature/pre_test_root_password' into develop
Merge branch 'feature/cronscript-piuparts-analyze' into develop
Merge branch 'feature/reschedule-old-logs' into develop
Merge branch 'config/no-eatmydata' into develop
set --no-eatmydata for lenny2squeeze
huge speedup grepping for unknown failues (from 40m to 2s)
we only care about .log files (not .kpr)
sort sid before wheezy
also report diskusage
dont reschedule stable distros
Merge branch 'feature/generate_daily_report' into develop
use /home/holgi consistently
Merge commit '293e487305f7decb0752cbdd6f9ff38df40badc5' into develop
explain that these packages will be automatically rescheduled
Finally, upgrades from squeeze to wheezy are also being tested. Yay
wheezy and squeeze2wheezy should have same precedence
also enable it ;)
--no-eatmydata for squeeze
also catch errors from running detect_well_known_errors
also catch errors from running piuparts-report
detect tests being terminated due to excessive output.
as we have precedences now, reserve more packages
as we have precedences now, reserve more packages
temp. disable rescheduling of old successful logs to test squeez2wheezy faster
auto-reschedule is a global option
add news entry about rescheduling of old logs
run some scripts from within generate_daily_report
only send mail if there is output
only cause output if there is meaningful output
test wheezy while developing
only cause output if there is meaningful output
make this work for fresh installs
only cause output if there is meaningful output
whitespace fixup
Holger Levsen (holger) (377):
initial (old) configuration for piatti
set PYTHONPATH for master command
use tmp directory on /org partition
tmpdir is a commandline option for the piuparts command..
try to workaround tzdata upgrade problems from etch to lenny
obsolete
some debugging and apt-updates here and there..
try harder
use even more force
disable this workaround temp.
disable upgrade tests, just do install/remove tests in sid for now
workaround missing graceful config handling
make relative stats
fix syntax error
set correct directory for master and turn on logging
move piatti:~/piupartsplan.txt into this README.txt, remove unused dir
configure piuparts-reports
piuparts works better if one uses amd64 packages when running on amd64
update todo list
also mention the right arch here
update with stuff from wiki page
switch to distro specific configuration
fix paths - it's really getting time for scripts now...
a good motto is like good breakfast or such
subdirs per section are created automatically
revert configuration change, try to enable logging for the slave
document strange setup with different slave dirs...
make more generic, remove redundant stuff
fix links
- improve look.
fix image location, thanks Kmos
add script to update the setup on piatti
various cleanups and improvements
remove bla
add sidebar
provide navigation bar
* piuparts-slave.py:
fix syntax errors
piuparts-report.py
provide working configs, add crontab, update readme + todo
activate piuparts-report cronjob, rename stats.html to index.html, update todo
turn off debugging output
install cronjob correctly
update ~piupartsm as piupartsm, more verbose output
only gerate reports four times a day
inform those who can restart piupartss about reboots
add .forward files
gah, forgot to add this one too..
generate reports only twice per day, improve output a bit
add user+sudoers, convert to asciidoc
more documentation, enhance update script to create initial configuration too, put $date in versionstring if distrinution=UNRELEASED
sudo make sandwhich
sudo sudo make sandwhich
some layout improvements
improve layout, create report only once per day
more layout
more layout improvements
fix syntax error
fix syntax error by using proper templates
lenny2squeeze is not needed after all. also fix a a small error in piuparts-reports.py
* Break backwards compatibility of the configuration files for master-slave-
* Install scripts for master-slave mode without .py extension.
fix cute but annoying typos
yet more typos
~piupartsm/bin/detect_network_issues is run daily from cron and should detect failed piuparts runs due to network issues on the host.
ignore more files
- improve layout, generate navigation for the pages.
dynamically create navigation, some more polishing
* piuparts-master.py: remove code to write statistics, that is done only in
test sid and squeeze at the same frequency
add scripts to run slave in screen, to join that session and a cronjob to send the tail-output every six hours to monitor piuparts
make it work ;-) aka fix logical errors
just test sid for development
- generate sources.txt (per section) with state summary per source package
Add build-dependencies on xmlto.
piatti needs sources-urls too
add cronjob to check for leftover processes
new cronjob to monitor the slave and report problems
dont bother to send mails on development machines
fix typo
nicer subject for daily reports
send one mail, not two
only send mail about leftover_processes once per day
avoid perm probs and be more verbose about the error detected
dont cleanup at the same time the data is used
- provide links to source status pages from state pages.
automatically report untestable packages, so that they can be investigated
fix perms
add cronjob to monitor for tmp cruft, plus some minor fixes
cleanup todo list
fix syntax error
fix syntax error
fix various syntax and logic errors in monitoring scripts.
notify about broken packages, which are most likely temporarily issues in the archive
document the broken packages issue in the news section
make pusling happy: less red, no yellow, more blue.
some more css improvements
run checks earlier at night so one can cleanup in the morning
fix syntax error
use mirror on piatti
make mirror independent
- automatically reschedule uninstallable packages after 5 days (so there is some time the issue gets resolved in the archive)
- use variables to ease changing intervals
improve errorhandling, some bugfixes, disable daily screenlog mail
- draw graphs of package states over time.
display contents of tmpfiles instead of their names and be more verbose about detection methods used
add gs to suggests and disable untestable/archive feature as it's buggy
killing slaves is fine
sleep a bit longer if there is nothing to do
run checks before the report, fix a syntax error
prepare backup, add news entry about testable archive done, link to todo
fix typo and make backup even smaller
- create maintainer centric pages for each email address found in
- if uploaders if empty, dont display it
- fix get_packages_in_state() to only return unique packages and not also
- improve and fix source status overview calculation
removing the untestable/archive feature as its not particulary useful (as just dumb rescheduling for testing every 7 days should be fine as well), but breaks piupartslib/packagesdb
also detect failures on this side of the network
new script to ease mass bug filing by analysing logs and grouping common errors together. so far it only knows about violations of policy 9.1.2 and 6.8
include number of affected packages per distro
add warning about false positives
parse piuparts.conf instead of hardcoding those two values taken from there
- detect packages with miss a depends or use non-essential in purge
parsing perl configuration files in shell is painful
detect yet unknown errors too
also report issues in packages which passed the piuparts test, but only once per week
i hate the smell of syntax errors in the morning
- lots of refactoring in piuparts-report
- include index.tpl (if it exists) from output-directory into the
include the grep-pipes used to detect the errors in the output
everybody loves typos
ignore 53 issues in squeeze and 63 in sid due to #530501
- piuparts-slave: call piuparts with --mirror when mirror is set in piuparts.conf.
dont use components with "mirror" setting in .conf...
add check to detect packages which leave processes running on the system
fix typo
detect packages which try to overwrite other packages files
detect packages which dont remove owned files on purge
moving code error can cause errors.
factorize script to detect known problems, to be able to create webpages and prepare moving this code in piuparts-report.py one day
work in progress
Reschedule testing for 163 successful and 27 failing packages in sid which were affected by #530501. Once openssh 1:5.1p1-6 has reached squeeze, this will be done again with 194 packages there.
fix typos
make cronjob quiet
format correctly
format correctly, missed one
fix output and logic, bash does have some severe limitations on arrays...
those are assumed to be failures, not issues
follow my own naming convention
htmlify
typolove
yat
nicer html, uglier code
generate pages for known issues, which are detected by bash helper
polishing
improve info text
fix regex
dont warn (wrongly) about a problem if piuparts-slave has been restarted
dont complain about intercepted error
schedule at more sensible times
yay. that bug was already filed, so we can refer to it
be more verbose in the analysis
also display packages on which bugs have been filed
detect 'Some index files failed to download...' problems
document that it was agreed that these bugs are serious
dont mention conflicts, thanks weasel
document that it was agreed to file these bugs with severity serious
correctly list logs with undetected errors
only report packages as buggy if the latest version was tested unsuccessfully
be less verbose and buggy
include failures of the day in the daily piuparts-report mail
bugs are lovely
sorting is nice
detect failures due to modified conffiles
fix silly syntax errors
fix a typo and only display the last failed log for the unclassified errors
everybody loves parsing command output
count correctly and only display the latest failure
explain why this is bad
syntax error, we love you
delete chroot-tarballs once a month, so that new ones are created automatically
start slave_run automatically after reboots
detect failed builds due to missing chroot tarballs
detect purge failures better
fix pattern matching
fix detection of non-fatal issues
more bugfixing to detect known issues
yet more bugfixing
be more verbose
document that these bugs are serious
include grep command used to find failed packages into the pages displaying them
no need to notify about reboots anymore, piuparts-slave is started automatically at reboot
document rescheduling of failed packages due to #545949
Reschedule testing for 17170 (successfully tested) packages in sid, to make sure they still install fine with dependency based booting enabled now in sid.
* piuparts-report.py: report packages failing due to insserv errors.
archive old logfiles, as they confuse the grouping displayed in the webinterface - /me waves to #547674
provide some more help
added missing mkdir
detect update-rc.d warnings
improve wordings
fix typo
lets call this a typo. fixed :-)
fix logic for real
bugfix
details, details
skip minimizing, as those chroots are created minimized already
bugfix
add news entry about rescheduling 2000 packages
fix logic bug
detect another kind of network issue
fix regex
silently start the slave
test for symlinks, but warn only
fix typo
report broken symlinks on the webpages
add changelog entry, improve helptext
split view
fix typo
announce rescheduling of all failed packages
emphasize
news: Reschedule testing for 14287 successfully tested packages in sid, those in squeeze will be rescheduled once all testable package in sid have been tested again. This is because piuparts now creates and maintains chroots securily (using gpg signed Release files for both debootstrap and apt-get) and because it warns if broken symlinks are found in a package.
correct regex to find broken symlinks
fix sed expressions
provide link to the configuration used on piatti and how the setup was done
After testing 14416 packages in sid in three days, reschedule 15944 packages in squeeze... see previous entry for an explanation why.
reminder for a useful option one day...
better describe tests done and 'merge' files
piuparts-report.py: report packages which failed due to broken maintainer
support less wide browser windows better
reword plans slighty
announce rescheduling of failed packages to catch broken symlinks
brag about circ depends being tested now :-)
- report packages which failed due to output from cron after removal and due to not enough force being used.
improve html output
detect more errors from cron and fix counting of detected errors
fix css error
delete (and thus reschedule) detected logfiles
restart the slave to recreate distro chroot.tar.gz's after they have been deleted
restructure about+overview page
Reschedule testing for 319 failed packages in sid and 544 in squeeze, since --warn-on-others is now used.
emphasize submitted bugs and thus, unsubmitted ones too
emphasize bugs to filed, not filed ones
fix typo, lighten up the blue
detect problems due to a piuparts problem not well understood yet
report packages which failed due to a problem with pre-depends
- support python 2.5 and 2.6 instead of 2.4 and 2.5
current test configuration
test squeeze and lenny2squeeze seperatly
add new check for stale reserved packages and prepare crontab for daily report webpage
emphasize squeeze over sid and maintainer over package
update news
* piuparts.py:
detect failures due to broken logrotate scripts
add news about leftovers after purge being ignored for squeeze and lenny2squeeze now
alarming over left over processes every 6h is enough
disable sid as long as kdebinding produces depends with a broken syntax, use lenny packages+sources for lenny2squeeze
ignore empty logs
add note about sid not being tested currently
- report packages which failed,
fix more grammar
this is faulty too, but at least produces better results (more packages being tested) for now
delete failed logs due to network issues
fix path
Finally enabled testing of sid again.
update history
From today on, broken logrotate scripts after purge are only reported in sid
fix typo
Most of these packages are probably not buggy but rather affected by #582630
news: Schedule all 27438 passed packages in squeeze for re-testing now that squeeze is frozen.
start work on putting the known circular depends into the configfile
more
more testing
is that the format we need
getting closer
getting closer
another attempt at reading known_circular_depends from the configfile
Finally, get rid of hardcoded circular depends in the library, instead read them from the configfile.
add comment from where the lists where taken and when they were last updated
fix errors in known circular depends lists
refactor screenlog handling
mail screenlog when the slave crashes
fix syntaxo
improve usability :)
explain the problem a bit better
fix speling, thanks KiBi
- report packages which failed
fix german dates spotted by KiBi
<b>2010-11-06</b>: The lists of known circular depends is now taken from http://debian.semistable.com/debgraph.out.html and maintained seperatedly (and maually) for each tested distribution in piuparts.conf - this is not optimal (which would be piuparts detecting them automatically) but much better than the hardcoded list which we had in the piuparts library since December 2009.
try to catch all logs with these problem
proper link
update list with data from Robert
<b>2010-11-12</b>: Schedule all 108 failed packages in squeeze for re-testing.
better variable names
fix script to really archive all old logs
improve code to handle bugged state
improve code to handle bugged state
fix typo
activate code \o/
make output more useful
adjust code to match comment
fix typo
less picky
adjust explaination to code
add links to bts next to the logfiles
also link to the bts
link to bugs in sid
merge changes from piatti to goldwasser
be less verbose
<b>2010-11-14</b>: Schedule all 429 failed packages in sid for re-testing.
dont believe statistics you didnt fake yourself
lenny2squeeze testing takes so long...
only alarm if slave log hasnt changed for more than 31min, the slave is allowed to sleep 30min once in a while
really find all new failed logs from the last 24h
Disable the logrotate test until #582630 is fixed and reschedule all 51 packages in sid failed due to it.
starting to play with hooks to deal with exceptions
be more verbose
debug output: which package(s) are being tested
fix
almost there
the first package treated specially: sudo
proper structure
the 2nd package treated specially: fai-nfsroot
fai-nfsroot needs cleanup
set root passwd for sudo
the 3rd package treated specially: ltsp-client-core
the 4th package treated specially: apt-listbugs
post_install is actually run after install+upgrade
shorter script names
restore /etc before purge to avoid conffile prompt
try harder to make buggy upgrades work with apt-listbugs installed :)
set a root password unconditionally as there are packages depending on sudo (and sudo refuses removal if no root passwd is set)
explain exceptions and announce rescheduling of all failures in lenny2squeeze
upstart and file-rc: force installation and removal of essential package (sysv-rc|sysvinit)
use more force: Yes, do as I say!
upstart and file-rc: reinstall essential package (sysv-rc|sysvinit) else removal wont work
i bet this aint enough
use more force: Yes, do as I say!
we need to cleanup, piuparts detects leftover files. d'oh.
hmm
deal with packages depending on upstart too
everybody loves typos
configure apt to use force-yes for these exceptions
revert changes. one can put 'force-yes' in apt.conf, but not 'Yes, do as I say'... see #605127
s/rc buggy/RC buggy/
also find and delete stale reservations from the slave
dont report failures due to remove being disabled anymore, as remove is now allowed
<b>2010-11-28</b>: debconf-english is the seventh package getting special treatment by piuparts: before removal, debconf-i18n is installed (see <a href="http://bugs.debian.org/539146" target="_blank">#539146</a> has the details and the news entry for 2010-11-25 lists the other six packages.)
doh. forgot to cleanup. see r829 for the rest of this commit
fix grammar
treat ltsp-client as ltsp-client-core as it depends on it
ignore warning which doesnt cause error
<b>2011-01-03</b>: Reschedule 12306 successful and 8 bugged logs in lenny2squeeze for re-testing. Those are logs older than 148 days, which refers to when Squeeze was initially frozen (2010-08-06). Deep freeze was announced on 2010-12-13 and there are 3800 logs older then that too, but for future deletions it's better to use 2010-01-03 (=commit r857), which fixes a bug in piuparts-slave resulting in using the sid packages file for lenny2squeeze tests.
lenny2squeeze is using the squeeze packages file, so it should also use squeeze's list of circular depends
update lists of known circular depends taken from http://debian.semistable.com/debgraph.out.html
ignore trash
<b>2011-01-15</b>: Reschedule 10123 successful and failed logs in lenny2squeeze for re-testing. Those are logs which have been tested before Squeeze was deep frozen or while there was still a bug in piuparts-slave, see last news entry for details.
<b>2011-01-25</b>: Reschedule 27655 successfully tested packages in Squeeze, since they were tested before the deep freeze. Yesterday all 70 failed and bugged packages were rescheduled too, which surprisingly led to
<b>2009-07-24</b>: <a href=http://bugs.debian.org/531349 target=_blank>#531349</a> has been fixed, pi
<b>2011-02-07</b>: Add <b>wheezy</b>! Whoohoo!
be more verbose
exit earlier
exit cleanly, if config doesnt exist
piatti has been upgraded to squeeze, thanks to zobel and the other DSA team members
new daily cronjob: reschedule 100 oldest logs per section (if they are older then 120 days)
update lists of circular depends
update
add news entry about rescheduling of old logs. only reschedule in sid+wheezy, after 180 days, 200 logs per day maximum
fix typo
'Hash Sum mismatch' is another failure option
lists of known circular depends taken from http://debian.semistable.com/debgraph.out.html
<b>2011-07-10</b>: systemd-sysv is the eighth package getting special treatment by piuparts as it needs removal of sysvinit before installation and installation of that package before removal...
switch to new pathes from dh_python2
* piuparts-slave:
switch to new pathes from dh_python2
spread the news about dpkg --force-unsafe-io and dont do that for lenny2squeeze.
- turn MAX_AGE_TGZ into a configuration option
also detect 'E: Error, pkgProblemResolver::Resolve generated breaks'
Luk Claes (luk) (6):
Move piuparts configuration into service directory
Move config from etc/piuparts to etc/ as we don't have any other configuration
Update path to new installation place
Update piuparts invokation to new location
Use other mirror for now
Also update the packages-url for now
-----------------------------------------------------------------------
--
piatti.debian.org configuration files
More information about the Piuparts-commits
mailing list