[Piuparts-commits] [SCM] piuparts git repository branch, develop, updated. 0.48-55-gc89d173
Holger Levsen
holger at layer-acht.org
Thu Dec 20 19:14:54 UTC 2012
The following commit has been merged in the develop branch:
commit 01eb1531c5b3ba07d6bffdda1f5beee20a04c0cf
Merge: a22376eaa361fb5edca08554f7b864777c649c43 75a4ce7273ef9f7c80fa61d288c936eeb24f0a8e
Author: Holger Levsen <holger at layer-acht.org>
Date: Thu Dec 20 20:09:51 2012 +0100
Merge remote branch 'andreas/preview/warn-on-debsums-errors' into develop
diff --combined debian/changelog
index 8e50d0d,547aa02..b6377e1
--- a/debian/changelog
+++ b/debian/changelog
@@@ -11,44 -11,28 +11,46 @@@ piuparts (0.49) UNRELEASED; urgency=lo
* piuparts.py:
- Run custom scripts only once if dependencies are installed via a
metapackage.
+ - Rename --no-debsums option to --warn-on-debsums-errors.
- Add --extra-repo=<sources.list line> option for adding extra
package sources, e.g. backports, security or local repositories.
The argument must be a valid line (including whitespace) that is added
to sources.list verbatim. The option may be repeated to add more lines.
+ - Qualify to the package names extracted from .debs with the version.
+ - Add --testdebs-repo=<sources.list line> option. Provide the package(s)
+ to be tested (and additional dependencies that are not yet in the
+ archive) in a repository to enable dependency resolution with apt-get.
+ Use for complex install and upgrade scenarios.
+ - Add --distupgrade-to-testdebs option. During distupgrade tests override
+ packages from the target distro with the packages from the testdebs
+ repository. This allows doing distupgrade tests before uploading.
* piuparts.conf:
* piupartslib/conf.py:
- Add get_final_distro() method.
- Add optional distro argument to get_mirror() method.
+ * piupartslib/packagesdb.py:
+ - Speed up LogDB by fetching existing files with os.listdir().
* piuparts-master.py:
* piuparts-slave.py:
- Disable X forwarding while connecting to master.
- Move checks for package existence to test_package().
+ * piuparts-analyze.py:
+ - Sort bugs and try the newest bug first.
* piuparts-report.py:
- Fix URLs to piatti's config.
- Hardlink the logfiles to the htdocs tree (with copy as fallback).
+ * post_setup_squeeze-fake-essential: Restrict fake-essential install-info to
+ the 4 packages that actually need this.
+ * New custom script: pre_remove_40_find_obsolete_conffiles, report obsolete
+ conffiles after package upgrades. Suggested by Michael Biebl.
* pre_remove_50_find_missing_copyright: Skip check for packages that are not
installed or have only config files remaining.
* post_remove_cleanup: Cleanup dovecot certificates.
+ * Add support for installing and upgrading to ia32-libs/wheezy/amd64.
* Merge known problem analyses excessive_output and runtime_exceeded into
new known problem resource_violation.
+ * New known problem: Leaving obsolete conffiles after upgrade.
+ * New known problem: Modifying conffiles or other shipped files (issue).
* New bug template: partial-upgrade-file-overwrite.
-- Andreas Beckmann <debian at abeckmann.de> Fri, 30 Nov 2012 13:11:25 +0100
diff --combined piuparts-report.py
index 02d7b87,82165f6..b8ecf38
--- a/piuparts-report.py
+++ b/piuparts-report.py
@@@ -342,6 -342,7 +342,7 @@@ state_by_dir =
linktarget_by_template = [
("initdscript_lsb_header_issue.tpl", "but logfile contains update-rc.d issues"),
("command_not_found_issue.tpl", "but logfile contains 'command not found'"),
+ ("debsums_mismatch_issue.tpl", "but logfile contains modified conffiles or other shipped files"),
("alternatives_after_purge_issue.tpl", "but logfile contains forgotten alternatives"),
("owned_files_after_purge_issue.tpl", "but logfile contains owned files existing after purge"),
("unowned_files_after_purge_issue.tpl", "but logfile contains unowned files after purge"),
@@@ -349,7 -350,6 +350,7 @@@
("installs_over_symlink_issue.tpl", "but package installs something over existing symlinks"),
("broken_symlinks_issue.tpl", "but logfile contains 'broken symlinks'"),
("packages_have_been_kept_back_issue.tpl", "but logfile contains 'packages have been kept back'"),
+ ("obsolete_conffiles_issue.tpl", "but logfile reports obsolete conffiles"),
("dependency_error.tpl", "due to unsatisfied dependencies"),
("packages_have_been_kept_back_error.tpl", "...and logfile also contains 'packages have been kept back'"),
@@@ -380,7 -380,6 +381,7 @@@
("logrotate_error_after_removal_error.tpl", "due to errors from logrotate after removal"),
("installs_over_symlink_error.tpl", "...and package installs something over existing symlinks"),
("broken_symlinks_error.tpl", "...and logfile also contains 'broken symlinks'"),
+ ("obsolete_conffiles_error.tpl", "...and logfile reports obsolete conffiles"),
("unknown_failures.tpl", "due to unclassified failures"),
]
diff --combined piuparts.1.txt
index 26a6406,8c1a005..2bb257c
--- a/piuparts.1.txt
+++ b/piuparts.1.txt
@@@ -76,11 -76,6 +76,11 @@@ The tarball can be created with the '-s
a new one with debootstrap. This is similar to '--basetgz', but the contents
are not archived.
+*--distupgrade-to-testdebs*::
+ Use the "testdebs" repository to override the packages in the distupgrade
+ target distribution. This allows to test complex upgrade paths before the
+ packages enter the archive.
+
*--extra-repo*='deb-line'::
Provide an additional (unparsed) line to be appended to sources.list, e.g.
'deb <URL> <distrib> <components>' or
@@@ -137,9 -132,6 +137,6 @@@ The 'components' that are used for a mi
+
Note that file: addresses works if the directories are made accessible from within the chroot with '--bindmount'.
- *--no-debsums*::
- Don't run debsums to check for modified or missing files.
-
*--no-diversions*::
Don't check for broken diversions.
@@@ -194,19 -186,6 +191,19 @@@
*--skip-logrotatefiles-test*::
Skip testing the output from the logrotate files left in the system after remove a package.
+*--testdebs-repo='deb-line'::
+ Provide an additional (unparsed) line to be appended to sources.list, e.g.
+ 'deb <URL> <distrib> <components>' or
+ 'deb file:// </bind/mount> ./'
+ The "testdebs" repository provides the packages to be tested (and some
+ additional dependencies, if needed, e.g. all packages built from the same
+ source package as the (binary) package being tested) and can be used for
+ testing complex installation and upgrade scenarios involving dependencies
+ that are not yet in the archive. This repository will be available only
+ for installing the target packages. Dependency resolution will be done by
+ apt-get. The packages to be tested can be passed as .debs or as package
+ names (with '--apt').
+
*-t directory, --tmpdir*='directory'::
Use directory as the place where temporary files and directories are created. The default is the environment variable *TMPDIR*, or _/tmp_ if not set.
+
@@@ -218,6 -197,9 +215,9 @@@ Note: the temporary directory must *not
*-V, --version*::
Write out the version number of the program.
+ *--warn-on-debsums-errors*::
+ Print a warning rather than failing if debsums reports modified files.
+
*--warn-on-leftovers-after-purge*::
Print a warning rather than failing if files are left behind after purge.
diff --combined piuparts.py
index c9b792c,15f8b26..27846cd
--- a/piuparts.py
+++ b/piuparts.py
@@@ -139,7 -139,6 +139,7 @@@ class Settings
# distro setup
self.debian_mirrors = []
self.extra_repos = []
+ self.testdebs_repo = None
self.debian_distros = []
self.keep_sources_list = False
self.do_not_verify_signatures = False
@@@ -164,13 -163,12 +164,13 @@@
self.extra_old_packages = []
self.skip_cronfiles_test = False
self.skip_logrotatefiles_test = False
- self.check_debsums = True
self.check_broken_diversions = True
self.check_broken_symlinks = True
self.warn_broken_symlinks = True
self.warn_on_others = False
self.warn_on_leftovers_after_purge = False
+ self.distupgrade_to_testdebs = False
+ self.warn_on_debsums_errors = False
self.ignored_files = [
# piuparts state
"/usr/sbin/policy-rc.d",
@@@ -909,18 -907,6 +909,18 @@@ class Chroot
create_file(self.relative("etc/apt/sources.list"),
"".join(lines))
+ def enable_testdebs_repo(self, update=True):
+ if settings.testdebs_repo:
+ logging.debug("enabling testdebs repository '%s'" % settings.testdebs_repo)
+ create_file(self.relative("etc/apt/sources.list.d/piuparts-testdebs-repo.list"), settings.testdebs_repo + "\n")
+ if update:
+ self.run(["apt-get", "update"])
+
+ def disable_testdebs_repo(self):
+ if settings.testdebs_repo:
+ logging.debug("disabling testdebs repository")
+ remove_files([self.relative("etc/apt/sources.list.d/piuparts-testdebs-repo.list")])
+
def create_apt_conf(self):
"""Create /etc/apt/apt.conf.d/piuparts inside the chroot."""
lines = ['APT::Get::Assume-Yes "yes";\n']
@@@ -1104,9 -1090,6 +1104,9 @@@
self.install_packages_by_name(packages, with_scripts=with_scripts)
def install_package_files(self, package_files, packages=None, with_scripts=False):
+ if packages and settings.testdebs_repo:
+ self.install_packages_by_name(packages)
+ return
if package_files:
self.copy_files(package_files, "tmp")
tmp_files = [os.path.basename(a) for a in package_files]
@@@ -1178,14 -1161,12 +1178,12 @@@
added = [ln for ln in post_install_diversions if not ln in pre_install_diversions]
return (removed, added)
- def check_debsums(self, ignore_errors=False):
- if not settings.check_debsums:
- return
+ def check_debsums(self):
(status, output) = run(["debsums", "--root", self.name, "-ac"], ignore_errors=True)
if status != 0:
logging.error("FAIL: debsums reports modifications inside the chroot:\n%s" %
indent_string(output.replace(self.name, "")))
- if not ignore_errors:
+ if not settings.warn_on_debsums_errors:
panic()
def list_paths_with_symlinks(self):
@@@ -1976,18 -1957,9 +1974,18 @@@ def get_package_names_from_package_file
vlist = []
for filename in package_files:
(status, output) = run(["dpkg", "--info", filename])
+ p = None
+ v = None
for line in [line.lstrip() for line in output.split("\n")]:
- if line[:len("Package:")] == "Package:":
- vlist.append(line.split(":", 1)[1].strip())
+ if line.startswith("Package:"):
+ p = line.split(":", 1)[1].strip()
+ if line.startswith("Version:"):
+ v = line.split(":", 1)[1].strip()
+ if p is not None:
+ if v is not None:
+ vlist.append(p + "=" + v)
+ else:
+ vlist.append(p)
return vlist
# Method to process a changes file, returning a list of all the .deb packages
@@@ -2121,8 -2093,6 +2119,8 @@@ def install_purge_test(chroot, chroot_s
# Install packages into the chroot.
os.environ["PIUPARTS_PHASE"] = "install"
+ chroot.enable_testdebs_repo()
+
chroot.run_scripts("pre_install")
if settings.warn_on_others or settings.install_purge_install:
@@@ -2208,8 -2178,6 +2206,8 @@@
file_owners = chroot.get_files_owned_by_packages()
+ chroot.disable_testdebs_repo()
+
# Remove all packages from the chroot that weren't there initially.
chroot.restore_selections(chroot_state["selections"], packages)
@@@ -2240,8 -2208,6 +2238,8 @@@ def install_upgrade_test(chroot, chroot
# Then from the package files.
os.environ["PIUPARTS_PHASE"] = "upgrade"
+ chroot.enable_testdebs_repo()
+
chroot.install_packages(package_files, packages)
chroot.check_for_no_processes()
@@@ -2249,8 -2215,6 +2247,8 @@@
file_owners = chroot.get_files_owned_by_packages()
+ chroot.disable_testdebs_repo()
+
# Remove all packages from the chroot that weren't there initially.
chroot.restore_selections(chroot_state["selections"], packages)
@@@ -2366,23 -2330,14 +2364,23 @@@ def install_and_upgrade_between_distros
os.environ["PIUPARTS_PHASE"] = "distupgrade"
chroot.upgrade_to_distros(settings.debian_distros[1:-1], distupgrade_packages)
+
+ if settings.distupgrade_to_testdebs:
+ chroot.enable_testdebs_repo(update=False)
+
chroot.upgrade_to_distros(settings.debian_distros[-1:], distupgrade_packages)
chroot.check_for_no_processes()
os.environ["PIUPARTS_PHASE"] = "upgrade"
+ if not settings.distupgrade_to_testdebs:
+ chroot.enable_testdebs_repo()
+
chroot.install_packages(package_files, [p for p in packages_qualified if not p.endswith("=None")])
+ chroot.disable_testdebs_repo()
+
chroot.check_for_no_processes()
file_owners = chroot.get_files_owned_by_packages()
@@@ -2490,10 -2445,6 +2488,10 @@@ def parse_command_line()
action='store_true',
help="Do not verify signatures from the Release files when running debootstrap.")
+ parser.add_option("--distupgrade-to-testdebs", default=False,
+ action='store_true',
+ help="Use the testdebs repository as distupgrade target.")
+
parser.add_option("-e", "--existing-chroot", metavar="DIR",
help="Use DIR as the contents of the initial " +
"chroot, instead of building a new one with " +
@@@ -2560,10 -2511,6 +2558,10 @@@
help="Additional (unparsed) lines to be appended to sources.list, e.g. " +
"'deb <URL> <distrib> <components>' or 'deb file://</bind/mount> ./'")
+ parser.add_option("--testdebs-repo",
+ help="A repository that contains the packages to be tested, e.g. " +
+ "'deb <URL> <distrib> <components>...' or 'deb file://</bind/mount> ./'")
+
parser.add_option("--no-diversions", action="store_true",
default=False,
help="Don't check for broken diversions.")
@@@ -2630,10 -2577,6 +2628,6 @@@
action="store_true", default=False,
help="Skip testing the output from the logrotate files.")
- parser.add_option("--no-debsums",
- action="store_true", default=False,
- help="Skip running debsums in the chroot.")
-
parser.add_option("--skip-minimize",
action="store_true", default=True,
help="Skip minimize chroot step. This is the default now.")
@@@ -2671,6 -2614,11 +2665,11 @@@
help="Print a warning rather than failing if "
"files are left behind after purge.")
+ parser.add_option("--warn-on-debsums-errors",
+ action="store_true", default=False,
+ help="Print a warning rather than failing if "
+ "debsums reports modified files.")
+
parser.add_option("--fail-on-broken-symlinks", action="store_true",
default=False,
help="Fail if broken symlinks are detected.")
@@@ -2724,21 -2672,19 +2723,21 @@@
[settings.extra_old_packages.extend([i.strip() for i in csv.split(",")]) for csv in opts.extra_old_packages]
settings.skip_cronfiles_test = opts.skip_cronfiles_test
settings.skip_logrotatefiles_test = opts.skip_logrotatefiles_test
- settings.check_debsums = not opts.no_debsums
settings.check_broken_diversions = not opts.no_diversions
settings.check_broken_symlinks = not opts.no_symlinks
settings.warn_broken_symlinks = not opts.fail_on_broken_symlinks
settings.warn_on_others = opts.warn_on_others
settings.warn_on_leftovers_after_purge = opts.warn_on_leftovers_after_purge
+ settings.warn_on_debsums_errors = opts.warn_on_debsums_errors
settings.ignored_files += opts.ignore
settings.ignored_patterns += opts.ignore_regex
settings.pedantic_purge_test = opts.pedantic_purge_test
if not settings.pedantic_purge_test:
settings.ignored_patterns += settings.non_pedantic_ignore_patterns
+ settings.distupgrade_to_testdebs = opts.distupgrade_to_testdebs
settings.extra_repos = opts.extra_repo
+ settings.testdebs_repo = opts.testdebs_repo
log_file_name = opts.log_file
--
piuparts git repository
More information about the Piuparts-commits
mailing list