[Piuparts-commits] [SCM] piuparts git repository branch, master, updated. eda668423fa87898c59d1075118693714aa5a053
Holger Levsen
holger at layer-acht.org
Fri Dec 23 10:25:34 UTC 2011
The following commit has been merged in the master branch:
commit aca8bd985bac2797582c62ab0056f16d4cbe0453
Author: Holger Levsen <holger at layer-acht.org>
Date: Thu Nov 10 10:22:25 2011 +0100
Remove whitespaces from whitespace-only lines.
diff --git a/ChangeLog b/ChangeLog
index b3cbb02..2d47953 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -11,23 +11,23 @@
2005-10-17 Lars Wirzenius <liw at iki.fi>
* Version 0.12.
-
+
* /var/log/faillog ignored by default.
-
+
* Documented that upgrade testing to experimental doesn't work.
2005-10-14 Lars Wirzenius <liw at iki.fi>
* Version 0.11.
-
+
* Summary of changes:
-
+
- Checks for missing package names on command line immediately,
not after creating chroot.
-
+
- apt-get in the chroot gets configured to allow unauthenticated
repositories.
-
+
- Tweaks to list of files ignored by default.
2005-09-15 Lars Wirzenius <liw at iki.fi>
@@ -37,9 +37,9 @@
2005-09-15 Lars Wirzenius <liw at iki.fi>
* piuparts.py: When creating policy-rc.d, make it executable.
-
+
* piuparts.docbook: Added missing "if". Thanks, Jonas Meurer.
-
+
* run-piuparts.py: Wrote.
2005-09-09 Lars Wirzenius <liw at iki.fi>
@@ -72,7 +72,7 @@
code. Also, log everything identically to stdout and log file,
since the old behavior was confusing to many people, as observed
on IRC and at Debconf5. This also obsoletes -v (--verbose).
-
+
* piuparts.docbook: Document that -v (--verbose) is obsolete.
2005-08-30 Lars Wirzenius <liw at iki.fi>
@@ -90,10 +90,10 @@
of requiring that they match everything. This is less surprising
to users and they can still use ^ and $ to anchor a match to the
ends, if they need it.
-
+
* Makefile: Ignore the fdmount temp files that cause upgrades to
etch/sid fail at the moments, and re-enable upgrade tests.
-
+
* piuparts.py, piuparts.docbook: Added option -t and changed the
default location for temporary files and directorys (including
the chroot) to be $TMPDIR (or /tmp if not set).
@@ -102,15 +102,15 @@
* Makefile: Upgrade testing from sarge to etch fails for me, because
of something to do with fdutils, so I have disabled it for now.
-
+
* piuparts.py: Only use the first mirror found in sources.list,
which is anyway what the manual page claims we are doing. This
should reduce the problems people have been having with piuparts
creating broken source.lists in the chroot when people have more
than a canonical Debian repository listed.
-
+
* piuparts.py: Remove temporary files upon error exit.
-
+
* piuparts.docbook: Use code names instead of
stable/testing/unstable in the example.
@@ -127,9 +127,9 @@
* piuparts.py: Create a /usr/sbin/policy-rc.d inside the chroot
when it is created. Suggested by Frank Lichtenheld.
-
+
* piuparts.py: Fixed a message from "FAIL:" to "PASS:". Oops.
-
+
* Makefile: Commented out an explicit mirror definition so it
should now use sources.list defaults.
@@ -138,7 +138,7 @@
* piuparts.py: Suggested by Frank Lichtenheld: don't run dpkg
and do other work if there are no files (cleans up log file,
saves a couple of seconds).
-
+
* piuparts.py, piuparts.docbook: Implemented and documented a
limit on the size of outputs of command piuparts runs.
@@ -213,7 +213,7 @@
* piuparts.py: Report version and command line arguments at
startup.
-
+
* piuparts.py: When calling install_purge_test, report package
list when command line args are package names, instead of empty
list.
@@ -290,7 +290,7 @@
* piuparts.py: Added /var/cache/debconf and
/var/cache/debconf/passwords.dat to the list of filenames that
are automatically ignored.
-
+
* piuparts.py: After unpacking a base tarball, run apt-get update
and clean on it, to be sure everything is up to date.
diff --git a/NEWS b/NEWS
index b2459fd..ad05cd8 100644
--- a/NEWS
+++ b/NEWS
@@ -31,16 +31,16 @@ New features
Piuparts now checks for symlinks whose target does not
exist.
-
+
Option parsing code has been rewritten, and --help now
works better.
-
+
The chroot is now minimized before used: all unnecessary
packages are purged.
-
+
/dev/MAKEDEV, /etc/nologin, /usr/doc/cpio,
/var/spool/cron added to default ignores.
-
+
A version number may now begin with a + character. There
was a package that did that and piuparts crashed.
@@ -52,11 +52,11 @@ Bug fixes
The configuration files of piuparts-master/slave are
now documented in the README.
-
+
The Python profiler is no longer used. It used to be,
but that was a leftover from development (also known
as failure to read diffs before committing).
-
+
When testing upgrades between distributions, piuparts
now makes sure that the packages being tested are upgraded,
even if it means removing an old version of a dependency.
@@ -70,11 +70,11 @@ New features
chroot after an installation or a purge has completed.
This check then finds packages that don't use invoke-rc.d
to start new processes.
-
+
A number of new default ignores have been added:
/etc/modprobe.d, compiled versions of debconf's Python
modules, papercut, ssl certificates.
-
+
/proc is now mounted (and unmounted) inside the chroot.
@@ -94,15 +94,15 @@ New features
for earlier versions of the same packages, and if so,
moves them automatically around. This saves a bit of
manual works. Thanks to Goswin Brederlow for the idea.
-
+
When piuparts creates a chroot from a tarball (option
-b), it now upgrades it before using it.
A number of new entries to the default ignores list.
-
+
Log files are now created with permissions that are
0666 modified with the process umask.
-
+
piuparts-report.py has been optimized somewhat.
Version 0.15
@@ -114,15 +114,15 @@ Bug fixes
< and > (they're deprecated but one or two packges still
use them). It also now allows underscores in package
names because of the type-handling package.
-
+
Small fixes to the manual page.
-
+
New features and significant user visible changes
piuparts-master now understands Provides: headers.
-
+
A number of new entries to the default ignores list.
-
+
New option --keep-sources-list from John Wright.
@@ -133,12 +133,12 @@ Bug fixes
Specifications for ignoring some directories were buggy
and have now been fixed: /var/spool/news, /var/lib/cvs.
-
+
When testing a .deb file given on the command line,
if any of its dependencies were missing, the package
itself would be removed instead of piuparts reporting
an error. This has been fixed.
-
+
The check for whether a package is marked untestable
for piuparts-master now works.
@@ -147,7 +147,7 @@ New features and significant user visible changes
New program piuparts-report.py produces some "statistics"
about packages and their status with regard to testing
with piuparts-slave.
-
+
The chroot is always set up for piuparts, even if it is
unpacked from a tarball. This reduces problems with
out-of-date chroots and with using the pbuilder base.tgz
@@ -173,10 +173,10 @@ New features and other user visible changes
piuparts-report.py. Since these are not useful for most
users, they're not installed on $PATH, but in
/usr/share/piuparts instead.
-
+
The slave part also runs upgrade tests between Debian
releases, which run-piuparts.py didn't.
-
+
Some additional files are ignored by default when
comparing the state of the chroot before and after
package installation.
diff --git a/README.txt b/README.txt
index 43921b3..dab3161 100644
--- a/README.txt
+++ b/README.txt
@@ -68,7 +68,7 @@ tarball with the command (again, as root):
----
# pbuilder create
----
-
+
then you only have to remember to update this tarball with:
----
@@ -234,7 +234,7 @@ don't have direct network access.
=== Distributed piuparts testing protocol
-
+
The slave machine and the piuparts-master program communicate
using a simplistic line based protocol. SSH takes care of
authentication, so there is nothing in the protocol for that. The
@@ -244,7 +244,7 @@ single line) or long (a status line plus additional data lines).
Simple commands and responses are of the following format:
'keyword arg1 arg2 arg3 ... argN'
-
+
The keyword is a command or status code ("ok"), and it and the
arguments are separated by spaces. An argument may not contain a
space.
@@ -270,7 +270,7 @@ the master responds with):
>> reserve
<< ok vorbisgain 2.3-4
----
-
+
Here the slave first reports a successful test of package liwc,
version 1.2.3-4, and sends the piuparts log file for it. Then it
reserves a new package to test and the master gives it
@@ -317,16 +317,16 @@ Command: fail <packagename> <packageversion>
.
Success: ok
----
-
+
Same as "pass", but package failed one or more tests.
-
+
----
Command: untestable <packagename> <packageversion>
log file contents
.
Success: ok
----
-
+
Slave reports that a particular package is untestable, possibly
because it insists on interacting with the user.
@@ -360,31 +360,31 @@ three they are all mandatory:
* "idle-sleep" is the length of time the slave should wait before querying the master again if the master didn't have any new packages to test. In seconds, so a value of 300 would mean five minutes, and that seems to be a good value when there are fairly few slaves per master. The default is 300 seconds.
* "max-tgz-age" is used to specify the maximum age (in seconds) after which basesystem tarballs will be recreated. If recreation fails, the old tarball will be used again. The default is 2592000 seconds, which is 30 days.
-
+
* "min-tgz-retry-delay" is used to specify the minimum time (in seconds) between attempts to recreate a tarball which was created more than "max-tgz-age" seconds ago. The default is 21600 seconds, which is 6h.
* "master-host" is the host where the master exists. The slave will give this host to ssh.
-
+
* "master-user" is the username of the master. The slave will log in using this username.
* "master-directory" is the directory where the master keeps its files. Can be relative to the master's home directory.
-
+
* "master-command" is the command to run on master-host to start the master. When the master has been installed from the Debian package, the command is 'python /usr/share/piuparts/piuparts-master'. If you want to use a section in the master configuration file other than "master", append the section name to this command. For example, if the master configuration file has a "sid-ia64" section that you want to use, the command should be 'python /usr/share/piuparts/piuparts-master sid-ia64'.
-
+
* "log-file" is the name of a file to where the master should write its log messages. In the default configuration file it is "/dev/null", that is, log messages are not put in a file.
==== section specific configuration
* "packages-url" is a URL to the Packages.bz2 file specifying what packages should be tested. This needs to be a Packages.bz2 file, other compression methods are not supported. For example, you might use 'http://ftp.debian.org/debian/dists/sid/main/binary-i386/Packages.bz2' but you really do want to replace "ftp.debian.org" with the name of your local mirror.
-
+
* "sources-url" is a URL to the Sources.bz2 file for your mirror. "sources-url" must match "packages-url", if it is not defined, piuparts-reports will not generate source centric html pages.
* "mirror" tells the slave which mirror it is to use. The slave gives this to piuparts when it runs it. Components must not be used here. "packages-url" defines which component to use. This setting is redundant and should go away.
-
+
* "piuparts-cmd" is the command the slave uses to start piuparts. It should include 'sudo' if necessary so that piuparts runs with sufficient priviledges to do its testing (and that means root priviledges).
-
+
* "distro" is the distribution the slave should tell piuparts to use for basic install/purge testing. This can be left empty if only upgrade tests should be run.
-
+
* "chroot-tgz" is the name of the file the slave should use for the tarball to keep the chroot for the basic install/purge testing. If the tarball doesn't exist, the slave creates it. This can be left empty if only upgrade tests should be run.
* "upgrade-test-distros" is the space delimited list of distributions the slave should use for testing upgrades between distributions (i.e., Debian versions). Currently, "lenny squeeze sid" is a good choice. Leave this unset if you do not want to run upgrade tests.
diff --git a/debian/changelog b/debian/changelog
index a3e1db1..1424ff2 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -13,6 +13,7 @@ piuparts (0.42) UNRELEASED; urgency=low
* debian/copyright: apply patch by Scott Schaefer to make it compliant with
DEP-5. Thanks Scott!
* piuparts.py and piuparts-report.py: Expand tabs to spaces.
+ * Remove whitespaces from whitespace-only lines.
[ Mika Pflüger ]
* piuparts-analyze.py: Rewrite to use python-debianbts to analyze if bugs
diff --git a/piuparts-analyze.py b/piuparts-analyze.py
index 9664e2d..8b917e7 100644
--- a/piuparts-analyze.py
+++ b/piuparts-analyze.py
@@ -105,9 +105,9 @@ def prepend_to_file(filename, data):
f.write(data)
f.write(old_data)
f.close()
-
+
shutil.copymode(filename, filename + ".tmp")
-
+
os.rename(filename, filename + "~")
os.rename(filename + ".tmp", filename)
os.remove(filename + "~")
diff --git a/piuparts-master.py b/piuparts-master.py
index 50f03fd..806f183 100644
--- a/piuparts-master.py
+++ b/piuparts-master.py
@@ -79,7 +79,7 @@ class Protocol:
line = self._input.readline()
logging.debug(">> " + line.rstrip())
return line
-
+
def _writeline(self, line):
logging.debug("<< " + line)
self._output.write(line + "\n")
@@ -198,12 +198,12 @@ def main():
section = sys.argv[1]
config = Config(section=section)
config.read(CONFIG_FILE)
-
+
setup_logging(logging.DEBUG, config["log-file"])
if not os.path.exists(os.path.join(master_directory, section)):
os.makedirs(os.path.join(master_directory, section))
-
+
logging.info("Fetching %s" % config["packages-url"])
packages_file = piupartslib.open_packages_url(config["packages-url"])
known_circular_depends = config["known_circular_depends"]
diff --git a/piuparts-report.py b/piuparts-report.py
index a5793eb..7bb5182 100644
--- a/piuparts-report.py
+++ b/piuparts-report.py
@@ -415,7 +415,7 @@ def setup_logging(log_level, log_file_name):
handler = logging.StreamHandler(sys.stderr)
handler.setFormatter(formatter)
logger.addHandler(handler)
-
+
if log_file_name:
handler = logging.FileHandler(log_file_name)
logger.addHandler(handler)
@@ -790,7 +790,7 @@ class Section:
sourcerows += "<tr class=\"normalrow\"><td class=\"labelcell\">Maintainer:</td><td class=\"contentcell2\" colspan=\"5\">%s</td></tr>" % (self.link_to_maintainer_summary(maintainer))
if uploaders:
sourcerows += "<tr class=\"normalrow\"><td class=\"labelcell\">Uploaders:</td><td class=\"contentcell2\" colspan=\"5\">%s</td></tr>" % (self.link_to_uploaders(uploaders))
-
+
source_summary_page_path = os.path.join(self._output_directory, "source", source_subdir(source))
if not os.path.exists(source_summary_page_path):
os.makedirs(source_summary_page_path)
@@ -866,7 +866,7 @@ class Section:
if source_state != "udeb":
sources += "%s: %s\n" % (source, source_state)
self.create_maintainer_templates_for_source(source, source_state, sourcerows, binaryrows, maintainer, uploaders)
-
+
write_file(os.path.join(self._output_directory, "sources.txt"), sources)
diff --git a/piuparts-slave.py b/piuparts-slave.py
index eb35a79..b73ad8e 100644
--- a/piuparts-slave.py
+++ b/piuparts-slave.py
@@ -48,7 +48,7 @@ def setup_logging(log_level, log_file_name):
handler = logging.StreamHandler(sys.stderr)
handler.setFormatter(formatter)
logger.addHandler(handler)
-
+
if log_file_name:
handler = logging.FileHandler(log_file_name)
logger.addHandler(handler)
@@ -110,12 +110,12 @@ class Slave:
self._master_user = None
self._master_directory = "."
self._master_command = None
-
+
def _readline(self):
line = self._from_master.readline()
logging.debug("<< " + line.rstrip())
return line
-
+
def _writeline(self, *words):
line = " ".join(words)
logging.debug(">> " + line)
@@ -125,19 +125,19 @@ class Slave:
def set_master_host(self, host):
logging.debug("Setting master host to %s" % host)
self._master_host = host
-
+
def set_master_user(self, user):
logging.debug("Setting master user to %s" % user)
self._master_user = user
-
+
def set_master_directory(self, dir):
logging.debug("Setting master directory to %s" % dir)
self._master_directory = dir
-
+
def set_master_command(self, cmd):
logging.debug("Setting master command to %s" % cmd)
self._master_command = cmd
-
+
def connect_to_master(self, log_file):
logging.info("Connecting to %s" % self._master_host)
if self._master_user:
@@ -244,7 +244,7 @@ class Section:
if self._config["upgrade-test-distros"] and tarball:
create_or_replace_chroot_tgz(self._config, tarball,
base_tgz_ctrl, self._config["upgrade-test-distros"].split()[0])
-
+
for rdir in ["new", "pass", "fail"]:
rdir = os.path.join(self._slave_directory, rdir)
if not os.path.exists(rdir):
@@ -349,14 +349,14 @@ def test_package(config, package, packages_files):
output.write("\n")
package.dump(output)
output.write("\n")
-
+
# omit distro test if chroot-tgz is not specified.
if config["chroot-tgz"]:
command = "%(piuparts-cmd)s -ad %(distro)s -b %(chroot-tgz)s" % \
config
if config["keep-sources-list"] in ["yes", "true"]:
command += " --keep-sources-list "
-
+
if config["mirror"]:
command += " --mirror %s " % config["mirror"]
command += " " + package["Package"]
@@ -459,7 +459,7 @@ def fetch_packages_file(config, distro):
f = piupartslib.open_packages_url(packages_url)
packages_file = piupartslib.packagesdb.PackagesFile(f)
f.close()
-
+
return packages_file
@@ -471,7 +471,7 @@ def create_file(filename, contents):
def main():
setup_logging(logging.INFO, None)
-
+
# For supporting multiple architectures and suites, we take a command-line
# argument referring to a section in configuration file.
# If no argument is given, the "global" section is assumed.
diff --git a/piuparts.py b/piuparts.py
index 4f0d35b..c8f3e99 100644
--- a/piuparts.py
+++ b/piuparts.py
@@ -58,13 +58,13 @@ except ImportError:
class Defaults:
"""Default settings which depend on flavor of Debian.
-
+
Some settings, such as the default mirror and distribution, depend on
which flavor of Debian we run under: Debian itself, or a derived
distribution such as Ubuntu. This class abstracts away the defaults
so that the rest of the code can just refer to the values defined
herein.
-
+
"""
def get_components(self):
@@ -72,10 +72,10 @@ class Defaults:
def get_mirror(self):
"""Return default mirror."""
-
+
def get_distribution(self):
"""Return default distribution."""
-
+
class DebianDefaults(Defaults):
@@ -104,13 +104,13 @@ class UbuntuDefaults(Defaults):
class DefaultsFactory:
"""Instantiate the right defaults class."""
-
+
def guess_flavor(self):
p = subprocess.Popen(["lsb_release", "-i", "-s"],
stdout=subprocess.PIPE)
stdout, stderr = p.communicate()
return stdout.strip().lower()
-
+
def new_defaults(self):
if not settings.defaults:
settings.defaults = self.guess_flavor()
@@ -126,7 +126,7 @@ class DefaultsFactory:
class Settings:
"""Global settings for this program."""
-
+
def __init__(self):
self.defaults = None
self.tmpdir = None
@@ -325,7 +325,7 @@ def setup_logging(log_level, log_file_name):
handler.setFormatter(formatter)
logger.addHandler(handler)
HANDLERS.append(handler)
-
+
if log_file_name:
handler = logging.FileHandler(log_file_name)
handler.setFormatter(formatter)
@@ -442,7 +442,7 @@ def remove_files(filenames):
def make_metapackage(name, depends, conflicts):
"""Return the path to a .deb created just for satisfying dependencies
-
+
Caller is responsible for removing the temporary directory containing the
.deb when finished.
"""
@@ -476,7 +476,7 @@ def make_metapackage(name, depends, conflicts):
def is_broken_symlink(root, dirpath, filename):
"""Is symlink dirpath+filename broken?
-
+
When resolving the symlink, pretend (similar to chroot) that root is
the root of the filesystem. Note that this does NOT work completely
correctly if the symlink target contains .. path components. This is
@@ -523,10 +523,10 @@ class IsBrokenSymlinkTests(unittest.TestCase):
self.symlink("absolute-broken", "absolute-broken-to-symlink")
self.symlink("/", "absolute-works")
self.symlink("/absolute-works", "absolute-works-to-symlink")
-
+
def tearDown(self):
shutil.rmtree(self.testdir)
-
+
def testRelativeBroken(self):
self.failUnless(is_broken_symlink(self.testdir, self.testdir,
"relative-broken"))
@@ -542,7 +542,7 @@ class IsBrokenSymlinkTests(unittest.TestCase):
def testAbsoluteBrokenToSymlink(self):
self.failUnless(is_broken_symlink(self.testdir, self.testdir,
"absolute-broken-to-symlink"))
-
+
def testRelativeWorks(self):
self.failIf(is_broken_symlink(self.testdir, self.testdir,
"relative-works"))
@@ -572,10 +572,10 @@ class IsBrokenSymlinkTests(unittest.TestCase):
class Chroot:
"""A chroot for testing things in."""
-
+
def __init__(self):
self.name = None
-
+
def create_temp_dir(self):
"""Create a temporary directory for the chroot."""
self.name = tempfile.mkdtemp(dir=settings.tmpdir)
@@ -778,7 +778,7 @@ class Chroot:
if settings.scriptsdir is not None:
self.run_scripts("post_distupgrade")
self.check_for_no_processes()
-
+
def apt_get_knows(self, package_names):
"""Does apt-get (or apt-cache) know about a set of packages?"""
@@ -802,7 +802,7 @@ class Chroot:
logging.error("Error copying %s to %s: %s" %
(source_name, target_name, detail))
panic()
-
+
def list_installed_files (self, pre_info, post_info):
"""List the new files installed, removed and modified between two dir trees.
Actually, it is a nice output of the funcion diff_meta_dat."""
@@ -871,11 +871,11 @@ class Chroot:
self.run(["dpkg", "--" + operation, name], ignore_errors=True)
self.run(["dpkg", "--remove", "--pending"], ignore_errors=True)
-
+
def restore_selections(self, changes, packages):
"""Restore package selections in a chroot by applying 'changes'.
'changes' is a return value from diff_selections."""
-
+
deps = {}
nondeps = {}
for name, state in changes.iteritems():
@@ -883,7 +883,7 @@ class Chroot:
nondeps[name] = state
else:
deps[name] = state
-
+
deps_to_remove = [name for name, state in deps.iteritems()
if state == "remove"]
deps_to_purge = [name for name, state in deps.iteritems()
@@ -892,7 +892,7 @@ class Chroot:
if state == "remove"]
nondeps_to_purge = [name for name, state in nondeps.iteritems()
if state == "purge"]
-
+
# Run custom scripts before removing all packages.
if settings.scriptsdir is not None:
self.run_scripts("pre_remove")
@@ -906,13 +906,13 @@ class Chroot:
if not settings.skip_cronfiles_test:
cronfiles, cronfiles_list = self.check_if_cronfiles(packages)
-
+
if not settings.skip_cronfiles_test and cronfiles:
self.check_output_cronfiles(cronfiles_list)
if not settings.skip_logrotatefiles_test:
logrotatefiles, logrotatefiles_list = self.check_if_logrotatefiles(packages)
-
+
if not settings.skip_logrotatefiles_test and logrotatefiles:
installed = self.install_logrotate()
self.check_output_logrotatefiles(logrotatefiles_list)
@@ -1056,7 +1056,7 @@ class Chroot:
panic()
else:
logging.debug("No broken symlinks as far as we can find.")
-
+
def check_if_cronfiles(self, packages):
"""Check if the packages have cron files under /etc/cron.d and in case positive,
it returns the list of files. """
@@ -1615,7 +1615,7 @@ def process_changes(changes):
def check_results(chroot, root_info, file_owners, deps_info=None):
"""Check that current chroot state matches 'root_info'.
-
+
If settings.warn_on_others is True and deps_info is not None, then only
print a warning rather than failing if the current chroot contains files
that are in deps_info but not in root_info. (In this case, deps_info
@@ -1702,7 +1702,7 @@ def install_purge_test(chroot, root_info, selections, package_list, packages):
apt_cache_args.extend(packages)
returncode, output = chroot.run(apt_cache_args)
control_infos = deb822.Deb822.iter_paragraphs(output.splitlines())
-
+
depends = []
conflicts = []
for control in control_infos:
@@ -1714,7 +1714,7 @@ def install_purge_test(chroot, root_info, selections, package_list, packages):
all_conflicts = ", ".join(conflicts)
metapackage = make_metapackage("piuparts-depends-dummy",
all_depends, all_conflicts)
-
+
# Install the metapackage
chroot.install_package_files([metapackage])
# Now remove it
@@ -1744,7 +1744,7 @@ def install_purge_test(chroot, root_info, selections, package_list, packages):
# Remove all packages from the chroot that weren't there initially.
changes = diff_selections(chroot, selections)
chroot.restore_selections(changes, packages)
-
+
chroot.check_for_broken_symlinks()
return check_results(chroot, root_info, file_owners, deps_info=deps_info)
@@ -1756,7 +1756,7 @@ def install_upgrade_test(chroot, root_info, selections, package_list, package_na
# First install via apt-get.
chroot.install_packages_by_name(package_names)
-
+
if settings.scriptsdir is not None:
chroot.run_scripts("pre_upgrade")
@@ -1764,14 +1764,14 @@ def install_upgrade_test(chroot, root_info, selections, package_list, package_na
# Then from the package files.
chroot.install_package_files(package_list)
-
+
file_owners = chroot.get_files_owned_by_packages()
# Remove all packages from the chroot that weren't there
# initially.
changes = diff_selections(chroot, selections)
chroot.restore_selections(changes, package_names)
-
+
chroot.check_for_no_processes()
chroot.check_for_broken_symlinks()
@@ -1830,7 +1830,7 @@ def install_and_upgrade_between_distros(filenames, packages):
else:
root_tgz = chroot.create_temp_tgz_file()
chroot.pack_into_tgz(root_tgz)
-
+
if settings.end_meta:
# load root_info and selections
root_info, selections = load_meta_data(settings.end_meta)
@@ -1841,11 +1841,11 @@ def install_and_upgrade_between_distros(filenames, packages):
# set root_info and selections
root_info = chroot.save_meta_data()
selections = chroot.get_selections()
-
+
if settings.save_end_meta:
# save root_info and selections
save_meta_data(settings.save_end_meta, root_info, selections)
-
+
chroot.remove()
dont_do_on_panic(cid)
chroot = get_chroot()
@@ -1871,7 +1871,7 @@ def install_and_upgrade_between_distros(filenames, packages):
chroot.install_package_files(filenames)
chroot.run(["apt-get", "clean"])
-
+
chroot.check_for_no_processes()
file_owners = chroot.get_files_owned_by_packages()
@@ -1882,7 +1882,7 @@ def install_and_upgrade_between_distros(filenames, packages):
result = check_results(chroot, root_info, file_owners)
chroot.check_for_no_processes()
-
+
if root_tgz != settings.basetgz:
remove_files([root_tgz])
chroot.remove()
@@ -1927,11 +1927,11 @@ def set_basetgz_to_pbuilder(option, opt, value, parser, *args, **kwargs):
def parse_command_line():
"""Parse the command line, change global settings, return non-options."""
-
+
parser = optparse.OptionParser(usage="%prog [options] package ...",
version="piuparts %s" % VERSION)
-
-
+
+
parser.add_option("-a", "--apt", action="store_true", default=False,
help="Command line arguments are package names " +
"to be installed via apt.")
@@ -1940,7 +1940,7 @@ def parse_command_line():
metavar='CMDLINE', default=None,
help="Use CMDLINE via autopkgtest (adt-virt-*)"
" protocol instead of managing a chroot.")
-
+
parser.add_option("-b", "--basetgz", metavar="TARBALL",
help="Use TARBALL as the contents of the initial " +
"chroot, instead of building a new one with " +
@@ -1949,7 +1949,7 @@ def parse_command_line():
parser.add_option("--bindmount", action="append", metavar="DIR",
default=[],
help="Directory to be bind-mounted inside the chroot.")
-
+
parser.add_option("-d", "--distribution", action="append", metavar="NAME",
help="Which Debian distribution to use: a code name " +
"(for example lenny, squeeze, sid) or experimental. The " +
@@ -1958,7 +1958,7 @@ def parse_command_line():
parser.add_option("-D", "--defaults", action="store",
help="Choose which set of defaults to use "
"(debian/ubuntu).")
-
+
parser.add_option("--debfoster-options",
default="-o MaxPriority=required -o UseRecommends=no -f -n apt debfoster",
help="Run debfoster with different parameters (default: -o MaxPriority=required -o UseRecommends=no -f -n apt debfoster).")
@@ -1972,7 +1972,7 @@ def parse_command_line():
default=False,
action='store_true',
help="Make dpkg use --force-confdef, which lets dpkg always choose the default action when a modified conffile is found. This option will make piuparts ignore errors it was designed to report and therefore should only be used to hide problems in depending packages. (See #466118.)")
-
+
parser.add_option("--do-not-verify-signatures", default=False,
action='store_true',
help="Do not verify signatures from the Release files when running debootstrap.")
@@ -1981,13 +1981,13 @@ def parse_command_line():
default=[],
help="Add FILENAME to list of filenames to be " +
"ignored when comparing changes to chroot.")
-
+
parser.add_option("-I", "--ignore-regex", action="append",
metavar="REGEX", default=[],
help="Add REGEX to list of Perl compatible regular " +
"expressions for filenames to be " +
"ignored when comparing changes to chroot.")
-
+
parser.add_option("-k", "--keep-tmpdir",
action="store_true", default=False,
help="Don't remove the temporary directory for the " +
@@ -1996,7 +1996,7 @@ def parse_command_line():
parser.add_option("-K", "--keyring", metavar="FILE",
default = "/usr/share/keyrings/debian-archive-keyring.gpg",
help="Use FILE as the keyring to use with debootstrap when creating chroots.")
-
+
parser.add_option("--keep-sources-list",
action="store_true", default=False,
help="Don't modify the chroot's " +
@@ -2020,16 +2020,16 @@ def parse_command_line():
parser.add_option("--lvm-snapshot-size", metavar="SNAPSHOT-SIZE", action="store",
default="1G", help="Use SNAPSHOT-SIZE as snapshot size when creating " +
"a new LVM snapshot (default: 1G)")
-
+
parser.add_option("-m", "--mirror", action="append", metavar="URL",
default=[],
help="Which Debian mirror to use.")
-
+
parser.add_option("-n", "--no-ignores", action="callback",
callback=forget_ignores,
help="Forget all ignores set so far, including " +
"built-in ones.")
-
+
parser.add_option("-N", "--no-symlinks", action="store_true",
default=False,
help="Don't check for broken symlinks.")
@@ -2038,7 +2038,7 @@ def parse_command_line():
action="store_true", default=False,
help="Skip testing the upgrade from an existing version " +
"in the archive.")
-
+
parser.add_option("--no-install-purge-test",
action="store_true", default=False,
help="Skip install and purge test.")
@@ -2051,13 +2051,13 @@ def parse_command_line():
parser.add_option("--pedantic-purge-test",
action="store_true", default=False,
help="Be pedantic when checking if a purged package leaves files behind. If this option is not set, files left in /tmp are ignored.")
-
+
parser.add_option("-s", "--save", metavar="FILENAME",
help="Save the chroot into FILENAME.")
parser.add_option("-B", "--end-meta", metavar="FILE",
help="Save chroot package selection and file meta data in FILE for later use. See the function install_and_upgrade_between_distros() in piuparts.py for defaults. Mostly useful for large scale distro upgrade tests.")
-
+
parser.add_option("-S", "--save-end-meta", metavar="FILE",
help="Load chroot package selection and file meta data from FILE. See the function install_and_upgrade_between_distros() in piuparts.py for defaults. Mostly useful for large scale distro upgrade tests.")
@@ -2068,7 +2068,7 @@ def parse_command_line():
parser.add_option("--skip-cronfiles-test",
action="store_true", default=False,
help="Skip testing the output from the cron files.")
-
+
parser.add_option("--skip-logrotatefiles-test",
action="store_true", default=False,
help="Skip testing the output from the logrotate files.")
@@ -2083,7 +2083,7 @@ def parse_command_line():
parser.add_option("--scriptsdir", metavar="DIR",
help="Directory where are placed the custom scripts.")
-
+
parser.add_option("-t", "--tmpdir", metavar="DIR",
help="Use DIR for temporary storage. Default is " +
"$TMPDIR or /tmp.")
@@ -2112,7 +2112,7 @@ def parse_command_line():
parser.add_option("--fail-on-broken-symlinks", action="store_true",
default=False,
help="Fail if broken symlinks are detected.")
-
+
parser.add_option("--log-level", action="store",metavar='LEVEL',
default="dump",
help="Displays messages from LEVEL level, possible values are: error, info, dump, debug. The default is dump.")
@@ -2149,11 +2149,11 @@ def parse_command_line():
settings.pedantic_purge_test = opts.pedantic_purge_test
if not settings.pedantic_purge_test:
settings.ignored_patterns += settings.non_pedantic_ignore_patterns
-
+
log_file_name = opts.log_file
defaults = DefaultsFactory().new_defaults()
-
+
settings.debian_mirrors = [parse_mirror_spec(x, defaults.get_components())
for x in opts.mirror]
settings.check_broken_symlinks = not opts.no_symlinks
@@ -2225,7 +2225,7 @@ def parse_command_line():
sys.exit(exitcode)
return args
-
+
def get_chroot():
if settings.adt_virt is None: return Chroot()
@@ -2266,7 +2266,7 @@ def process_packages(package_list):
else:
logging.error("FAIL: Installation, upgrade and purging tests.")
panic()
-
+
chroot.remove()
dont_do_on_panic(cid)
else:
diff --git a/piupartslib/conf.py b/piupartslib/conf.py
index 6ff3bae..20f47d2 100644
--- a/piupartslib/conf.py
+++ b/piupartslib/conf.py
@@ -40,7 +40,7 @@ class Config(UserDict.UserDict):
for key, value in defaults.iteritems():
self[key] = value
self._mandatory = mandatory
-
+
def read(self, filename):
cp = ConfigParser.ConfigParser()
cp.read(filename)
diff --git a/piupartslib/dependencyparser.py b/piupartslib/dependencyparser.py
index ad30eb3..5c17e82 100644
--- a/piupartslib/dependencyparser.py
+++ b/piupartslib/dependencyparser.py
@@ -37,10 +37,10 @@ class DependencySyntaxError(Exception):
self._msg = "Error: %s: %s (text at error: '%s', full text being parsed: '%s')" % \
(cursor.get_position(), msg, cursor.get_text(10),
cursor.get_full_text())
-
+
def __str__(self):
return self._msg
-
+
def __repr__(self):
return self._msg
@@ -48,7 +48,7 @@ class DependencySyntaxError(Exception):
class _Cursor:
"""Store an input string and a movable location in it"""
-
+
def __init__(self, input):
self._input = input
self._len = len(self._input)
@@ -62,7 +62,7 @@ class _Cursor:
"""Are we at the end of the input?"""
self.skip_whitespace()
return self._pos >= self._len
-
+
def next(self):
"""Move to the next character"""
if self._pos < self._len:
@@ -87,9 +87,9 @@ class _Cursor:
def match(self, regexp):
"""Match a regular expression against the current position
-
+
The cursor is advanced by the length of the match, if any.
-
+
"""
m = regexp.match(self._input[self._pos:])
if m:
@@ -98,10 +98,10 @@ class _Cursor:
def match_literal(self, literal):
"""Match a literal string against the current position.
-
+
Return True and move position if there is a match, else return
False.
-
+
"""
if self.get_text(len(literal)) == literal:
self._pos += len(literal)
@@ -132,17 +132,17 @@ class SimpleDependency:
class DependencyParser:
"""Parse Debian package relationship strings
-
+
Debian packages have a rich language for expressing their
relationships. See the Debian Policy Manual, chapter 7 ("Declaring
relationships between packages"). This Python module implements a
parser for strings expressing such relationships.
-
+
Syntax of dependency fields (Pre-Depends, Depends, Recommends,
Suggests, Conflicts, Provides, Replaces, Enhances, Build-Depends,
Build-Depends-Indep, Build-Conflicts, Build-Conflicts-Indep), in a
BNF-like form:
-
+
depends-field ::= EMPTY | dependency ("," dependency)*
dependency ::= possible-dependency ("|" possible-dependency)*
possible-dependency ::= package-name version-dependency?
@@ -168,18 +168,18 @@ class DependencyParser:
name-char ::= alphanumeric | "+" | "-" | "." | "_"
version-char ::= alphanumeric | "." | "+" | "-" | ":" | "~"
debian-version-char ::= alphanumeric | "." | "+"
-
+
White space can occur between any tokens except inside package-name,
version-number, or arch-name. Some of the headers restrict the syntax
somewhat, e.g., Provides does not allow version-dependency, but this is
not included in the syntax for simplicity.
-
+
Note: Added "_" to name-char, because some packages (type-handling
in particular) use Provides: headers with bogus package names.
-
+
Note: Added upper case letters to name pattern, since it some of the
Mozilla localization packages use or used them.
-
+
"""
def __init__(self, input_string):
@@ -188,19 +188,19 @@ class DependencyParser:
def get_dependencies(self):
"""Return parsed dependencies
-
+
The result is a list of lists of SimpleDependency objects.
Let's try that again.
-
+
The result is a list of dependencies, corresponding to
the comma-separated items in the dependency list. Each dependency
is also a list, or SimpleDependency objects, representing
alternative ways to fulfill the dependency; in other words,
items separated by the vertical bar (|).
-
+
For example, "foo, bar | foobar" would result in the following
list: [[foo], [bar, foobar]].
-
+
"""
return self._list
@@ -227,7 +227,7 @@ class DependencyParser:
break
dep = self._parse_possible_dependency()
return vlist
-
+
def _parse_possible_dependency(self):
name = self._parse_package_name()
if not name:
@@ -290,7 +290,7 @@ class DependencyParser:
self._cursor.skip_whitespace()
if self._cursor.get_char() == "[":
self._cursor.next()
-
+
vlist = []
while True:
self._cursor.skip_whitespace()
@@ -302,7 +302,7 @@ class DependencyParser:
raise DependencySyntaxError("Expected architecture name",
self._cursor)
vlist.append(m.group())
-
+
return vlist
else:
return None
diff --git a/piupartslib/packagesdb.py b/piupartslib/packagesdb.py
index 75100c6..58e62b5 100644
--- a/piupartslib/packagesdb.py
+++ b/piupartslib/packagesdb.py
@@ -45,7 +45,7 @@ def rfc822_like_header_parse(input):
else:
headers.append(line)
return headers
-
+
def unique (s):
# taken from http://code.activestate.com/recipes/52560/ - thanks to Tim Peters
n = len(s)
@@ -93,7 +93,7 @@ class Package(UserDict.UserDict):
name, value = header.split(":", 1)
self[name.strip()] = value.strip()
self._parsed_deps = {}
-
+
def _parse_dependencies(self, header_name):
if header_name in self._parsed_deps:
depends = self._parsed_deps[header_name]
@@ -160,7 +160,7 @@ class LogDB:
def listdir(self, dirname):
return dircache.listdir(dirname)
-
+
def exists(self, pathname):
try:
cache = self.exists_cache
@@ -170,10 +170,10 @@ class LogDB:
if pathname not in cache:
cache[pathname] = os.path.exists(pathname)
return cache[pathname]
-
+
def open_file(self, pathname, mode):
return file(pathname, mode)
-
+
def remove_file(self, pathname):
os.remove(pathname)
@@ -186,7 +186,7 @@ class LogDB:
if self.exists(os.path.join(subdir, log_name)):
return True
return False
-
+
def any_log_exists(self, package, subdirs):
try:
cache = self.basename_cache
@@ -202,11 +202,11 @@ class LogDB:
if len(parts) == 2 and parts[0] == package_name:
return True
return False
-
+
def create(self, subdir, package, version, contents):
(fd, temp_name) = tempfile.mkstemp(dir=subdir)
os.close(fd)
-
+
# tempfile.mkstemp sets the file mode to be readable only by owner.
# Let's make it follow the umask.
umask = os.umask(0)
@@ -229,7 +229,7 @@ class LogDB:
full_name = os.path.join(subdir, self._log_name(package, version))
if self.exists(full_name):
self.remove_file(full_name)
-
+
class PackagesDB:
@@ -248,7 +248,7 @@ class PackagesDB:
"unknown-preferred-alternative",
"no-dependency-from-alternatives-exists",
]
-
+
_dep_state_to_state = {
"failed-testing": "dependency-failed-testing",
"cannot-be-tested": "dependency-cannot-be-tested",
@@ -272,7 +272,7 @@ class PackagesDB:
self._package_state = {}
self.set_subdirs(ok="pass", fail="fail", evil="untestable",
reserved="reserved", morefail=["bugged"])
-
+
def set_subdirs(self, ok=None, fail=None, evil=None, reserved=None, morefail=None):
# Prefix all the subdirs with the prefix
if self.prefix:
@@ -290,12 +290,12 @@ class PackagesDB:
if morefail:
self._morefail = [pformat % s for s in morefail]
self._all = [self._ok, self._fail, self._evil, self._reserved] + self._morefail
-
+
def create_subdirs(self):
for sdir in self._all:
if not os.path.exists(sdir):
os.makedirs(sdir)
-
+
def read_packages_file(self, input):
self._packages_files.append(PackagesFile(input))
self._packages = None
@@ -326,13 +326,13 @@ class PackagesDB:
deps.append(dep)
if dep in self._packages:
more += self._packages[dep].dependencies()
-
+
# Break circular dependencies
if break_circles and package["Package"] in deps:
deps.remove(package["Package"])
return deps
-
+
def _compute_package_state(self, package):
if self._logdb.log_exists(package, [self._ok]):
return "successfully-tested"
@@ -433,7 +433,7 @@ class PackagesDB:
deps.remove(pkg)
if package["Package"] in deps:
return "circular-dependency" # actually, it's a unknown circular-dependency
-
+
# treat circular-dependencies as testable (for the part of the circle)
state = "unknown"
if package["Package"] in self._known_circular_depends:
@@ -451,7 +451,7 @@ class PackagesDB:
def _compute_package_states(self):
if self._in_state is not None:
return
-
+
todo = []
unpreferred_alt = []
@@ -493,7 +493,7 @@ class PackagesDB:
self._in_state["unknown"] = todo
self._in_state["unknown-preferred-alternative"] = unpreferred_alt
-
+
for state in self._states:
self._in_state[state].sort()
@@ -503,11 +503,11 @@ class PackagesDB:
def get_pkg_names_in_state(self, state):
self._compute_package_states()
return self._in_state[state]
-
+
def get_packages_in_state(self, state):
self._compute_package_states()
return unique([self._packages[name] for name in self._in_state[state]])
-
+
def get_all_packages(self):
self._find_all_packages()
return self._packages
diff --git a/unittests.py b/unittests.py
index fd8e294..9d68632 100644
--- a/unittests.py
+++ b/unittests.py
@@ -41,10 +41,10 @@ class DependencyParserTests(unittest.TestCase):
class FakeLogDB(piupartslib.packagesdb.LogDB):
"""A fake version of the LogDB class, for testing
-
+
This version simulates filesystem actions so that there is no need
to do actual I/O. Cleaner, although not quite as thorough.
-
+
"""
def __init__(self):
@@ -55,22 +55,22 @@ class FakeLogDB(piupartslib.packagesdb.LogDB):
"reserved": [],
"bugged": [],
}
-
+
def listdir(self, dirname):
return self.dict[dirname]
-
+
def _parse(self, pathname):
return os.path.dirname(pathname), os.path.basename(pathname)
-
+
def exists(self, pathname):
vdir, base = self._parse(pathname)
return base in self.dict[vdir]
-
+
def open_file(self, pathname, mode):
vdir, base = self._parse(pathname)
self.dict[vdir].append(base)
return StringIO.StringIO()
-
+
def remove_file(self, pathname):
vdir, base = self._parse(pathname)
if base in self.dict[vdir]:
--
piuparts git repository
More information about the Piuparts-commits
mailing list