Zope2.12 tarball

Fri Oct 22 21:20:29 UTC 2010

Hey Michael,

On 22/10/2010 Michael Mulich wrote:
> Thanks for the response. Even a short email is better than none. So
> I appreciate all the feedback you can give.

i intended to spend more time on zope in debian, but at the moment
university keeps me busy. but i'll try to help you if possible ...

> On 10/19/10 5:26 PM, Jonas Meurer wrote:
> >Hey Michael,
> >
> >First, great to see you still working on that issue. Unfortunately i'm
> >far to busy to do any major work regarding zope2.12 packaging.
> >
> >On 18/10/2010 Michael Mulich wrote:
> >>I've got a few questions hopefully someone can answer. But first,
> >>let me inform you about what I've been doing to package Zope 2.12
> >>and any subsequent minor versions.
> >>
> >>Summary of this message: I have created a Zope2 tarball for us to
> >>use in packaging and I could use your help.
> >
> >great. Could you elaborate on what this tarball contains? I guess all
> >zope 2.12 dependencies plus some buildout magic you wrote to build
> >zope2.12 from this tarball without internet connection?
> >
> 
> The tarball contains all the necessary libraries to build and
> install Zope without an internet connection. The build uses buildout
> under the hood. There is a custom buildout recipe I wrote to remove
> any build dependencies. The end result of a build (via 'make') will
> stage three pieces of information: the libraries themselves,
> generated scripts, and a pth file.

ok, that sounds good so far.

> >>I started work on a zope2.12 branch a couple of weeks ago called
> >>buildout-built.[0] This branch has been successful in packaging
> >>Zope2 and its dependencies. However, this implementation has one
> >>major flaw. That flaw is that the source can't be patched before the
> >>build, which is important when making the zope-common customization
> >>for dzhandle support. Also, patching the source before the build is
> >>critical for security fixes. So the branch is dead, but it
> >>illuminated a better more generic solution.
> >
> >indeed, the ability to patch sources before build is important. why is
> >it impossible with that branch?
> >
> 
> This branch depends on internet connectivity, mostly to pypi and
> zope.org. It is impossible to patch the source before the build
> because buildout doesn't allow us to do this. Buildout is an all in
> one process, rather than a source, configure, build and install
> process.

all right. an internet connection at build time  is no option for
official debian packages. the packages need to be built _only_ from
installed build-dependencies and the package source. nothing more.
packages installed by the meta-package 'build-essential' don't need to
be listed in the build-depends.

> >>Upon thinking about the issue a bit more and having communication
> >>with others about packaging Zope2 on systems other than Debian, I
> >>decided to resurrect the Zope2 tarball. The tarball is generic
> >>enough that it should work across all Unix platforms. I'll be
> >>hosting the tarballs out of weblion.psu.edu.[1] The source for the
> >>tarball can also be found in the WebLion subversion repository.[2]
> >>The tarball for Zope 2.12.10 is complete and can be found up on the
> >>WebLion web server.[3] Also, this is a maintainable solution, see
> >>the docs online or in the tarball for more info.[4]
> >>
> >>I've started work on another branch, named with-revived-tarball,
> >>which hopefully we can get working with this new tarball.[5] So far,
> >>I've touched nothing more than the watch, changelog and control
> >>files.
> >>
> >>My main question at this point is... How do I get the rules file to
> >>work with the tarball?
> >
> >the rules file is a Makefile. It's the Makefile which is used to
> >configure, compile, build and install the software. In simple cases, all
> >it does is invoking configure, make, make install.
> >
> 
> Ok, this I can do. The problem I'm having has to do with actually
> finding tarball files, like configure, to run these commands on.
> 
> I keep getting "dpkg-source: warning: ignoring deletion of directory
> <file_name>" for all the files that have been unpacked, when I run
> dpkg-buildpackage on the branch.

ok, i guess i can help you with that. the debian source consists of
three files:

package_1.2.orig.tar.gz - the upstream tarball without any debian changes
package_1.2-1.diff.gz - the diff that contains the debian/ subdirectory
package_1.2-1.dsc - the description file containing meta-information

if you want to build a debian package, easiest way is to rename upstream
source to package_1.2.orig.tar.gz, unpack that tarball and add the
debian/ subdirectory to the new folder 'package-1.2', change into that
folder, modify all the debian/* files accordingly and invoke
dpkg-buildpackage. this will create the diff.gz file as a diff from
orig.tar.gz and the build folder (which is not allowed to contain any
data but the debian/ subdirectory). dpkg-buildpackage requires the
package tarball of upstream version specified in the changelog.

as we use svn for source management, you can use svn-buildpackage as
well. svn-buildpackage searches for the tarball at
../tarballs/package_1.2.orig.tar.gz'.

i tried to build your package the following way:

- change into folder 'zope2.12/branches/' (checkout from svn)
- create a subfolder called 'tarballs'
- copy your selfmade tarball to 'tarballs/zope2.12_2.12.10.orig.tar.gz
- change into folder 'with-revived-tarball'
- invoke 'svn-buildpackage --svn-ignore-new'

svn-buildpackage copies and unpacks the upstream tarball to
'../build-area', copies the debian subdir from current folder into the
unpacked build directory, and invoked dpkg-buildpackage.

all these are very rough information which only contain the information
relevant for you. i suggest you take a look at the documentation:

http://www.debian.org/doc/maint-guide/
	Debians New Maintainers' Guide

http://www.debian.org/doc/debian-policy/
	Debian Policy Manual

http://www.debian.org/doc/developers-reference/
	Debian Developer's Reference

> >>Can I get a hand putting this together? The hard part is done. We
> >>just need to take the tarball and put it in the package.
> >
> >I just gave this tarball a quick try. First thing I recognized was that
> >compilation failed as it tries to write at a place outside the build
> >dir:
> >
> >localhost - - [19/Oct/2010 23:12:27] "GET /docutils/ HTTP/1.1" 200 128
> >Getting distribution for 'docutils'.
> >install_dir /home/jonas/debian/zope/zope2.12/branches/build-area/zope2.12-2.12.10.obsolete.0.0208947901419414/build/eggs/tmp87e50d
> >error: /usr/lib/pymodules/python2.6/numpy/egg-dist-tmp-5fAqrj: Permission denied
> >An error occured when trying to install docutils 0.7. Look above this message for any errors that were output by easy_install.
> >While:
> >   Installing zope2.12.
> >   Getting distribution for 'docutils'.
> >Error: Couldn't install: docutils 0.7
> >
> 
> I'm not sure how you got this far. I can't get past the unpacking
> and configuration.

see above.

> >Unfortunately I don't have time to dig deeper into this issue right
> >now, but here are some comments based on unchecked assumptions:
> >
> >- the build process must not write anywhere but inside the build
> >   directory
> >
> 
> I think this might have to do with my use of tempfile in the build
> process. This should be relatively easy to resolve.

that would be great.

> >- the build process must not depend on network connection
> >
> 
> How about a localhost connection? I fake an package index for
> buildout's sake. This was the easiest work around for for version
> pinning and source acquisition, but I plan to eventually factor this
> out of the process in a later package version.

not sure about that. which kind of connection is it? I saw some HTTP GET
connections to local files. i guess this is no problem as it doesn't
depend on any network connectivity. but e.g. a HTTP GET to
http://localhost would depend on a webserver on the build system, which
obviously is not ok.

> >- if c/c++ is compiled during build process, setting CFLAGS for the
> >   compiler needs to be possible. Crosscompiling needs to be supported.
> >   usually this is done by passing confflags like --build and --host as
> >   well as CFLAGS to the configure script, which adds them to the
> >   Makefile.
> >
> 
> I can't say anything about this at the moment.
> 
> >- if paths and/or links are modified/written during build process, this
> >   must be configurable. for example options like --prefix are useful in
> >   configure script, and 'make install' should know environment variables
> >   like $DESTDIR.
> >
> 
> The configuration script has two options '--prefix' and
> '--with-python'. For our case, we will be using the '--prefix'
> option.
> 
> I do not have a $DESTDIR variable, but I am now using ROOT during a
> 'make install'. This seems to be how the older tarballs once did it.

--prefix is good. but the point is: --prefix need to be set to something
like '/usr', and still 'make install' needs to be able to install into
$DESTDIR (which is ./debian/zope2.12-2.12.10 for the build process).
'make install' is not allowed to install into '/usr' directly.

the reason for that is, that $DESTDIR will be packed and compressed in
the debian package, and the package installation simply unpacks it again
at the target system.

> >In other words: a way to build and install the software into a custom
> >directory is required. later when the files are copied to the filesystem
> >all links and paths need to be correct.
> >
> 
> Yeah, I think this can be handled with the ROOT variable during
> 'make install'.
> 
> Thanks Jonas!

hope to help, and thanks for your great work on debian zope packages!

greetings,
 jonas
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://lists.alioth.debian.org/pipermail/pkg-zope-developers/attachments/20101022/8ef9eca3/attachment.pgp>