Zope2.12 tarball

[Michael Mulich]

Jonas-

Thanks for the response. Even a short email is better than none. So I 
appreciate all the feedback you can give.

On 10/19/10 5:26 PM, Jonas Meurer wrote:
> Hey Michael,
>
> First, great to see you still working on that issue. Unfortunately i'm
> far to busy to do any major work regarding zope2.12 packaging.
>
> On 18/10/2010 Michael Mulich wrote:
>> I've got a few questions hopefully someone can answer. But first,
>> let me inform you about what I've been doing to package Zope 2.12
>> and any subsequent minor versions.
>>
>> Summary of this message: I have created a Zope2 tarball for us to
>> use in packaging and I could use your help.
>
> great. Could you elaborate on what this tarball contains? I guess all
> zope 2.12 dependencies plus some buildout magic you wrote to build
> zope2.12 from this tarball without internet connection?
>

The tarball contains all the necessary libraries to build and install 
Zope without an internet connection. The build uses buildout under the 
hood. There is a custom buildout recipe I wrote to remove any build 
dependencies. The end result of a build (via 'make') will stage three 
pieces of information: the libraries themselves, generated scripts, and 
a pth file.

>> I started work on a zope2.12 branch a couple of weeks ago called
>> buildout-built.[0] This branch has been successful in packaging
>> Zope2 and its dependencies. However, this implementation has one
>> major flaw. That flaw is that the source can't be patched before the
>> build, which is important when making the zope-common customization
>> for dzhandle support. Also, patching the source before the build is
>> critical for security fixes. So the branch is dead, but it
>> illuminated a better more generic solution.
>
> indeed, the ability to patch sources before build is important. why is
> it impossible with that branch?
>

This branch depends on internet connectivity, mostly to pypi and 
zope.org. It is impossible to patch the source before the build because 
buildout doesn't allow us to do this. Buildout is an all in one process, 
rather than a source, configure, build and install process.

>> Upon thinking about the issue a bit more and having communication
>> with others about packaging Zope2 on systems other than Debian, I
>> decided to resurrect the Zope2 tarball. The tarball is generic
>> enough that it should work across all Unix platforms. I'll be
>> hosting the tarballs out of weblion.psu.edu.[1] The source for the
>> tarball can also be found in the WebLion subversion repository.[2]
>> The tarball for Zope 2.12.10 is complete and can be found up on the
>> WebLion web server.[3] Also, this is a maintainable solution, see
>> the docs online or in the tarball for more info.[4]
>>
>> I've started work on another branch, named with-revived-tarball,
>> which hopefully we can get working with this new tarball.[5] So far,
>> I've touched nothing more than the watch, changelog and control
>> files.
>>
>> My main question at this point is... How do I get the rules file to
>> work with the tarball?
>
> the rules file is a Makefile. It's the Makefile which is used to
> configure, compile, build and install the software. In simple cases, all
> it does is invoking configure, make, make install.
>

Ok, this I can do. The problem I'm having has to do with actually 
finding tarball files, like configure, to run these commands on.

I keep getting "dpkg-source: warning: ignoring deletion of directory 
<file_name>" for all the files that have been unpacked, when I run 
dpkg-buildpackage on the branch.

>> Can I get a hand putting this together? The hard part is done. We
>> just need to take the tarball and put it in the package.
>
> I just gave this tarball a quick try. First thing I recognized was that
> compilation failed as it tries to write at a place outside the build
> dir:
>
> localhost - - [19/Oct/2010 23:12:27] "GET /docutils/ HTTP/1.1" 200 128
> Getting distribution for 'docutils'.
> install_dir /home/jonas/debian/zope/zope2.12/branches/build-area/zope2.12-2.12.10.obsolete.0.0208947901419414/build/eggs/tmp87e50d
> error: /usr/lib/pymodules/python2.6/numpy/egg-dist-tmp-5fAqrj: Permission denied
> An error occured when trying to install docutils 0.7. Look above this message for any errors that were output by easy_install.
> While:
>    Installing zope2.12.
>    Getting distribution for 'docutils'.
> Error: Couldn't install: docutils 0.7
>

I'm not sure how you got this far. I can't get past the unpacking and 
configuration.

>
> Unfortunately I don't have time to dig deeper into this issue right
> now, but here are some comments based on unchecked assumptions:
>
> - the build process must not write anywhere but inside the build
>    directory
>

I think this might have to do with my use of tempfile in the build 
process. This should be relatively easy to resolve.

> - the build process must not depend on network connection
>

How about a localhost connection? I fake an package index for buildout's 
sake. This was the easiest work around for for version pinning and 
source acquisition, but I plan to eventually factor this out of the 
process in a later package version.

> - if c/c++ is compiled during build process, setting CFLAGS for the
>    compiler needs to be possible. Crosscompiling needs to be supported.
>    usually this is done by passing confflags like --build and --host as
>    well as CFLAGS to the configure script, which adds them to the
>    Makefile.
>

I can't say anything about this at the moment.

> - if paths and/or links are modified/written during build process, this
>    must be configurable. for example options like --prefix are useful in
>    configure script, and 'make install' should know environment variables
>    like $DESTDIR.
>

The configuration script has two options '--prefix' and '--with-python'. 
For our case, we will be using the '--prefix' option.

I do not have a $DESTDIR variable, but I am now using ROOT during a 
'make install'. This seems to be how the older tarballs once did it.

> In other words: a way to build and install the software into a custom
> directory is required. later when the files are copied to the filesystem
> all links and paths need to be correct.
>

Yeah, I think this can be handled with the ROOT variable during 'make 
install'.

Thanks Jonas!

-Michael Mulich (pumazi)