[ECLIPSE] "State of the Art"

Wed Dec 17 07:18:34 UTC 2008

Hello :)

Since we seem to be getting new team members every day (which is
already very cool and more than I expected),
I thought to post a "state of the art" to try and get you guys up to
speed without you having to do the "reverse-engineering"
that I had to.

As you all know it 's been about 2 years (or more?) since eclipse
packages have seen any updates that found their way
into "official" debian/ubuntu or that are even reasonably working
IMHO. This is what we want to change :)

Here 's what we have to work with:

First, there is the preliminary debian package work, the packaging
files of which (the debian/ folder) can be found
in:

  * svn co svn://svn.debian.org/pkg-java/trunk/eclipse

I have not thoroughly studied this work to be honest, mostly because I
have looked at the below.

Then, there are 2 unofficial ubuntu packages of 3.4.0 (both by Nathan
aka Rockwalrus):

  * The one in https://launchpad.net/~eclipse-team/+archive

    This seems to be largely based on the above debian work, and for
its status I quote rockwalrus himself:

"I've been working on a package for 3.4 since RC1. I just uploaded a
source package to my [[https://launchpad.net/~rock-gimp/+archive
ppa]].
It is based off of the never-released packaging for 3.3 in debian's
pkg-java svn. There are a few major issues:

 * Help indexing causes memory corruption, which leads to a crash.
This seems to be a gcj-4.2 issue; other jvms don't seem to have this
problem.
    Unfortunately, the source package is set up to build with gcj, so
some of the help file packages never get built. Furthermore, if you
try to search
     help files installed using the update manager while running under
gcj, eclipse crashes.

 * The Eclipse source build lost the ability to compile the SWT native
libraries sometime between RC1 and the final release. This package
uses
    the .so files that come in the upstream tarball.

 * There are several jars that were replaced with symlinks to jars in
/usr/share/java. Since the 3.4 versions have had osgi properties added
to their manifests,
    this isn't a safe transformation anymore, so those files aren't
replaced with symlinks anymore.

 * P2 doesn't work. If you enable classic update, you can still use it
to install new plugins from update sites.

Despite all that, I've found it to be reasonably stable and much more
useful to me than the 3.2 packages in Hardy.
"

As you probably imagine, all of the above are non-starters :-)

  * And the one in https://launchpad.net/~rockwalrus/+archive

This seems to be a new/fresh effort, with 2 major differences from the above:

    * Use eclipse's build tools as much as possible (i.e., pdebuild
instead of raw Ant)

    * Use split source packages (i.e., different source packages for
eclipse-rcp, eclipse-platform etc)

This work is thus split to an "eclipse-bootstrap" package that
contains the core OSGi runtime (equinox) and some Ant machinery and
this in turn builds
eclipse-rcp, eclipse-platform etc.

Unfortunately, this work was never completed and I couldn't get the
generated eclipse platform to start successfully (but then maybe I
just didn't try
hard enough).

The above solution seems to be leading to *much* cleaner
packaging/building code which is good for obvious reasons, but I 'm
scared by the fact
that
        a) This means we pull from CVS, which might get us different
code than the one in the source zip

        b) We seem to need to write various .xml  (assemble*.xml etc)
files that are included in the source release ourselves  (perhaps with
every new
            version?)

I have already sent 2 emails with the above questions to Nathan but
still no answer (which is OK, since not even a week has passed and he
is probably
very busy right now).

Nevertheless, we have already managed to "steal" his icu4j package
that already has some of the work needed to be useful as an eclipse
dependency
and we 'll be looking at this approach for a more long-term
maintainable solution as well, just probably not now, because right
now our goal should
be IMHO to get a first working package out fast, sacrificing policy
compliance and maintainability if need be :-)

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Which brings us to the good news :-)

The good news is that we have a number of factors on our side now that
previous teams  did not:

                * There is a free JDK out there now (openjdk) that
works fine with eclipse and is free enough for debian main even !!!
                   This means we no longer have to wrestle against
GCJ, which is HUGE

                * Fedora has done most (all) of the work, to get
Eclipse 3.4.1 (Ganymede SR1) into Fedora 10. This means we have a
clear path to
                   follow for guaranteed success: "Copy Fedora where
possible" :-)

                   Moreover, they have largely documented what the
problems were that they had to solve and how they have solved them.

                            This documentation can be found here:
                                            *
http://fedoraproject.org/wiki/Features/Eclipse34

                            And the packaging files can be found here:
                                              * cvs -d
:pserver:anonymous at cvs.fedoraproject.org:/cvs/pkgs co devel/eclipse

                 * Even better, Andrew Overholt (aka overholt) and
Alexander Kurtakov (aka akurtakov), the main drivers behind the fedora
effort have been
                    very friendly and approachable and have already
answered quite a few of our questions.

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Now, a few things about Eclipse Ganymede itself and the differences
from previous versions:

                  * There is a new packaging system, called P2 that
deals with provisioning eclipse plugins. Docs for that are in
http://wiki.eclipse.org/P2
                     In the long  term this will enable all sorts of
interesting interactions between Eclipse Update Manager and Apt but
for now we just have
                     to cooperate with it as well as we can, otherwise
you will not be able to install plugins from the Update UI or they
won't work and let's
                     face it, both of those options Just Suck :-)

                     To work well with P2, we have to make our package
look like a P2 repository, i.e., we have to generate some XML metadata
(typical java thinking ;-)
                     And for that purpose we call the Director
application during package creation
(org.eclipse.equinox.p2.director.app.application).
                     Let's say P2 is not the easiest thing to figure
out right now and the fact that the other day we were helping in
#eclipse the CTO of Zend
                     get it right sort of proves the point ;-)

                   * As you probably know, Eclipse uses JNI (native
libraries) for a few functions and the worse thing is it sometimes
tends to keep those .so
                     files inside .jar archives (!). This is obviously
not good for linux distros for various reasons (see e.g.,

https://bugs.eclipse.org/bugs/show_bug.cgi?id=90535 - .so files
installed in a strange location)

                     Therefore various people joined forces and wrote
a small application called fileInitializer. We call this app during
package creation as well
                     and it neatly extracts .so files under
/usr/lib/eclipse/configuration and this mostly solves the problem. Our
problem now is we need a proper
                     upstream for this piece of code :-)

                    * Another thing about P2 is that it needs at least
parts of ECF (eclipse communications framework) for downloading files
and such. This is
                      kind of a problem since ECF in turn depends on
Eclipse. The fedora people have an ecf-filetransfer.tar.bz2 in their
sources in which they
                      have just the necessary bits for P2 to work and
this is the approach I think we should follow as well for now, but a
more maintainable solution
                      for the long term is still needed.

                   * Eclipse tends to include its dependencies to
"external" (i.e., non-eclipse.org) jars in the SourceBuild package and
in the binary package produced
                      by it, to be standalone installable. This is a
good strategy for e.g., Windows and MacOS but we are actually trying
to build a HIGH QUALITY OS here :-)
                      This means we need to follow DRY and have only
one jar for each function and symlink to it from the other places
where its needed.

                      Unfortunately, Eclipse wants again some
Eclipse-specific metadata in its dependencies, so we 'll have to track
each one of them and either add this extra
                      info to the package (OSGified Manifest.MF's
mostly)   or create a -eclipse subpackage ala Fedora's icu4j-eclipse
where we put the feature.xml and such.
                      Again "Copy Fedora" is our Mantra in this phase :-)

Hope that gives you a rough picture of how things are right now.
Please point newcomers to this thread and add your questions, things
that are missing etc.

Cheers and Sorry for the long post :-)
Pantelis