[Soc-coordination] GSoC - Apt Ordering - Report 5

[Christopher Baines]

Hello,

So over the last two weeks I have been working on smarting up the code
(including comments), improving the code debuging and plenty of bug
fixes. Michael has been helping me out enormously with the testing, 

One particular bug was more challenging than others though,  so, as I
might have said before SmartUnPack and SmartConfigure are the two
important methods. To stop loops from forming, the Loop flag is used,
the problem with my original implementation was that it was quite crude.
If the loop flag was set on a package, it was simply assumed that it was
going to be unpacked or configured at the right time. 

So, take the rather complicated example, there are three packages: A, B
and C, being installed on the system, A depends on B and C, B and C both
depend on A. So when run this would happen:
SmartUnPack A               - First A is unpacked 
  SmartConfigure A          - Then configured (Loop set on A)
    SmartUnPack B           - B is unpacked as A depends on B
      SmartConfigure B      - Now as B depends on A, SmartConfigure
                              would be called, except the Loop 
                              flag is set on A, so its ignored 
    SmartUnPack C           - A also depends on C, so C is now unpacked 
      SmartConfigure C      - Then configured

This would result in these dpkg commands:
dpkg –unpack A B
dpkg –configure B           - Error here
dpkg –unpack C
dpkg –configure A C

So, you can see the problem above, A is not configure before or at the
same time as B (same time meaning for APT same dpkg call). Initially, I
could not work out how to fix this, but luckily David Kalnischkies was
on hand to help. David suggested that instead of just ignoring the
dependency when the loop flag is set, instead any problems with
satisfying it are addressed. Practically this meant allowing SmartUnPack
and SmartConfigure to be called more than once, and if they are called
on a package with the loop flag set, they will just address any checks.

So instead the above example would look like:
SmartUnPack A
  SmartConfigure A 
    SmartUnPack B
      SmartConfigure B
        SmartConfigure A
          SmartUnPack C
            SmartConfigure C
              SmartConfigure A

Resulting in:
dpkg –unpack A B C
dpkg –configure A B C

I have also been doing a lot of thinking on the implementation of
parallel downloading and installing of packages. The current thinking is
to extend the acquire systems model to also encompass package management
allowing the systems to be brought together. 

So apt would have a dpkg that could process queues of packages either
for installation or  configuration, matching the model behind the
downloading of packages. As queued packages are downloaded they can be
grouped correctly, then added to the correct installation or
configuration queues. I hope to begin coding properly on this perhaps in
the coming week (outside of GsoC). This is where in my opinion one of
the main benefit of Immediate Configuration for all packages lies, as
without Immediate Configuration for all packages this would not be
possible. 

On the subject I raised last report on dpkg –command-fd, Michael asked
on the debian-dpkg irc channel and the response given was that this
functionality is planned in libdpkg, but that's still a bit away. When
this is implemented APT will be able to hopefully use this to avoid dpkg
loading its database many times during a single apt call (this has a
much greater effect when using Immediate-Configure-All).

So in conclusion, the current improved APT (currently avalible from my
bzr branch [2]) will perform immediate configuration on all packages, if
given the argument -o APT::Immediate-Configure-All=true. The changes I
made to the package manager should also improve the reliability and
consistency of APT when faced with complex dependency scenarios,
irrespective of Immediate Configuration. The main downside at the moment
is because of the dpkg overhead explained at the end of the above
paragraph, running using Immediate-Configure-All is much slower. 

Thanks,

Chris

1.
http://lists.alioth.debian.org/pipermail/soc-coordination/2011-July/001054.html | http://lists.debian.org/deity/2011/07/msg00119.html
2. https://code.launchpad.net/~cbaines/apt/apt-chris
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: This is a digitally signed message part
URL: <http://lists.alioth.debian.org/pipermail/soc-coordination/attachments/20110821/0d591db2/attachment.pgp>