yada yada, new_deb822.py

[John Wright]

On Sat, Aug 19, 2006 at 05:46:01PM +0200, Adeodato Simó wrote:
> Hey.
> 
> John, I've recently realized that deb822.py is unable to cover one of my
> use cases, which is _fast_ iteration through e.g. unstable's Packages file.
> This is expected, of course, since the implementation is pure Python,
> but whilst I can see the advantages of having a pure python parser for
> rfc822 files, I still need the speed.
> 
> Yesterday Jeroen van Wolffelaar pointed out to me that python-apt itself
> has an interface to parse rfc822-like files, but a rather rudimentary one.
> It is very fast, though.
> 
> Since speed is what I need, I'll be probably using something similar to
> this in my application:
> 
>   http://people.debian.org/~adeodato/tmp/2006-08-19/new_deb822.py
> 
> A comparison on my system:
> 
>   % =time -p python -c 'import deb822 as deb822; [ None for x in deb822.Deb822.iter_paragraphs(file("/dev/shm/Packages"))]'
>   real 13.76
>   user 12.98
>   sys 0.11
> 
>   % =time -p python -c 'import new_deb822 as deb822; [ None for x in deb822.Deb822.iter_paragraphs(file("/dev/shm/Packages"))]'
>   real 0.98
>   user 0.89
>   sys 0.02
> 
> I want to work in my application for a bit now, but if you'd like, I can
> put down some time to merge both implementations, so that apt_pkg is
> used if available, and the Python one if not.

Wow; I hadn't bothered to see how long it took deb822 to parse
unstable's Packages...  I've used ParseTagFile before, but I was always
frustrated with the lack of documentation.  Still, wrapping deb822
around it isn't a bad idea.  I'd be willing to work on integration.
Would you prefer that I wait for a new_deb822 branch, or can I start
with new_deb822.py above?

> BTW, this are my pending branches against the old deb822.py:
> 
>   http://people.debian.org/~adeodato/code/branches/deb822/use_iterables/
>   http://people.debian.org/~adeodato/code/branches/deb822/limit_fields/

Great, I've merged these into trunk.

-- 
John Wright