[Reportbug-maint] Bug#564112: cachefile srcpkgcache.bin is discarded without good reason

[David Kalnischkies]

package apt
retitle 564112 cachefile srcpkgcache.bin is discarded without good reason
tag 564112 confirmed
thanks

Hello Ben Hutchings,

> Each invocation of apt-cache was taking about 3 seconds (with a warm
> disk cache) and it was invoked once for each of ~400 binary package
> names.  I think that must be a bug in APT, and indeed it runs much
> faster following an 'apt-get update'

Yeah, the disk cache is "warm", but the APT cache is somehow cold as it
resists in /var/cache/apt and is only writable for root - so APT needs to
parse all files in /var/lib/apt/lists and the status files for each run...
"apt-get update" helps in this regard as it rebuilds the cache(s) (as root)
so the next APT runs have an up-to-date cache. Every other APT command
executed as root would have achieved the same (apt-cache gencaches).
We can't relax the write-access to the cache as APT depends on this
information and an evil user modifying this cache files would open all
sort of problems...

But the real question is: Why is the APT cache no longer up-to-date?
For APT a cache is no longer valid than the mtime of the cached file
has changed - as this happens more often for the status files than
the lists-file APT builds two caches:
srcpkgcache.bin - Cache of the /var/lib/apt/lists files
pkgcache.bin - Cache of srcpkgcache.bin + status files
The status files are pretty small and therefore fast to parse.
The real timesucker is the parsing of the lists files, so i guess,
(as long as your machine isn't really slow) APT thinks the
srcpkgcache.bin file is out-of-date - as APT can't write the cache
to the file it is only build in memory - 400 times in a row in your case...

I noticed these "no longer up-to-date" from time to time but
always thought the cause is my APT setup... I will take a deeper
look at it than it hits me the next time - and as the next APT will
include a debug option for it you are all invited to do the same. :)
(The option will be called Debug::pkgCacheGen)

>> I agree we can optimize a bit the code (but there's probably a reason
>> is done one-by-one)
While on it: As reportbug is a python app i would suggest to have
a look at APTs python interface "python-apt" - using it could
obsolete the whole calling and output-parsing (overhead) ...
(and maybe also look at reportbug-ng which seems to use this already).


Best regards / Mit freundlichen Grüßen,

David "DonKult" Kalnischkies