[Popcon-developers] Bug#574743: popularity-contest: all-popcon-results.txt.gz contains bogus data

[Bill Allombert]

On Sat, Mar 20, 2010 at 05:59:33PM +0000, Stuart Prescott wrote:
> Package: popularity-contest
> Version: 1.46
> Severity: normal
> 
> The popcon summary data at http://popcon.debian.org/all-popcon-results.txt.gz
> contains bogus data on lines 85993 to 85995 (at present):
> 
> Package: py<F4>hon-central                     0     0     0     1
> Package: /usr/lib/mime/packages/mime-suprort     0     0     0     1
> Package: grof<E6>-base                         0     1     0     0
> 
> This is presumably all dodgy data from just one submitter... perhaps the popcon
> aggregation scripts should filter such data that has package names that are
> clearly incorrect like these? (i.e. the package names are non-conformant with
> policy §5.6.7/§5.6.1)

Well, I have removed the broken entry. I do not know how far I want to remove
non-policy compliant package names: popcon never rejected non-Debian packages
before, though I suppose we have to remove package with 8bit characters to
avoid trouble wiht UTF-8 display.

> I presume that there is a simple checksum included in the data as it
> submitted by popcon so that issues with corruption in transit aren't an issue
> and that the data in question here indicates some poor user with a very badly
> broken status file.

There is no checksum in popcon submission unfortunately. This looks like
more like a broken TCP frame.

> Dodgy data like this is an issue for consumers of the popcon results such as
> the UDD (which obviously needs to be made more robust to such bad input).

Agreed, but note that malicious people can forge popcon report easily so
you still need to check for bad input.

Thanks for your bug report!
-- 
Bill. <ballombe at debian.org>

Imagine a large red swirl here.