[Qa-debsources] SPDX generation issues

[Stefano Zacchiroli]

On Thu, Jan 07, 2016 at 11:04:58AM +0100, Orestis Ioannou wrote:
> - Missing upstream name: native packages do not have an upstream name.
> this is an easy fix i ll do it in my PR.
> 
> - better error handling when somebody tries to generate the document but
> gives incorrect or no version. I am at it and will fix it in the PR.

OK, great!

> - files paragraphs missing copyright field (required in the docs). for
> example:
> http://sourcesdev.debian.net/src/matplotlib/1.5.0~rc2-1/debian/copyright/#L67
> 
> What's the best solution here? Ignore the paragraph? Give just none
> fields? right now it just breaks but on the license rendering we provide
> None fields for the ones we miss
> http://sourcesdev.debian.net/copyright/license/matplotlib/1.5.0~rc2-1/

I think we need more generally to decide what to do for invalid
machine-readable d/copyright files. My proposal is to ignore them, and
return an error indicating that they are invalid according to the spec.
In an ideal world this work should be done in Debian more generally by
an unrelated QA process/tooling. But right now we're probably the
biggest consumer of machine-readable d/copyright, so it is kinda normal
that we hit those errors earlier than others.

Being conservative will give an incentive to our users to fix broken
d/copyright files, or at least to report them as bug.

To that end, however, we need to do better than fail with an "internal
server error", we need to return a proper, user understandable error,
embedded in a flask template.  Relatedly, and in view of d/copyright
parsing at package extraction time by Debsources, we should also think
about loggging an appropriate error, so that we can easily grep the logs
and report bugs accordingly.

> - timeout errors on >medium sized packages.For example testing:
> http://sourcesdev.debian.net/copyright/spdx/python-django/1.9-1/
> [warn] [client 192.168.17.254] mod_fcgid: read data timeout in 40 seconds
> [error] [client 192.168.17.254] Premature end of script headers:
> debsources.fcgi
> 
> As we discussed in the other email thread, the errors due to timeout
> will probably be fixed for some packages when we will have the DB since
> we will just need to do one query in the DB instead of querying the
> d/copyright file for each file. Not sure though this will be ok for huge
> packages like linux, chromium etc.

I don't seen what we can do about this right now. The timeout seems like
a decent enough failure mode.

> Should i bug report the timeout error? The other i can fix them in the
> next days.

Please do report a bug about this, so that we can keep track of the
issue.

Thanks!
Cheers.
-- 
Stefano Zacchiroli  . . . . . . .  zack at upsilon.cc . . . . o . . . o . o
Maître de conférences . . . . . http://upsilon.cc/zack . . . o . . . o o
Former Debian Project Leader . . . . . @zacchiro . . . . o o o . . . o .
« the first rule of tautology club is the first rule of tautology club »
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 811 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/qa-debsources/attachments/20160107/6c31866e/attachment.sig>