[Collab-qa-commits] r516 - / bapase
lucas at alioth.debian.org
lucas at alioth.debian.org
Mon Nov 26 07:46:09 UTC 2007
Author: lucas
Date: 2007-11-26 07:46:09 +0000 (Mon, 26 Nov 2007)
New Revision: 516
Added:
bapase/
bapase/Makefile
bapase/Notes
bapase/README
bapase/Template.proposed-orphan
bapase/Template.proposed-removal
bapase/Templates
bapase/bugsummary.rb
bapase/gen_html.rb
bapase/merge_bugs_popcon.rb
bapase/package-actions.txt
bapase/pc_bugs_summary.rb
bapase/popcon_src
bapase/wnppsummary.rb
Log:
added bapase, BAd PAckages SEarch
Added: bapase/Makefile
===================================================================
--- bapase/Makefile (rev 0)
+++ bapase/Makefile 2007-11-26 07:46:09 UTC (rev 516)
@@ -0,0 +1,77 @@
+all: scores.html
+
+scores.html: Sources_ok testing-status.txt popcon_sources.txt package-actions.txt bugsummary wnppsummary
+ ./gen_html.rb > $@
+
+Sources_ok: testing-main-Sources testing-contrib-Sources testing-non-free-Sources unstable-main-Sources unstable-contrib-Sources unstable-non-free-Sources stable-main-Sources stable-contrib-Sources stable-non-free-Sources oldstable-main-Sources oldstable-contrib-Sources oldstable-non-free-Sources experimental-main-Sources experimental-contrib-Sources experimental-non-free-Sources
+ touch Sources_ok
+
+################# SOURCES
+
+experimental-main-Sources:
+ wget -O - http://ftp.debian.org/debian/dists/experimental/main/source/Sources.gz | gunzip > $@
+
+experimental-contrib-Sources:
+ wget -O - http://ftp.debian.org/debian/dists/experimental/contrib/source/Sources.gz | gunzip > $@
+
+experimental-non-free-Sources:
+ wget -O - http://ftp.debian.org/debian/dists/experimental/non-free/source/Sources.gz | gunzip > $@
+
+
+oldstable-main-Sources:
+ wget -O - http://ftp.debian.org/debian/dists/oldstable/main/source/Sources.gz | gunzip > $@
+
+oldstable-contrib-Sources:
+ wget -O - http://ftp.debian.org/debian/dists/oldstable/contrib/source/Sources.gz | gunzip > $@
+
+oldstable-non-free-Sources:
+ wget -O - http://ftp.debian.org/debian/dists/oldstable/non-free/source/Sources.gz | gunzip > $@
+
+
+stable-main-Sources:
+ wget -O - http://ftp.debian.org/debian/dists/stable/main/source/Sources.gz | gunzip > $@
+
+stable-contrib-Sources:
+ wget -O - http://ftp.debian.org/debian/dists/stable/contrib/source/Sources.gz | gunzip > $@
+
+stable-non-free-Sources:
+ wget -O - http://ftp.debian.org/debian/dists/stable/non-free/source/Sources.gz | gunzip > $@
+
+testing-main-Sources:
+ wget -O - http://ftp.debian.org/debian/dists/testing/main/source/Sources.gz | gunzip > $@
+
+testing-contrib-Sources:
+ wget -O - http://ftp.debian.org/debian/dists/testing/contrib/source/Sources.gz | gunzip > $@
+
+testing-non-free-Sources:
+ wget -O - http://ftp.debian.org/debian/dists/testing/non-free/source/Sources.gz | gunzip > $@
+
+unstable-main-Sources:
+ wget -O - http://ftp.debian.org/debian/dists/unstable/main/source/Sources.gz | gunzip > $@
+
+unstable-contrib-Sources:
+ wget -O - http://ftp.debian.org/debian/dists/unstable/contrib/source/Sources.gz | gunzip > $@
+
+unstable-non-free-Sources:
+ wget -O - http://ftp.debian.org/debian/dists/unstable/non-free/source/Sources.gz | gunzip > $@
+
+################# / SOURCES
+
+testing-status.txt:
+ wget http://qa.debian.org/~lucas/testing-status.txt
+
+popcon_sources.txt: Sources_ok
+ ./popcon_src > popcon_sources.txt
+
+bugsummary: bts2ldap-fullindex
+ ./bugsummary.rb > bugsummary
+
+bts2ldap-fullindex:
+ wget -O bts2ldap-fullindex http://qa.debian.org/data/bts2ldap/fullindex
+
+wnppsummary: bts2ldap-fullindex
+ ./wnppsummary.rb > wnppsummary
+clean:
+ rm -f testing-status.txt *-Sources popcon_sources.txt scores.txt Sources_ok bts2ldap-fullindex RC+patch.txt bugsummary wnppsummary
+
+.PHONY: clean
Added: bapase/Notes
===================================================================
--- bapase/Notes (rev 0)
+++ bapase/Notes 2007-11-26 07:46:09 UTC (rev 516)
@@ -0,0 +1,187 @@
+Packages Police
+
+Objectives:
+- reduce the number of useless packages in Debian
+ rationale:
+ - useless packages eat DD time:
+ - RC bugs to fix
+ - RC bugs to file (time lost filing bugs against useless packages when doing archive-wide QA)
+ - useless packages eat users time:
+ - better alternatives
+ - need to choose between more packages
+- reduce the number of poorly maintained packages in Debian
+ rationale:
+ - obviously, improve the general quality of Debian
+
+Objectives similar as MIA, but package-centric approach.
+Might allow to partially avoid the sensitive personal issues.
+
+Things we can do:
+- Triage packages orphaned for a long time
+- Try to find suspicious packages using different metrics, and propose solutions
+
+Mix of automated and manual actions
+-> define workflow, processes, to be efficient
+-> discuss them so they are widely accepted
+
+Orphaned packages:
+- lots of packaged orphaned for a long time
+- we release them!
+- orphaned should only be a temporary state between two maintainers, or between one maintainer and the removal from Debian.
+
+Orphaned packages: workflow (proposal)
+
+t0: package is orphaned
+
+After 6 months, ping previous maintainer, potential adopters (bug commenters, relevant teams, maintainers of reverse dependencies)
+set O bug to severity: serious and remove package from testing
+
+If 2 months later, no answer from anybody (bug still O, not ITA), remove the
+package from Debian (and store the last package somewhere, so it can easily be
+re-added without starting from scratch).
+
+ITA bugs:
+watch bugs with no action for more than 2 months. Ask the adopter.
+If no answer for a month, revert to O.
+
+Important points:
+- we can start with the "easy" packages:
+ orphaned for > 2 years, popcon < 500
+
+----------------
+Suspicious packages
+Goal: find useless and/or broken packages.
+Using different metrics:
+- popcon
+- RC bugs
+- bugs
+- time of last maintainer upload
+- time of last upload
+- testing status (in testing ? trying to migrate ? for how long ?)
+- WNPP status (O, RFH, RFA) (for how long ?)
+- MIA status of the maintainer
+- number of packages maintained by the maintainer (1 is bad, 100 as well)
+- number of maintainers for the package
+- team maintained?
+
+Combine those metrics using a scoring system, and find suspicious packages.
+
+Workflow:
+(1)
+- File a bug, explaining to the maintainer that we believe there's a problem.
+- Propose solutions (finding co-maintainers, joining a team)
+- Threaten to orphan the package or to remove it
+- Bug severity: serious if really worrying, or important.
+
+(2)
+- after 2 months, review the package again.
+- do appropriate action.
+ - before removal, try to find 1 seconder.
+
+----------
+Subject: Handling of poorly maintained and useless packages
+
+Hi,
+
+This mail tries to address the problem of not orphaned, poorly maintained
+or useless packages in Debian. The proposal below tries to address both
+cases, because they are often related. (useless packages tend to be poorly
+maintained, and vice-versa).
+
+[I originally planned to discuss this during the QA meeting. But it might
+be a better idea to first start the discussion on the mailing list, so we
+get a rough idea of the things that have to be discussed in the meeting.
+Also, this was originally posted to -qa at . Since it didn't degenerate into
+a flamewar, I'm posting this to -devel@ after doing some minor changes to
+get opinions from a larger audience.]
+
+Why is it bad to have those packages in Debian?
+-----------------------------------------------
+Some distributions have a "the more, the better" policy. I don't think
+that it is what we want for Debian, because:
+- we (try to) support all packages the same way. Which means we have to
+ support those packages as well.
+- such packages eat DD time. Bugs are filed against them when doing
+ archive-wide QA, people fix RC bugs in them, etc.
+- some poorly maintained packages are actually useful, and have people
+ willing to take over maintenance, but it's currently difficult to
+ hijack packages, especially when the maintainer is unresponsive.
+- such packages eat user time: when trying to find a package doing X,
+ it's better if users only have to evaluate 4 packages, instead of 6
+ packages, with 2 packages being clearly inferior solutions. Even
+ worse, some users might be using the inferior packages, ignoring that
+ there are better alternatives.
+
+Current status
+--------------
+Michael Ablassmeier and me filed some bugs some time ago on packages
+that were good candidates for orphaning or removal from Debian. The list
+can be viewed at [1]. However, we haven't orphaned/removed the packages
+so far.
+In 2005, Marc Brockschmidt did the same kind of bug filing, but was
+brave enough to orphan/file removal requests.
+
+Why do we need a workflow for that?
+-----------------------------------
+There's an authority problem in Debian. Even if nobody disagrees that a
+package should be removed, if the maintainer is unresponsive, usually,
+nobody takes the decision to remove it. Having some "rules" one could
+refer to would help. Also, we have to agree on common "rules", so
+everybody processes this stuff the same way.
+
+Proposed workflow
+-----------------
+Suspicious packages are found by combining different metrics into a
+scoring system:
+- popcon score
+- RC bugs
+- number of bugs (possibly per popcon inst)
+- age of last maintainer upload
+- testing status (in testing? trying to migrate? for how long?)
+
+Some additional info might also be useful:
+- age of last upload
+- WNPP status (O, RFH, RFA) (for how long?)
+- Maintainer's MIA status
+- number of packages maintained by the maintainer
+- number of maintainers for the package
+- is the package team maintained?
+
+Step 1:
+- Based on the scoring system, find a suspicious package.
+- Review the package manually (look at bugs, etc)
+- If the package needs action, file a bug:
+ - severity: serious
+ - explaining what are the problems with the package
+ - proposing a solution (orphaning the package, or removing it
+ from Debian, or finding co-maintainers)
+ - make it clear that, without answer, the proposed solution will
+ be carried out
+
+Step 2: (when the problems haven't been solved)
+- Review the package again
+- Take the proposed actions
+
+Delays and control:
+-------------------
+It's important to decide on reasonable delays.
+- If the procedure takes too long, it will be discouraging
+- If the procedure is too short, decisions will be contested
+I think that the following makes sense:
+- For packages where orphaning was proposed: 50 days
+- For packages where removal was proposed: 100 days
+Additionally, before removals, at least one DD should second the removal
+request (after reviewing the package).
+
+Rationale: orphaning can be easily reversed in case the maintainer
+suddenly wakes up again. Removal is a bit harder to reverse (need to fetch
+the package from snapshot.d.n), thus the longer delay and the seconder.
+
+So, what do you think about that? Any proposed changes?
+
+[1] http://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=proposed-orphan,proposed-removal;users=debian-qa@lists.debian.org;nam0=Proposals;pri0=tag:proposed-removal,proposed-orphan;ttl0=Proposed%20to%20be%20removed,Proposed%20to%20be%20orphaned;nam1=Status;pri1=severity:serious,normal;ttl1=No%20answer%20yet,acknowledged;nam2=Progress;pri2=pending:pending,done;ttl2=In%20progress,Solved
+
+
+
+
+/org/qa.debian.org/data/bts2ldap/fullindex
Added: bapase/README
===================================================================
--- bapase/README (rev 0)
+++ bapase/README 2007-11-26 07:46:09 UTC (rev 516)
@@ -0,0 +1,4 @@
+bapase -- BAd PAckages SEarch
+-----------------------------
+This directory contains a set of scripts which use various metrics to determine
+packages that require some actions (removal, orphaning, maintainer ping, etc)
Added: bapase/Template.proposed-orphan
===================================================================
--- bapase/Template.proposed-orphan (rev 0)
+++ bapase/Template.proposed-orphan 2007-11-26 07:46:09 UTC (rev 516)
@@ -0,0 +1,36 @@
+To: submit at bugs.debian.org
+Subject: XXX: should this package be orphaned?
+
+Package: XXX
+Version: XXX
+Severity: serious
+User: debian-qa at lists.debian.org
+Usertags: proposed-orphan
+
+Hi,
+
+While reviewing some packages, your package came up as a package that
+should maybe be orphaned by its maintainer, because:
+
+[ XXX include as many reasons as possible here ]
+
+If you think that it should be removed from Debian instead of being
+orphaned, please reply to this bug and tell so.
+
+If you agree that it should be orphaned, sending the following commands
+to control at bugs.debian.org should do it (after replacing nnnnnn with
+this bug's number):
+
+severity nnnnnn normal
+reassign nnnnnn wnpp
+retitle nnnnnn O: <packagename> -- <short package description>
+thanks
+
+For more information, see
+http://www.debian.org/doc/developers-reference/ch-pkgs.en.html#s-archive-manip
+http://www.debian.org/devel/wnpp/
+
+If you disagree and want to continue to maintain this package, please
+close this bug, preferably in an upload also fixing the other issues.
+
+Thank you,
Added: bapase/Template.proposed-removal
===================================================================
--- bapase/Template.proposed-removal (rev 0)
+++ bapase/Template.proposed-removal 2007-11-26 07:46:09 UTC (rev 516)
@@ -0,0 +1,35 @@
+To: submit at bugs.debian.org
+Subject: XXX: should this package be removed?
+
+Package: XXX
+Version: XXX
+Severity: serious
+User: debian-qa at lists.debian.org
+Usertags: proposed-removal
+
+Hi,
+
+While reviewing some packages, your package came up as a possible
+candidate for removal from Debian, because:
+
+[ XXX include as many reasons as possible here ]
+
+If you think that it should be orphaned instead of being removed from
+Debian, please reply to this bug and tell so.
+
+If you agree, sending the following commands to control at bugs.debian.org
+should do it (after replacing nnnnnn with this bug's number):
+severity nnnnnn normal
+reassign nnnnnn ftp.debian.org
+retitle nnnnnn RM: <packagename> -- RoM; <reasons>
+thanks
+
+For more information, see
+http://wiki.debian.org/ftpmaster_Removals
+http://ftp-master.debian.org/removals.txt
+
+If you disagree and want to continue to maintain this package, please
+just close this bug, preferably in an upload also fixing the other
+issues.
+
+Thank you,
Added: bapase/Templates
===================================================================
--- bapase/Templates (rev 0)
+++ bapase/Templates 2007-11-26 07:46:09 UTC (rev 516)
@@ -0,0 +1,25 @@
+Hi,
+
+Since the situation didn't improve, and the maintainer never answered
+this bug report, I'm now orphaning the package, as originally suggested.
+
+Thank you,
+
+
+-------------
+clone n -1
+reassign -1 wnpp
+severity -1 normal
+retitle -1
+thanks
+
+Hi,
+
+Since the situation hasn't evolved, I am now orphaning this package.
+I still plan to request its removal in the near future, but this orphaning could allow someone to adopt the package if appropriate.
+
+Please reply to this bug report if you feel that this package shouldn't be removed.
+
+Please also reply if you think that this package _should_ be removed, as this will allow to make the process quicker.
+
+Thank you,
Added: bapase/bugsummary.rb
===================================================================
--- bapase/bugsummary.rb (rev 0)
+++ bapase/bugsummary.rb 2007-11-26 07:46:09 UTC (rev 516)
@@ -0,0 +1,104 @@
+#!/usr/bin/ruby -w
+
+bugid = nil
+bugsrcpkg = nil
+bugseverity = nil
+bugtitle = nil
+bugtags = []
+bugaffect = []
+
+f = File::new('RC+patch.txt', 'w')
+
+$pkgs = {}
+
+class BugStat
+ attr_accessor :rc_testing, :rc_unstable, :imp_testing, :imp_unstable, :normal_testing, :normal_unstable, :all_testing, :all_unstable
+
+ def initialize
+ @rc_testing = 0
+ @rc_unstable = 0
+ @imp_testing = 0
+ @imp_unstable = 0
+ @normal_testing = 0
+ @normal_unstable = 0
+ @all_testing = 0
+ @all_unstable = 0
+ end
+
+ def to_s
+ "#{@rc_testing} #{@rc_unstable} #{@imp_testing} #{@imp_unstable} #{@normal_testing} #{@normal_unstable} #{@all_testing} #{@all_unstable}"
+ end
+end
+
+IO::read('bts2ldap-fullindex').each_line do |l|
+ l.chomp!
+ key, val = l.split(' ', 2)
+ if key == 'dn:' and bugid != nil
+ # FIXME fin du bug en cours
+ if bugtags.include?('patch') and ['serious', 'grave', 'critical'].include?(bugseverity) and bugaffect.include?('testing') and bugaffect.include?('unstable')
+ f.puts "#{bugid} #{bugsrcpkg} #{bugtitle}"
+ end
+
+ if $pkgs[bugsrcpkg].nil?
+ $pkgs[bugsrcpkg] = BugStat::new
+ end
+
+ if bugaffect.include?('testing')
+ if ['serious', 'grave', 'critical'].include?(bugseverity)
+ $pkgs[bugsrcpkg].rc_testing += 1
+ $pkgs[bugsrcpkg].imp_testing += 1
+ $pkgs[bugsrcpkg].normal_testing += 1
+ $pkgs[bugsrcpkg].all_testing += 1
+ elsif bugseverity == 'important'
+ $pkgs[bugsrcpkg].imp_testing += 1
+ $pkgs[bugsrcpkg].normal_testing += 1
+ $pkgs[bugsrcpkg].all_testing += 1
+ elsif bugseverity == 'normal'
+ $pkgs[bugsrcpkg].normal_testing += 1
+ $pkgs[bugsrcpkg].all_testing += 1
+ else
+ $pkgs[bugsrcpkg].all_testing += 1
+ end
+ end
+
+ if bugaffect.include?('unstable')
+ if ['serious', 'grave', 'critical'].include?(bugseverity)
+ $pkgs[bugsrcpkg].rc_unstable += 1
+ $pkgs[bugsrcpkg].imp_unstable += 1
+ $pkgs[bugsrcpkg].normal_unstable += 1
+ $pkgs[bugsrcpkg].all_unstable += 1
+ elsif bugseverity == 'important'
+ $pkgs[bugsrcpkg].imp_unstable += 1
+ $pkgs[bugsrcpkg].normal_unstable += 1
+ $pkgs[bugsrcpkg].all_unstable += 1
+ elsif bugseverity == 'normal'
+ $pkgs[bugsrcpkg].normal_unstable += 1
+ $pkgs[bugsrcpkg].all_unstable += 1
+ else
+ $pkgs[bugsrcpkg].all_unstable += 1
+ end
+ end
+
+ bugtags = []
+ bugaffect = []
+
+ elsif key == 'debbugsID:'
+ bugid = val
+ elsif key == 'debbugsSourcePackage:'
+ bugsrcpkg = val
+ elsif key == 'debbugsSeverity:'
+ bugseverity = val
+ elsif key == 'debbugsTag:'
+ bugtags << val
+ elsif key == 'debbugsTitle:'
+ bugtitle = val
+ elsif key == 'debbugsAffected:'
+ bugaffect << val
+ end
+end
+
+f.close
+
+$pkgs.each_pair do |k,v|
+ puts "#{k} #{v}"
+end
Property changes on: bapase/bugsummary.rb
___________________________________________________________________
Name: svn:executable
+ *
Added: bapase/gen_html.rb
===================================================================
--- bapase/gen_html.rb (rev 0)
+++ bapase/gen_html.rb 2007-11-26 07:46:09 UTC (rev 516)
@@ -0,0 +1,318 @@
+#!/usr/bin/ruby -w
+
+require 'date'
+
+DATEZERO = Date::parse('0000-01-01')
+CURDATE = Date::today
+
+# hash for bin->src resolution
+$srcpkg = {}
+
+# hash for src->bins resolution
+$pkgs = {}
+
+# hash for testing status
+$testing = {}
+
+# read *-Sources and feed $srcpkg and $pkgs
+def read_sources
+ Dir::glob('{testing,unstable}-*-Sources') do |s|
+ src = nil
+ IO::read(s).each_line do |l|
+ if l =~ /^Package:/
+ src = l.split(' ')[1].chomp
+ $pkgs[src] = [] if $pkgs[src].nil?
+ elsif l =~ /^Binary:/
+ l.split(' ',2)[1].split(', ').each do |b|
+ b.chomp!
+ if $srcpkg[b].nil? # else, we already know that binary pkg
+ $srcpkg[b] = src
+ $pkgs[src] << b
+ end
+ end
+ end
+ end
+ end
+end
+
+class TestingStatus
+ attr_accessor :testingversion, :unstableversion, :firstinunstable, :testingdays, :intesting, :syncdays, :sync, :syncversion
+end
+
+# read testing-status.txt and feed $testing
+def read_testing
+ IO::read('testing-status.txt').each_line do |l|
+ ts = TestingStatus::new
+ p, ts.testingversion, ts.unstableversion, ts.firstinunstable, ts.testingdays, ts.intesting, ts.syncdays, ts.sync, ts.syncversion = l.chomp.split(' ')
+ ts.firstinunstable = Date::parse(ts.firstinunstable)
+ ts.intesting = Date::parse(ts.intesting)
+ ts.sync = Date::parse(ts.sync)
+ ts.syncdays = ts.syncdays.to_i
+ ts.testingdays = ts.testingdays.to_i
+ $testing[p] = ts
+ end
+end
+
+class Actions
+ attr_reader :actions, :act_todo, :act_status, :act_comment
+ def initialize
+ @actions = []
+ @act_status = ""
+ @act_todo = false
+ @act_comment = ""
+ end
+
+ def add(desc)
+ desc.chomp!
+ date, who, act, comment = desc.split(' ', 4)
+ date = Date::parse(date)
+ if act =~ /^(.+)\((.+)\)$/
+ act_name, act_arg = $1, $2
+ if [ 'PROP_RM', 'PROP_RM_O', 'PROP_O', 'O', 'REQ_RM', 'SEC_RM' ].include?(act_name)
+ # FIXME check bug
+ elsif act_name == 'OK'
+ act_arg = act_arg.to_i
+ else
+ puts "Unknown action: #{act} (#{desc})"
+ end
+ @actions << [date, who, [act_name, act_arg], comment]
+ else
+ puts "Unparseable action: #{act} (#{desc})"
+ exit(1)
+ end
+ end
+
+ def analyze_actions
+ @actions.sort! { |a,b| b[0] <=> a[0] }
+ idx = 0
+ rm_o = false
+ while idx < @actions.length
+ if @actions[idx][2][0] == 'OK'
+ if @actions[idx][0] + @actions[idx][2][1] <= CURDATE
+ idx += 1
+ next # OK not valid anymore, consider next action
+ else
+ # nothing to do except waiting
+ @act_status = "OKed (until #{@actions[idx][0] + @actions[idx][2][1]})"
+ @act_comment = @actions[idx][3] if not @actions[idx][3].nil?
+ break
+ end
+ elsif @actions[idx][2][0] == 'REQ_RM'
+ @act_status = "<a href=\"http://bugs.debian.org/#{@actions[idx][2][1]}\">Removal was requested</a>"
+ @act_comment = @actions[idx][3] if not @actions[idx][3].nil?
+ break
+ elsif @actions[idx][2][0] == 'O'
+ @act_status = "<a href=\"http://bugs.debian.org/#{@actions[idx][2][1]}\">Was orphaned</a>"
+ @act_comment = @actions[idx][3] if not @actions[idx][3].nil?
+ break
+ elsif @actions[idx][2][0] == 'SEC_RM'
+ @act_status = "<a href=\"http://bugs.debian.org/#{@actions[idx][2][1]}\">Removal request needed by !#{@actions[idx][1]}</a>"
+ @act_todo = true
+ @act_comment = @actions[idx][3] if not @actions[idx][3].nil?
+ break
+ elsif @actions[idx][2][0] == 'PROP_RM_O'
+ @act_status = "<a href=\"http://bugs.debian.org/#{@actions[idx][2][1]}\">Was orphaned, will need removal</a>"
+ rm_o = true
+ idx += 1
+ @act_comment = @actions[idx][3] if not @actions[idx][3].nil?
+ next
+ elsif @actions[idx][2][0] == 'PROP_RM'
+ ok = false
+ if @actions[idx][0] + 50 <= CURDATE and !rm_o
+ @act_status = "<a href=\"http://bugs.debian.org/#{@actions[idx][2][1]}\">Should be orphaned before removal (since #{@actions[idx][0] + 50})</a>"
+ @act_todo = true
+ ok = true
+ end
+ if @actions[idx][0] + 100 <= CURDATE
+ @act_status = "<a href=\"http://bugs.debian.org/#{@actions[idx][2][1]}\">Should be removed (since #{@actions[idx][0] + 100})</a>"
+ @act_todo = true
+ ok = true
+ end
+ if !ok
+ @act_status = "<a href=\"http://bugs.debian.org/#{@actions[idx][2][1]}\">Removal suggested (since #{@actions[idx][0]})</a>"
+ end
+ @act_comment = @actions[idx][3] if not @actions[idx][3].nil?
+ break
+ elsif @actions[idx][2][0] == 'PROP_O'
+ if @actions[idx][0] + 50 <= CURDATE
+ @act_status = "<a href=\"http://bugs.debian.org/#{@actions[idx][2][1]}\">Should be orphaned (since #{@actions[idx][0] + 50})</a>"
+ @act_todo = true
+ else
+ @act_status = "<a href=\"http://bugs.debian.org/#{@actions[idx][2][1]}\">Orphaning suggested (since #{@actions[idx][0]})</a>"
+ end
+ @act_comment = @actions[idx][3] if not @actions[idx][3].nil?
+ break
+ else
+ puts "Unknown act: #{@actions[idx][2][0]}"
+ exit(1)
+ end
+ end
+ end
+
+ def Actions::read(file)
+ pkgs = {}
+ IO::read(file).each_line do |l|
+ next if l =~/^\s*#/ or l =~/^\s*$/
+ pkg, rest = l.split(' ',2)
+ if pkgs[pkg].nil?
+ pkgs[pkg] = Actions::new
+ end
+ pkgs[pkg].add(rest)
+ end
+ pkgs.each_pair { |k, v| v.analyze_actions }
+ pkgs
+ end
+end
+
+# Popcon
+$popcon = Hash::new { 0 }
+def read_popcon
+ IO::read('popcon_sources.txt').each_line do |l|
+ pkg, v = l.split
+ $popcon[pkg] = v.to_i
+ end
+end
+
+# BUGS
+$rcbugs = Hash::new { 0 }
+$bugs = Hash::new { 0 }
+
+def read_bugs
+ IO::read('bugsummary').each_line do |l|
+ f = l.split(' ')
+ $rcbugs[f[0]] = f[2].to_i
+ $bugs[f[0]] = f[8].to_i
+ end
+end
+
+# WNPP
+$wnpp = Hash::new
+
+def read_wnpp
+ IO::read('wnppsummary').each_line do |l|
+ p, t, n = l.split(' ')
+ if $wnpp[p].nil?
+ $wnpp[p] = []
+ end
+ $wnpp[p] << [ t, n ]
+ end
+end
+
+# ACTIONS
+$actions = Actions::read('package-actions.txt')
+read_sources
+read_testing
+read_popcon
+read_bugs
+read_wnpp
+
+$score = {}
+$pkgs.keys.each do |pkg|
+ score = 0
+ if $testing[pkg]
+ ts = $testing[pkg]
+ score += ts.testingdays
+ score += (ts.syncdays * 0.5) if ts.syncdays > 10
+ else
+ score += 1000000
+ end
+ if $actions[pkg] and $actions[pkg].act_todo
+ score += 100000 # bump score if action needed
+ end
+ if $popcon[pkg] < 500
+ score += (500 - $popcon[pkg]) * 2
+ end
+ if $rcbugs[pkg] > 0
+ if $rcbugs[pkg] > 4
+ score += 5 * 300
+ else
+ score += $rcbugs[pkg] * 300
+ end
+ end
+ # score bugs
+ if $bugs[pkg] > 0
+ pc = $popcon[pkg]
+ bugs = $bugs[pkg]
+ if pc < 4000
+ norm = 1 + 0.002 * pc
+ elsif pc < 60000
+ norm = 7.85 + (16.0/56000) * pc
+ else
+ norm = 25
+ end
+ if bugs > norm
+ ret = (bugs - norm)/norm * 100 * 10 # 10 % en trop -> 100 pts
+ score += (ret > 2000 ? 2000 : ret)
+ end
+ end
+ $score[pkg] = score if score > 0
+end
+
+puts "<html><head>
+<style type=\"text/css\">
+ td, th {
+ border: 1px solid gray;
+ }
+ tr:hover {
+ background-color: #ccc;
+ }
+ table {
+ border-collapse: collapse;
+ }
+</style>
+<title>PP</title>
+</head><body>"
+puts "<table border=\"1\"><tr>"
+puts "<th></th><th>Package</th><th>Score</th><th>Action</th><th>Testing</th><th>Migrate</th><th>Popcon</th><th>WNPP</th><th>RC</th><th>Bugs</th><th>Comments</th>"
+puts "</tr>"
+
+n = 0
+$score.to_a.sort { |a,b| b[1] <=> a[1] }.each do |e|
+ pkg = e[0]
+ n += 1
+ break if n >= 1000
+ puts "<tr><td>#{n}</td>"
+ puts "<td><a href=\"http://packages.qa.debian.org/#{pkg}\">#{pkg}</a></td>"
+ puts "<td>#{$score[pkg]}</td>"
+ if $actions[pkg]
+ if $actions[pkg].act_todo
+ puts "<td><b>#{$actions[pkg].act_status}</b></td>"
+ else
+ puts "<td>#{$actions[pkg].act_status}</td>"
+ end
+ else
+ puts "<td></td>"
+ end
+ if $testing[pkg]
+ puts "<td>#{$testing[pkg].testingdays}</td><td>#{$testing[pkg].syncdays}</td>"
+ else
+ puts "<td>?</td><td>?</td>"
+ end
+
+ puts "<td>#{$popcon[pkg]}</td>"
+
+ puts "<td>"
+ if $wnpp[pkg]
+ $wnpp[pkg].each do |e|
+ puts "<a href=\"http://bugs.debian.org/#{e[1]}\">#{e[0]}</a> "
+ end
+ end
+ puts "</td>"
+
+ puts "<td><a href=\"http://bugs.debian.org/src:#{pkg}\">#{$rcbugs[pkg]}</a></td>"
+ puts "<td><a href=\"http://bugs.debian.org/src:#{pkg}\">#{$bugs[pkg]}</a></td>"
+ # comments
+ if $actions[pkg]
+ comment = $actions[pkg].act_comment.gsub(/#\d+/) do |bug|
+ bugn = bug.gsub(/^#/, '')
+ "<a href=\"http://bugs.debian.org/#{bugn}\">#{bug}</a>"
+ end
+ puts "<td>#{comment}</td>"
+ else
+ puts "<td></td>"
+ end
+ puts "</tr>"
+end
+puts "</table>"
+puts " -- #{$score.length} packages listed."
+puts "</body></html>"
Property changes on: bapase/gen_html.rb
___________________________________________________________________
Name: svn:executable
+ *
Added: bapase/merge_bugs_popcon.rb
===================================================================
--- bapase/merge_bugs_popcon.rb (rev 0)
+++ bapase/merge_bugs_popcon.rb 2007-11-26 07:46:09 UTC (rev 516)
@@ -0,0 +1,25 @@
+#!/usr/bin/ruby -w
+
+bugs = {}
+IO::read('bugsummary').each do |l|
+ p, bs = l.chomp.split(' ', 2)
+ bugs[p] = bs
+end
+
+popc = {}
+bugs.keys.each do |p|
+ popc[p] = 0
+end
+
+IO::read('popcon_sources.txt').each do |l|
+ p, pc = l.chomp.split(' ', 2)
+ popc[p] = pc
+end
+
+(popc.keys + bugs.keys).sort.uniq.each do |p|
+ if bugs[p].nil?
+ bugs[p] = "0 0 0 0 0 0 0 0"
+ end
+ puts "#{p} #{popc[p]} #{bugs[p]}"
+end
+
Added: bapase/package-actions.txt
===================================================================
--- bapase/package-actions.txt (rev 0)
+++ bapase/package-actions.txt 2007-11-26 07:46:09 UTC (rev 516)
@@ -0,0 +1,140 @@
+topal 2007-05-10 lucas PROP_RM(423202)
+topal 2007-10-14 lucas REQ_RM(423202)
+
+estraier 2007-05-10 lucas PROP_RM(423242)
+estraier 2007-10-13 lucas PROP_RM_O(446514)
+estraier 2007-10-14 lucas OK(20)
+estraier 2007-11-05 lucas REQ_RM(423242)
+
+liblinux-aio-perl 2007-05-11 lucas PROP_RM(423327)
+liblinux-aio-perl 2007-10-13 lucas PROP_RM_O(423294)
+liblinux-aio-perl 2007-10-14 lucas OK(20)
+liblinux-aio-perl 2007-11-05 lucas REQ_RM(423327)
+
+php-image-canvas 2007-05-11 lucas PROP_RM(423353)
+php-image-canvas 2007-10-14 lucas OK(30)
+
+php-image-graph 2007-05-11 lucas PROP_O(423310)
+php-image-graph 2007-10-14 lucas OK(30)
+
+gutenbrowser 2007-06-20 lucas PROP_RM(429795)
+gutenbrowser 2007-10-12 lucas OK(30)
+
+pnetc 2007-05-11 abi PROP_RM(423354)
+pnetc 2007-10-13 lucas PROP_RM_O(423316)
+pnetc 2007-10-14 lucas OK(30)
+
+ttyrec 2007-09-30 lucas PROP_RM(444704)
+
+phpqladmin 2007-09-30 lucas PROP_RM(444709)
+
+skippy 2007-09-30 lucas PROP_RM(444715)
+
+sope 2007-09-30 lucas PROP_RM(444718)
+
+tinysnmp 2007-09-30 lucas PROP_RM(444723)
+
+tmda 2007-09-30 lucas PROP_RM(444724)
+
+quake2 2007-05-10 lucas PROP_O(423200)
+quake2 2007-10-13 lucas OK(15)
+quake2 2007-11-05 lucas OK(20)
+
+ppscsi 2007-05-11 lucas PROP_O(423317)
+ppscsi 2007-10-13 lucas OK(15)
+ppscsi 2007-10-14 lucas PROP_RM(423317)
+
+swi-prolog 2007-05-11 lucas PROP_O(423322)
+swi-prolog 2007-10-13 lucas OK(30)
+
+parrot 2007-09-30 lucas PROP_O(444708)
+
+pugs 2007-09-30 lucas PROP_O(444711)
+
+prc-tools 2007-09-30 lucas PROP_O(444713)
+
+irssi 2007-10-08 lucas PROP_O(445840)
+
+crystalspace 2007-10-02 lucas OK(30)
+crystalspace-data 2007-10-02 lucas OK(30)
+
+zangband 2007-10-14 lucas OK(100)
+
+gabber2 2007-10-10 lucas REQ_RM(444725)
+
+jabberoo 2007-10-10 lucas REQ_RM(444726)
+
+interchange 2007-10-14 lucas OK(50) see discussion in #340576
+
+gcc-snapshot 2007-10-10 lucas OK(1000)
+
+hurd 2007-10-10 lucas OK(1000)
+
+gnumach 2007-10-10 lucas OK(1000)
+
+gtklookat 2007-10-14 lucas OK(20)
+
+ghc-cvs 2007-10-10 lucas OK(1000)
+
+twutils 2007-10-10 lucas OK(20)
+
+bayonne 2007-10-14 lucas OK(20) needs a sparc build, ok otherwise
+
+mercury 2007-10-14 lucas PROP_RM(446665)
+
+inform 2007-10-14 lucas PROP_RM(446664)
+
+supercollider 2007-10-14 lucas PROP_RM(446667)
+
+jmagick 2007-10-14 lucas OK(100) has been orphaned but ITAed, so things might improve
+
+vegastrike 2007-10-14 lucas OK(20) an upload should take place soon according to #426872
+
+arb 2007-10-14 lucas PROP_RM(446666)
+
+lessdisks 2007-10-17 lucas REQ_RM(389498)
+
+defrag 2007-10-17 lucas REQ_RM(446691)
+
+twin 2007-10-15 lucas PROP_O(446692)
+
+libgstreamer-perl 2007-10-15 lucas OK(30) Pinged HE in #422822
+
+uclibc 2007-10-15 lucas PROP_O(446693)
+uclibc 2007-11-13 lucas REQ_RM(438738)
+
+crystalspace 2007-11-05 lucas OK(30) Pinged bayle in #358545
+
+basilisk2 2007-11-05 lucas OK(30) Pinged Jonas in #337887
+
+gclcvs 2007-11-05 lucas PROP_RM(449352)
+
+bibletime 2007-11-05 lucas PROP_O(449353)
+
+llvm 2007-11-05 lucas PROP_RM(449355) discussion in #451106 (ITP llvm2)
+
+elfsh 2007-11-05 lucas PROP_O(449356)
+
+libapache2-mod-xmlrpc2 2007-10-13 lucas O(423293)
+
+vegastrike 2007-11-05 lucas OK(30) Pinged debian-games team on IRC
+
+harbour 2007-10-13 lucas O(423287)
+
+pixie 2007-11-05 lucas OK(20)
+
+oskit 2007-10-13 lucas O(423308)
+
+metalog 2007-10-13 lucas O(423299)
+
+dak 2007-11-05 lucas PROP_RM(449429)
+
+pgapack 2007-11-05 lucas PROP_RM(449431) pinged on #379388
+
+misdn-kernel 2007-11-05 lucas OK(20) new version in experimental...
+
+gnue-appserver 2007-11-05 lucas OK(20) maintained by ajmitch. let's wait for MIA.
+
+mobilemesh 2007-11-21 lucas PROP_RM(452102)
+
+cfengine2 2007-11-21 lucas PROP_O(452352)
Added: bapase/pc_bugs_summary.rb
===================================================================
--- bapase/pc_bugs_summary.rb (rev 0)
+++ bapase/pc_bugs_summary.rb 2007-11-26 07:46:09 UTC (rev 516)
@@ -0,0 +1,22 @@
+#!/usr/bin/ruby -w
+
+n = 0
+s = 0
+bugs_normal = 0
+bugs_all = 0
+pc = 0
+IO::read('popcon_bugs_sorted.txt').each_line do |l|
+ f = l.split(' ')
+ bugs_normal += f[7].to_i
+ bugs_all += f[9].to_i
+ pc = f[1].to_i
+ n += 1
+ if n == 500
+ puts "#{s} #{pc} #{bugs_normal.to_f/n} #{bugs_all.to_f/n}"
+ s = pc
+ n = 0
+ bugs_normal = 0
+ bugs_all = 0
+ end
+end
+puts "#{s} #{pc} #{bugs_normal.to_f/n} #{bugs_all.to_f/n}"
Added: bapase/popcon_src
===================================================================
--- bapase/popcon_src (rev 0)
+++ bapase/popcon_src 2007-11-26 07:46:09 UTC (rev 516)
@@ -0,0 +1,54 @@
+#!/usr/bin/ruby -w
+
+# hash for bin->src resolution
+$srcpkg = {}
+
+# hash for src->bins resolution
+$pkgs = {}
+
+# read *-Sources and feed $srcpkg and $pkgs
+def read_sources
+ Dir::glob('{testing,unstable}-*-Sources') do |s|
+ src = nil
+ IO::read(s).each_line do |l|
+ if l =~ /^Package:/
+ src = l.split(' ')[1].chomp
+ $pkgs[src] = [] if $pkgs[src].nil?
+ elsif l =~ /^Binary:/
+ l.split(' ',2)[1].split(', ').each do |b|
+ b.chomp!
+ if $srcpkg[b].nil? # else, we already know that binary pkg
+ $srcpkg[b] = src
+ $pkgs[src] << b
+ end
+ end
+ end
+ end
+ end
+end
+
+read_sources
+
+popcon = {}
+`wget -q -O /dev/stdout http://popcon.debian.org/by_inst`.each_line do |l|
+ l.chomp!
+ next if l =~ /^#/
+ next if l =~ /^-----/
+ t = l.split
+ next if t[1] == "Total"
+ src = $srcpkg[t[1]]
+ next if src.nil?
+# puts "# #{src} #{t[2..-1].join(' ')}"
+ insts = t[2].to_i
+ if popcon[src]
+ if popcon[src] < insts
+ popcon[src] = insts
+ end
+ else
+ popcon[src] = insts
+ end
+end
+
+popcon.to_a.sort { |a,b| a[1] <=> b[1] }.reverse.each do |e|
+ puts e.join(' ')
+end
Property changes on: bapase/popcon_src
___________________________________________________________________
Name: svn:executable
+ *
Added: bapase/wnppsummary.rb
===================================================================
--- bapase/wnppsummary.rb (rev 0)
+++ bapase/wnppsummary.rb 2007-11-26 07:46:09 UTC (rev 516)
@@ -0,0 +1,40 @@
+#!/usr/bin/ruby -w
+
+bugid = nil
+bugpkg = nil
+bugseverity = nil
+bugtitle = nil
+bugtags = []
+bugaffect = []
+
+IO::read('bts2ldap-fullindex').each_line do |l|
+ l.chomp!
+ key, val = l.split(' ', 2)
+ if key == 'dn:' and bugid != nil
+ next if bugpkg != 'wnpp'
+ # Analyze bug.
+ if bugtitle =~ /^(ITA|ITP|O|RFA|RFP|RFH): ([^\s]*) -- (.*)$/
+ next if not ['ITA', 'O', 'RFA', 'RFH'].include?($1)
+ puts "#{$2} #{$1} #{bugid}"
+ else
+ STDERR.puts "NOMATCH: ##{bugid}: #{bugtitle}"
+ end
+
+ bugtags = []
+ bugaffect = []
+
+ elsif key == 'debbugsID:'
+ bugid = val
+ elsif key == 'debbugsPackage:'
+ bugpkg = val
+ elsif key == 'debbugsSeverity:'
+ bugseverity = val
+ elsif key == 'debbugsTag:'
+ bugtags << val
+ elsif key == 'debbugsTitle:'
+ bugtitle = val
+ elsif key == 'debbugsAffected:'
+ bugaffect << val
+ end
+end
+
Property changes on: bapase/wnppsummary.rb
___________________________________________________________________
Name: svn:executable
+ *
More information about the Collab-qa-commits
mailing list