[Piuparts-commits] rev 379 - piatti piatti/org/piuparts.debian.org/htdocs piatti/org/piuparts.debian.org/htdocs/images trunk trunk/debian

Holger Levsen holger at alioth.debian.org
Tue Apr 28 22:37:48 UTC 2009


Author: holger
Date: 2009-04-28 22:37:47 +0000 (Tue, 28 Apr 2009)
New Revision: 379

Added:
   piatti/org/piuparts.debian.org/htdocs/images/w3c-valid-css.png
Removed:
   piatti/org/piuparts.debian.org/htdocs/images/vcss.png
Modified:
   piatti/README.txt
   piatti/org/piuparts.debian.org/htdocs/images/valid-html401.png
   piatti/org/piuparts.debian.org/htdocs/style.css
   trunk/
   trunk/README.txt
   trunk/TODO
   trunk/debian/changelog
   trunk/debian/control
   trunk/piuparts-report.py
Log:
 - draw graphs of package states over time.
 - suggest python-rpy which is needed for piuparts-reports.
 - layout and overview page improvements.



Modified: piatti/README.txt
===================================================================
--- piatti/README.txt	2009-04-28 12:49:29 UTC (rev 378)
+++ piatti/README.txt	2009-04-28 22:37:47 UTC (rev 379)
@@ -4,7 +4,6 @@
 == ToDo
 
 - review sudoers and come up with (a bit) more restrictive one
-- use local mirror
 - there should be a 2nd group of piuparts-people. those who can sudo into piupartsm to process logfiles. maybe make that the qa-group
 
 == Installation
@@ -29,7 +28,7 @@
 
 === piuparts installation from svn source
 
-* sudo apt-get install apt python debootstrap lsof lsb-release python-debian make dpkg-dev python-support asciidoc xmlto
+* sudo apt-get install apt python debootstrap lsof lsb-release python-debian make dpkg-dev python-support asciidoc xmlto python-rpy r-recommended r-base-dev
 * you need a webserver too, if you run the master
 * Copy 'svn://svn.debian.org/svn/piuparts/piatti/home/piupartss/bin/update-piuparts-setup' on the host and run it. It assumes you want to set it up in '/org/piuparts.debian.org' and does all further svn checkouts as well as source code installation. It needs the piupartss and piupartsm user set up as described below, though.
 * sudo ln -s /org/piuparts.debian.org/etc/ /etc/piuparts

Modified: piatti/org/piuparts.debian.org/htdocs/images/valid-html401.png
===================================================================
(Binary files differ)

Added: piatti/org/piuparts.debian.org/htdocs/images/w3c-valid-css.png
===================================================================
(Binary files differ)


Property changes on: piatti/org/piuparts.debian.org/htdocs/images/w3c-valid-css.png
___________________________________________________________________
Added: svn:mime-type
   + application/octet-stream

Modified: piatti/org/piuparts.debian.org/htdocs/style.css
===================================================================
--- piatti/org/piuparts.debian.org/htdocs/style.css	2009-04-28 12:49:29 UTC (rev 378)
+++ piatti/org/piuparts.debian.org/htdocs/style.css	2009-04-28 22:37:47 UTC (rev 379)
@@ -35,7 +35,6 @@
 p.text {
  font-family: sans-serif;
  padding: 5px;
- font-size: 105%;
  font-weight: normal;
 }
 
@@ -163,7 +162,7 @@
 }
 
 td {
- font-size: 105%;
+ font-size: 100%;
  border: 0px solid #000;
  padding: 4px;
  padding-left: 6px;
@@ -264,6 +263,12 @@
     padding: 0.2em 0.3em 0.2em 0.3em;
     border-bottom: 1px dotted #999;
 }
+td.lastcell {
+    font-size: 80%;
+    text-align: center;
+    padding: 0.2em 0.3em 0.2em 0.3em;
+    border-bottom: 1px dotted #999;
+}
 td.contentcell2 {
     text-align: left;
     padding: 0.2em 0.3em 0.2em 0.3em;


Property changes on: trunk
___________________________________________________________________
Modified: svn:ignore
   - README.html
piuparts.1.html
piuparts.1
piuparts
docbook-xsl.css

   + README.html
piuparts.1.html
piuparts.1
piuparts
docbook-xsl.css
README.xml
piuparts.1.xml


Modified: trunk/README.txt
===================================================================
--- trunk/README.txt	2009-04-28 12:49:29 UTC (rev 378)
+++ trunk/README.txt	2009-04-28 22:37:47 UTC (rev 379)
@@ -394,6 +394,6 @@
 Some of the configuration items are not required, but it is best to set them all to be sure what the configuration actually is.
 
 
-=== About the setup on piuparts.debian.org
+=== Running piuparts in master-slave mode, piuparts-report and the setup on piuparts.debian.org
 
-See link:http://svn.debian.org/viewsvn/piuparts/piatti/README.txt?view=markup[svn://svn.debian.org/svn/piuparts/piatti/README.txt]
+If you want to run piuparts-report (which is only+very useful if you run piuparts in master-slave mode), you need to 'apt-get install python-rpy r-recommended r-base-dev'. For more information see link:http://svn.debian.org/viewsvn/piuparts/piatti/README.txt?view=markup[svn://svn.debian.org/svn/piuparts/piatti/README.txt].

Modified: trunk/TODO
===================================================================
--- trunk/TODO	2009-04-28 12:49:29 UTC (rev 378)
+++ trunk/TODO	2009-04-28 22:37:47 UTC (rev 379)
@@ -3,41 +3,47 @@
 
 for 0.36:
 
-- do fancy R graphs with the data from $distro/counts.txt
-
-- take care of old conf files on upgrades
-	/etc/piuparts/piuparts-master.conf
-	/etc/piuparts/piuparts-slave.conf
-
 - report:
  - refactor code, sources.txt should not be calculated where it currently is
  - get rid of old log pages and put stats in the index page
+ - do fancy R graphs with the data from $distro/counts.txt
+  - refactor..! search for FIXME
+  - also per state
+ - include links to http://qa.debian.org/developer.php?login=$maintainer_email 
+   or probably better: create maintainer centric pages.
+ - include uploaders in maintainer centric pages :-)
+ - include pts state (as an icon) in source summary pages
 
-- provides are not handled correctly: ie libapt-pkg-perl has a dependency on 
-  libapt-pkg-libc6.7-6-4.6 which piuparts believes is an unknown-package while 
-  in reality its provided by apt
-- break circular dependencies when there are only circular dependencies left for 
-  testing. prefer circular dependencies from the same source package until there are
-  none of this kind left.
-  warn visibly in logfile (but without error) when testing a packages with
-  circular dependencies.
-- http://piuparts.debian.org/sid/state-failed-testing.html lists linux-image-2.6.29-1-amd64 
-  four times :-(
 - http://piuparts.debian.org/sid/state-dependency-does-not-exist.html#addresses-goodies-for-gnustep 
   links to http://goldwasser.athome/source/a/addresses-for-gnustep (0.4.7-1).html 
   obviously doesnt exist.
-- bug or feature: udebs turn up as a state ;-) see http://piuparts.debian.org/source/u/udev.html
+
+- udebs turn up as a state - see http://piuparts.debian.org/source/u/udev.html
+
 - untestable packages are not recognicable as such, see http://piuparts.debian.org/source/a/apache2.html 
   atm. (log is N/A, even though its there, even if empty) - http://piuparts.debian.org/sid/state-cannot-be-tested.html
   aint helpful neither.
+
 - confirm, that killing the slave results in retesting the currently tested package. 
 
+- take care of old conf files on upgrades
+	/etc/piuparts/piuparts-master.conf
+	/etc/piuparts/piuparts-slave.conf
+
+once 0.36 is released:
+
+- integration with packages.qa.debian.org
+- announcement on d-d-a
+
 for 0.37:
 
 - probably include patch for #523950
 - probably include patch for #519192 - not sure if i like that aproach
 
-- report: 
+- master:
+ - write stats about number of daily tests (-> report: create fancy graphs)
+
+- report:
  - write stats about the reasons for failures:
   - installation hangs and uses all cpu when installing with 
     DEBIAN_FRONTEND="noninteractive"
@@ -53,8 +59,6 @@
   - unclassified
  - write stats for total number of logs (when a package has more than a log 
    in fail/ or bugged/)
- - include links to http://qa.debian.org/developer.php?login=$maintainer_email 
-   or probably better: create maintainer centric pages.
  - RSS feeds of logs
 
 - expire old logs a month after testing if newer logs exists (except if the new log is fail
@@ -83,8 +87,10 @@
 - write reportbug wrapper for mass bug filing (start simple, make it more sophisticated 
   in 0.38)
 
-- publish FAI classes to setup piuparts.$fqdn automatically
+- publish FAI classes to setup piuparts.$fqdn automatically 
 
+- release 0.37 before debconf9
+
 for 0.38 and on:
 
 - multi-arch:

Modified: trunk/debian/changelog
===================================================================
--- trunk/debian/changelog	2009-04-28 12:49:29 UTC (rev 378)
+++ trunk/debian/changelog	2009-04-28 22:37:47 UTC (rev 379)
@@ -49,6 +49,7 @@
     - provide links to logfiles in statistics page.
     - provide links to source packages pages from state pages and back, as well
       as links to the dependencies state.
+    - draw graphs of package states over time.
     - improve layout, generate navigation for all pages.
   * Break backwards compatibility of the configuration files for master-slave-
     mode. Merge those three into one: /etc/piuparts/piuparts/piuparts.conf.
@@ -64,6 +65,7 @@
     - depend on python (>>2.4), make dependency to python-debian unversioned
     - add build-dependencies on debhelper, asciidoc and xmlto.
     - remove build-dependencies on docbook2x and docbook-xml.
+    - suggest python-rpy which is needed for piuparts-reports.
     - set Homepage: to http://piuparts.debian.org
     - bump Standards-Version to 3.8.1, no changes neccessary.
   * Rewrite debian/rules from scratch using debhelper.

Modified: trunk/debian/control
===================================================================
--- trunk/debian/control	2009-04-28 12:49:29 UTC (rev 378)
+++ trunk/debian/control	2009-04-28 22:37:47 UTC (rev 379)
@@ -20,6 +20,7 @@
 Architecture: all
 Depends: apt, python (>> 2.4), debootstrap (>= 0.3.3), lsof, lsb-release, 
     python-debian
+Suggests: python-rpy
 Description: .deb package installation, upgrading, and removal testing tool
  piuparts tests that .deb packages (as used by Debian) handle
  installation, upgrading, and removal correctly. It does this by

Modified: trunk/piuparts-report.py
===================================================================
--- trunk/piuparts-report.py	2009-04-28 12:49:29 UTC (rev 378)
+++ trunk/piuparts-report.py	2009-04-28 22:37:47 UTC (rev 379)
@@ -31,6 +31,7 @@
 import urllib
 import shutil
 import string
+from rpy import *
 
 import piupartslib
 
@@ -82,62 +83,66 @@
       <a href="http://wiki.debian.org/piuparts/FAQ" target="_blank">FAQ</a> 
      </td>
      </tr>     
-    <tr class="titlerow">
-     <td class="bluetitlecell">
-      Available reports
-     </td>
-    </tr>
-    $section_navigation
-    <tr>
+    <tr class="normalrow">
      <td class="contentcell">
-      <a href="http://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=piuparts;users=debian-qa@lists.debian.org&archive=both" target="_blank">Bugs filed</a> 
+      <a href="http://bugs.debian.org/piuparts" target="_blank">Bugs</a> 
      </td>
     </tr>     
     <tr class="titlerow">
      <td class="titlecell">
-      Other Debian QA efforts
+      Documentation
      </td>
     </tr>
     <tr class="normalrow">
      <td class="contentcell">
-      <a href="http://edos.debian.net" target="_blank">EDOS tools</a>
+      <a href="/doc/README.html" target="_blank">piuparts README</a>
      </td>
     </tr>
+    <tr class="titlerow">
     <tr class="normalrow">
      <td class="contentcell">
-      <a href="http://lintian.debian.org" target="_blank">Lintian</a>
+      <a href="/doc/piuparts.1.html" target="_blank">piuparts manpage</a>
      </td>
     </tr>
     <tr class="normalrow">
      <td class="contentcell">
-      <a href="http://packages.qa.debian.org" target="_blank">Package Tracking System</a>
+      <a href="http://www.debian.org/doc/debian-policy/" target="_blank">Debian policy</a>
      </td>
-    <tr class="normalrow">
+    </tr>
+    <tr class="titlerow">
+     <td class="bluetitlecell">
+      Available reports
+     </td>
+    </tr>
+    $section_navigation
+    <tr>
      <td class="contentcell">
-      <a href="http://udd.debian.org" target="_blank">Ultimate Debian Database</a>
+      <a href="http://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=piuparts;users=debian-qa@lists.debian.org&archive=both" target="_blank">Bugs filed</a> 
      </td>
-    </tr>
-    </tr>
+    </tr>     
     <tr class="titlerow">
      <td class="titlecell">
-      Documentation
+      Other Debian QA efforts
      </td>
     </tr>
     <tr class="normalrow">
      <td class="contentcell">
-      <a href="http://www.debian.org/doc/debian-policy/" target="_blank">Debian policy</a>
+      <a href="http://edos.debian.net" target="_blank">EDOS tools</a>
      </td>
     </tr>
     <tr class="normalrow">
      <td class="contentcell">
-      <a href="/doc/README.html" target="_blank">piuparts README</a>
+      <a href="http://lintian.debian.org" target="_blank">Lintian</a>
      </td>
     </tr>
-    <tr class="titlerow">
     <tr class="normalrow">
      <td class="contentcell">
-      <a href="/doc/piuparts.1.html" target="_blank">piuparts manpage</a>
+      <a href="http://packages.qa.debian.org" target="_blank">Package Tracking System</a>
      </td>
+    <tr class="normalrow">
+     <td class="contentcell">
+      <a href="http://udd.debian.org" target="_blank">Ultimate Debian Database</a>
+     </td>
     </tr>
     <tr class="titlerow">
      <td class="titlecell">
@@ -145,7 +150,7 @@
      </td>
     </tr>
     <tr class="normalrow">
-     <td class="contentcell">
+     <td class="lastcell">
       $time
      </td>
     </tr>
@@ -165,15 +170,12 @@
   <div>
    piuparts was written by <a href="mailto:liw at iki.fi">Lars Wirzenius</a> and is now maintained by 
    <a href="mailto:holger at debian.org">Holger Levsen</a>,  
-   <a href="mailto:luk at debian.org">Luk Claes</a> and others. GPL2 licenced.
-   <br>
-  </div>
-  <div>
+   <a href="mailto:luk at debian.org">Luk Claes</a> and <a href="http://qa.debian.org/" target="_blank">others</a>. GPL2 licenced.
    <a href="http://validator.w3.org/check?uri=referer">
-    <img border="0" src="/images/valid-html401.png" alt="Valid HTML 4.01!" height="31" width="88">
+    <img border="0" src="/images/valid-html401.png" alt="Valid HTML 4.01!" height="15" width="80" valign="middle">
    </a>
    <a href="http://jigsaw.w3.org/css-validator/check/referer">
-    <img border="0" src="/images/vcss.png" alt="Valid CSS!"  height="31" width="88">
+    <img border="0" src="/images/w3c-valid-css.png" alt="Valid CSS!"  height="15" width="80" valign="middle">
    </a>
   </div>
  </div>
@@ -299,15 +301,13 @@
     </tr>
     <tr class="normalrow">
      <td class="contentcell2">
-      To make sure piuparts is run on all packages, piuparts.debian.org was set up.
-      <br>
-      piuparts.debian.org is a service running on <a href="http://db.debian.org/machines.cgi?host=piatti" target="_blank">piatti.debian.org</a>,
-      generously donated by <a href="http://hp.com/go/debian/" target="_blank">HP</a> and hosted at piuparts.cs.helsinki.fi by 
-      the University of Helsinki, at the <a href="http://cs.helsinki.fi/index.en.html" target="_blank">Department of Computer Science</a> in Finland.
-     </td>
-    </tr>
-    <tr class="normalrow">
-     <td class="contentcell2">
+      To make sure piuparts is run on all packages, piuparts.debian.org was set up as a service running on 
+      <a href="http://db.debian.org/machines.cgi?host=piatti" target="_blank">piatti.debian.org</a>. 
+      This machine was generously donated by <a href="http://hp.com/go/debian/" target="_blank">HP</a> 
+      to run piuparts on the Debian archive and is hosted as piuparts.cs.helsinki.fi by 
+      the University of Helsinki, at the 
+      <a href="http://cs.helsinki.fi/index.en.html" target="_blank">Department of Computer Science</a>
+      in Finland.
       This setup is currently still being polished. Better reports and statistics as well as PTS integration is
       planned. Join #debian-qa if you want to help.
      </td>
@@ -741,6 +741,20 @@
                           (html_protect(state), html_protect(state),
                           len(self._binary_db.get_packages_in_state(state)),
                           dir_link)
+
+        # create and include graph
+        # FIXME: refactor!  
+        countsfile = os.path.join(self._output_directory, "counts.txt")
+        pngfile = os.path.join(self._output_directory, "monthly-states.png")
+        r('t <- (read.table("'+countsfile+'",sep=",",header=1,row.names=1))')
+        r('cname <- c("date",rep(colnames(t)))')
+        r('v <- t[(nrow(t)-28):nrow(t),0:12]')
+        r('palette(c("green4", "red", "green", "brown", "black", "skyblue4", "yellow", "darkred", "salmon", "purple", "lightgreen",  "blue","darkgray"))')
+        r('bitmap(file="'+pngfile+'",type="png16m",width=12,height=9,pointsize=10,res=100)')
+        r('barplot(t(v),col = 1:13, main="Packages per state in '+self._config.section+' (past 4 weeks)", xlab="", ylab="Number of packages",space=0.1,border=0)')
+        r('legend(x="bottom",legend=colnames(t), ncol=2,fill=1:13,xjust=0.5,yjust=0,bty="n")')
+        tablerows += "<tr class=\"normalrow\"> <td class=\"contentcell\" colspan=\"3\"><a href=\"%s\"><img src=\"/%s/%s\" height=\"450\" width=\"600\" alt=\"Package states in the 4 weeks\"></a></td></tr>\n" % ("monthly-states.png", self._config.section, "monthly-states.png")
+
         tablerows += "<tr class=\"normalrow\"> <td class=\"labelcell\">Total</td> <td class=\"labelcell\" colspan=\"2\">%d</td></tr>\n" % \
                           self._binary_db.get_total_packages()
         htmlpage = string.Template(HTML_HEADER + SECTION_STATS_BODY_TEMPLATE + HTML_FOOTER)




More information about the Piuparts-commits mailing list