[Openstack-devel] OpenStack summit April 2012 post mortem
loic at enovance.com
Fri Apr 20 06:37:32 UTC 2012
Debian GNU/Linux was one of the few volunteer based projects represented during the OpenStack design summit held 16th - 18th April 2012
When you look at the stats of "Who wrote Essex", two companies show: stackops and eNovance. Three members of the OpenStack packaging team ( Ghe Rivero, Julien Danjou and myself ) are paid by these companies and I think it's fair to say that a majority of the contributions were made for the sake of Debian GNU/Linux packaging.
We are tiny compared to other distributions Fedora, Ubuntu or even Suse who were represented by over sixty people during the summit. But in term of efficiency and the result accomplished, we've done very well collectively.
The format of the design summit is somewhat unusual. The goal is to actually design "blueprints" that will influence the next release. Over three hundred developers discuss pre-defined topics in short sessions (30 minutes to 60 minutes) and try to reach a consensus on something that matters to them. Design decisions can't really be decided in such a short time but in many cases the direction in which the design process is heading has been heavily influenced by the heated discussions from the audience. I will focus on the sessions that, in my opinion, are related to packaging and deployment.
I originaly thought the discussions will occur while sitting at the Debian GNU/Linux table. But I did not figure out where the table was when I arrived and I quickly realized that Debian GNU/Linux should be represented during the sessions themselves. I've not heard Suse speak up, although they were present. But in many occasions the discussion involved packagers from Fedora or Ubuntu and I had many opportunities to speak for Debian GNU/Linux.
* Stable maintainance release
The overall impression is that every company present is much more concerned than I expected about the stable release life cycle. To the extent that HP, for instance, is deeply commited to maintaining a Diablo based release that works for them, despite the huge amount of work it involves. The discussion was about the process outlined here:
A branch is actively maintained (until the next release is out) or passively maintained (when the next release becomes actively maintained). There has been a discussion about what happens afterwards. Mark McLoughlin (Fedora) proposed that it is orphaned. Dave Walker (Ubuntu) and Loic Dachary (Debian GNU/Linux) advocated that they need to be maintained as long as there is someone to backport the security fixes. Thierry Carrez (Release Manager) proposed that after being passively maintained by the OpenStack project, the branch shows in a list with all other branches and a clear information about wether or not someone is caring for it. Of course it would be better if the OpenStack project itself could commit to maintain stable branches for a longer period. But nobody was ready to commit to this at this point in time. The solution proposed is a good interim since old stable branches stay in the same framework and do not move to a cimetery where they would become second class citizen.
I did not attend the sessions specific to documentation. However, I heard many times that the lack of an update documentation is a major problem. I met Anne Gentle and congratulated her for the hard work. She said that there is currently a shortage of people working on documentation. I've not heard anything about a solution to fix this problem. There has been a proposals in the context of a common library to handle the openstack flags / options to also use it to generate documentation such as manual pages. The idea of having a common library for this did not reach a consensus though.
* Integration tests
In the course of a search for ways to check regressions after an upgrade of OpenStack, I discovered Tempest https://github.com/openstack/tempest which is a set of integration tests. It proved useful to spot problems on an existing installation. During the "Test Strategy, Processes, and Quality Metrics" session, I discovered that it becomes an increasingly important part of the continuous integration process. Jaypipes has advocated on many occasions that it should be required to run before a commit is accepted in trunk. The trystack.org platform could be the reference infrastructure against which it is run.
The gerrit workflow http://wiki.openstack.org/GerritWorkflow is familiar to all OpenStack developers. But integration tests are equaly important for ops who are expecting to deploy successfully. Ghe made https://github.com/GheRivero/debostack/wiki and Ubuntu and Fedora also created similar custom environments. The ambition of Tempest is to be run by the OpenStack project itself, so that at least one deployment is proved to work.
That ambition raised the question of using packages to deploy for integration tests. I proposed that whenever a script has to be written in order to deploy the installation against which Tempest is run, it highlights the fact that the existing packages are missing an important use case.
The default Debian GNU/Linux use case may not be fit for integration tests. They will favor a situation where each component is deployed with a diversity of options activated in order to exercises all combinations of the OpenStack. As opposed to the default Debian GNU/Linux use case which is made for a casual system administrator willing to give OpenStack a try. However, the packages must not be hostile to the integration test / Tempest use cases because they are an essential part of the Q&A process.
* Stress test
The cambridge based research center for Quanta is the home of the stress test part of Tempest as found in https://github.com/openstack/tempest/tree/master/stress . It could be used but with caution according to the author: it cannot yet reliably figure out if something succedeed or not because the current logs are difficult to parse.
Dan Bode who is leading the effort on OpenStack puppet modules held a session on "Making the configuration of openstack easier". The core of the discussion was about the difficulties he encountered in creating cross distribution puppet modules. Namely, the constant state of flux of the configuration files, the lack of uniformity ( Fedora using one format + Unbutu / Debian GNU/Linux using another format for nova ) and the need for .d files to ease the generation of configuration stanzas as opposed to editing a single configuration file.
These are the topics I wanted to discuss with all distribution packagers and to my suprise they are adressed by someone depending on all distributions and challenging the OpenStack upstream for configuration practices. As long as Dan Bode is assigned to the making of OpenStack puppet modules, I am optimistic about the fact that the people in charge of coding will have our concerns in mind.
During the session about openstack-common, the same points ( .d directories for each file for instance) were already raised.
In a nutshell I think we're in good shape. The OpenStack upstream is aware of the requirements for better configuration and agrees to implement them. Integration tests are in the process of becoming standard practice and will ensure that at least one deployment is working according to the specifications. Essex is our target release for Wheezy and we can rely on the fact that it will be actively / passively maintained for a year. We can then package Folsom for experimental and follow the stable branch release cycle.
The pain points are the documentation, the fact that the integration tests are not yet complete and the fact that the maintainance cycle is too short compared to the Debian GNU/Linux release cycle. These should be addressed during the next summit.
To improve our presence during the next summit, there should be two of us to attend the sessions occuring at the same time. It would also help to discuss during lunch / parties. I had the pleasure to meet Faidon Liambotis who is a Debian developper, but he was representing his new employer and we did not have the opportunity to discuss before the summit. We could sit at the table and take turns, thus taking advantage of the generous offer of the OpenStack fundation. I discussed the OpenStack conference schedule with Stefano Maffulli and it is so dense that it is unrealistic to attend them all. I'd be happy if we can be represented propertly on one important occasion : twice a year for the design summit.
In conclusion that was a great summit and I'm happy that the work of Ghe Rivero and Julien Danjou for the packaging of the Essex release was acknowledged properly. I probably forgot a few things and I'd be happy to discuss over the phone or on IRC if you'd like.
Loïc Dachary Chief Research Officer
// eNovance labs http://labs.enovance.com
// ✉ loic at enovance.com ☎ +33 1 49 70 99 82
More information about the Openstack-devel