[parted-devel] automating the release process
Joel Granados
jgranado at redhat.com
Wed Jul 15 09:27:28 UTC 2009
On Wed, Jul 15, 2009 at 10:56:57AM +0200, Jim Meyering wrote:
> Joel Granados wrote:
> > You can see the updated version on
> > http://jgranado.fedorapeople.org/packages/parted/do_major_release
> > On Tue, Jul 14, 2009 at 08:36:09PM +0200, Joel Granados wrote:
> >> Hello list.
> >>
> >> Been working on a script that will automate part of the release process.
> >> The local part. After talking to Jim it seemed logical to generalize
> >> what I had been using to make my life easier.
> >>
> >> The script is attached. As always, any suggestions are greatly
> >> appreciated.
> >>
> >> FYI. The script runs as follows:
> >>
> >> ./script --version 1.9.0 --key-id YOUR_KEY_ID
> >>
> >> Regards.
> >>
> >> PS: I'm thinking of calling it parted-release or something similar.
>
> Hi Joel,
>
> parted-release is a good name.
> Better than do_major_release, because we prefer "-" to "_"
> and because you might want to make it work also for non-major releases.
Well, I have not envisioned a non major script yet. But its worth
considering in the future. Don't feel very strongly about the name.
I'll do the change for the next version.
>
> The rest of the feedback is regarding this,
> http://jgranado.fedorapeople.org/packages/parted/do_major_release
> (a copy included below, for reference)
>
> I have a strong preference for not using all-caps variable names.
> Mostly for readability, but also for name-space-related issues.
> All-caps are typically reserved for applications, while lower-case
> ones are not, so in using lower-case names we're less likely to
> impinge on any application name-space.
I don't really feel strongly about this. The change is strait forward
enough. It will be in the next version. (comming up in a couple of
minuts). I'll basically change all the vars that are only caps to the
same var name but in lower case.
>
> The first test here is unnecessary,
>
> if [ "x$KEY_STRING" = "x" -o "x$KEY_ID" = "x"] ; then
> KEY_STRING="-s"
> KEY_ID="FIXME: YOUR_KEY"
> fi
I did this because I want to keep the variables related, even in the
event of someone changing the script in the wrong way. I would like to
keep the two checks because of this.
>
> since those two variables are always set together.
> Just testing the key ID is sufficient.
>
> Besides, don't you just want to give a diagnostic
> and exit nonzero when $KEY_ID is not specified or empty?
No. When the key is specified I want to use it. When it is not I want
to use the -s option of gpg. So the user does not *have* to put his gpg
ID if he has the default configured correctly.
>
> ----------------------------------
> By using the full absolute name,
>
> STAGE_DIR=$(pwd)
>
> and with unquoted uses below, e.g., the derived PARTED_DIR,
>
> PARTED_DIR="$STAGE_DIR/parted"
> req_ac=$( (cat $PARTED_DIR/bootstrap.conf \
>
> you have made the script slightly vulnerable.
> If it is run from a poorly-named directory (a name containing
> spaces or shell meta-characters), it will fail, at least,
> and may even do something nasty.
>
> It's best to avoid absolute names, when possible.
True. I'll quote the paths and use relative paths.
>
> ----------------------------------
> You can probably remove the _do_mail_template function as well as
> the those manual version checks, (the do_check_*_ver functions).
> The latter are already performed by running ./bootstrap.
> And when you run e.g., "make major" an announcement email template is
> written to /tmp/announce-parted. (see the "announcement" target in maint.mk)
hrm, hadn't noticed the /tmp/announce-parted file. Thx for pointing
that out.
I'll remove the function and create a README-release. The readme
release
>
> ------------------------------------
> For your _do_success file, please just cat a separate file, README-release
> which would contain the same information. Then, it's slightly easier to
> merge changes between it and other files like the one in coreutils.
>
> Thanks for working on this!
> Jim
>
> Here's a copy of
> http://jgranado.fedorapeople.org/packages/parted/do_major_release
> --------------------------------------------
> #!/bin/bash
>
> v=""
> USER=$(id -u)
> DATE=$(date +%F)
> STAGE_DIR=$(pwd)
> LOGFILE=$STAGE_DIR/release.log
> MAIL_TEMPLATE=$STAGE_DIR/mail_template
> PARTED_DIR=""
> GNULIB_DIR=""
> KEY_STRING=""
> KEY_ID=""
> AUTOCONF_VER=""
> AUTOMAKE_VER=""
> GNULIB_VER=""
> GIT_VER=""
>
> usage()
> {
> echo "These are the options for $0"
> echo "$0 --version VERSION [--key-id KEYID]"
> }
>
> # Get all the input values
> while [ $# -gt 0 ] ; do
> case $1 in
>
> --key-id)
> KEY_STRING="-u $2"
> KEY_ID="$2"
> shift; shift
> ;;
>
> # The version that is to be released
> --version)
> v="$2"
> shift; shift
> ;;
>
> # The default
> *)
> usage
> exit 1
> ;;
>
> esac
> done
>
> if [ "x$v" = "x" ] ; then
> usage
> exit 1
> fi
> if [ "x$KEY_STRING" = "x" -o "x$KEY_ID" = "x"] ; then
> KEY_STRING="-s"
> KEY_ID="FIXME: YOUR_KEY"
> fi
>
>
> # Shamelessly taken from bootstrap.
> # Note this deviates from the version comparison in automake
> # in that it treats 1.5 < 1.5.0, and treats 1.4.4a < 1.4-p3a
> # but this should suffice as we won't be specifying old
> # version formats or redundant trailing .0 in bootstrap.conf.
> # If we did want full compatibility then we should probably
> # use m4_version_compare from autoconf.
> sort_ver() { #sort -V is not generally available
> ver1="$1"
> ver2="$2"
>
> #split on '.' and compare each component
> i=1
> while : ; do
> p1=$(echo "$ver1" | cut -d. -f$i)
> p2=$(echo "$ver2" | cut -d. -f$i)
> if [ ! "$p1" ]; then
> echo "$1 $2"
> break
> elif [ ! "$p2" ]; then
> echo "$2 $1"
> break
> elif [ ! "$p1" = "$p2" ]; then
> if [ "$p1" -gt "$p2" ] 2>/dev/null; then #numeric comparision
> echo "$2 $1"
> elif [ "$p2" -gt "$p1" ] 2>/dev/null; then #numeric comparision
> echo "$1 $2"
> else #numeric, then lexographic comparison
> lp=$(printf "$p1\n$p2\n" | LANG=C sort -n | tail -n1)
> if [ "$lp" = "$p2" ]; then
> echo "$1 $2"
> else
> echo "$2 $1"
> fi
> fi
> break
> fi
> i=$(($i+1))
> done
> }
>
> # Returns 0 if $1 greater than $2, 1 otherwise.
> gt_ver()
> {
> biggest=$(sort_ver $1 $2 \
> | awk '{print $2}')
>
> if [ "x$biggest" = "x$1" ] ; then
> return 0
> else
> return 1
> fi
> }
>
> # Returns 0 if $1 is less than $2, 1 otherwise.
> lt_ver()
> {
> smallest=$(sort_ver $1 $2 \
> | awk '{print $1}')
>
> if [ "x$smallest" = "x$1" ] ; then
> return 0
> else
> return 1
> fi
> }
>
> _do_git_clone()
> {
> git clone -q http://git.debian.org/git/parted/parted.git
> git clone -q git://git.sv.gnu.org/gnulib
> PARTED_DIR="$STAGE_DIR/parted"
> GNULIB_DIR="$STAGE_DIR/gnulib"
> }
>
> _do_check_autoconf_ver()
> {
> # Required autoconf (req_ac)
> req_ac=$( (cat $PARTED_DIR/bootstrap.conf \
> | grep autoconf \
> | awk '{print $2}') \
> || echo -n)
>
> # Available autoconf (ava_ac)
> ava_ac=$( (autoconf -V \
> | head -n 1 \
> | awk '{print $4}') \
> || echo -n)
>
> if [ "x$req_ac" = "x" -o "x$ava_ac" = "x" ] ; then
> return 1
> elif [ $(lt_ver "$req_ac" "$ava_ac") ] ; then
> return 1
> else
> AUTOCONF_VER="$ava_ac"
> return 0
> fi
> }
>
> _do_check_automake_ver()
> {
> # Required automake (req_am)
> req_am=$( (cat $PARTED_DIR/bootstrap.conf \
> | grep automake \
> | awk '{print $2}') \
> || echo -n)
>
> # Available automake (ava_am)
> ava_am=$( (automake --version \
> | head -n 1 \
> | awk '{print $4}') \
> || echo -n)
>
> if [ "x$req_am" = "x" -o "x$ava_am" = "x" ] ; then
> return 1
> elif [ $(lt_ver "$req_am" "$ava_am") ] ; then
> return 1
> else
> AUTOMAKE_VER="$ava_am"
> return 0
> fi
> }
>
> _do_check_git_ver()
> {
> # Required git (req_git)
> req_git=$( (cat $PARTED_DIR/bootstrap.conf \
> | grep git \
> | awk '{print $2}') \
> || echo -n)
>
> ava_git=$( (git --version \
> | head -n 1 \
> | awk '{print $3}') \
> || echo -n)
>
> if [ "x$req_git" = "x" -o "x$ava_git" = "x" ] ; then
> return 1
> elif [ $(lt_ver "$req_git" "$ava_git") ] ; then
> return 1
> else
> GIT_VER="$ava_git"
> return 0
> fi
> }
>
> _do_check_gnulib_ver()
> {
> cd $GNULIB_DIR
> # Available gnulib.
> ava_gl=$( (./gnulib-tool --version \
> | head -n 1 \
> | awk '{print $6}')
> || echo -n)
>
> if [ "x$ava_gl" = "x" ] ; then
> return 1
> else
> GNULIB_VER=$ava_gl
> return 0
> fi
> cd $STAGE_DIR
> }
>
> _require_git()
> {
> ( git --version ) > /dev/null 2>&1 ||
> {
> echo "Could not find git. Pls install from http://git-scm.com/download."
> exit 1
> }
> }
>
> _do_mail_template()
> {
> # Find out the sizes of the gz and xz files.
> gz_size=$(ls -sh $PARTED_DIR/parted-$v.tar.gz \
> | awk '{print $1}')
> xz_size=$(ls -sh $PARTED_DIR/parted-$v.tar.xz \
> | awk '{print $1}')
>
>
> cat >> $MAIL_TEMPLATE << EOF
> This is to announce parted-$v, a release we're calling "stable",
> because we think it's solid enough for general use.
>
> For a summary of changes and contributors, see:
> http://git.debian.org/?p=parted/parted.git;a=shortlog
>
> Thanks to all people who have been testing pre-release, reporting bugs,
> contributing code, suggesting enhancements, and answering user questions
> on the mailing lists!
>
> ----------
>
> Here are the compressed sources:
> ftp://ftp.gnu.org/gnu/parted/parted-$v.tar.gz ($gz_size)
> ftp://ftp.gnu.org/gnu/parted/parted-$v.tar.xz ($xz_size)
>
> Here are the GPG detached signatures[*]:
> ftp://ftp.gnu.org/gnu/parted/parted-$v.tar.gz.sig
> ftp://ftp.gnu.org/gnu/parted/parted-$v.tar.xz.sig
>
> [*] You can use either of the above signature files to verify that
> the corresponding file (without the .sig suffix) is intact. First,
> be sure to download both the .sig file and the corresponding tarball.
> Then, run a command like this:
>
> gpg --verify parted-$v.tar.xz.sig
>
> If that command fails because you don't have the required public key,
> then run this command to import it:
>
> gpg --keyserver keys.gnupg.net --recv-keys $KEY_ID
>
> and rerun the \`gpg --verify\' command.
>
> This release was bootstrapped with the following tools:
> Autoconf $AUTOCONF_VER
> Automake $AUTOMAKE_VER
> Git $GIT_VER
> Gnulib $GNULIB_VER
>
> *****************
> How can you help?
> *****************
> If you're interested in lending a hand, or just want to use
> the latest versions right now, you can build these programs
> and run the test suite like this:
>
> gzip -dc parted-$v.tar.gz | tar xf -
> cd parted-$v
> ./configure
> make
> make -k check | grep FAIL
>
> [If you downloaded the much-smaller .xz tarball, then just
> substitute this for the gzip... line above:
> xz -dc parted-$v.tar.xz | tar xf -
> If you don't have "xz" installed yet, first try getting it via
> your distribution, e.g., "aptitude install xz" or "yum install xz"]
>
> Be sure to use make's -k option so that make doesn't stop
> just because one of the earlier tests fails.
> Please report any build problems or test failures to the
> address at hidden mailing list.
>
> **************************
> NEWS (for parted-$v)
> **************************
>
> FIXME: COPY THE NEWS CONTENTS THAT IS RELATED TO parted-$v HERE.
> EOF
> }
> _do_sanity_check()
> {
> cd $PARTED_DIR
>
> # Might be a good idea to include code that checks the versions of autotools.
> ./bootstrap --gnulib-srcdir=../gnulib >> $LOGFILE 2>&1
> ./configure >> $LOGFILE 2>&1
>
> make >> $LOGFILE 2>&1
> sudo make check RUN_VERY_EXPENSIVE_TESTS=yes RUN_EXPENSIVE_TESTS=yes >> $LOGFILE 2>&1
> sudo chown $USER.$USER . -R >> $LOGFILE 2>&1
>
> make dist-check >> $LOGFILE 2>&1
> make maintainer-clean >> $LOGFILE 2>&1
>
> cd $STAGE_DIR
> }
>
> _do_release()
> {
> cd $PARTED_DIR
>
> # Change the NEWS flie
> sed -e \
> "s/^.*in release 1.9.0.*/* Noteworthy changes in release $v ($DATE) [stable]/" \
> -i $PARTED_DIR/NEWS
>
> # Make a signed commit
> git commit -F <(printf 'version '$v'\n\n* NEWS: Record release date.\n') -a >> $LOGFILE 2>&1
> git tag $KEY_STRING -m "parted $v" v$v HEAD
>
> # Run make major
> ./bootstrap --gnulib-srcdir=../gnulib >> $LOGFILE 2>&1
> ./configure >> $LOGFILE 2>&1
> make >> $LOGFILE 2>&1
> make major >> $LOGFILE 2>&1
>
> _do_mail_template
>
> cd $STAGE_DIR
> }
>
> _do_success()
> {
> echo "\
> The release process has finished successfully. You are encouraged to follow
> the following steps:
>
> 1. Test the tarball. Copy it to a few odd-ball systems and ensure that it
> builds and passes all tests.
>
> 2. Write the release announcement that will be posted to the mailing mailing
> lists. You can look at similar messages in
> http://lists.gnu.org/mailman/listinfo/info-gnu for guidance.
> You can also look at the mail_template file created by the release script.
>
> 3. Run the gnupload command that was suggested by the release script. You can
> find this at the end of release.log.
>
> 4. Wait a few minutes (maybe up to 30?) and then use the release URLs to
> download all tarball/signature pairs and use gpg --verify to ensure that
> they're all valid. You will also need these URLs for the announcement mail.
>
> 5. Push the new tag with the following command:
> git push origin tag v$v
>
> 6. Send the gpg-signed announcement mail, e.g.,
> To: info-gnu at gnu.org, parted-devel at lists.alioth.debian.org
> Cc: coordinator at translationproject.org, bug-parted at gnu.org
> Subject: parted-$v released [stable]
>
> 7. Announce it on Savannah, too:
> From here:
> https://savannah.gnu.org/projects/parted/
> click on the \"submit news\", then write something like the following:
>
> Subject: coreutils-7.2 released [stable]
> The announcement is here:
> URL_OF_THE_ANNOUNCEMENT
>
> Then go here to approve it:
> https://savannah.gnu.org/news/approve.php?group=parted
> "
> exit 0
> }
>
> _do_fail()
> {
> echo "\
> The release process has returned an error
> please check the $LOGFILE for more information.
> "
> exit 1
> }
>
> _require_git
> echo "git is installed..."
>
> echo "Cloning parted & gnulib (this might take a few minutes)..."
> _do_git_clone
> echo "parted & gnulib cloned..."
>
> _do_check_autoconf_ver \
> || $( echo "check_autoconf failed, update your autoconf version."
> exit 1 )
> echo "autoconf version OK..."
>
> _do_check_automake_ver \
> || $( echo "check_automake failed, update your automake version."
> exit 1 )
> echo "automake version OK..."
>
> _do_check_git_ver \
> || $( echo "check_git failed, update your git version"
> exit 1 )
> echo "git version OK..."
>
> _do_check_gnulib_ver \
> || $( echo "check_gnulib failed, run `git clone git://git.sv.gnu.org/gnulib` "
> echo "in your stage directory ($STAGE_DIR) "
> echo "and execute the script once more."
> exit 1 )
> echo "gnulib version OK..."
>
>
> _do_sanity_check
> _do_release
>
> if [ $? -eq 0 ] ; then
> _do_success
> else
> _do_fail
> fi
--
Joel Andres Granados
Brno, Czech Republic, Red Hat.
More information about the parted-devel
mailing list