bdb 4.2->4.3 upgrade

[Branko Čibej]

Peter Samuelson wrote:
> In Debian, we have to move subversion from Berkeley DB 4.2 to DB 4.3
> for reasons beyond the scope of this mail.  Therefore we have to
> provide instructions for upgrading old repositories.  It is also not
> really practical to keep a DB 4.2 'svnadmin' binary around after the
> transition, so I can't just copy the steps in
> http://subversion.tigris.org/faq.html#bdb43-upgrade directly.
>   
Those steps are summarized from BDB's documentation. Have you read it?


> I'm having trouble finding anyone who knows enough about Berkeley DB to
> give me solid answers here, so I've got a couple of questions: (1) The
> README attached below - does it seem accurate and safe?  (2a) Is the
> part about deleting log files (taken from faq.html) necessary,
Yes.

>  or
> incidental?  I'd like to keep the whole thing simple.  (2b) Is it
> possible to simplify the steps somewhat by having the user delete *all*
> the log files unconditionally? 
No.

>  It seems to me that after a successful
> db_recover, this should be safe ... true or false?
>   
Definitely false. There's no guarantee that some log files won't still 
be in use after a db_recover (or svnadmin recover).

> Upgrading a repository to BDB 4.3
> ---------------------------------
>
> Debian subversion packages prior to version 1.2.3dfsg1-3 were compiled
> to use Berkeley DB 4.2; newer releases use BDB 4.3.  If you have
> created any subversion repositories in BDB format (not the default
> format, but it was the default prior to subversion 1.1.0), you must
> upgrade them to use BDB 4.3 before subversion will be able to use them.
>
> If you try to use a BDB 4.2 repository, you will get an error message
> including the lines:
>
>   DB_VERSION_MISMATCH: Database environment version mismatch
>   bdb: Program version 4.3 doesn't match environment version
>
> To upgrade the repository, follow the steps below.  We shall assume
> your repository is in /srv/svn/repo1.
>
> First make sure the 'db4.2-util' package is installed.  Then, ensure
> that nobody is using or will use the repository while you're updating
> it.  The "mv" commands below will help with this:
>
>   mv /srv/svn/repo1 /srv/svn/repo1.x
>   lsof +D /srv/svn/repo1.x
>   
This looks like a dangerous thing to do, because one of the still-active 
processes might want to create a new log file and would fail miserably 
in the middle of a transaction. Instead of moving things away, it would 
be better to disable any SVN-related Apache services and/or svnserve 
daemons, and lock down the svn and svnserve binaries so that local and 
svn+ssh users can't execute them. Then wait until lsof reports no uses. 
You can do this in a script.

> If the lsof line reports any processes using files in that directory,
> either wait for them to finish what they are doing, or kill them.  Feel
> free to back up the repository before continuing, although you can't
> use "svnadmin", since it won't read BDB 4.2 environments.  You can use
> "cp", though, e.g. "cp -a /srv/svn/repo1.x /srv/svn/repo1.bak".
>
> Next, make sure the repository databases are in a consistent state
> (normally they will be, but you are about to destroy the data needed to
> fix them if they aren't):
>
>   db4.2_recover -h /srv/svn/repo1.x/db
>
> Delete the database "environment":
>
>   rm /srv/svn/repo1.x/db/__db.???
>
> Now svnadmin can be used, and if there are any unused database logs,
> you may as well delete them:
>
>   svnadmin list-unused-dblogs /srv/svn/repo1.x
>   
You can avoid messing with the new svnadmin and do everything with the 
BDB utilities:

   1. db4.2_checkpoint -1 -h /srv/svn/repo1.x/db
      Force a checkpoint of the database environment and logs. This will
      minimize the number of in-use log files after recovery.
   2. db4.2_recover -h /srv/svn/repo1.x/db
      Recover the database
   3. db4.2_archive -d -h /srv/svn/repo1.x/db
      Delete unused database log files
   4. rm /srv/svn/repo1.x/db/__db.???

Once again, this is something you can write a script for.

> If any such files are found, you can delete them with "rm".  Finally,
> put the repository back where clients can see it again:
>
>   mv /srv/svn/repo1.x /srv/svn/repo1
>
> Alternatively, you may wish to take this opportunity to change the
> format of your repository from 'bdb' to 'fsfs'.  Of course, this
> requires enough disk space to store both formats at once:
>
>   svnadmin create /srv/svn/repo1.new
>   svnadmin dump -q /srv/svn/repo1.x | svnadmin load -q /srv/svn/repo1.new
>   mv /srv/svn/repo1.new /srv/svn/repo1
>
> (and after verifying that everything went well and the new repository
> is usable...)
>
>   rm -r /srv/svn/repo1.x
>
> Why would you want to switch to 'fsfs' format?  'fsfs' is now the
> default format because it has several advantages over 'bdb', though
> also a few disadvantages.  The Subversion Book discusses the
> differences:
>
>   http://svnbook.red-bean.com/en/1.1/ch05.html#svn-ch-5-sect-1.3
>   
-- Brane