Simple backup and recovery with the ‘snapshot’ command

Backup, disaster recovery and migrations of Gitorious has been a somewhat manual process up till now. Though all you really need to do is to dump the mysql database and save the repository folders, setting up and performing those backup tasks has been an ad-hoc task for each installation. Furthermore any disaster recovery work has been quite the forensic process.

To address this we’ve added convenience commands for simple snapshots and restore of Gitorious installations. The snapshot/restore commands are available on the Gitorious mainline now, and are present if you update to the newly released v2.3.

Using the snapshot/restore commands

If you’re on a recent version of Gitorious you’ll find two new commands under the scripts directory: scripts/snapshot and scripts/restore. Each of them takes a single parameter: the path of the tar file you want your Gitorious installation backed up to, or restored from. Note that you'll need to launch the commands from the root directory of your Gitorious install (the directory where you find your Rakefile, Gemfile, config directory etc).

How to perform snapshots in older Gitorious versions

It's fairly straightforward to add these new snapshot|restore commands to an older Gitorious instance a well: you only need to clone the latest version of the Gitorious code, and copy the following files to the same locations in your current, older installation:

script/restore

script/snapshot  

lib/tasks/backup.rake

Assumptions and caveats

  • For disaster recovery, you'll first need to get a functional installation of Gitorious up and running, after which you can run the restore command to bring in your data again. Note that the snapshot tarball also includes the configuration files from your old Gitorious installation. They are not automatically copied into your new installation, but can be used to recreate your old config in your new Gitorious installation (useful if your previous Gitorious installation had custom LDAP integration, custom hooks etc).
  • Snapshots will not preserve any custom code or theming you may have added to your Gitorious installation: any such local modifications will need to be backed up and restored separately
  • Some operations in Gitorious are asynchronous, meaning they are performed by publishing/consuming messages on a queue. The snapshot/restore commands don't currently capture or restore the state of the queue, so any current work on the queue will be lost.
  • The snapshot/recovery commands assume that you have the time and disk-space to slurp down all your hosted repos into a local tarball. Sites with huge amounts of git repository data may therefore need more custom backup schemes.
  • The restore command assumes that no breaking changes have happened between the version you snapshot from, and the version you restore your data into. In the future, major Gitorious version jumps may necessitate a more manual restore procedure due to changes in configurations, db schema, folder structure etc.

Usage scenarios

The snapshot|restore commands are useful for multiple tasks beyond just standard backups.

Backup and disaster recovery

Run periodic backups, for instance via cron. Execute the snapshot command, for example like this script/snapshot /tmp/todays-snapshot.tar. Transfer the newly created tarball to another server, offsite location or something like Amazon S3.

If disaster strikes on your Gitorious installation, install a new instance of Gitorious, then run script/restore todays-snapshot.tar to restore your old state in the new installation.

Migrating

The same procedure is useful if you just need to move or clone your Gitorious state from one server to another. Snapshot the old one, copy the tarball file over to the other installation and restore there. Simple.

Upgrading

Different scenario, same procedure: if you need to upgrade from an older version of Gitorious and don't want to bother with carefully upgrading your Gitorious instance across multiple versions, you can simply snapshot the state of your old Gitorious site, install a fresh new Gitorious instance and just restore your state there.

Snapshots and rollbacks for development and testing

If you're a developer extending, customizing or developing new features for Gitorious, you can use the snapshot|restore commands to simply setting up and restoring multiple standard database/repository states in order to simplify testing.

Let us know if you hit any snags

We've been testing and using these features internally at Gitorious AS for a while so they should be reliable by now, but do let us know if you find any issues.

2 Comments

  1. Posted December 20, 2012 at 3:55 am | Permalink

    “perform snapshots in older Gitorious versions”
    I did as your suggest, but get error message “: No such file or directory” when running snapshot command

  2. John
    Posted July 3, 2013 at 8:32 pm | Permalink

    This is a neat tool, but I have a password on my mysql root user and it doesn’t work unless I remove the password. Is there any way to add mysql root user credentials for the snapshot script to work?


Follow

Get every new post delivered to your Inbox.

Join 811 other followers

%d bloggers like this: