Server migration the coming Tuesday

The servers powering have been running for more than 3 years, and it’s time to move to new servers. The migration will start Tuesday November 27th at 10.00CET, and we expect to be done before noon. All services will be unavailable while we’re doing the migration.

We’re making some changes in how Gitorious runs on the new servers, most notably:

Apart from the potential speed improvements and stability provided by these changes in our underlying infrastructure will continue working like it does today after the upgrade. We will post reminders about the migration on our status site before we start, and post updates as the migration proceeds.


  • 10:40 CET: We have exported the database from the old database server, which has been shut down. We’re currently importing the database to the new database server.
  • 10:50 CET: The take has come to say goodbye to the old frontend server
  • 11:14 CET: We’ve imported the database and are currently running health checks on the new servers
  • 12:35 CET: A little behind our schedule, we’re just about ready to open up
  • 12:39 CET: And we’re back up

Rails 3 progress

Quick update: Gitorious is now running Rails 3.2.8 and all tests are passing. There are still a few minor “TODO”‘s left to tackle, but we’ll be upgrading our internal dogfooding server on Monday. When it’s running smoothly, we’ll get back to the UI upgrade.

We’re a bit behind schedule as I forgot to account for being away at Øredev for two days (the results of which can be seen on

Have a nice weekend!

Rails 3 upgrade

After a week of working on the Rails 3 upgrade, this is the current status: 3358 tests, 4712 assertions, 5 failures, 0 errors. That’s pretty close, and may I say, pretty exciting to us. Once the last 5 test failures have been fixed, there are still some aspects of the application that needs to be either verified manually, or added automatic tests for, but for now, things are looking good.

The upgrade has taken a little longer than initially expected, but good things are coming out of it, and the end is in sight. I’ll post a new update next week, at which point we should be back to working on the UI upgrades and the new repo browser.

Have a nice weekend!

Gitorious 2.3.2 released (small hotfix for installation/db setup issue)

We just tagged version 2.3.2 of Gitorious.

This is only a minor hotfix. The update resolves an intermittent issue in some Gitorious environments/database setups due to a database migration script from earlier this year. There is some discussion of the issue at the Gitorious Google Group.

From the version tag description:

Resolve issue with an older database migration.

An old migration (committed february 2012) caused intermittent issues
in some Gitorious installations depending on how the
environment/database is set up. This fix makes the db migration in
question play nice.

We’ve added a description of how to upgrade to this minor version in (see Upgrades -> “Upgrading from 2.3.1 to 2.3.2”).

Short story: this is only an important change for you if you’re seeing the below message during setup or upgrade of your Gitorious rig:

==  MakeProjectMembershipPolymorphicOnSubject: migrating ======================

— add_column(:project_memberships, :content_type, :string)

   -> 0.0062s

rake aborted!

An error has occurred, all later migrations canceled

Mysql::Error: Table ‘gitorious_production.content_memberships’ doesn’t exist: SELECT * FROM `content_memberships`

Quick reboot of the servers Tuesday at 11AM CET [completed]

We’ll need to do a quick reboot of our servers to update their kernels tomorrow at 11AM CET. We expect it to take no more than 5 minutes, and will keep this post updated as we proceed.


  • 11:05: We’re taking down the first server
  • 11:06: The servers are down
  • 11:12: The servers are back up again

UI upgrade and Rails 3

A while ago we announced some UI upgrades we are working on, and people have been asking when these changes will land in master. The short answer is that it will be a little while still. Read on for the longer answer.

If you’ve been following along, I’m sure you have seen our new repository browser. Dolt (a stand-alone tool to browse source code, view Readme files and more) is more or less done.

During the past couple of weeks I have made a couple of attempts at integrating Dolt back into Gitorious. Unfortunately these attempts have not yet been as successful as I’d hoped: there have been yaks to shave along the way.

Gitorious is currently based on a fairly old version of Rails. Working with old software imposes some constraints on what version of various third-party libraries can be used, which again causes trouble when trying to also use newer software (Dolt uses EventMachine and related libraries). In an attempt to resolve some issues related to this, I finally got my hands dirty and upgraded our Rails version from 2.3.5 to 2.3.14 last week.

This week, we decided to go one further and complete the long pending upgrade to Rails 3. I’m happy to report that this process is coming along nicely enough that we’re confident we’ll complete the upgrade next week. In doing this, we gain access to newer features in Rails that will help us in several ways. Most importantly, we will be able to integrate the source code browser properly.

We know that many of you are waiting for the new UI, and apologize for the delay. I wanted to post this update to let you know that we’re working on it, even if we have taken a detour to get there. When it all lands, we can all look forward to a more usable, prettier, safer and faster Gitorious.

Have a nice weekend!

LDAP authorization lands in Gitorious mainline

Gitorious started with a developer scratching his own itch. The first commit was done more than five years ago, and every commit since then has been about developers implementing features we need in a software tool we use all the time.

Gitorious is different from most other Git frontends and collaboration tools out there by being free software. Anyone using Gitorious is free to make changes to Gitorious itself; scratch their own itch. Over the last five years, 65 committers have committed to Gitorious and had their commits merged into the Gitorious mainline repository.

Gitorious is central in the software development process of hundreds of organizations, ranging from the very small to the very large. And as these organizations discover features they need in Gitorious, they are free to implement such features.

A year ago we announced three great new features in Gitorious, sponsored by a company using Gitorious internally. A few months later, we asked for help from the community in implementing support for private repositories in Gitorious, a feature which landed in Gitorious mainline last February.

LDAP Authorization

A few months ago, we were contacted by a company using Gitorious internally. They use LDAP for authentication, and wanted to discuss whether it was possible to use their LDAP server for authorization as well as authentication. They were willing to sponsor the development required to make this happen, and today we’re proud to announce that Gitorious now supports using an LDAP backend for authorization.

This is how it works

Granting access to a group of users in Gitorious is easy: you just define a Team of Gitorious users and grant access to your repositories and projects to that team. For a local Gitorious installation you would typically add Gitorious Teams for developers, QA/testing, project management and operations.

However, most larger organizations already has such groups defined in their LDAP/Active Directory directory, so duplicating this effort seems pointless. Furthermore, updating the Gitorious Teams as people join the company, change jobs or leave the company is a lot of extra work.

With LDAP authorization enabled, Gitorious no longer keeps track of which users are part of which teams. As users are added to and removed from LDAP groups, they will automatically be granted access (or have their access revoked) to any projects and repositories allowing access to those groups. As a project or repository owner you still grant access to teams, but the actual members of those teams are managed by your LDAP directory.

Since Gitorious needs to maintain the relationships between projects/repositories and (LDAP) Teams, you still define Teams in Gitorious – but those teams have LDAP groups as members, not users. With LDAP authorization enabled, each Gitorious Team has one or more LDAP groups as members, and any Gitorious user who is member of any of those LDAP groups will be granted access to anything allowed by that Gitorious Team. Technically, we switch the Team implementation in Gitorious between either database-backed or LDAP-backed teams based on whether LDAP authorization is enabled or not.

A scenario

Let’s say Bill is a new employee at BigCorp inc, and has never logged into their Gitorious server before. His LDAP username is bill, and he is member of the developers group in the LDAP directory. That group has commit access to the utilities repository in the tools project on the Gitorious server. Here’s what he needs to do to start committing to that repository:

  • Visit the login page on the server
  • Enter his LDAP username and password in the login form. The Gitorious server will try to authenticate him using the provided credentials. Once this succeeds, a new user record is created in the Gitorious database
  • Bill is prompted to upload an SSH key to the server
  • Bill can start pushing code to the repository

There is no registration step, no groups to update, it all Just Works™.

When is this available?

This feature was merged into the master branch of the Gitorious mainline repository a few days ago. If your organization uses LDAP we encourage you to try it out, and report any issues you find on the Gitorious issue tracker. We’ll release this as part of Gitorious v2.4.0 as soon as any issues have been fixed.

To get started, have a look at the sample authentication.yml file shipping with Gitorious. You’ll probably want to add the following options to config/authentication.yml:

  • bind_user: (username/password): a username/password to use for binding while looking up LDAP groups and memberships. Note: Specifying a bind user/password will cause Gitorious to use authenticated bind, another feature that has been lacking in Gitorious for some time.
  • membership_attribute_name: the name of the attribute your LDAP server uses to list groups a given user is member of.
  • members_attribute_name: the name of the attribute your LDAP server uses to list users who are member of a group.
  • base_dn: The base DN for users in your LDAP directory
  • group_search_dn: The base DN for groups in your LDAP directory

And in config/gitorious.yml, you’ll need to add:

  • use_ldap_for_authorization: true. The use_ldap_for_authorization option will replace the built-in Team provider with the LDAP Team provider.

Migrating to LDAP authorization

It is currently not recommended to simply turn on this feature on your existing Gitorious server, as the data in your database needs to be updated from the internal Team backend to the LDAP Team backend. However, setting up a new Gitorious server to try this out is easier than ever. Simply creating a database and connecting your Gitorious server to your LDAP server will allow you to start using Gitorious immediately, as user accounts are automatically created on first login.

We will start working on a tool to help you migrate to LDAP authorization, and hope to have this ready by the time we craft a new version of Gitorious supporting this new feature. We’ll be documenting how to use this feature and make it available at the Gitorious documentation site, launching really soon.

Maintenance window Thursday at 13:00 CET [DONE]

We’ll need to take down the servers for ~15 minutes the coming Thursday (October 25th) starting at 1PM CET to do another shot at migrating from our old SAN infrastructure. We’re sorry about the unforeseen troubles during our last two attempts at the same, but we should be better prepared this time.

We’ll update this blog post and our status site as we proceed.


  • 13:04: We’re taking down the servers
  • 13:15: We’re bringing up the first server temporarily, will be rebooted again afterwards
  • 13:28: We’re back up

Short maintenance Monday October 15th at 12 CET [complete]

We need to restart our servers the coming Monday starting 12:00 CET to reconfigure the storage pools on the physical hosts. We expect the process to take no more than 15 minutes, and will update this blog post as we proceed.


  • 12:04 CET: We’re taking down the servers
  • 12:09 CET: The servers are down, SAN is being reconfigured on the host OS
  • 12:13 CET: We’re bringing the servers back up again
  • 12:16 CET: We’ll need to take down one of the servers again to verify the partition tables.
  • 12:25 CET: We’re seeing some messages in the logs that we’re not comfortable with; investigating
  • 12:35 CET: We’re rolling back, should be back up in a few minutes
  • 12:39 CET: We’re back up.

Gitorious 2.3.1 released

We just tagged version 2.3.1 of Gitorious.

To upgrade your installation, follow these instructions.

The highlights from the changelog:

Gitorious CLI scripts, unicorn/nginx support, bug fixes

This release introduces command line scripts for interacting with a
production Gitorious environment. These scripts reside in
$GITORIOUS_ROOT/bin, and (contrary to what’s in script) all will:

* Assume a production environment (specify RAILS_ENV for overriding
* Switch to the user/group specified in gitorious.yml – no need use
* Set up the correct environment (no need for bundle exec)
* Change directory to your Gitorious root
* Will work with exec; you do not need to spawn a shell script to set up
environment variables etc. This means you can use these scripts directly
from cron, upstart, sysvinit, systemd

We also added support for running Gitorious under Nginx by sending
X-Accel-Redirect headers instead of the X-Sendfile headers used by
Apache with mod_xsendfile. Look at the frontend_server setting in
gitorious.yml and doc/templates/nginx.conf.sample for how to set up

Other bug fixes and minor improvements:
* use `git gc –auto` from script/repo_housekeeping
* start using a replacement pagination library
* provide templates for Upstart configuration files in
* patch Rails to not send empty Set-Cookie headers when running behind
Unicorn. This makes Gitorious play nicely with eg. Varnish
* make rake resque:work load the processors
* provide a sample Unicorn configuration file


Get every new post delivered to your Inbox.

Join 871 other followers