Fast and Safe Module Updates with Drush and SVN

SecurityOne of the primary ways of keeping a Drupal site of any size running securely and at peak performance is to ensure that all of its modules stay updated. With thousands of modules in the Drupal eco-system, updates are released literally every day. Luckily, Drupal core's Update Status module helps site administrators keep notified of modules in need of updating.

In this article, you'll learn how to use the power of Drush and Apache Subversion (aka "svn"), a revision control tool, to update your site's module quickly and safely. You'll be presented with two methods for solving the problem. The second method is much faster, but comes with a caveat. Read on.
What does it mean to perform a module update "safely"? For the context of this article, "safely" means the ability to rollback a module update quickly if it breaks something on your site. This often requires a two-step approach: both reverting to the previous version of the module and reverting to a previous version of the site's database, as module updates often involve database updates. The use of a source code respository (svn, for the sake of this article) and timely database backups fulfills this saftey requirement.
Of course, it goes without saying that module updates should always be tested on a development site before being deployed on a live site.
The Fast, Unsafe Way (no drush or svn)
Back when dinosaurs roamed the earth, many Drupal web sites were maintained in a very cavelier manner with no source code repository utilized. This made module updates very fast, but also very dangerous. To update a module, all you had to do was FTP the new version up to your server (overwriting the old one), run update.php, cross your fingers, and hope that nothing went wrong. Thankfully, most site maintainers have learned (the hard way) that this can lead to disaster. While this practice is still possible, certainly no one in the Drupal community upgrades modules this way anymore.
A Safe, but Tedious Method (drush and svn, separately)
Subversion logoAs source code repositories became more accessible (friendly), more and more developers realized their benefits and started using them. While Drupal.org currently uses CVS, one of the most widely used source code repository systems in the Drupal community (for now) is svn. This article isn't going to talk about how to implement svn, as there are many resources available.
Drush logo

Assuming you haven't been living under a rock for the past year or so, you're probably aware that the Drupal community has embraced Drush, a command-line interface for interacting with your Drupal web site. Drush allows you to quickly perform tasks that you'd normally have to wade through Drupal's user inteface to accomplish (clearing caches, downloading modules, etc...) The real power of Drush comes into play when you start creating command line scripts that combine multiple Drush commands.

Using svn and Drush together to perform module updates is common practice. Unfortunately, it can be a bit tedious. Here's a high-level view of the process:
  • backup your database (I still like to use the Backup and Migrate module for this, although it is possible using Drush)
  • remove the module to be upgraded using svn
  • commit the change to svn
  • use drush to download the new version of the module
  • add the module to svn
  • commit the change to svn
  • run update.php
From the command line (after backing up the database), the process looks like this (for the Devel module):
$ svn del devel
D     devel/ui.mouse.js
D     devel/devel_node_access.info
... lots of files in here ...
D     devel/README.txt
D     devel
$ svn commit -m 'removing old devel module'
Deleting       modules/devel
Committed revision 147.
$ drush dl devel
Project devel (6.x-1.21) downloaded to /sites/demo/sites/all/modules/devel.   [success]
$ svn add devel
A         devel
A         devel/devel-rtl.css
A         devel/devel.css
A         devel/devel.info
... lots of files in here ...
A         devel/ui.draggable.js
A         devel/ui.mouse.js
$ svn commit -m 'adding updated devel module'
Adding         modules/devel
Adding         modules/devel/LICENSE.txt
Adding         modules/devel/README.txt
... lots of files in here ...
Adding         modules/devel/ui.draggable.js
Adding         modules/devel/ui.mouse.js
Transmitting file data .....................................
Committed revision 148.
$ drush updatedb
The following updates are pending:

 devel module                                                        

 6003 - As per issue #813132: change schablon.com to white for krumo.

Do you wish to run all pending updates? (y/n): y
Executing devel_update_6003                              [success]
'all' cache was cleared                                  [success]
Finished performing updates.                             [ok]
$ 
It is even possible to use Drush to perform the database backup via "drush sql-dump", something which I have not yet configured for my particular system. The "drush updatedb" command not only performs the database update (update.php), but it even clears that cache for you.
Another method for removing the old module files is to remove the existing module files (but not directories) via the rm command, then using Drush to download the new module, then commit all the changes at once using svn. This method is a bit more tedious than the method outlined above, but gives you the advantage of improved change tracking in svn.
While this method is perfectly sound, there are quite a few steps involved.
The Safe, Fast Method (drush and svn combined!)
Luckily, the current maintainers of the Drush module (Moshe Weitzman, Owen Barton, Adrian Rossouw, and greg.1.anderson have included SVN support to Drush (next time you see them on IRC, throw them some karma)! This means that we can reduce the number of steps significantly:
  • backup your database
  • use drush to download the new version of the module, commit the changes to svn, and perform the database update all in a single step!
From the command line (after backing up the database), the process looks like this (for the Transliteration module):
$ drush pm-update --svnsync --svncommit transliteration
Refreshing update status information ...
Done.
Update information last refreshed: Wed, 08/11/2010 - 12:44

Update status information on all installed and enabled Drupal projects:
 Name                         Installed version          Proposed version           Status                          
 AddThis Button               6.x-2.9                    6.x-2.9                    Up to date                      
 Advanced help                6.x-1.2                    6.x-1.2                    Up to date                      
 Automatic Nodetitles         6.x-1.2                    6.x-1.2                    Up to date                      
 Backup and Migrate           6.x-2.2                    6.x-2.2                    Up to date                      
 Better Formats               6.x-1.2                    6.x-1.2                    Up to date                      
 Acquia Drupal                1.2.25 (Drupal 6.17 core)  1.2.26 (Drupal 6.17 core)  SECURITY UPDATE available       
 CCK Private Fields           6.x-1.1                    6.x-1.1                    Up to date                      
 CCK Redirection              6.x-1.2                    6.x-1.2                    Up to date                      
 Content Profile              6.x-1.0                    6.x-1.0                    Up to date                      
 Chaos tool suite             6.x-1.6                    6.x-1.7                    Update available                
 Deployment                   6.x-1.x-dev                6.x-1.x-dev                Update available                
 Devel                        6.x-1.21                   6.x-1.21                   Up to date                      
 Domain Access                6.x-2.5                    6.x-2.5                    Up to date                      
 Embedded Media Field         6.x-1.20                   6.x-1.24                   Update available                
 Environment Indicator        6.x-1.0                    6.x-1.0                    Up to date                      
 Flag                         6.x-1.3                    6.x-1.3                    Up to date                      
 Flag Friend                  6.x-1.0-rc4                6.x-1.0                    Update available                
 getID3()                     6.x-1.3                    6.x-1.4                    Update available                
 Global Redirect              6.x-1.2                    6.x-1.2                    Up to date                      
 Hierarchical Select          6.x-3.1                    6.x-3.3                    SECURITY UPDATE available       
 Image Resize Filter          6.x-1.9                    6.x-1.9                    Up to date                      
 IMCE                         6.x-2.0-beta3              6.x-2.0-rc1                Update available                
 IMCE Wysiwyg bridge          6.x-1.1                    6.x-1.1                    Up to date                      
 Invite                       6.x-2.0-alpha1             6.x-2.0-beta2              Update available                
 Migrate                      6.x-1.0                    6.x-1.0                    Up to date                      
 Migrate Extras               6.x-1.0-beta1              6.x-1.0-beta1              Up to date                      
 Nodequeue                    6.x-2.9                    6.x-2.9                    Up to date                      
 Organic groups               6.x-2.1                    6.x-2.1                    Up to date                      
 Panels                       6.x-3.5                    6.x-3.7                    Update available                
 Path redirect                6.x-1.0-beta6              6.x-1.0-beta7              Update available                
 Quick Tabs                   6.x-2.0-rc4                6.x-2.0-rc4                Up to date                      
 Radioactivity                6.x-1.3-rc1                6.x-1.3                    Update available                
 Scheduler                    6.x-1.7                    6.x-1.7                    Up to date                      
 Schema                       6.x-1.7                    6.x-1.7                    Up to date                      
 Skinr                        6.x-1.5                    6.x-1.5                    Up to date                      
 Taxonomy Image               6.x-1.6                    6.x-1.6                    Up to date                      
 Transliteration              6.x-2.1                    6.x-3.0                    Installed version not supported 
 Table Wizard                 6.x-1.2                    6.x-1.3                    Update available                
 Views Bulk Operations (VBO)  6.x-1.9                    6.x-1.9                    Up to date                      
 Views cycle                  6.x-1.0-beta4              6.x-1.0                    Update available                
 Wysiwyg                      6.x-2.1                    6.x-2.1                    Up to date                      
 Copywrite                    6.x-1.0                    6.x-1.1                    Update available                
 Fusion                       6.x-1.0-rc1                6.x-1.0                    Update available                

Code updates will be made to the following projects:
Transliteration [transliteration-6.x-3.0]

Note: Updated projects can potentially break your site. It is NOT recommended to update production sites without prior testing.
Note: A backup of your package will be stored to backups directory if it is not managed by a supported version control system.
Note: If you have made any modifications to any file that belongs to one of these projects, you will have to migrate those modifications after updating.
Do you really want to continue? (y/n): y
Project transliteration was updated successfully. Installed version is now 6.x-3.0.
Project committed to Subversion successfully                                                                                      [ok]
'all' cache was cleared                                                                                                           [success]
No database updates required                                                                                                      [success]
'all' cache was cleared                                                                                                           [success]
Finished performing updates.                                                                                                      [ok]
$ 
The first thing that the drush pm-update command does is ensure that the update information is up-to-date - it performs the same function as the "Check manually" link on the admin/reports/updates page of your Drupal site.

One the update information is refreshed, the updates can begin. Let's look at the command in detail:

  • pm-update is the function that will actually download, install, and perform the database updates for any modules that are out-of-date ("pm" stands for "package manager")
  • --svnsync tells drush to add/delete any new or obsolete files in the module from the SVN repository
  • --svncommit tells drush to commit the changes to the site's SVN repository
  • transliteration is the name of the module to be updated. If you run the command without a module name, it will attempt to update all modules that are out-of-date.

A-ha! Gotcha!

There is one "gotcha" using this method that you should be aware of. This method doesn't remove the out-of-date module before downloading the new one. This means that files in the out-of-date module that should not be present in the new version are still present. Confused? Here's an example: Ubercart was recently upgraded from version 2.2 to 2.3. The "uc_order" module of Ubercart recently changed they way it handles email template files to the standard Drupal template method. This change removed two files from the ubercart/uc_order/templates/ directory:
  • admin.itpl.php
  • customer.itpl.php
and added three files:
  • uc_order-admin.tpl.php
  • uc_order-customer.tpl.php
  • uc_order.tpl.php
When performing the module upgrade using drush pm-upgrade --svnsync --svncommit ubercart, the three new files were added, but the two obsolete files were not removed. Luckily, this was a harmless oversight, but may not be for some module updates (Views has been a culprit on several occasions). A Drush feature request has been added but no work has been completed for this as of yet.
Consider yourself warned.

Conclusion

Module updates are an important part of maintaining any Drupal site. Performing them in a safe and secure manner can help you avoid headaches in the future. Drush and SVN are great tools to help you perform updates when used separately. Combine them, and you'll have something that is more than the sum of its parts, and save yourself a boat-load of time.
Thanks to Ryan Price and Damien McKenna for reviewing this post.

Comments

Use the CVS package handler - this fixes the "gotcha" described, files are added and removed as upstream (and svn added/removed also). Detecting file removals using wget/tarballs is complex. Using CVS also provides a useful reference point when developing, you can quickly diff against the "clean" module in addition to the SVN trunk. Note that you can't upgrade from wget downloaded modules with cvs, you will need to make a fresh dl.

Submitted by Owen Barton (not verified) on Wed, 08/18/2010 - 02:29

I prefer to be a bit closer to the metal so that I can see things go wrong early on:


cd some_module
cvs up -dPr DRUPAL-X--Y-Z
# Look for any conflicts or other errors.
drush updatedb
# Look for errors and fix them.
# Depending on the site run unit tests and/or manual testing.
svngrep
# this is a small custom script that adds new files to SVN and removes missing files
svn st
# make sure everything is being added/updated/removed as expected
svn commit -m"Security update for module foo to version X--Y-Z"

Then on the staging server

svn up
drush updatedb
# Depending on the site run unit tests and/or manual testing.
# Get the client to review if it's bigger than a trivial update
# Deploy code to production.

Might seem like it's implied here but I'd mention this too before doing anything:


drush vset site_offline 1

Also generally it's best to deploy the updates on only a development server and use your preferred version control to pull in the new tag/release.

Submitted by webkenny (not verified) on Sun, 08/29/2010 - 14:07

I liked the method described under and am writing a custom drush command.

What would be the procedure to rollback when things go wrong? I have tried

svn merge -r N:N-2

(the revision numbers are two apart because there were two commits)
and I get error when I commit after the reverse merge. It looks like [merge] cannot handle the case where a same folder is removed and added back.

Wow, thanks for saving me hours of time updating modules!

Submitted by Ben Leivian (not verified) on Wed, 06/29/2011 - 15:42

Add new comment

Restricted HTML

  • Allowed HTML tags: <a href hreflang> <em> <strong> <cite> <blockquote cite> <code> <ul type> <ol start type> <li> <dl> <dt> <dd> <h2 id> <h3 id> <h4 id> <h5 id> <h6 id>
  • Lines and paragraphs break automatically.
  • Web page addresses and email addresses turn into links automatically.