Polka Tables, originally uploaded by markrocky.
From our photography excursion (with Harvest Digital) on the South Bank using “Lomolitos” (mini Lomos), on Wednesday evening…
Agile web app deployment with SVN, rsynch, Phing and more
We are a small development team, at Harvest Digital, handling multiple tasks throughout the day, across different environments and projects. It makes for an exciting challenge. Our roles/ skills are clearly defined and our small size means we can react quickly to change, with little margin for communication errors and so on (see Getting Real and Less is more: Jumpstarting Productivity with Small Teams). But, when it comes to deploying to a staging or production server, I am ashamed to admit that we had often introduced a huge potential for human error by doing much of the work manually. Each project is different and so there is no consistent deployment process across the board.
Deployment is often the last step in the process to get any attention and it is one that can be unnecessarily tedious and prone to error. With more agile development projects, where deployments may be required several times a day, we obviously cannot afford the time nor the potential mistakes that may be made by manually deploying a project.
At its simplest level, deployment may just involve making sure that the target server (i.e. staging or production) has the latest revision of files from the project’s source control repository (we use SVN). But factor in automated tests and database migration and you can see how easily things can go wrong. Especially if you are deploying several times a day. You need to automate as many steps in the process as possible, thereby eliminating any manual tasks that introduce potential error and duplication.
Whenever we reach a point in a project where the code can be considered stable, we should tag it in SVN as a release. This gives you the ability to deploy a particular release based on its tag, instead of just the latest revision in the repository. By convention you never create revisions in the tag folder – tags are simply named snapshots of the source repository. We typically use version numbers as tags with some scheme based on feature-set or milestone and release date that can be ordered alpha-numerically e.g. “rel_belfast_2.04″. This allows us to say with certainty what is deployed at any one time, without having to scan logs etc. Tags give context to a release name (you can easily associate the release with project milestones) that revision numbers and timestamps do not. Tags are usually kept in a ‘tags’ subfolder, in the top level of the repository.
For a while now at Harvest, we’ve been working with symfony – an excellent PHP framework – that includes a few tools to help simplify the deployment process (such as Pake). These tools allow for remote syncronisation of files with rsync (the benefits of synchronisation over standard FTP are much touted elsewhere) via SSH (with the shell command: e.g. symfony sync staging) and semi-automate the build process (including database schema changes, data dumps etc: e.g. symfony propel-build-all myApp) by using Pake on the target machine.
But there is still a lot of room for improvement, particularly when it comes to database migrations and SVN integration. Plus, symfony is not our only set-up. We need an automated deployment process that can be applied across all our projects. This is where I’ve previously seen shell scripts being used but Phing can do much of this heavy lifting work for us, as it has many standard deployment tasks built-in to it. According to the Phing blurb:
Features include file transformations (e.g. token replacement, XSLT transformation, Smarty template transformations), file system operations, interactive build support, SQL execution, CVS operations, tools for creating PEAR packages, and much more.
Phing (PHing Is Not GNU make) is a project build system based on Apache Ant and is available through the PEAR installer. If you use --alldeps on Phing, it’ll grab, funnily enough, all the dependencies. But most are good packages (e.g. PHPUnit, PHPdoc and VersionControl_SVN). It can be used in a multitude of ways, including unit testing, creating docs, running SQL – which allows us to keep database schema changes in version control. The beauty of Phing is everything can be configured easily through the build.xml file (and then further with properties files) – so that the project’s deployment configuration can be kept in version control too.
In short, we love this thing called Phing!
For more detail on advanced tasks such as database migration take a look at these posts from Federico Cargnelutti and Dave Marshall.
Open source: utopian-dreams, recession-proofing and socialism…
Those who know me will undoubtedly have at some point had the pleasure of listening to me ranting on about the power of the open source movement – or, more generally, social collaboration online. On the bus yesterday I read an editorial piece – “Collaboration is the new revolution” – in the Guardian newspaper that certainly struck a few chords. Here’s a few extracts, by way of a summary:
1) Open source paves the way for the odd utopian dream:
Sir Thomas More’s description of Utopia as a place where “nobody owns anything, but everyone is rich” is not a bad way to describe the open source movement in which people around the world collaborate with each other to produce services that anyone can use – or improve on – for nothing.
2) Open source is recession proof:
One of the interesting things about the collaborative movement is that it is probably recession-proof, though you won’t see it in economic statistics because it mostly does not involve cash transactions.
3) Open source is robust and well-trusted by large successful organisations:
Big corporations, such as IBM, Google and Amazon, are devourers of open source software because they find it cheap, efficient, low-maintenance and reliable. But UK government departments, including health and the foreign office, have proved risk-averse with hardly any open source in their infrastructure.
4) Open source is socialism (well, social collaboration at least):
…open source combines the cooperative spirit that was at the heart of the Labour party in the past with the entrepreneurial skills needed today.
And with nothing more to add to that summary, I’m off to join the revolution (well, okay, I’m off down the pub to carry on ranting to anyone who’s too polite to stop me).
No more mobile tweets?
Twitter has announced they will cut outbound SMS alerts for users in the UK because it was costing too much!
Whatever you say about Twitter, I was enjoying the ability it gave me to receive updates via SMS. In fact, it was this feature that won me over to using it and helped demonstrate what it was all about to mates. I figured that the initial free SMS alerts that appeared in my Twitter account would run out one day and then I’d have to pay for more credits – easy business model. I’d happily have paid too. So why pull it? Maybe its all part of some ploy by Twitter to gather a crowd reaction – so that when they announce payment plans for SMS alerts, they already have the crowds’ support. Yeah – okay, that may be a tad cynical but I do hope they listen to their UK users (and, in fact, anyone who used the UK based SMS alerts). The reaction is already gathering pace – with comments and tweets a plenty – and now a Facebook group. SMS is huge here and it seems to be a glaringly obvious way to actually turn Twitter into a revenue making business, in the UK at the very least.
