Idiot-Proof Deployment with Phing
Once upon a time, a long time ago, I went onto a conference stage for the very first time and said that I thought I might be the world’s ditsiest PHP developer. I actually still think that is pretty true, and if you work with me then you will know that I mostly break and fix things in approximately equal measure. With this in mind, when I launched my own product recently (BiteStats, a thing to automatically email you a summary of your analytics stats every month), I knew that I would need a really robust way of deploying code. I’ve been doing a few different things for a few years, and I’ve often implemented these tools with or for other organisations, but I don’t have much code in production in my own right, weirdly. I decided Phing was the way to go, got it installed, and worked out what to do next.
build.xml
I created a file called build.xml in the root of my project directory, and initially it contained only the following:
<?xml version="1.0" encoding="UTF-8"?> <project name="bitestats" basedir="." default="deploy"> <property name="builddir" value="./build" /> </project>
That default attribute says which task to run if no task is specified, and the property tag initialises a variable I’ll need in a couple of places later on. In fact it’s the build directory, a directory that will be used to hold all the artefacts of the build.
Tasks
Tasks are defined in the build file, inside the project tag. I have two really simple ones which I think most projects have: build and clean. These create the build directory ready for me to use when I run other tasks, and remove it again. So with those included, my file now looks like this:
<?xml version="1.0" encoding="UTF-8"?> <project name="bitestats" basedir="." default="deploy"> <property name="builddir" value="./build" /> <target name="clean"> <echo msg="Clean..." /> <delete dir="${builddir}" /> </target> <target name="prepare"> <echo msg="Prepare..." /> <mkdir dir="${builddir}" /> </target> </project>
To run the tasks, you just do phing prepare or phing clean respectively.
Real Deploy Tasks
To run the deployment itself, I have the default task that you saw declared in the first snippet, deploy. The task definition looks like this:
<target name="deploy"> <echo msg="Next Tag ..." /> <exec command="echo $(($(hg tags | egrep -o '^deploy\-([0-9]*) ' | egrep -o '[0-9]* ' | sort -nr | head -n 1) + 1))" outputProperty="nextTag" /> <echo msg="... is ${nextTag}. Tagging ..." /> <exec command="hg tag deploy-${nextTag}" /> <echo msg="Grab code ..." /> <exec command="hg archive -r deploy-${nextTag} build/deploy-${nextTag}" /> <echo msg="Zip code ..." /> <exec command="tar zcf deploy-${nextTag}.tgz deploy-${nextTag}" dir="build" /> <!-- copy to live --> <echo msg="Put code onto live server ..." /> <exec command="scp -i /path/to/ssh/key build/deploy-${nextTag}.tgz [email protected]:/path/to/bitestats/htdocs/" /> <!-- make live--> <exec command="ssh -i /path/to/ssh/key [email protected] 'cd bitestats/htdocs && tar -zxf deploy-${nextTag}.tgz && ln -s deploy-${nextTag} next && ln -s /usr/local/Zend/ZendFramework-1.11.2-minimal/library/Zend/ public/library/Zend && php -f public/cli.php -- -a updatedbpatch && mv -fT next public'" logoutput="yes"/> <exec command="ssh [email protected] 'supervisorctl restart worker'" logoutput="yes"/> </target>
A quick recap of what actually happens here:
- We work out what number the next tag would be and tag the codebase. I use Mercurial for source control and host my code in a private repo on BitBucket
- We use
hg archive, which is basically like SVN export, to get the appropriate code version and then tar and zip it. - The code is scp-ed to the server, by the apache user
- On the remote server, a few more commands get run all in one go. Firstly, the code is unpacked
- A symlink called “next” is created and pointed to the new folder (I blogged about my symlink strategy already if you are interested)
- A symlink is created to the library code, in this case ZF but you’d also create symlinks to config files, upload/cache directories, and anything else that’s needed at this point too
- Any database patches needed are run at this point (again, I blogged this script before in case you want to see it)
- We switch the symlink so that the new content is live
- Finally, we stop and start the gearman worker so it picks up new code (one worker, all on the one server, there really isn’t a lot of load on this site!). My gearman processes are overseen by supervisord, but it only answer to my user. It actually prompts me for my password at this point
Having a Deployment Process
The whole process takes a few seconds and so far it has worked like a charm. I’m really happy with it, and really happy I took the time to put this in place. Every time I deploy I feel a little bit smug :) I know it can be hard to get deployment stuff set up, after all we can do all the individual steps pretty easily and everyone has plenty of other stuff to worry about – but I think it’s important and even though the “right” solution will be different for every application and platform, I wanted to share what works for me. What works for you? Leave me a comment!
I use the same approach as you do but without phing and inspired by WePloy.
https://github.com/rlerdorf/WePloy
It’s great to see how other people do it :)
A couple of differences from the way we do it here:
1. Our build directory is completely separate from our source code.
2. I’m lazier than you though and use a date/time string for my tag names .
Like you, I like having a deployment process in place as it makes the entire processes of putting a new version of a site live much much easier and stress-free.
Also, consider upgrading to ZF 1.11.5 :)
Regards,
Rob…
Phing is okay, but if you haven’t already, you ought to check out Webistrano. It’s a web-based deployment GUI built atop Capistrano with robust capabilities, including support for passwordless SSH authentication, OS-level atomicity of deployment (change a symlink), catalog of and rollback to previous deployed versions, multiple projects, multiple users, deploying to multiple hosts, database migrations, git, Subversion, and more. Basically, login and click deploy. Really nice. Yes, it is done in Ruby for Rails applications, but I’ve been using it successfully with PHP projects for quite some time now.
Thanks for the mention of webistrano, it is certainly a giant in this area. I’ve used it and liked it but for me, to install Ruby and write my scripts in it just for deployment didn’t seem worth it – reckon it’s a good fit particularly for more complex sites however
Glad to see you using such an ace tool (as you know I’m definitely an advocate)!
With quite a few of your methods, you can use inbuilt functionality within Phing, which allows you to use the built in PHP functionality of Phing, rather than using it as just a wrapper for execs.
For others it’s definitely worth checking the built in tasks, especially if they’re not a full on command line buff =)
http://www.phing.info/docs/guide/stable/chapters/appendixes/AppendixB-CoreTasks.html
http://www.phing.info/docs/guide/stable/chapters/appendixes/AppendixC-OptionalTasks.html
Dom
Hello :) Thanks for the useful links, I am bad at reinventing these kinds of wheels and resolve to read the documentation first in the future, I didn’t know half of that stuff was available!
I have a similar setup, only I don’t use SSH commands on the server. Instead, I have another part of my Phing file dedicated to the deployment tasks on the server and call Phing on the server via SSH.
The SSH task of Phing did not work right for me: It silently fails when the server is not in the known hosts list amd on some other commands (unzip in my case) it also silently fails. For SSH-related things I’d rather use exec. The Phing SCP task works, though.
Do you think for the command for the deployment “” should be rolled into a shell script. That way you can write to a log using stderr/stdout?
I always wondered why tools like Phing, Ant and the likes are so popular… most build configurations I’ve seen so far contain mostly exec statements, so essentially you’re writing a shell script but wrap your commands in XML.
Is there a genuine advantage over writing plain shell scripts that I am missing?
Markus,
The main advantage for me is the platform independence.
Regards,
Rob..
Markus, I just like it for familiarity and an easy way to package stuff that belongs together. So I’ll have a phing build file with a few different available tasks and yes, they basically run shell scripts, but in a way I can easily find those jobs and that other people can understand. So in technical terms, no difference, but in user terms, it works for me.
Nice article!
Have you ever considered using liquibase to handle your database updates?
Check it out here: http://www.liquibase.org/
—Jaco
Pingback: Programowanie w PHP » Blog Archive » Lorna Mitchell’s Blog: Idiot-Proof Deployment with Phing
Hi,
I try many tools to automate deployment application , e.g JBoss, tomcat, packages (based on copying new packages and so on).
I see phing is so promise to handle my projects and deployment, But I want to handle phing from a web gui using YII framework
Is any one may guide me,
Thanks
Lorna,
Your code is not visible in browser (IE, Firefox)
Thanks! I knew I’d broken one of the many syntax highlight plugins used in these older posts but I guess I didn’t find all the places that used it. Fixed and it should look much better now.