Category Archives: tech

22 Apr
2010

svn+ssh with specific identity file

Here is a quick howto on using a config file to associate an SSH connection with a particular identity file. When I ssh directly from the command line, I can specify the port, identity file to use, and a whole raft of other options. However these options aren’t available when checking out from subversion using svn+ssh, and I needed to specify a particular private key to use. These instructions are for my kubuntu 9.10 installation but should work on every flavour of *nix as far as I know.

I was trying to check out using something like this:

svn co svn+ssh://[email protected]/desired/path

To do this, I created this file: ~/.ssh/config and in it I put the following:

host lornajane
HostName svn.example.com
IdentifyFile /path/to/private_key

Now I update my checkout syntax to make use of my alias rather than specifying the host directly, like this:

svn co svn+ssh://lorna@lornajane/desired/path

SSH realises that it should pull the host information from your local config file, and uses your HostName, IdentityFile and any other settings you specify (there are many options, see this reference for more information. For example I often use this for port numbers if I’m tunnelling my SSH between places, the possibilities are endless.

Hope this helps, if you’ve got anything to add then I’d be delighted to see a comment from you.

19 Apr
2010

Book Review: Making it Big in Software

Recently the kind folks at Prentice Hall contacted me to ask if I’d like to read a new publication of theirs , “Making it Big in Software“. It sounded like something I’d be interested in and the copy duly arrived. My first impressions were of a wordy book, very much aimed at large organisations, or those working in academia. And although I’m 20-something myself, the advice in the first sections of the book is aimed more at students or recent graduates than, I felt , at me.

Moving on into the book I found more that applied to me, including some great advice on communicating in different settings and with different types of people. I was also interested to find some thoughts on debugging as a skill to be used instead of, or before, launching a built-in debugger. This makes sense to me because my first programming languages weren’t the kind that have IDEs or other advanced tools.

The chapter on time management starts with the words “if you only read one chapter in this book, this should probably be the one” – and I must say I agree. I consider myself to be organised but there were some great strategies outlined in that particular chapter. Also running through the book like a thread is a series of interviews with software luminaries, each of whom were asked mostly the same questions, including how they manage their time and work-life balance. So there were some real-life stories in there too – I was encouraged that almost all of the people interviewed said they didn’t think they found balance, because personally I know I haven’t.

The interviews provided a great structure on which to hang a book full of good advice. One chapter advises how to identify and avoid “career killers” – those dead-end projects that will hold up your progression. Later on the focus moves away from the big organisation that the author and interviewees have most experience of, to look at starting software companies and the skills and attitude needed for this. Certainly once I got past the first third of the book I was much more drawn in; maybe because that dealt with an early stage of career that I felt I had already mudddled through?

Overall this is quite a dense book to read over a short period of time, but an excellent one to keep around and dip into or read another chapter every few weeks perhaps. Certainly its one that will be going on my bookshelf for me to look back at and refresh my thoughts on various topics covered for some time to come.

24 Mar
2010

Ada Lovelace Day 2010: Mrs Maginnis

This is a post in celebration of Ada Lovelace Day, recognising women in science and technology. Last year I wrote about Kathy Sierra for Ada Lovelace Day; this year my subject is someone much closer to my own life.

I’d like to write about someone who influenced me greatly – in my choice of study and in my attitude to the gender imbalance in that area and in the industry in which I now work. Mrs Maginnis taught me advanced maths when I was 16-18, and making decisions about my next move. I was educated at an all-girls grammar school, which in some ways was an advantage. The experience of those all-female maths classes taught me a lot about how different women are when they meet their individual challenges in the company of one another.

The classes were very tough academically (as they should be!), and I did struggle during the course. At the start of the summer holiday, Mrs Maginnis handed me a textbook with a list of questions to try, and her phone number. All summer I wrestled those questions, with the occassional phone call for help. I went back to school, completed the year, and left the following summer with two A grades in A-level maths – undoubtedly because of the help and support I had been given by this one teacher. Those two A grades landed me my first choice of university place – I thanked her and off I went.

However, the strong influence of this character has stayed with me long since the memorable day I opened the results envelope. I was headed for a course in Electronic Engineering and Mrs Maginnis had been a Mechanical Engineer herself and I think had a good idea of what lay in store for me. The thing is – I was educated entirely with women … I genuinely did not know that there were subjects that girls just don’t do. So when I arrived at university with my academic grades from the all-girls school, I was confronted by a course that was 95% male and contained mostly people with grades at least as good as mine and usually including more practical subjects such as Electronics, which I hadn’t had the opportunity to study. So it was a struggle from that point of view.

The teacher who had, on paper, taught me maths, had in fact taught me so much more. Even now I sometimes remember stories, anecdotes, and advice that all drifted across the classroom along with the hyperbolic functions and calculus (which is a much more distant memory now). She had been an engineer herself, and through her stories she showed us all how it would be done, something on what to expect and some tips on how to handle it. Even now I don’t think I’ve had a stronger female role model for industry – and she managed all this from academia, when I was a teenager. So – I raise my glass to Mrs Maginnis. Thankyou for all you taught me.

23 Mar
2010

GETting RESTful collections – may I filter?

At work at Ibuildings recently, I’ve been teaching some classes on web services, and its a topic that I’ve spoken about once or twice at conferences. But something has always bothered me, so I find myself in the unusual position of blogging a question.

RESTful collections

So, when you are retrieving information from a RESTful service. You have two options: retrieve a specific resource, whose URL you know; or retrieve a collection, which may contain a list of resources. I’ve also seen some nice ways of filtering collections, by creating kind of “sub collections” or “views”, similar to what twitter does with the URL of lists, for example http://twitter.com/dpcon/speakers10 which is like a filtered list of twitter users.

What if I want to search and sort?

Is it RESTful to add GET parameters to a collection in order to add functionality such as filtering, sorting, or pagination? What I have in mind is a URL that looks something like this:

  • http://example.com/users?orderby=firstname&start=0
  • http://example.com/users?start=0&limit=25
  • http://example.com/users?active=1&orderby=join_date&limit=12

This is what I would do with a search results page in a web application, and I use the same approach to web services which works really well and I recommend it to everyone! But is it RESTful?

I am also wondering where OpenSearch would fit into the answer for all this, I only noticed it recently but the more I look at it the more I think it could be an interesting addition!

Thoughts, links for me to RTFM, and all other additions are welcome in the comments box :)

01 Mar
2010

Simple Database Patching Strategy

One problem that seems to need solving too often is how to keep databases in sync across different platforms, different developers working on a project, and deploying database changes along with code changes. There are lots of ways of approaching this, none of them are really excellent however and personally I tend to err on the side of simple being better. Fewer dependencies means a solution more likely to work on every platform (and no additional complications for the live platform). Usually this means patch files of some kind – here’s an outline of my usual approach. For the impatient:

  • add a table for meta data to the database, set a database version parameter to 0
  • export structure and any essential data (lookup tables, etc) into an sql file
  • for any changes, create a numbered patch file e.g. patch-1.sql, including the change and an update statement to bring the version meta data to match the patch file
  • store all of the above in source control
  • for bonus points, create another sql file with some nice sample data

Give the Database Some Version Awareness

Naming is completely changeable but mine looks something like this:

CREATE TABLE `meta_data` (entry varchar(255) primary key, value varchar(255));

INSERT INTO `meta_data` SET entry="version", value="0";

This new row will hold information about what patch version the database is up to. Every patch that is created will update this number so that it is possible to tell which patches have and have not been applied.

Create an Initial Data Structure

Firstly create a database directory for all these files to live in. This should be outside your web root but inside your source control project.

Take your database and dump just the structure of the tables using the –no-data switch to mysqldump like this:

mysqldump -u  -p  --no-data > create.sql

You will also want to populate tables which hold things like lookup values, country lists, that sort of thing, so that these are set up. People starting to develop with this project, or if the application needs to be deployed to somewhere new, can use this file as a starting point.

Create Patches for Changes

This is where the discipline element comes in – no direct edits on the database are allowed! Instead, write the SQL for your change and place it in the patch file, then run it against the database. If that sounds too much like hard work then copy/paste the SQL you use to make changes, or the SQL generated by whatever SQL tool you use, and place it in the file.

Every file must have its own incrementing version number in its file name, e.g. patch-1.sql, patch-2.sql etc. Within the file the version must also be updated to match, with a statement like:

UPDATE `meta_data` SET value="1" WHERE entry = "version";

Recommended Practice

Here are a few pointers on getting the most out of something like this:

  • Under no circumstances is it acceptable to edit a patch file that has been committed to source control. Someone might have run it already and you’ll confuse everyone completely.
  • Having matching undo scripts alongside the patches can be really useful in case a change gets deployed and needs to be undone.
  • Make a script to look up the database settings in the config file of your application, query the database for what version it is at, and run any outstanding scripts. This makes life much easier especially if you have large numbers of scripts (I’ve seen the patch numbers hit the hundreds)
  • A possible adaptation of this approach is to create patch files for use for updating a database, but to also update the install.sql file to make it correct at any point in time, this means a much shorter and easier setup time for new deployements/developers. The version awareness works in the same way regardless
  • Creating a sample database which creates a few records in each table can really help for development purposes – its quicker for people to get set up and attain a working system that they can make progress with.

I’m sure there are many improvements and variations on this theme of simple database patching, leave a comment and let me know what works for you and why!

23 Feb
2010

Supermondays: Recap

Last night I travelled to the northeast of England to speak to a thriving technical community up there called Supermondays. They contacted me some time ago asking if I could get there to speak one Monday, and last night was the night! It was a very civilised gathering, with sandwiches and cups of tea, and using a lecture theatre at the university for space. As a speaker the best thing about this is that its a space designed for addressing people in, unlike most user groups (and indeed conferences!) where two steps away from the lectern sees you standing in the dark, falling off the stage, or getting projected on to. Last night was a different story with lots of space to wander, slides projected well above me on the wall so everyone could see clearly, and relatively good acoustics despite no amplification.

My talk was entitled “PHP and Web Services: Perfect Partners” – the slides are on slideshare if you want to take a look. There was also a talk about android development by Alex Reid, including a live coding demo which went surprisingly well! Judging by the various events that were plugged and discussed on the night, at the main event and in the pub afterwards, this is a diverse and vibrant technical community – so if you are in the northeast, get along to Supermondays!

19 Feb
2010

Open Office Presenter Console

I’ve been having issues with the presenter console on both my ubuntu machines since upgrading to Karmic (9.10). One is a Thinkpad T400 running kubuntu and the other is an aspireone netbook running ubuntu netbook remix. Neither wanted had a working installation after upgrade and I couldn’t get the plugin installed using the open office plugin manager.

I discovered that this plugin is now available through apitude – simply install the package openoffice.org-presenter-console and it should all work splendidly! I use the presenter console when I am speaking (which is quite often) to show the time and the upcoming slide, its a great tool.

08 Feb
2010

Word Count

There’s a little command line utility on *nix which I use a lot – it’s wc or “word count”. This is especially useful to because I live in a world where everything is plain text right up until I have to send it to someone else (and sometimes not even then). Despite its name, word count can count more than just words – it can do characters, words, lines and can tell you the length of the longest line while its at it.

Counting Lines

The biggest problem with counting lines is remembering the name of the utility, since its called “word count” and not “line count”. I tend to use this for doing things like piping grep to wc and counting the lines to give me an idea of how many occurrences of something there are. I also use it to count errors in weblogs or really anything else that I could do with summarising. The syntax is something like:

grep -R TODO * | wc -l

Using a count like this is especially good for things like auditing code, where I need to know how prevalent something is – or refactoring, where I’m looking for how many of a particular pattern are outstanding. Counting lines is also very compatible with my habit of making lists in text files.

Counting Words

This is the feature that the utility was originally designed for, and as you can imagine, its pretty good at that. As with most things, this blog post started life as a text file and when I got to this point I saved it and ran:

wc -w wc_article.txt

It outputs the number of words (272) and the name of the file, which is useful if you’re giving it a pattern to match.

Word Count

Its a really convenient and versatile little program; I use it often and I hope others will find it useful too.

26 Jan
2010

Contributing to Projects on GitHub

Recently I’ve been contributing to the code project behind joind.in, the event information and feedback site. I rely on joind.in a lot and after putting up with a frankly astonishing volume of feature requests from me, its owner Chris Cornutt very politely suggested that I might like to fix some of them myself. The project is hosted on github and I haven’t traditionally been much of a git fan, but I wanted to contribute so I set off to work out how to begin.

Register on Github

To do anything useful I first needed to sign up for an account. Github has a range of accounts but I found that with one of their free accounts I would be able to get started and contribute to the project. This gives me a project space of my own and a user to tie all my activities to.

Set up SSH Key

In order to authenticate against the github servers, you need to set up an ssh key and give them your public key so they know you are you. You’ll then need to tell git to use this key whenever it makes contact with the github servers. I do quite a bit with ssh and ssh keys myself so I was comfortable with this step. Even if you are totally new, its still pretty straightforward and they have a great howto on github itself which will help.

I had issues with git not picking up that it needed to use a non-standard ssh key, but I found the answers in this entry on the git website. In a nutshell, set up an ssh alias, set the key in there and then use the alias instead of the actual URL when giving the repo location to git. This now works like a charm for me.

Fork the Project

Now, github uses “fork” where I might choose to say “checkout” – fork in my world means something else completely. But in this case you’re just making your own copy of the project repository. This is where you will commit your changes to and it retains its link with the original repository making it easy for anyone with commit access to that to pull in your changes. Patch files are nowhere to be seen, and although I was wary at first, this is project collaboration at its most painless, I’m impressed! Forking was relatively simple and again there was great documentation on the github site. In particular I recommend that you take the time to follow the bit about adding an alias for the “upstream” repository – this made committing my changes to the main joind.in repo really easy.

The forking instructions linked above also gave a description of how to actually use git, how to get my changes applied to my local repo, and how to push them to my remote repo on github itself.

Make a Pull Request

Once I’d fixed a few things, I was ready to push the code back to the main project so that Chris could consider it for inclusion. This is done by making a pull request from the main project page – you can add a comment about the changes you are supplying to help the maintainers to manage all the incoming patches.

Go Forth and Contribute

It was easier than I expected to get set up to contribute to a project using github, so find something you want to improve and/or be involved with, and do it. I began by fixing the docs for joind.in, which was a great place to start since it allowed me to make a useful contribution without touching the code in the first instance :)

15 Jan
2010

Screen in Ubuntu Karmic

I have written about screen quite often, mostly including my .screenrc file and showing how to have named tabs for the various screen tabs you have open. When Ubuntu Jaunty came out, I found it had some quite cool enhancements that made the customisations for screen really easy by default – and I wrote about these.

In Karmic Koala, Ubuntu 9.10, the packages are still there but they’ve changed names! So if you want to use screen with Ubuntu Karmic or later, install packages byobu and byobu-extras, and uninstall screen-profiles and screen-profiles-extras (they were broken on my system after upgrade anyway) and you should find everything works as expected. To run screen with the new features, you should run “byobu” instead – although screen commands seem to work to detach and reattach the screens that result, weirdly.

I’m mostly posting about it because I have been very frustrated and there’s no way I could have guessed, or probably ever will remember, what these packages are called. Apparently a byobu is a japanese room screen … you learn something new every day!