I’ve introduced Gearman into a project I’m working on, and since a few people have asked I thought I’d share my experiences. Basically, this application generates some PDFs from a variety of data sources, makes images, and emails it. Since the whole data processing, image handling, PDF generation process is fairly heavy, I’m putting the requests to generate these onto a gearman queue and having some workers process the jobs. The eventual aim is to bring up EC2 instances (or php-specific cloud hosting perhaps? Recommendations gratefully received!) to do this work but right now I have one worker and it’s all on one server.

I installed Gearman and the pecl module for it, and then looked at how to integrate it into my application. I was replacing an existing synchronous script that generated the output, so I already had the code I wanted to be performed by the worker, and since this was a ZF application I set up a CLI bootstrap for it and ran my existing functionality from a controller called that way.

Adding Jobs to Gearman

Adding jobs to the gearman queue was easy, but I struggled to understand which function I needed from the documentation. It turns out that if you just want a job to wait for some resource to be available to do it, then GearmanClient::doBackground() is the way forward. My code to add jobs looks like this:

$client = new GearmanClient();
$client->addServers('127.0.0.1:4730');
$client->doBackground('basicReport', json_encode($job_data));

The $job_data variable holds an array of data that is needed by the worker to know what to generate.

Writing a Gearman Worker in PHP

As I say, I bootstrapped my ZF application from the commandline and created a controller action to be called. It contains code like this:

        $worker = new GearmanWorker();
        $worker->addServer('127.0.0.1');
        $worker->addFunction('basicReport', [callback]);
        while ($worker->work()) {
            echo "working ... \n";ob_flush();
            if (GEARMAN_SUCCESS != $worker->returnCode()) {
                echo "Worker failed: " . $worker->error() . "\n";
            }                                                                                                                                                                       
        }

I named the model class and function in my callback (wrote about callbacks the other day) and the job data that I json_encoded and passed in when I put the job in the queue is passed as the parameter to the callback, which enabled me to basically hook into my original synchronous code and have it pick up where it left off!

I’ve been very impressed with gearman so far and I’m looking forward to putting the workers onto the cloud and scaling the application as it needs it. As I say, I’m literally using doBackground() to put jobs into a queue for the workers to come along and perform asynchronously, and gearman seems like a great tool for that.

Edit (Feb 2013): This post was edited to reflect changes in the addServers() method in GearmanClient