JAXL - Connecting to Google's Cloud Connection Service (CCS)

I recently implemented the server side of a system to send notifications to iOS apps through APNS. This was extremely easy to implement and opened my eyes to the benefits of having a persistent streaming connection to Apple's servers. That is to say my backend is a constantly running service, and as/when new 'notifications' are stored in our database they are immediately sent through to APNS.

For Android notifications I was utilizing GCM and a simple HTTP connection to Google's servers (using CURL). I ran the script as a cron every x minutes and it would send out the notifications as appropriate.

Whilst this worked perfectly fine.. the idea had now crossed my mind and it was inevitable that I'd have to implement a similar persistent connection to Google's servers for GCM.

Cloud Connection Service (CCS) and XMPP

Google have a service called the Cloud Connection Service to achieve this. It utilizes the XMPP protocol (originally an instant messaging xml based data transmission format) to communicate in both directions with a client.

As you may have noticed (if you are a regular reader of our blog) a lot of our products are PHP based. For this particular project it made sense to execute this functionality using PHP.

Google do not provide any examples of utilizing PHP to connect to CCS and in fact there are very few well maintained, well used, generally solid implementations of XMPP communication in PHP.

A brief 'Google' of XMPP presented a lot of information about the protocol. I would assume that Google opted to utilize XMPP because it is a 'standard', and has been used and developed over 15 years. It is seemingly well known within the engineering community and it allows for two way communication such that GCM can supply 'Receipts' for messages. This is something that iOS and APNS can not provide.

On a personal level I have absolutely no experience with XMPP. In its entirity it is quite a complicated protocol. For a GCM implementation the only things that are relevant are essentially authenticating, and sending messages. Even though CCS allows for bi-directional communication, this was of little interest to me.

Getting to work

I wrote a script to connect to Google's CCS servers and send messages using JAXL. This was really easy to do:

//Initialize
$this->client = new JAXL(array(
         'jid'=> $this->senderId .'@gcm.googleapis.com',
         'pass'=> $this->googleApiKey,
         'auth_type'=> 'PLAIN',
         'host' => $this->host,
         'port' => $this->port, 
         'strict' => false,
         'force_tls' => true,
         'log_level' => JAXL_DEBUG,
         'protocol' => 'tls'
    ));
    
    //add a callback for authorisation success
        $this->client->add_cb('on_auth_success', function() { 

      $this->client->set_status("available!", "dnd", 10);

		//send your messages
      $this->sendYourMessages();
    });
    
    //start the client
    $this->client->start();

Within my sendYourMessages method I was loading some notifications from my database, looping through them, and sending them utilizing $this->client->send() / $this->client->send_raw().

This worked perfectly.

Given the number of Stack Overflow questions and Google Code discussions about the difficulty of connecting to CCS with PHP I was somewhat bemused to say the least.

Unfortunately however I had been a little too optimistic. What I wanted to do was maintain a persitent connection to the CCS server and constantly poll for new notifications in my database.

Given that PHP and asynchronicity are rarely found in the same sentence together, this was going to be a little tougher to achieve.

My intention was to utilize a continuous while loop to continually poll my database:

while (true) {
	//poll database
    //send messages over XMPP connection
    
    //take a nap
    sleep(5);
}

The problem is that this is blocking - nothing below this block will ever execute until the while loop completes (which is never).

If you look into the internals of JAXL it works in a slightly more complex yet similar way. That is to say there is a continous blocking loop checking the connected streams to see if it can/should read/write to them, and then acting accordingly.

When you execute start() on the client it configures things and then executes JAXLLoop::run() which starts this continuous blocking loop.

The long and the short of it is that you cannot continuously poll the XMPP connection and continuously poll your own data source.

Read the source

I made the foolish mistake of going in blind and trying to hack together an appropriate resolution.
A fear of the complexities of XMPP and a smidgen of laziness ironically meant that a resolution took significantly longer than it should have.

After a number of hours of futility I decided to step back and read through the JAXL source in its entirity. At this point everything slotted into place and a suitable resolution (see below) was relatively easy to come by.

The JAXL source is a little 'hmm ok'.. but it is pretty simple to get your head around.

As for XMPP.. whilst a lot of the information on the web is very much all or nothing, I did find this: How XMPP Works Step By Step which I found to be the most concise explanation of the relevant workings.

If you utilize the JAXL_DEBUG log_level in your configuration, the output matches up almost perfectly to that outlined in the above link.

The resolution

I wanted a resolution that could work on top of JAXL without requiring a significant time investment or refactoring.

Conceptually the resolution was as follows: Implement batch data polling within the JAXL Loop.

We can remove the issue of one loop blocking the second by.. only having one loop :)

During my research into the problem I stumbled upon this StackOverflow answer which suggests using UDP sockets. This seems like a complete 'overengineering' in the sense that it would work but the complexities and problems associated with it beg the question 'Why not do something easier?' (like the below).

I have forked JAXL and committed my changes to github here.

What i have essentially done is manipulate the 'periodic jobs' concept already contained within the JAXL codebase. If you look here there is a very brief explanation.

In that same file is the following message:

"Since cron jobs are called inside main select loop, do not execute long running cron jobs using
JAXLClock else the main select loop will not be able to detect any new activity on
watched file descriptors. In short, these cron job callbacks are blocking."

This is important. Essentially the clock (loop) is executed every second and we say 'has 15 seconds passed since we last got data'. If it has, we execute the callback which passes through the message to the JAXL event handler.

Within your implementation of the handler callback you load your data. Whilst you are doing this, the clock is not ticking. That is to say the streams are not being monitored. Make sure your data loads quickly !

This is by no means ideal but it is about as good as it gets with PHP.

Usage

The usage of this setup is as follows:

• Pass the configuration parameter batched_data when you instantiate JAXL

• Add the get_next_batch callback to your client.

• Within the get_next_batch callback load your data and 'send' it using send / send_raw.

Alternatives

During my research I found a number of discussions questioning how to approach doing similar things with JAXL.

For example this one, and this one.

Abhinav Singh (the creator of JAXL) historically was quite active in responding to threads about JAXL.

In a number of threads I have found he mentions jaxlctl, and utilizing pipes. I investigated both of these options and found them to be not worth pursuing. That said.. have a play, and if you produce anything interesting I would be intrigued to see it.

As for JAXL in general. A post on Google Code and the lack of commits in the past few years suggests that Abhinav has stopped working on JAXL :(

Conclusion

It is a pretty simple conclusion really.. JAXL can be used with Google CCS, and a solution for a persistent connection whilst polling for new data is possible. It has a few drawbacks, but this code is being utilized without issue in a production environment.

Hopefully some of the people trying to implement such functionality will stumble upon this post.

If you have any questions/comments, I would be happy to answer them :)