Elegant Reconnection with Phoenix WebSockets

In Chris McCord’s excellent Programming Phoenix, he describes how to handle disconnections in a WebSocket application. While the strategy shown in the book works, it suffers a tight coupling between the server and the Javascript-side of the WebSocket connection.

Here, I’ll show a more general purpose solution that can be used with any client, whether it be Javascript or any other platform.

The problem – getting the client state up to date after it reconnects.

Let’s quickly look at the problem and the solution outlined in the book.

In Chapter 10: Using Channels -> Handling Disconnects, the section begins with:

Any stateful conversation between a client and server must handle data that gets out of sync. This problem can happen with unexpected disconnects, or a broadcast that isn’t received while a client is away. We need to handle both cases. Let’s find out how.

Our JavaScript client can disconnect and reconnect for a number of different reasons. Our server might be restarted, a rumbler might drive under a bridge, or our Internet connection may just be poor. We simply can’t assume network reliability when designing our real-time systems”

The Solution

The book then goes on to create a system that uploads annotation entities, and their database id, per-entity. The Javascript client cleverly notices each annotation id, and keeps track of the last one received. If the connection goes away and returns, upon reconnection, the client sends the id of the last received annotation with its re-connection attempt.

In this way, the server is able to know to only upload annotation entities with ids greater than the id sent in the re-connection attempt.

Criticism

This is a perfectly workable solution but suffers a few shortcomings.

1. The Javascript code has to process the data it has received in a special way to keep track of the last id.
2. That special way has to be rewritten for every single client that uses the WebSocket.

A More Generic Solution

Thinking about the problem in a generic way, we can see that it boils down to this:

1. The server is in a certain state at the time it sends its message to the client.
2. During a disconnect, the server’s state will continue to be updated without the client receiving the updates.
3. During a reconnect, the server needs to know the old state and send the interim states back to the client to get it up to speed.

With that in mind, we can borrow some ideas from the “Memento” pattern.

Here’s the solution:

1. When the server sends a message to the client, it includes with the message, an opaque object (the memento) which contains the minimum information relevant to the current state of the server.
2. The memento is stored with the client unmodified and unexamined.
3. When the client disconnects and reconnects, it sends the memento to the server with the reconnection attempt.
4. The server interprets the memento and applies the new server state to the client.

In this scenario, the only contract between the client and server to implement this is the address of the memento (perhaps a special json key?). That small contract is the only piece of information to be shared among client implementations.

Revisiting Programming Phoenix

Modifying the Programming Phoenix example, we can modify the server to send up a map under the key __memento__. We can subsequently modify the Javascript client to save data from the __memento__ key every time it receives a message that contains it. When calling channel.join(), the saved memento is always sent down (again, under the __memento__ key).

Now we modify the server code so that every time it sends up a new annotation entity, it also sends a __memento__ object that includes the id of the last annotation it sends up. It knows it will receive that in the future should a reconnection attempt occur.

Advantages

1. All clients can implement this small piece of code to handle reconnection.
2. To support future features, the server can modify the structure of the memento without incurring a change to the client code.