The @protocol synchronization

By Muralidharan Padmanaban, Naresh Gurijala and Intiser Ahmed

Heading with title

What is synchronization?

In the @protocol world, your personal data is encrypted with your own private key and stored on your mobile device. Periodically, this data is copied securely over to a dedicated cloud server which only you can decrypt and read since you have sole access to your private key. Nobody else, including The @ Company can read your data. The process of maintaining identical copies of this data on the server and your mobile devices is known as synchronization.

Commit Log

Commit logs play a crucial role in the @protocol synchronization process. When you perform an action (create, update, delete) in an @pp, the changes get saved in your handheld device as key-value pairs. Each key-value pair is assigned a commit ID from the server, which is a unique number returned from the server for an update/delete operation in the @pp.

Real-time Synchronization in the @protocol

There are two main scenarios that utilize synchronization. The first scenario is when your device is offline. Updates are saved in your device but not synced to the server. If the device goes online, the saved updates are then synced to the server. The second scenario is when you have multiple devices: one device is offline, and the other is online. Updates from the device which is online are periodically synced to the server, but the device that is offline will not pull these changes from the server until it goes online. For example, let’s say you are reading this article on your handheld device and you’ve added 5 claps. The number of added claps will be synced to the cloud server. If you view the same article from another handheld device which was offline then comes back online, you will see that the claps number is updated.

Sync from device to secondary server.

Let’s describe a use case: @alice inserts her phone number, which is saved as a key-value pair on the device and added as an entry to the device’s commit log without a commit ID. When the @pp checks the latest commit on the server, it receives a null response because the server has not created a commit ID yet. Then the @pp sends an update command to the server to update the phone number @alice inserted. The server commit log saves this update with a commit ID and returns the commit ID to the local (handheld device) commit log. The @pp updates the commit log with the commit ID sent from the server. Now, the device and server have identical commit logs and are synchronized.

Diagram 1.1 depicts synchronization from a local device to a server.
Diagram 1.1 depicts synchronization from a local device to a server.
Diagram 1.1 depicts synchronization from a local device to a server.

Sync from secondary server to device

In this scenario, the cloud has the latest changes but the local storage is out of sync.

Let’s describe a use case: @alice has two devices. One device is offline, and the other is online. On both devices, the latest commit ID is 2. On device #2, which is online, @alice updates her location value as ‘california,’ creating a new key-value pair. The @pp syncs this key from device #2 to the server, updating the server’s commit ID with the latest value, 3. Now, device #1, which was offline, comes back online. The latest commit ID in this device is 2, whereas the latest commit ID on the server is 3. To resolve this discrepancy, the @pp updates device #1 with the unsynced key, adding a commit entry. After synchronization, both device #1 and device #2 should have 3 as their latest commit ID.

Diagram 1.2 depicts synchronization from a server to a local device.
Diagram 1.2 depicts synchronization from a server to a local device.
Diagram 1.2 depicts synchronization from a server to a local device.

Syncing only @pp specific data

The same @sign can be used to log into multiple @pps, each of which has a unique namespace. The @pp can sync data in two different ways: (1) syncing all the keys (different namespaces) and (2) syncing keys that are specific to an @pp(single namespace). The default behavior is syncing all the keys, but an @pp can override the default behavior by syncing only specific keys. To achieve this, we have to provide regex to sync data from the remote secondary. Ideally, app developers should use the app namespace as regex.

Diagram 2.1 depicts the steps that take place during an @pp specific sync.
Diagram 2.1 depicts the steps that take place during an @pp specific sync.
Diagram 2.1 depicts the steps that take place during an @pp specific sync.
Diagram 2.2 shows what happens when all @pp keys are synced together.
Diagram 2.2 shows what happens when all @pp keys are synced together.
Diagram 2.2 shows what happens when all @pp keys are synced together.

Device offline sync

Apps built on the @protocol remain responsive even when offline. When you update information on a device that is offline, the key-value pair is stored in the device. Once connectivity is reestablished, your device sends an update to the server, synchronizing it with the current server state.

Let’s describe a use case: @alice’s mobile device goes offline. After that, @alice adds her email address, which is saved on @alice’s mobile device as a key-value pair without any commit ID. When @alice’s device comes back online, a manual sync is triggered, sending an update command to the server, which in turn responds with a commit ID. The commit ID returned by the server is then updated on the local commit log on @alice’s device, ensuring that the device and server are in sync.

Diagram 3.1 explains what happens during an offline sync.
Diagram 3.1 explains what happens during an offline sync.
Diagram 3.1 explains what happens during an offline sync.

Data Resiliency in the @protocol

Data resiliency is the ability to recover data in situations when an @pp is uninstalled or a device is reset, including when your device is lost or the device suddenly resets. With the @protocol, you receive a key file during the initial onboarding process. When your device is reset and the @pp is reinstalled, the synchronization process ensures that your previous data is pulled from the server to the @pp so that the @pp can be restored to its original state.

Performance considerations

When data is synced from the cloud server to an @pp or vice versa, there are several performance considerations that have to be factored in to make the sync process seamless. We will cover the performance aspects in a separate article.

Authors

Intiser Ahmed (intiser@atsign.com) is a Student Ambassador for The @ Company

Murali Dharan (murali@atsign.com) is a Backend Developer for The @ Company

Naresh Reddy Gurijala (naresh@atsign.com) is a Senior Software Engineer for The @ Company

Learn more about The @ Company from our GitHub repo.

Now for some internet optimism

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store