Frequently Asked Questions about our private, open-source protocol
The atPlatform is a clever abstraction machine. Perhaps the most abstracted component in the entirety of the atPlatform is its backend service. Very often, we like to say that the atPlatform is an “automatic backend service” with no more “database management”, but what do these features specifically entail? In today’s episode of “Ask the Founders”, Philippe Cutillas, an atPlatform developer and one of the winners of our @ppathon II event, had an excellent question for two of our co-founders Kevin Nickels and Colin Constable to answer on this topic.
P: I remember hearing that currently, servers tied to atSigns are 4 MB, and I was wondering what the plan for expansion is in the future? I could think of a couple, like just expanding storage as demand grows, having different yearly subscription price tiers for different storage levels, or even selling your own standalone servers that people could have next to their routers or something, but I just wanted to check what your guys’ plan is so we can plan our growth with it too!
K & C: First, as to the storage question:
Secondary Server Data:
Our Secondary Servers currently use a native Dart database for persistence (hiveDB) and they both run in individual docker containers. Storage for these containers comes from a shared NFS (Network File System) file system. This means that there is a very large amount of disk space available to store information and we do not use that much of it, so storage is not much of an issue as far as data is concerned. Currently, this includes the data for the Secondary Server itself as well as server logs.
Binary Data:
Large amounts of storage are, however, needed for binary data (images, videos, etc.) and we must have a strategy for securely storing encrypted binary data.
While the database that we use, hiveDB, can support a limited amount of storage for encrypted binary data, it will not provide the performance and scale needed.
We have recently been working with the developer of FileBin to deploy their service in Kubernetes as an option for securely storing binary data. We will be publishing SDK updates and some new libraries very soon. Keep an eye out for news on Discord.
We are pretty excited about The Interplanetary File System and have a POC for that integration currently working as well.
K & C: Second, as to the memory question:
Our default memory resource allocation for each secondary server has been recently increased to 6MB each, which is only the minimum amount. The docker container has the ability to flex that up to as much as 24MB as needed. The database is an in-memory database and the amount of storage used directly affects the amount of RAM needed for in-memory persistence operations.
Essentially, we have taken the responsibility to provide infrastructure resources to maintain performance and minimize the cost. We continue to tweak the configuration to strike the right balance without compromising the integrity of the service.
K & C: Third, as to where Secondary Servers can be deployed:
As you know, the local development environment runs the self same root and secondary code that we run in production. Check out the GitHub repos as well as the dockerhub and pub.dev packages.
We have always hoped that others would choose to run Secondary Servers on their own. This applies to a wide range of use cases:
- Individuals that want to directly monitor and control their Secondary Server (note: this requires either a static IP address or dynamic DNS service to be configured to work either at home or in the cloud).
- Infrastructure providers that may want to provide value added services (such as more memory, storage, etc.) on their infrastructure and charge for those services. We do not make any money from this, but it would be a useful thing and we are all for it.
- Applications (likely yours is one of those) that offer shared services for a large number of others and need more scalability for those services.
- Attestation providers that provide “bridges” to other data which can validate the truth of some information without having to expose the underlying personal identifiers and can even be done anonymously.
We are aware of a couple of developers that are working on “enterprise” class Secondaries (one written in Java and another considering a NodeJS implementation). It would be great to work together with you to figure out the best solution for your application and whether one of these projects might fit the bill or if we need to work on something new.
Our preferred solution is to stick with Dart but have the backend database be something more robust like Redis and use Kubernetes to scale up the number of Secondary Servers pointing back at the shared Redis DB. This is close to completion and means the same code gets reused and can be scaled horizontally.
Atsign is a tech startup committed to transforming how the modern Internet treats people’s data. To learn more about Atsign and their mission, check out their website.
