Scaling to the moon
By Jeffrey M. Barber
Here I am with two servers and a service in early access, and I’m thinking about massive scale. And, by massive scale, I mean the entire planet filled will billions of devices. This kind of scale is the one thing that I’ll probably never achieve again since leaving Meta where I was the architect of how Meta does real-time. Have you read my former team’s SOSP 21 paper on BladeRunner? It’s a fun read. Regardless, I was wondering if Adama could achieve massive scale, so let’s think loudly.
Cleaning up a multi-region mess
By Jeffrey M. Barber
I’ve started to make a mess of things. Fortunately, it is recoverable, so come along as I untangle my multi-region mess. If I haven’t mentioned it here, then the internal plan is to have Adama run on the edge close to people. Ideally, I want a heterogeneous fleet of clusters spread across the globe running on different vendors. I’m evaluating using PlanetScale for the control plane, and it’s looking like the control plane will be located in the US where I may shard at a tenant level. I’m still thinking about the database, but the open question at hand is how to build the multi-region data plane. As a design detail, documents are stored with Amazon S3 and then rehydrated into one region with periodic snapshots, so keep that in mind. In particular, documents will live on a single host which I’ll write about more at a later time.
One does not simply call a function
By Jeffrey M. Barber
July was exceptionally productive on many fronts. The IDE is coming along, and I’m shooting for a second early access launch with the web portal. RxHTML is shaping up, and the tension between the purity of doing everything with RxHTML and needing a few escape hatches here and there has been illuminating.
Adama now offers basic website hosting, limited CDN functionality, secret management, collaborative text editing with operational transforms, and has the foundations for calling other services. As I’m exploring how to have Adama talk to itself to become a full actor model, trust has become a central issue.
Towards a serverless Actor Model
By Jeffrey M. Barber
As I was building the IDE with RxHTML, I ran into a problem that I’ve been thinking about for a while: escaping the silo with service calls. Suddenly, a nice-to-have or neat-to-think-about feature became a priority zero. The motivation came from attempting to keep RxHTML minimal without adding too many new components or javascript-code. For example, just creating a space and document at the same time is problematic.
The less I want RxHTML and the front-end to do, the more I need Adama to pick up the slack. Unfortunately, Adama documents are isolated from the rest of the world and even other documents.
Woe betide anyone that uses MQTT (at scale)
By Jeffrey M. Barber
As I wander, I’m drafting new marketing around the real-time aspects of this platform. This approach is more relatable and easier to explain, but I feel it comes with expectations around integration. One integration would be slapping an MQTT front-end to adapt MQTT clients to the platform. The benefits would be to simplify integrations as there are many people familiar with MQTT and have it within their stack already. The drawback is that I spent multiple years strangling MQTT at Meta where I (regrettably) patented a new protocol.
The path of Monastic Code Machine
By Jeffrey M. Barber
Ever since I was in sixth grade, I’ve been drawn to the machine like a moth to the flame. Last year, I turned 40 and am trying to stay retired from big tech. There are things that I miss like helping engineers blow past their limits, and there is the intoxication of massive scale. There are plenty things that I don’t miss which is why I recently completed six rounds of ketamine infusions which has been a fantastic introduction of psychedelics.