December 22nd, 2022 Inventing an ugly programming language By Jeffrey M. Barber

Compared to some languages, my language is ugly; just an ugly baby that only a code-father like myself could love. First, I’m embracing curly braces which for many may be insulting. Second, my type system is not as great as I would like; this irks me, but I’m working on it. Third, I’m really accepting a large and complex set of built-in types into my heart, and I’m reminded about RISC vs CISC instruction sets. Compared to many languages, my language appears to be a step backwards. Today, I want to illustrate three things my language can do that your language can’t and why the steps backwards are worth it.

Memory management

As of now, I’m cheating and using the JVM. That’s right, I’m a dirty cheater just translating from my language to Java on the fly. I thought about rewriting this and using WASM as the core platform. I mean, all the cool kids are using WASM, and I want to be a cool kid, right?

The reason I stopped pushing this direction was because going the distance with a language is beyond a full time endeavor. Instead, I got to a “good enough” point which allowed me to make progress on the total picture. By focusing on producing secure Java code, I can avoid many security pitfalls that befoul language designers which prevent productizing the language in a multi-tenant server-less context with shared memory.

There are a multitude of opinions of memory management from the wild-west of C, the cyberpunk dystopia of C++, the borrowing prison of Rust, and the la-la land of Java. I am being careful in not requiring Java nor garbage collection for the long term. Instead, I have a precise service boundary which allows me to spike memory usage and then wipe memory in bulk. This is very similar to a typical LAMP stack setup.

The long term memory storage in Adama is akin to storing a giant JSON object, and the interesting aspect is there is no “new” operator. Memory management becomes a problem precisely because you can bring memory into existence and then immediately forget about it like a red-headed stepchild. My view is that memory management should behave like a relational database. Instead of creating “new” objects, you insert data into a table, and then remove it. There is much wisdom in the relational model as you never really lose track of data.

Beyond the traditional reasons for simplifying and rethinking memory management, memory in Adama is tracked, transacted, and differentiable. Differentiable transactional memory is a super-power.

Moving beyond the process model

For a variety of reasons, system languages all look the same. They run from command line, use sockets, read/write files, and have similar standard libraries. The artifacts of these languages are homeomorphic to each other!

A key problem with the existing model is that state management is your problem as a developer. There is a tower of babel of sort in taking data to and from a disk or network which too many of us have had to invent layers for. Instead, the runtime should make it beyond easy to pause and resume an application such that state is preserved. Otherwise, developers are always contending with the durability of their state.

This requires the runtime to handle state, but this becomes a tricky problem because programs change. The OS is not much help as this solution will look like some VMware features where the block memory is preserved. Preserving block memory may help with reliability issues, but developers still have to contend with code changes during a process restart. There are multitudes of problems trying to solve persistence at the OS level for the current process model.

There are legions of ideas, services, techniques, and libraries to contend with state preservation. Adama solves this by changing the operating model for a program where the state is a first class structured citizen. This is a core reason the language is ugly as I’m exploring this space, and the exploration trumps everything cosmetic.

Conceptually, an Adama “program” has a memory model backed by a giant JSON object. The inputs to the “program” are messages, and this creates a precise service boundary. This service boundary allows the programs the freedom to move around. For instance, failures (whether a system failure or a platform deployment) manifest in the user experience as latency blips.

This also has the side-effect that programs can operate in space with solar radiation as ground control operators can send an erasure-encoded message, and the program can exploit determinism to process the message such that the state is driven to consensus in spite of bit flips by replaying the message against different regions of memory.

With Adama, there is no need to load or unload data. Furthermore, networking to users is built in at the service boundary, so there is no distraction to building products around the plumbing from users to data.

The discipline of the developer

Adama does not remove the need for developers to have discipline around changes to data. However, the difficulty of such schema changes is ameliorated in a few ways.

First, the tight coupling between the language and data means the bindings from the logic to data are statically checked. This means no SQL injection attacks, no adhoc queries scattered amongst the code base, no need to audit every query, and no need to coordinate with other teams.

Second, deploying an Adama “program” can be done slowly via different documents. The space of all “program instances” (i.e. documents) can be fractured by different versions of a program via consistent hashing. This allows deployments to be slow to limit blast radius and gate features.

Third, deploying to Adama provides the ability to lint across a spectrum of versions. The system can statically warn you about data changes which may be problematic. For example, changing a long variable to an integer variable may cause truncation. Developers are warned about potential one-way doors.

Fourth, Adama’s service boundary allows shadowing such that changes can be done in secret to understand the effects of every change. Developers can look for drift in state between two versions to look for bugs.

The core thesis is that the more the language and tooling do, then the less developers need to do. However, it does become more important that developers do the right thing.

The platform is what matters

Building any kind of digital experience has a lot of cruft to connect people to state and each other. At the end of the day, no one really cares what language a product was built it. They don’t care about the services used.

They care about the experience, whether it solves their problem, and the reliability of the solution. That’s it.

Adama is embracing a total vertical platform for shipping digital products, and I believe the languages we have today along with the infrastructure tooling to manage the artifacts of the language can be radically simplified.

The way build today is just too much for too little, and I believe it requires a rethink. While there are things that I’m not happy with, I’m curious about feedback and getting more people to play with the platform. Alternatively, you can follow me on twitter to keep updated on the platform, or join the discord and become part of the community I’m building.