January 10th, 2024 Happy Data Flow By Jeffrey M. Barber

I have a strong desire to radically improve all my documentation from the ground up. Since that’s a fantasy project right now, I wanted to take a second to document how data flows in Adama. This question was asked by members of a local community that I’m forming, so I made picture with the fantastic Excalidraw.

Happy Data

Now, let’s walk through these steps.

1. A user interacting with Adama has some software running in the browser (or mobile app) that connects to a document identified by a space and a key. A space is a collection of documents under the same specification (a text file written in the Adama Language). Here, the key represents a single document which is powered by an instance of the generated class from the Adama Language (the single Adama Specification powers potentially infinite documents identified by the key).

1.note If you are using RxHTML, then the channel name from the rx:action attribute on forms.

Current Winner: <lookup path="current_str" /> <br />
<form rx:action="send:my_channel">
    <input type="text" name="a_str" /> <br />
    <input type="text" name="some_id" /> <br />
    <button type="submit"> Attempt to Write </button>
</form>

2. This connection then allows the software to send in the language of the product (rather than a data mutation specific language). Since this is a short example to fit within a single image, the domain is some kind of string with a last winner write model.

3. The message send lands on the web server host which is then routed to the core stream after sanity validation; the WebSocket load balancer is at core a proxy and protocol adapter between WebSocket and the Adama Network Protocol. Both the space and key are leveraged for a routing decision against capacity management since the plan for Adama cloud is to allow the intermixing of small tenants and massive tenants as this reduces risk for massive tenants and lowers cost for small tenants.

4. The message send lands on the machine hosting the document.

5. The document is running bytecode for the Adama specification provided for the space. The message send will trigger the associated channel logic which will mutate the document. The state of the document is being monitored.

6. Once channel logic has completed without exception, the monitored document yields a JSON delta. In this trivial example, it’s the document as everything changed, but delta process allows for small changes in the document to emit a small change out. This change is then appended to a log for the given key.

7. The append goes into a write-ahead log on the disk and awaits an fsync().. Caravan is a simple single-machine data logger that can be thought of as a (space, key) map to a log of appends/snapshots. Each log has a configurable limit per space of the number of entries, and the standard TRIM operation is executed on a snapshot to provide a relatively efficient query (i.e. a list of updates followed up by a snapshot minimizes load() costs as the snapshot ignores the prior updates for satisfying a load()).

8. -> 13 After the fsync completes, every callback is succeeded up the entire stack and the user gets an end to end commitment that the write was successful. If the channel handler is async, then the the acknowledgement is still sent to indicate the job has started.