Over the years, I’ve thought much about UIs from a low level, and for whatever mysterious reason I love building my own scene graphs, engines, frameworks, and what-not. In retrospect, this is an immature approach to a common problem which is the suck of development. Specifically, web development sucks tremendously for so many reasons. Worse yet, as we answer the siren call to make it better, we build a messy brittle unstable empire. We can take a look at the mess that is the current JavaScript front-end engineering body of work, and you don’t have to search far for frustration with npm, dependencies, poor quality, supply chain attacks, broken shit, abandonware, etc, etc, and etc. Generating complaints is like shooting fish in a barrel with a bazooka.
Worse yet, you learn it and it feels brittle as you may be one upgrade away from an evening full of despair. I experienced enough pain with Docusaurus that I just gave up as the time investment felt wasted. I switched to both mdbook for the reference site and Jekyll for this site. I’m also finding Jekyll as a nice way to just throw up a server to render some HTML, so I’m exceptionally happy with it. Suffice it to say, I’m not a happy camper with the majority of options to buy in the front-end space via npm.
Maybe, I’m just a grumpy infrastructure engineer. Maybe, I just want all these damn kids off my lawn with their shiny dog-soft-serve tools since vanilla.js was good enough back in the day.
So, what do I do? At the risk of angering the framework gods, incur the wrath of xkcd 927, and build yet another framework; I’m going to ask the hard question: where did we go wrong?
Well, to a certain extent, we have fully embraced the worse is better philosophy with the browser. The spirit of going to a site, reading a document, navigating links, submitting forms, and what not have proved to be exceptionally fertile ground. The critical cynic may claim that the fertile ground is full of weeds, and that’s OK and fair.
Instead of criticism of where we are, let’s go back in time and take a different path: suppose JavaScript was never invented because something else was available, then what would that something be?
My claim is that if bandwidth was similar to what we have now during the birth of the internet, then all web developers would be citrix developers making apps that are simply delivered via the browser inside a frame buffer app stream. Maybe that is madness, but this is not too far off of where games may headed if stadia or other streaming efforts make substantial inroads over the coming decades. While I’m doubtful of streaming games for many reasons, I believe it is a worth while thought experiment to ask what is the gap between HTML and a frame buffer stream. The faults of the frame buffer stream are opportunities to fix HTML.
The stream is important, and WebSocket is a potential ally. However, woe unto you for using a WebSocket.
Fixing HTML must start with the task that every HTML developer eventually invents: templates. Recently, I did a deep dive into a variety of template engines. One big pattern is that many template engines rely on a host language to execute along with some new (and sadly novel) meta language. The host language tends to the be the major difference between them. A key exception are templates which are logic-less which have the supreme advantage that they are data with only a lean meta language to take a data object into HTML.
Now, at the end of the day, all of these offerings are a tool to turn your data into HTML. That’s it. That is the bulk of the entire front-end technical game: writing lots of code to turn data into HTML and capturing interactions into data. Fortunately, HTML is a tree and we play an ontological game of classifying the elements/components that get rendered. Roughly speaking, there are three types to contend with: built-in, canvas, and a composition.
The built-ins are what you have by default like h1, div, video, span, etc. They have predefined rendering rules which can be overridden by CSS. Canvas elements are where the developer have complete control to render anything within a box. Composition are the mixture of built-in and canvas elements to produce a built-in like experience, and with Web Components allowing developers to ship new things that behave like built-ins. Web Components are an exceptionally important and critical puzzle piece, and they are the corner stone of the solution envision. I’m a big fan of Web Components, and I believe React/Vue were stepping stones towards this as a standard.
Since software is a people business, I believe we can see a fault-line where many computer savvy individuals can weave together the built-ins and build a static HTML page; these people can also include the need dependencies to have custom or complex components via a marketplace. People can share snippets and templates to their hearts content. I’m basically building all my websites now using tailwind by copying and pasting snippets together. However, developers are really needed to build the rich components and custom canvas solutions since that involve deep code and highly tuned state machines.
If most people can build a static HTML page, then what is the path towards building a full application? What minimal things do they need? We return focus to what empowers a streaming application over a remote desktop protocol of WebRTC video stream like stadia. If we replace the frame buffer with HTML, then the results could be something like HTMX or Hotwire, and I do believe these push us forward. However, they both rely on back-end programming.
If I wasn’t building Adama, then I would definitely explore the differences between HTMX and Hotwire deeper. However, I’m building Adama and this blog, at its core is a content funnel, so I need a different solution because Adama fundamentally is a real-time document store for richly typed JSON documents. I am deriving inspiration from HTMX as it illustrates that HTML can be fixed with a minimal framework.
This has me leaning towards mustache or handlebars which are great for turning JSON into HTML, but they have a serious problem in that they require yet another language. This additional language makes the templates generic and sharable between different languages in different context, but it creates a problem for building HTML correctly. Fortunately, the language is simple enough that I can copy core ideas and avoid creating another language. Instead, I’ll just extend HTML and invent a small language for HTML attributes as needed.
Since I’ve started to play with this more, let’s build a table of how ideas can translate from mustache to RxHTML. This starts the design document where the game is to introduce new HTML elements which influence some state behind the scenes.
| mustache | what it does | HTML element | HTML attributes | requirements |
|---|---|---|---|---|
| {{path}} | Brings the data from the given path into the document escaped. | <lookup path=”path” /> | <div class=”prefix [path] suffix”> … </div> | None |
| {{{path}}} or {{&path}} | Brings the HTML from the given path in the document | <lookup path=”path” html /> | N/A | Requires a single parent |
| {{#path}} … {{/path}} | If the path is a Boolean that is true, then render the enclosed text | <test path=”path” /> … </test> | <div class=”prefix [#path]active[/path]”> … </div> | Requires a single parent |
| {{#path}} … {{/path}} | If the path is an object, then scope into the object and render the enclosed text | <scope path=”path” /> … </scope> | N/A | Requires a single parent |
| {{#path}} … {{/path}} | If the path is a list, then iterate the children and scope into each child rendering the enclosed text | <iterate path=”path” /> … </iterate> | N/A | Requires a single parent and a single child. |
| {{^path}} … {{/path}} | If the path is a Boolean that is true, null, undefined, or an empty list then render the enclosed text. | <test path=”path” invert /> … </test> | <div class=”prefix [^path]active[/path]”> … </div> | Requires a single parent |
| {{>template}} | Use the given template as a placeholder for sharing common aspects | <use name=”template” /> … <template name=”template”> … </template> | N/A | Requires a single parent and no children |
The requirements are design-smell since they illustrate some non-trivial errors, and so these warrant study. For example, instead of introducing new elements I can introduce new attributes which HTMX does. I’ll backtrack when I start to execute and play with it.
This table is fairly small since Mustache is exceptionally minimal (and stable), so that’s nice. At a core level, the new HTML elements along with the tiny language for attributes allow for a near bijection between RxHTML and Mustache when the medium is HTML. The open question then is how to make it interactive, so we start with a focus on forms. Fundamentally, a form should flow from the browser to server and then reactively update the HTML.
With Adama as the backend, we simply need to have a form send a message to Adama, and then Adama will update the JSON document via reactive magic. This implies that the Rx in RxHTML has the HTML tree bound reactively to changes at the DOM level. We create a new element called <message> which will capture the input elements, and we will introduce a <send> element to create a button to submit. We introduce new elements to minimize confounding factors (like embedding a form to execute a contact us against a third party). Maybe, I just need to fix form? The game that is being played will require tasting it when it can be played with.
With just these minimal extensions to HTML and the power of Adama’s reactivity, entire classes of applications are possible because the read-write loop is present. However, we can go further asking a few more questions.
The answers to the questions are interesting as I ponder them, and instead of wandering around them, I’ll just work on it and release a specification.
Well, I’m hacking on rxhtml within the Adama repo, so you can see my current progress. When I start to get 100% coverage, the semantics and behaviors are precisely what I want and I’ll release a specification. I feel it will be fertile ground, and the key escape hatch is WebComponents. As an experiment, I’ve already wrapped CodeMirror 6 into a WebComponent, and I intend to build all of the board game IDE within RxHTML.
The board game IDE will enable making webpages as that allows me to move faster, and if I achieve full mutability then there is a minimal bootstrapping period where the entire IDE can be built within the browser. There are varying degress of meta-ness, but my hope is then build an IDE which can then build another IDE for making a no-code solution which can then pump out entire websites. That is, I’m making a website which can create website makers which can then go and make websites.
I’m excited, and if you are too then join the community and stay tuned.