Emacs lisp as an interface definition language

Then your vocabulary of UI interactions isn't restricted to what is available in the chosen systems programming language, but is domain-specific. A component is different to a data structure. It also isn't a function. It's its own thing.

Most languages would struggle with representing this as a concept, but the Lisp-family has a very natural way of representing these ideas. Everything is a form. A form can be a function call, an aggregated list, a structured slotted object, an instruction to change the value of a variable, a macro… the list goes on and isn't at all restricted to what I just described.

What's more, the forms in Lisp are not restricted by the parser. You can define new kinds of forms as long as you can explain to the interpreter what those forms mean.

Take for instance use-package. Syntactically speaking it is a macro. Semantically this is not what we see, though. Just like the famous character in "the Matrix", we don't see macros, we see packages, we see configuration, we see customisation variables. And this is exactly what QML strives to achieve and what it is mostly praised for.

The main area where QML indeed struggles is when it comes to interactions with the rest of Qt. You are not supposed to go and change the UI from the backend. The trouble then is how do you create complex interplay. How do you load an abstract representation into memory and signal that to the system.

So in Elisp, the equivalent to the view.qml would look something like this.

  (application-window
   :id 'main-window
   :width '(:min 400 800)
   :heigh '(:min 200 600)
   :visible-p t
   :header
   (tool-bar
    :visible-p (lambda () (gt (get-property 'height 'main-window) 400))
    (tool-button
     :id 'decrement-current-index
     :visible-p (item-at (- (get-property 'current-index 'main-window) 1))
     :text (when (visible-p 'decrement-current-index) (item-at (- (get-property 'current-index 'main-window) 1)))
     :on-clicked
     (lambda () (decrement-current-index 'main-view))
     :anchors (:left (left 'parent)))))

One can argue the readability of the proposed API. For a zeroth-order approximation, one should note that a surprising amount of problems is simply resolved. Elisp already has symbols. Elisp already lets you define polymorphic inputs as well: the :visible-p keyword is set to the boolean true in the first place, and to a function that evaluates to a boolean in the second.

Next, we attach properties in a structured fashion. The :anchors and :width keywords are set to structures. Except we are making use of polymorphic input. If we give it a number, it becomes a number. If we give it a structured value, it must follow a specific pattern. We can depart from this further, by the way, we can specify a single :size that can be set to a list of lists and contain the right data.

In case of QML, the best way to do this is with language bindings and a "real" programming language. You are not expected to extend the JavaScript parts of the language, you are supposed to create global-scope variables that behave a certain way. This means that a considerable amount of preparation has to be done by interacting with the interpreter on the back end. In our case, it is the definition of what a model is for the LoadWindow.

Specifically, consider this snippet of code.

  LoadWindow{
  	id: loadWindow
  	fileModel: samplesModel
  	anchors.fill: parent
  	onBrowseForFile:{
	  fileBrowse.visible=true
  	}
  	onRequestLoadSamples:{
	  fileModel.appendRow(filename)
	  samplesView.visible = true
  	}
  	anchors.centerIn: parent
  	anchors.leftMargin: 8
  	anchors.rightMargin: 8
  	anchors.topMargin: 8
  	anchors.bottomMargin: 8
  }

We are creating a re-usable component known as a LoadWindow. This component is initialised with the global value samplesModel as its fileModel parameter. Internally this gets forwarded to the ListView component, which expects an object with certain behaviours. These behaviours are consistently the worst aspect of programming in Qt, and this is partly due to the API being designed in a very peculiar fashion, but exasperated by lack of decent documentation. I'm still not sure I'd be able to implement a model for a generic list view. In principle the way to avoid this problem is to load the data from JavaScript, but the question then becomes how to translate these components and how to call backend code.

Simply put JavaScript's understanding of objects is not sufficient to manage objects. As such, it is often neater to spend a bit more time creating a purpose-built abstract model for something that can be done with off-the-shelf components, rather than trying to embed the domain and fill the data from the JavaScript side. This is not how it was intended, but is often how it is done.

So how does the model work? In Python, the implementation is the least ugly in my experience.

class ParameterModel(QtCore.QAbstractListModel):
    """A model representing parameters in a nested sampling run."""

    nameRole = QtCore.Qt.UserRole + 1000 + 0
    texRole = QtCore.Qt.UserRole + 1000 + 1
    selectedRole = QtCore.Qt.UserRole + 1000 + 2

You create an object that inherits from QtCore.QAbstractListModel and override the methods that you need. This unfortunately means that you still need to define enumerations for the different data roles and this is a concept that is easy to understand once you see that the QAbstractListModel enforces a specific conceptual framework. You are dealing with data in a table, it has rows and columns. Nesting is suitably complex to represent and is also the most common pattern. As such, to define the columns, it is not enough to rely on the structures being identical, they need to play a certain kind of QtCore.Qt.UserRole, so we do that.

Next we define some properties. The easiest to understand is probably isEmpty. We create a new QtCore.Signal(), something that gets sent when something happens that says "the value of isEmpty is no longer what it was, so any UI logic that needs to update, should update." We want the value to not be stored as a boolean. In C++, this is done by a getter method, but in our case, Python already has indirection for non-slot members, so we can create a short-hand function. This returns a Python boolean that is transparently converted into a JavaScript boolean.

    isEmptyChanged = QtCore.Signal()

    def _isEmpty(self):
	  return self.rowCount() < 1

    isEmpty = QtCore.Property(bool, _isEmpty, notify=isEmptyChanged)

Next for the model to work as expected, i.e. to be readable by all the APIs that expect a Qt model, we need to implement a few functions: the role names need to be accessed via standard API; which we do here.

    def roleNames(self):
	roles = {
	    ParameterModel.nameRole: b'name',
	    ParameterModel.texRole: b'tex',
	    ParameterModel.selectedRole: b'selected',
	}
	return roles

Our data is that there is a name for a dataset, a (La)TeX representation tex and a boolean selected. This is meant to represent different parameters in different datasets. For most of the work I used it for, this would include things such as the Cosmological constant, the baryon density parameter, the dark matter parameter among a few other calibration parameters for the particular telescope.

Now suppose we found a way to internally represent the data. The API needed to access it for display is where we need to spend the largest amount of time.

    def data(self, index, role=QtCore.Qt.DisplayRole):
	if 0 <= index.row() < self.rowCount() and index.isValid():
	    if role == ParameterModel.nameRole:
		return self.names[index.row()]
	    if role == ParameterModel.texRole:
		return self.parent.tex[self.names[index.row()]]
	    if role == ParameterModel.selectedRole:
		return self.names[index.row()] in self.displayNames

Because we don't want to handle different columns, we simply look at different rows and postulate that every element of this model has data representing their name their tex and their selected status. Incidentally if you are wondering why I need to store this information in the model, this is because I want to be able to synchronise the behaviour of different components. Selecting a parameter in one dataset should scan for another parameter of the same name and same tex in a different dataset.

Convoluted, right?

It gets worse. Especially if you are nesting data. Especially if you are writing the program in C++, which has stronger typing and fewer convenience features. There are no decorators in C++, for example.

This is very much a problem caused by the rigid API imposed for compatibility's sake. Qt needed a way of representing the same data in multiple ways. This had to work around C++'s constraints and typing system. This meant that you had to do things a certain way. This meant that you needed the concept of a Role and it had to be implemented the way it is in the code.

This also meant that you needed to comply with Qt's other peculiarities. Qt has an ownership system. Unlike Rust, however, this is not enforced at the language level, it is something that is enforced by coding convention. In other words, you have a few opportunities to forget to properly initialise the parent, and if you do it wrong, congratulations, you have leaked memory.

    def __init__(self, parent=None, columns=[], tex={}):
	"""Construct."""
	super(ParameterModel, self).__init__(parent)
	self.names = columns
	self.tex = tex
	self.displayNames = {}

The C++ for this would be more verbose, but no less convoluted. There are quite a few things that you cannot do in Python that you would be able to do in C++, which would entice you to bite the bullet and do it "properly" but there is no chance that you could use a program that used dynamic libraries, and QML.

Lisp, by contrast, can do away with most of these problems.

What would a similar API look like in Lisp? Everything is a list, so most likely just lumping things into a nested list should work.

((:name "lambda" :tex "\\Lambda" :selected false))

is simply a natural thing to pass. Lisp is polymorphic. It can advise functions. So instead of relying on a rigid QML interpreter, we can rely on changing the behaviour both on the sending and receiving end. The polymorphic specification here gives us that flexibility. Indeed we would enjoy the same flexibility if we were working on the JavaScript end only. Unlike Elisp, however, the version of JavaScript embedded into QML has a limited extensibility, and even more limited set of packages.

So what's next?

Suppose we indeed needed to do a bit more work. We needed a custom made model. How would one go about communicating the information in a particularly reliable fashion?

One way is to expose the low-level API for the most general case and ensure that the end user is able to understand this well. The Qt model/view paradigm gets the first part but not the second.

In Elisp, however, one has a third option. Elisp macros are routinely used to create forms that can do multiple things, like set up and wrangle global state, register something when the time comes and add a hook when one needs to. Unlike C++ where your only choice to create a parameterised model is to subclass, Elisp gives you a wide array of tools to create a good user experience. So for example, to represent a nested model, one could have a macro:

  (make-model
   :roles ('name 'tex 'selected)
   :properties
   ('is-empty . (booleanp (lambda (self) (row-count self)) :notify ('is-empty-changed)))
   ;; Realistically, we will do some trickery to fill in the mentioned, but empty bits automatically.
   :data
   (lambda (index role)
     ;; todo
     ))

Note that this approach is:

1. Much easier to interpret. Roles are symbols and won't overlap with other symbols. Unless we explicitly ask them to.
2. Much easier to write. You can literally tab through the macro and it the documentation will tell you what each keyword is doing.
3. Much easier to debug. Literally, if something is not sensible, we can raise an exception in the macro itself. In Qt, you must first run this and maybe get a view that shows nothing. Maybe get a segfault instead.
4. Much easier to extend. For example, notice that we lumped the roles into the definition. Things pertaining to the same definition will exist within the definition. Want to add more keywords? Well, you're not breaking anyone else's code in the process, and you can still have a sensible default that depends on the other parts of the input.

The real meat and potatoes of the QML system is its rendering code. Simply put, it is the most efficient one can come up with. It outperforms most browsers. The main reason is that there is a scene graph and retained mode updates. There's also hardware acceleration. There's also a sophisticated convention on properties that allow you to know when to update certain parts of the retained mode GUI, when to make parts of the model/view invisible and de-load the objects from memory.

These can be ported to Elisp, but more likely re-implemented. I firmly believe it would be in everyone's best interest to have a diversity of implementation.

That being said, there's a lot from QML that we don't necessarily need to replicate rendering-wise. For one, the update system is contingent on values going through the property interface. In QML, because of the native code that is being run, and the ABI constraints, one can only rely on one form of interoperability – a rigid interface. A property has to follow a certain convention. There needs to be a notification that a value has been updated. How these updates are propagated have to be controlled by the programmer, so there is a zoo of functions that realistically only exist as functions, because the lambdas can't cross the ABI barrier. Further, the limited polymorphism is further limited.

In Elisp, there isn't a compile-time run-time barrier. All Lisp code exists in both kingdoms. This gives us a great advantage. In QML, the frontend and the backend are usually segregated into code that has to comply with a rigid ABI and a segment of code that lives in the dynamically typed land, but can't precisely make use of that freedom, because it doesn't exactly do much. In Elisp, the situation is radically different.

Whether or not we can make use of this fact is another matter. This, if anything makes us more flexible than e.g. QML. This may come at a considerable performance cost, but has a few benefits as well.

Consider where the QML development experience is lacking. Installing third-party components is not very clear cut. Emacs, by contrast offers use-package as well as facilities to make specialised package managers. Registering new types is a pain. Emacs lisp lets you do that much more easily and hide the installation behind a macro.

As yet another example, consider how a complex negotiation between the model and its view can be negotiated. In Qt, the pre-existing model/view segregation is rather challenging to wrap your head around. It helps to think of these as tables, and having done this twice in different languages (Python and C++) I will say, it is not a wholly inscrutable sphinx, just mostly inscrutable. These limitations arise from prior constraints of prior systems that need backwards compatibility, but also because of the limited ability of most programming languages to generate code at compile time. Lisp and Zig can create wonderfully simple methods of creating better more customised models.

Let's be more precise. Something like sending a signal whenever a value is updated is not something that is absolutely necessary, but if our system were to keep it, those can be accomplished with advice. As could the handling. There is a natural correspondence between signals and slots as well as hooks and pretty much everything that can be attached to them, namely functions, callables, and lambdas. Assuming we can retain the generality, QML's requirements become baroque.