XUL Templates in Mozilla and compatible technologies are a waste of time. They exhibit none of their supposed benefits and exact a high cost on all who would dare to use them, both due to the use of a unusual paradigm and near complete inability to create or use them without superhuman patience.

What should you use instead of XUL? With the exception of browser extensions to Mozilla, you should use HTML, JS, XBLinJS as needed, and JSON-type technologies.

That's the summary, let's examine and substantiate these claims. (Note this is about XUL templates... while XUL doesn't exactly get my blood pumping it has by far the least critical essay written about it.)

(Also note this essay was written against Mozilla 1.7; some of the precise technical complaints have been at least partially addressed since then, although I can not vouch for their utility or stability. I think the general thrust holds true, though.)

• My Experience With XUL
• XUL Template Claims
• Template Technology
• 1: XUL Templates Have The Shittiest Implementation of a Logic Language I Have Ever Seen, Part 1: No Predicates
• 2: Multiple Rules May Not Interleave
• 3: XUL Templates Have The Shittiest Implementation of a Logic Language I Have Ever Seen, Part 2: Only One "members" Allowed
• 4: Error Reporting?
• 5: Bugs Relating To Not Being First-Class Data
• 6: The Champion Has Left
• 7: Misc Other Weaknesses
• The Solution To XUL Templates
• 8: Mozilla's RDF Implementation Seems To Suck
• Conclusion
• Less Extreme Solutions

I worked with XUL for the six months, fulltime, building a highly adaptive application that like much else in the world boils down to a database modifier.

While I may only have been using XUL seriously for four months, I would not use that as grounds to discount what I have to say here. Little or none of what I have to say is of the nature where you can solve it with emailing me a series of magic incantations.

In these four months, I have tried to use XUL for complex layouts, trees with dynamic columns (quirky), trees with dynamic data (works but just barely), trees with both dynamic columns and dynamic data (completely impossible due to fundamental flaw in Mozilla), data binding with XBL widgets, data binding with XBL widgets with multiple template rules (failed due to fundamental flaw in Mozilla, XUL template specification, or both), and many other relatively complicated scenarios. Over and over again it is the same story: If you try anything beyond the simplest case, you will fail because XUL Templates are not powerful enough.

I have concluded that the theoretical XUL template system is marginal at best (assuming a perfect and complete implementation), and that the currently existing implementation in Mozilla is borderline useless.

Here's what XUL templates are supposed to be able to do, straight out of Creating Applications With Mozilla:

XUL templates are dynamically generated XUL elements and groups of XUL elements. They are often used to render lists and tables that display mutable, frequently updated data, such as your Inbox, your list of bookmarks, and user profiles. A XUL template can be used to create something as simple as a list of menu items, as you will see here, but it can also be used in much more exciting ways, as shown at the end of this chapter. You should consider using a XUL template instead of XUL when you want to create an interface that displays data, such as a roster of names, when the set of data is very large, when the data may change frequently, or when you create a display that you want to use for different sets of data.

Mark that list of how they are often used well, because that is about all they are useful for, and even then only because the Mozilla project has put so much effort into optimizing them for those uses. (Somehow, other languages and environments manage to do all of those things without basing the system on "templates"...)

Understanding XUL templates based on official documentation can be a pain because of the huge lump of terminology you have to absorb all at once, so let me break the technology down for you. XUL templates specify a logical matching language on top of (RDF) graph data. You can create several rules and match a given piece of data on the first rule, allowing "fall-through" to an appropriate rule. Each rule will then perform an action based on the data it matched.

Conceptually, let's suppose you had a list of your bank account transactions in one of the graph nodes, which of course has some positive and some negative transactions. You can create a XUL list template that says this:

• If the amount is negative, display the transaction and color it red.
• Display the transaction.

For a list of transactions, then, it might display this:

As you update the data, this table will update as well. Sounds pretty cool, eh? Logical matching languages have never been popular, although they have been around for a long time and have a mature theory behind them; if they are the best tool for the job then so be it.

Isn't that an easy example? If you just read the XUL template chapter of Creating Applications With Mozilla, don't XUL templates sound like the way to go?

Unfortunately, this is impossible to do with XUL templates for multiple distinct reasons.

The whole core of a logic language is the predicates it supports. Ideally, you can even define your own. A "predicate", in case you don't know, is something that looks at a value and returns true or false.

In the example above, I implicitly referred to the predicate "isNegative", which will return true for a negative number.

But you see, XUL templates actually don't support that predicate. In fact, think of a useful predicate, and XUL doesn't support it. String comparisions? Not supported. Any numeric comparisions at all? Nope. ... uh, well, that pretty much runs the gamut so I'm out of rhetorical points here.

The only predicates that XUL templates support is "existance" and "equals"; you can either match that an attribute exists, or that an exact value exists for that attribute. Therefore, if you want the behavior in the example above, you must ship an extra attribute on the negative data, "isNegative" perhaps, and completely omit this attribute from positive data. That's right, there isn't even a predicate for non-existance; the predicates aren't even closed under logical negation, to dip briefly into math terminology. In a lot of datasets, this is unnatural, and impossible if you're trying to use pre-existing RDF data other than Mozilla's internal RDF data, which is tuned for this unusual weakness.

It therefore goes without saying that you can not create your own composite predicates, unless you can munge the data in advance to contain exactly that predicate as a piece of data.

We would actually be better off even if we had Brainfuck available for predicate specification, and that is saying something indeed.

To the best of my knowledge, this is fundamental in Mozilla right now and there are no XUL-based workarounds.

If you have multiple rules in the dataset, they may not interleave; Mozilla will first display all data for one rule, then all of another, and so on down the line.

If we assume for the moment in my bank account example above that those transactions are in temporal order, and we want them to stay that way on the screen, we have a problem. Mozilla will first display all of the negatives in order, then all of the positives. As you might imagine there are only a limited number of datasets where the order doesn't matter at all.

It is this point in particular that at least as of this writing I find the Creating Applications With Mozilla book to border on the outright deceptive, as all of their sample data and sample rules are already in order relative to each other. It isn't until you download the examples and start playing with the order of the data in the RDF file that you learn the order can not change. (Of course, including this information in the chapter would make the technology look significantly less enticing.)

This is technically a Mozilla implementation flaw, but XUL has some wonky aspects here that may contribute to this flaw. (It probably ought to have the uri="?whatever" move out of the template, make the template a direct child of that element, and make all action specifications directly relative to that. Yes, there are implementation issues with that, but it is the only way I can see to make logical sense of multiple rules without hacks or kludges.) To the best of my knowledge, this is fundamental in Mozilla right now and there is no XUL-based workaround.

In order to "loop" through the data, you can use the "members" declaration, which says "match things that are a member of something else". This is the only way to have more than one match, so in the bank example above, each "transaction" is a member.

However, you are only allowed one "members" clause. I tried to create a general "search" results viewer, where the user would run a search and I would recieve a list of objects, where each object is itself a list of a varying number of properties and metadata about those properties.

Conceptually, it is relatively easy and I am reasonably confident that I wrote the XUL template that would be required correctly, with two "members" clauses and two looping elements inside the action. However, Mozilla can not go down two levels like that, and I ultimately had to pull one level up to the server side.

So not only is the logical language we are forced to use incapable of using or expressing any sort of advanced predicate, it can only "loop" down one level? Is there anything this language can actually do?

(Oh, bye the bye, not going to call this a seperate issue, but while you can bind values and use them later in the XUL template, you of course can not manipulate them in any way or use them in any composite fashion, so if you intend to use a value for an attribute on something in the XUL template, it had better damned well be exactly what you need it to be already in the data or you are out of luck. Minor exception: Values used as "text" nodes in the action can be interleaved with other text nodes.)

Error reporting? Who needs that?

Is your XUL file not quite compliant with XML? Good news, that will be reported. Don't get used to that, though....

Is your RDF file not correctly formatted? Silent failure. ("Silent failure" means the template is not rendered at all, with no feedback to the user or developer whatsoever.)

Did the RDF file in fact fail to load at all, because of a server error? Silent failure.

Is the node you're looking for not actually in the RDF file you're trying to load? Silent failure.

Is your "action" statement trying to do something unsupported? Silent failure.

Are your predicates incorrect at all? Silent failure.

Did you use two members statements despite my warnings above? Silent failure.

And the best part of all for a developer: You take samples off of the web somewhere. You modify them to fit your data. You experience silent failure. Pop quiz: Which of the above issues, or the others I didn't think to name, all causing silent failure, is the real cause? Good luck. (This is where I expect Mozilla loses a lot more developers than anyone will ever know. I know I tried to work with this back in 2000-ish, and I bailed out then over this issue. I kinda thought that four+ years later the situation would have improved...)

Templated data is not first class DOM data. Explaining this is hard, because it is a combination of little things. For instance, it is possible to use a template to define the columns of a tree, but you won't get the column picker. I don't fully know the reasons why, but ultimately it is because the template generated DOM nodes aren't "real".

You can see this by looking at the result of a template in the DOM viewer.

This also messes up the DOM and just generally makes templated content difficult, or, under some data models, impossible to deal with via scripts. The nodes don't really belong.

This shows all the classic signs of being a kludge: Unclean theoretical behavior ("fake" DOM nodes), an implementation that seems to have an ever-receding "doneness" point as you put more work in as more special cases are revealed (there's always another way those "fake" nodes aren't quite the same as real ones), and errors that have to be smacked down one at a time with the constant danger of regressions because you are basically jamming an impedance mismatch into the fundamental architecture.

This is somewhat speculative as I try to piece some history out of scattered old slide presentations and bugzilla listings, but it looks to me like the whole idea of XUL templates was Championed by Chris Waterson. Champion is capitalized here because it means not just saying "hey, this is a good idea", but actually coding it up and making it work.

He has since (as near as I can tell) left the project for some reason, which has left the XUL template code without a champion. Looking at the bugs filed against "xul templates" in Bugzilla, it seems to me that since then, none of the conceptual bugs have been fixed (for instance this notable one, although there are many in that vein). Crashes and failures have been fixed to keep the system going, but there has been no fundamental forward motion. Nothing I say in here is a crash issue, so barring another Champion emerging, none of these are likely to ever get fixed.

A fundamental aspect of Mozilla has been abandoned, and nobody is doing anything about it, not even clearing the rubble out of the way.

There is no way to provide status feedback with regard to loading the RDF. Above and beyond the near-complete lack of error messages mentioned before, there appears to be no way to signal a failure in the RDF loading to the user, other than the template silently not appearing. While a minor point in a technical sense, it is very important when dealing with remote data.

XUL templates can not be tested, because you can not reach into the middle of the process; they are a black box. This is a minor point for most developers, but for the (seemingly) select few of us who judge the testability as one of its most important traits, this alone can be a stopper. For that matter, this completely prevents any sort of debugging, too, even of the alert/print kind. You just have to magically divine what went wrong.

You've seen me say several times now "To the best of my knowledge, there are no XUL-based workarounds." The reason for this is to forestall the inevitable objection "But you can script a solution.", at least in some cases. To which I reply, yes, yes I can.

The good news is that the "solution" to the XUL template problem has already been provided to us, in the form of the DOM and the RDF reading interface, especially as reflected in JSLib. Instead of trying to force your idea into the XUL template, call for the RDF data directly and manipulate the DOM to do what you need. With a handful of utility functions to simplify the creation of the types of nodes you need (as all the attribute setting gets tedious if it is inline), you have all the flexibility of XUL templates, and you immediately gain access to everything I have been bitching about here.

You have the full power of Javascript to decide whether or not to display a node. You also get to stop munging your RDF data to satisfy Mozilla's inability to use predicates.

You may freely and arbitrarily interleave any "rule sets" you wish, even to add things in an order not reflected in the RDF data, such as dynamically sorting.

You may loop into the RDF data via the methods exposed in the JSLib as much or as little as you like. Consequently, you get the full power of Javascript on this data, too; you may organize things other than a list, you may re-position things, re-structure things, in general, you aren't stuck with a list of essentially homogenous things, opening a whole class of applications that XUL templates can't do.

You get the error reporting that you're used to with the Javascript aspect of the code, although it is dynamic code and may still be hard to follow. RDF errors can still occur but they aren't as silent in my experience.

The DOM nodes are first-class data. Any bugs in this area are general DOM bugs and will likely get more attention.

Javascript solutions are testable via standard methodologies. It can also be debugged.

As long as Mozilla continues to read RDF, this will work. Nobody has to champion this. Progress will be made as Mozilla's DOM progresses, instead of sitting out in left field with nobody working on it.

You can provide status feedback like "Loading..." while the RDF file is loading, and hide or change the status message when it is complete.

And to top it all off, a Javascript implementation is shorter! ...which surprised me. Even with counting the utility function and the special casing to handle a couple of special cases (which I intend to factor out later) in the final Javascript count, my XBL file started out at 4,873 bytes for the template code and ended at 3,680 bytes. This is a "bare-bone" conversion and the Javascript code will grow... but the shorter Javascript code also correctly handles loading failure, can handle heterogenous RDF (no more munging the outputted RDF to make Mozilla happy), and to top it all off, it can develop in the future, unlike the XUL template solution.

Yes, you lose automatic updating of the template if the underlying RDF data changes. This is solvable by adding "data change" callbacks into Mozilla, which as far as I can tell is the only aspect of everything discussed in this essay that requires something that isn't already implemented in Mozilla. Fortunately for my app I don't really care, but this could present problems for certain advanced uses. (Of course, compare this one relatively-easily-fixed shortcoming of the Javascript approach against the laundry list of problems with the XUL template approach...)

Taken all together, I think we have to conclude that even if you have one of the trivial cases that can today be accomplished via XUL templates, you should still read the RDF via JSLib and manipulate the DOM, because tommorow, your boss is going to want one of the fancy features that they can't give you. (And who knows how the parts of Mozilla that are currently built on templates would work if they weren't already limited by this? Apps and frameworks have a tendency to lock each other into a worldview that may not be optimal; seeing templates as fundamentally "a mechanism for displaying your message list with incidental other functionality" may be holding back everybody.) Ultimately, XUL templates are not maintainable.

Therefore: In your code, do not use templates. Use Javascript.

The Mozilla project should officially deprecate XUL templates. Frankly, until another champion emerges, it has already been de facto deprecated. This really isn't "removing" a feature, it is acknowledging it never properly existed. If that is too radical, undocument it. If that is too radical, at least tone the advocacy way, way down, ideally to the point of warning the developers away unless their app meets a very specific checklist.

JSLib should be expanded as necessary to make this easier. IMHO it is already pretty much there, but I am a relatively advanced user and it may be possible to make it easier for newer users. (It is hard enough to get into Mozilla development as it is, which is easy to see if you just start listing acronyms: HTML, XUL, XBL, RDF, XML, the list goes on, and you still typically need still more server side to do something interesting.) I am willing to contribute to this, but I can not lead this.

Callbacks for changing RDF data nodes, sequences, and arcs need to be added into Mozilla.

(Because I have hacked up my version of the JSLib, I defer to their documentation on how to use it. I'm not overwhelmed by it, but if I tried to give an example out of my code it wouldn't work on your system anyhow.)

But wait... this is assuming that you have to read RDF data. This is true only if you are reading internal Mozilla RDF data sources, or if you are reading external RDF data. Since this doesn't rate a full page:

Mozilla's RDF implementation is horrifyingly slow, leaks memory, and is surprisingly hard to deal with. (I've worked with RDF several times now and each time, I am confused as to how the RDF folk manage to take this simple concept of a graph and make it so unbelievably complicated to use. Even JSLib was a pain to get right and I had to add several utility methods that should have been used by every RDF developer... i.e., already in the library.) By "horrifyingly slow" I mean around two full orders of magnitude slower than other techniques for getting data into your application.

In particular, an (asynchronous) XMLHttpRequest loading up JSON data was around a hundred times faster than RDF loading the equivalent data, took less bandwidth, was cross-platform compatible, and was easier to generate on the server side.

XUL templates were a good idea and worth a try. This is not an idle concessionary closing point; I like multi-paradigm programming and use it all the time. However, the experiment ultimately did not turn out anywhere near as useful as first hoped. It is now an albatross around Mozilla's neck, scaring away an unknowable number of developers, trapping the one that stay into a sub-optimal solution, and consuming a disproportiate amount of documentation time and space for how little they can actually do.

It is time for the Mozilla project to face up to these facts and move on, away from XUL templates. (Strategy note: There is a certain Unnamed Competitor who does have the time and energy to make this Mostly Work^TM; it would be best to avoid competing with them on this field and moving the goalposts back into an area you can afford to compete in...)

Worst of all, I think, the XUL templates are a trap for developers. They promise the moon, at least in the (scanty) documentation, but deliver mud. Perhaps the most deceptive aspect is the implicit promise that this is somehow easier than a Javascript solution in the first place, which is untrue: The time taken to learn how to manipulate the DOM will pay off in other ways. The time taken to learn templates, which is much larger, is wasted.

In closing, I note that many, although not all, of these objections to XUL templates could in theory be fixed in a perfect world with infinite developer resources. However, we do not live in that world and they already appear to be languishing without any attention. Remember that the time spent salvaging them could instead be spent empowering Javascript developers as I outline here, creating the necessary helper functions and providing and documenting lazy callbacks for whatever you are trying to avoid rendering (like the canonical 100,000 element tree). Fixing the templates will be an immense amount of work. Polishing and documenting the last few bits of Javascript work necessary to make the Javascript hum will be IMHO about an order of magnitude less work.

Not everybody can just dump their XBL overboard and start over, and not everyone can leave behind their XUL. For such a person, XBLinJS is an extreme solution, but the nice thing is you can still apply some of its general principles and the principles presented in this critique as needed. Selectively replacing XUL templates in XBL with something else is a low-hanging fruit that can be done easily.

Russell Tyndall created something he calls a "repeater", which replaces an RDF/XUL template solution with Javascript data structures and an XBL tag that iterates along the data structure and creates new DOM nodes where the XBL tag used to be. No code is included, but if you've gotten to the point where you've tried to embed a XUL template into an XBL tag, the description is enough to implement something useful for your application.

You can also create XBL tags dynamically with any data source and code you want, it just gets ugly(/crashes) if you want to try to have the child and parent interact before they render, which many apps will not need. (You can lob string-only arguments to the child XBL node via .setAttribute calls after the .createElement call, and if you create some sort of ID and registry for your nodes you can then pass the string ID of the parent to the child to interact back with it. It's about at this point I threw my hands up and declared this crazy, but it does work.)

(Finally, if anyone is interested, I would be interested to see if XBLinJS could be used with XUL, in other words, replacing just XBL. Theoretically, it should be easy... conceivably it might even already work. All we might need issome namespace declaration support, which eventually even the HTML stuff is going to want if you ever want embedded SVG or something.)