Public Announcement: Iron Lute, a Python/TK Outliner

This is the first public posting regarding my next major project, Iron Lute (links to a screenshot). Iron Lute is an outliner, written in Python and using Tk as its toolkit; as a result it should run on Mac, Linux, Windows, and anything else that supports Python and Tk. (Off hand I don't know how many platforms that is, but it is plausible this would work on palmtops and some other obscure ones with few changes.) The license is currently undetermined, but will initially be open barring surprises; more details in a forthcoming post.

This category of my blog will be posted to http://www.jerf.org/ironlute/, and an Iron Lute-specific RSS file can be picked up from http://www.jerf.org/ironlute/rss.xml. (You can also subscribe to iRi RSS file.)

Initial postings will be in the form of a "developer travelogue", showing the development process as I see it. My goal will be to provide a view inside the head of a software architect, understandable enough to non-software folks to understand at least the thought patterns. It'll be up to you to decide how good I am at development ;-), but it should at least be interesting to some folk. These postings will start out at one a week and I'll evaluate the frequency from there; time spent writing the posts isn't entirely lost (forcing yourself to write thoughts down almost always clarifies them), but it is time not spent coding.

Now, about Iron Lute. Iron Lute was inspired by some of the recent innovations in Radio Userland, specifically Node Types. I saw (and see) a lot of potential in the idea, and immediately set about using it.

Unfortunately, I almost immediately ran into multiple serious shortcomings of Radio Userland's support for the idea. Don't get me wrong; I like the Radio Userland environment and I still like programming it; this isn't an anti-Userland screed. In fact what you are reading right now is Radio Userland generated. Nevertheless, I was driven by many shortcomings Radio Userland had for my purposes to create my own outliner to work with.

The number one shortcoming isn't even Userland's fault per se, but Radio Userland's outliner doesn't run worth a toot under Linux (WINE isn't good enough), and except for Radio Userland, I am trying to be an all-Linux house. (Mostly because I can no longer honestly agree to Microsoft's licenses.)

Other shortcomings include poor multithreading while opening remote files (opening a remote OPML file locked that window until the file loaded), hard-coded HTML styling support with no extensibility (RU's outliner can do bold, italic, underline, and links, but there is no user-extensible mechanism for adding anything else, or using that with anything other then HTML), general difficulties in extended it as easily as I'd like (for instance, can't extend the undo/redo support), and, as I'll elaborate on in my next postings, a very weak concept of what an "outline" is.

(Note that while I can't claim to have exhaustively examined all other options, RU is probably still the most customizable commercial outliner. Also I emphasize that these are not intended as major criticisms; if you use the outliner right now, can you honestly say you miss these features, except maybe multi-level undo/redo?)

As a result of these weaknesses and others, I had a very difficult time coming up with a project I could do that would not be fatally crippled by the limitations inherent in the outliner. As a for instance, I started designing something like Channel Z in my head, but at every turn, it was stymied by the outliner. I wanted to transclude deeply into the outline, not just at the surface; too hard, not implemented. I wanted to be able to leverage OO concepts, which is a major pain in Radio Userland. I wanted to be able to implement some other features someday that required tight coupling and intimate knowlege of data structures that don't really even exist in Radio Userland.

Perhaps I could have gotten some of this into Radio Userland eventually... but looking at it from Userland's POV, why would they have? It's one thing to make claims, but I have no proof that what I think can happen will. It's really my responsibility to show why what I think is true is true, and there were no mechanisms for me to extend Radio Userland from where I'm sitting, outside of Userland. So, the natural next step is to write my own outliner.

Since I was looking in the Open Source domain, I looked around for good outliners to start with. The only good Open Source outliner that I could find was JOE (Java Outline Editor). Unfortunately for me, Java is about my least favorite language, and while it's a good program, I would still need to do a lot of architectural work to get where I want to be, work that I don't want to do in that language.

Existing Python outliners are best not spoken of, and are mostly abandoned anyhow. One exception, LEO, is not an outliner in the sense I am looking for. It uses outlines for its organizational structure but it then edits the majority of its text in flat text mode. Indeed, I hope that someday Iron Lute will be good enough that the LEO project will want to use the outline widget I will be developing in their project. Then they can give their users the option of using outlines "all the way down".

Here's an interesting question: So where are the Open Source outliners? As I've discovered the hard way, the answer lies in the graphics toolkits that have been created for Linux and other operating systems. (For a non-programmer, a Graphics Toolkit is the set of on-screen widgets you see when you open a program, like text boxes, tabs, floating toolbars, etc. Windows has a couple of standard ones, Mac OSX has a couple, and Linux has an amazing profusion of them, some of which work on the other two OSes. Tk itself is by now largely an OS-agnostic toolkit.) Tree widgets (such as you see in Windows Explorer or the old Macintosh finder) inevitably work on only one line at a time; multi-line text is not supported. Nor can you ask a text widget how big the text in it is; you can ask how many characters it has in it but now how much screen space they occupy. This makes it impossible to try to string together text widgets in an outline manner, since you generally need to know the height of a widget on the screen before you can put one immediately underneath it.

Existing Python outliners, which I don't want to mention by name in a critical manner (no point being mean to mostly-defunct projects) have been significantly colored by what is even possible with the provided widgets.

One major part of Iron Lute to date has simply to get the outline widget largely working in a way I approve of, so it's useful to me as I write.

Iron Lute is intended to be an outliner with a strong, clean design, extensible by other programmers to load and understand other types of files and network loading methods (such as loading from a Manila site in a non-hacky way) without too much effort, resulting in features for users that currently people may not even believe are possible in an outliner format because they seem to difficult to make them work... but they can.

At the moment, I have a lot of work to do before even an alpha release can be made, but I daresay I'm over 50% of the way there, so I am confident that this will be released as alpha at some point over the next few months, hopefully by next summer although this is probably optimistic. Despite the general futility of hoping for external developers, I hope to attract some people to extend Iron Lute by making it easy to do so, and writing some good tutorials on how to do so.

The next Iron Lute post will tackle an issue at the heart of any outliner: What is an outline? One of the most fundamental aspects of an outliner is what data structure it uses to represent the outline. To answer the question requires a journey of exploration and iterative refinement.