making widgets from data in javascript

Here is a convoluted, extra-long tale of real-world software evolution. I'm one of those odd programmers who doesn't mind writing documentation because he likes to expound on his own code in fits of self-aggrandizement. I was working on (rewriting an existing app as) an Ajax application when I soon realized that the task of generating a widget from data was generally applicable to many parts of the interface. I imagine many others have thought the same. I created a function with these parameters: a parent node of the widget-to-be, a 'mapping object', a 'data object', and an optional 'context object' (the context object parameter wasn't in the initial design). The function interpreted the properties of the mapping object to create a widget out of the properties of the data object, and then it added the widget as a child of the passed parent node.

The mapping object had a reserved set of "special" properties with specific meanings to the widget-maker function, like 'widgetType' to specify which widget to create or 'nodeposition' to override the default append so the widget could be inserted at any point in the parent node's existing children. Any properties of the mapping object other than these "special" properties became properties of the widget. If the mapping property's value was a string, the widget-maker looked up that string as a property name of the data object, and then set a widget property with the same name as the mapping property to the property value of the data object - the mapping object's property served as a connecting link or data lookup-index ("to assign this widget property, look at this data property"). I thought this would work fine, for a few minutes. Then I realized I would want to set a widget caption to, say, data property A concatenated with data property B. So I set up an alternate behavior for mapping object properties. If the value was a function rather than a string (checked for using typeof), then the widget-maker evaluated the function with the data object as the parameter, and the return value became the property value on the widget. I abstracted the string-vs-function behavior into a separate function that let me write the equivalent of "I don't care if the mapping object property is a string or function, just do what you need to do for this mapping object property and give me a result to assign".

Was that enough? Not nearly. Even the best of widgets may not be complete as is. What if I had to add more text to the widget, or stick an icon on it, etc.? Generally speaking, a widget may need to have children of its own. I don't mean child widgets, which I'll explain the solution for later, but child content. I expanded the widget-maker to interpret a new special property, childNodes, which was a mixed array of strings or functions. Each element would be evaluated and then concatenated into the widget's innerHTML. (I started out by adding a text node for each element, but discovered that prevented me from having markup in the strings - the markup would actually become literal text in the text nodes, not HTML tags.) In practice, I often ended up just writing one big function, which meant childNodes was a one-element array, and that one element was a function! Eh, hindsight.

In addition to childNodes, some other noteworthy "complex" properties that I added were a domAttrs property for specifying a series of properties to apply to the widget's primary DOM node (not used much except for specifying a data-dependent node ID), a styles property for specifying style rules to apply, and a postCreation function for miscellaneous widget-related code that had to be run immediately after widget creation. The postCreation function might connect up some callbacks, for instance (this couldn't happen until after the widget was created). The callbacks created in the postCreation function could use any variables within the postCreation function, although by the time the callback ran the postCreation function would of course be long finished - closures! At first the postCreation function received the data object and the primary DOM node of the widget. Later there were two more parameters: a context object and the widget itself. The postCreation function had to be passed the widget because I found cases in which treating a widget as a child of a DOM node was incorrect - for everything to work properly, the widget had to be passed to another widget. To defeat the widget-maker's default behavior for those cases, I added yet another special property to the mapping object, manualAppend.

If making one widget from one data object is a common operation, then so is making one widget for each member object of an array. I made another function, an array-to-widgets function, that called the widget-making function as it walked the array. And if making an array of widgets from a data array is a common operation, then so is making a tree of widgets! After assuming that the data array was preordered by the relevant groups (meaning if any two elements are in the same group or subgroup then those two elements are contiguous in the array), the array-to-widgets function gained the capability to create trees of widgets by determining where the group breaks were in the array and then creating the right widget for each group or subgroup (and the right widget for each individual array element as children of those). Besides the data array and the parent DOM node, the parameters then included a list of properties the data was grouped on (contiguous elements with the same value for that property were in the same group or subgroup so the algorithm only add to check for changes in each group's "running value"), in grouping order going from the property for the "highest" or most-inclusive group to the property for the "lowest" or least-inclusive group, and another list of mapping objects that contained one mapping object for each of the groups/subgroups and one last mapping object to always run on each individual data element. It's less complicated than it may sound, once you get it right. A hierarchical tree list of cities grouped by state/province and country, in which each element in the data array is a city with 'country' and 'state' properties and the cities are preordered/presorted so that cities in the same country are contiguous and cities in each state/province are contiguous, might be created by passing the two lists ['country','state'] and [countryTreeMapper,stateTreeMapper,cityLeafMapper]. Naturally, I usually called the array-to-widgets function, rather than calling the widget-making function directly. (And just in passing, any element in the mapper list can be null, which the widget-making function handles by merely returning a 'placeholder' empty HTML element).

Ideally, the widget-maker function would be sufficiently distinct from the array-to-widgets function for reuse purposes, and it was - sometimes the postCreation function for a mapping object would call the widget-maker (so at some level the widget-maker was calling itself). But chances are, a group widget will need to summarize information about its group - showing its member count as part of some childNodes content, maybe. To do this, the functions in the mapping object must know 1. the array it is being generated from, 2. the index of itself (the starting index of the group) in that array. Although I wondered if there was a better way, I stuck a 'context object' parameter on the end of the widget-maker function but made sure the widget-maker function could work without it (remember, javascript functions can take a variable number of arguments). The array-to-widgets function updated the context object as it iterated, and passed the context to the widget-maker function. The widget-maker function would pass the context on to the functions in the mapping object (which also could be written to not take the additional context parameter at all).

I still wasn't done. Creating gobs of widgets out of a data array was a happy accomplishment, but consider what happens when the data array changes ever-so-slightly to have one more element. Is it reasonable to demolish and recreate all those widgets to accommodate one new red-headed stepchild? (The answer is no.) I started by trying to make the array-to-widgets function perform a data/widget "diff", but I gave up on that quite quickly. Instead, I modified the widget-maker function to first compute what the ID of the widget would be, check if that ID was taken, and only create the widget if the ID was available. Since IDs must be globally unique anyway, the procedure seemed reasonable. As for updates, if the widget-maker function found that the ID was taken, it reran the mapping object functions after setting a property on the context object to signal that the function was called for updating not creating. Otherwise, a new total might be appended on to the end of the old total instead of replacing it ("1011" instead of changing "10" to "11")!

For handling removals, I took a similar tack. The array-to-widgets function now kept track of the IDs of the widgets it created while iterating, and returned the list of created IDs as properties of an "old ID" object. It also took a new (optional) parameter, the object returned from the previous time the array-to-widgets function ran. If the old-ID-object was present, then after the array-to-widgets function was done iterating it checked to see if each of the properties of the old-ID-object were properties of the ID-tracking object it just created (the object it created for its return value anyway). Any IDs that were property names of the old ID object but not property names of the just-created ID object represented obsolete or outdated widgets, so the array-to-widgets function destroyed them (using the proper API so nothing was left dangling).

The refinement of my functions happened gradually as I better understood the problems the functions had to solve. Writing widget-creation code for any given chunk of data would be simpler or just more straightforward code, but I would then be forced to rewrite variations of that in perpetuity. I prefer a solution that makes use of javascript's no-fuss lists (javascript's "arrays"), hashes (javascript's "objects"), and anonymous functions/closures to let me compose program elements at runtime.

Bonus observation: It would be criminal for me to not give credit to Firebug. Apart from its impressive CSS features (dynamically adjusting style rules and immediately seeing the result is a favorite of mine), its javascript debugging is better than sliced bread. Literally. I would forgo sliced bread, chopping loaves with a cleaver, rather than forgo Firebug javascript debugging. Setting breakpoints in javascript code, even in anonymous functions, is my hero. The network activity tracking is also exceedingly convenient for finding out exactly what the asynchronous server request and reply looked like.