of metaprogramming, monads, DSLs

Update: I wrote a follow-up as feedback to the feedback.

This must be what happens to your mind after reading too many blogs.

Like the writer of "I'm not tired of Java yet", I'm one of those many people who still (must) predominantly use Java at work for all the usual boring reasons, like platform stability and understandable syntax and 3rd-party APIs, but who also enjoy learning about other ways of doing things. The functional programming paradigm is one I've covered before (e.g. in this comparison of FP's and other paradigms' answers to the fundamental concerns of programming). Much of functional programming still appeals to me (e.g. when I implemented an Ajax callback that, if necessary, would make a data-specific number of additional Ajax calls before doing its original task(s) such as updating the DOM). However, over time I'm gaining more appreciation for another paradigm: metaprogramming, programs that manipulate programs, especially when the manipulating and manipulated program are in the same language and the manipulation happens at run-time. Metaprogramming can enable a higher level of abstraction and drastically reduce the need for code to repeat itself, though I'd advise first exhausting the plain reuse mechanisms of the language: namespaces, modules, classes, traits, functions, etc. (On the other hand, in cases in which a complex OO design pattern's careful balance between rigor and flexibility isn't vital and metaprogramming isn't absurdly difficult or error-prone, I prefer the metaprogramming option...)

A broad collection of techniques fall into the metaprogramming category. In LISP-like languages, homoiconicity languages in which the language syntax is a language data structure, read-write metaprogramming is natural. In other languages, metaprogramming could be supported through implementation-provided reflection objects. If the language implementation supports loading/linking additional code at run-time, a program could generate source, prepare it, and load it. With some limitations, metaprogramming might be possible through byte-level operations performed by a library, like ASM. When the language (or the platform it runs on, such as .Net 3.5) offers "expression tree" objects, the creation of new code at run-time is less messy but not necessarily easier. In still other languages that as a matter of course interpret/compile source at run-time (or immediately before), a program could just synthesize source as a string and run an "eval" function on it. Finally, a comparatively primitive yet widely-used technique is a preprocessing step that applies templates/macros to the source to produce the complete source.

One language characteristic blurs the line between metaprogramming and programming somewhat like homoiconic languages: classes and objects that are modifiable without restriction. Learning some ways to exploit this characteristic has been called "Basic Training" for Ruby, but other languages with the characteristic are applicable. Its relationship to metaprogramming is clear. Given that classes and objects combine code and data, and classes and objects constitute the structure of program B (so OOP is a prerequisite), and program A can freely modify the classes and objects of program B (and vice-versa), then program A can manipulate program B--the definition of metaprogramming, in a limited sense. Nevertheless, the modification of classes and objects can accomplish much of the effect of metaprogramming rather simply. (Freedom languages with unrestricted object modifications seem to usually have an "eval", too.)

Regardless of the specific metaprogramming technique, two applications of the paradigm have been getting publicity: monads (Monads in Python is a good example) and DSLs (find an example by your preferred Ruby cheerleader). The DSLs under discussion are internal, not external, which means the DSLs act as an API short-hand that uses the syntax and semantics of a host general-purpose programming language. (But monads in the form of a parser combinator like Parsec can implement external DSLs, even an external DSL like Perl 6...) Monads and DSLs are both applications of metaprogramming because each of the two effectively transforms minimal code into a complete form for execution. In fact, as monad tutorials instruct, Haskell's "do-notation" is syntax sugar, i.e. a DSL, for using monads...

The surprising revelation I came to a short while ago is that these two applications of the metaprogramming paradigm are opposites. In a DSL, statements and expressions of the source code "expand out" into larger blocks of code. The DSL implementation translates the source code and extrapolates a context of other code around it and from it. A monad implementation, by contrast, constitutes a context of other code around the source code that translates the source code and makes the monad's purpose "focus in" on the source code. A monad is an inside-out DSL. In the DSL application of metaprogramming, one observes and implements "operations" and "entities" in a domain, then creates a DSL--a language for those operations and entities. In a monad application of metaprogramming, one observes and implements "operations" and "entities" in a domain, figures out how to interweave those operations and entities with code in general (via the monad "return" and "bind" definitions), then creates a monad--a "domain" for those operations and entities, in which code has extra significance/side-effects. Obviously, the opposite function of monads and DSLs also reflects opposite purposes for the host programming language: an internal DSL acts as a highly-specific dialect and a monad acts as an extension. DSLs could perhaps be friendly to non-programmers (presuming a minimum level of aptitude), but monads would be definitely unfriendly to them.

I'd like to try employing a monad application of metaprogramming to make certain tasks easier. Of course, outside of the pure functional paradigm, where both mainstream languages and reality have state, monads are less essential. Monads could still be beneficial whenever I'll need to model a complicated commonality among several statements, without repeating myself. Those who think monads are still a long way off should note that F# is on its way to becoming an officially-supported .Net language (it also works on Mono), and F# has "computation expressions" whose similarity to monads and list comprehensions is no accident...