clarifications of metaprogramming, monads, DSLs

The last post led to some interesting feedback that has prompted me to make some clarifications. Specifically, I'm responding mostly to the entry on chromatic's use.perl.org Journal (I often enjoy pondering what chromatic writes there and over at O'Reilly Network Blogs). I'm also including a short response to the reddit comments, and tossing in a couple other thoughts I had after posting.

• I remembered a few other metaprogramming techniques. My quick reference to a "preprocessing step that applies templates/macros to the source" doesn't give due recognition to the (sometimes headache-inducing) tricksy acrobatics performed by template metaprogramming. The general category of compiler-compilers was a gaping omission, considering how many of them that Wikipedia lists and how frequently people use these great tools. Camlp4 is a snazzy way to create parsers in OCaml, even for extending OCaml syntax (see the slides for the One-Day Compilers presentation). And F# has a "quotations" feature for treating code like data.
• Perl 6 isn't a DSL, of course. When I wrote that "Parsec can implement external DSLs, even an external DSL like Perl 6", I meant something more along the lines of "Parsec can implement languages that are much different from the host language Haskell, even a language as nontrivial as Perl 6". My intent was merely to comment in passing that "monads in the form of a parser combinator like Parsec" can and do function as an effective technique for external DSLs. Those of us who're familiar with Higher-Order Perl (the later chapters...) aren't (much) fazed by the concept of parser combinators...
  
• Monads aren't DSLs. Actually, I said the two were opposites. The line "A monad is an inside-out DSL" (and in bold, no less) is using DSLs somewhat as a metaphor or counterpoint for monads, although as I wrote a while ago my preferred monad metaphor is an abstract algebra group or ring (and if you see what the origin for monads was, the suitability of this metaphor isn't surprising).
• However, I did write "In fact, as monad tutorials instruct, Haskell's 'do-notation' is syntax sugar, i.e. a DSL, for using monads..." My intent was merely to comment in passing that do-notation is like a DSL within Haskell syntax for the >>= operator, through which the domain "using monads" is made easier to work in. It reminds me of how LINQ's SQL-ish syntax, with its keywords like "from" and "where", acts like a DSL for data queries within applicable languages. As parts of a language's intrinsic syntax, are "do-notation" or extra-special LINQ syntax true stand-alone DSLs? No. Do those language features, like DSLs, make it easier to program for a specific purpose, a specific task's "domain"? Darn tootin' (yes).
• Are monads a genuine example of metaprogramming, or programming in a language and to an API? Given that a monad is a feature of the language syntax, a monad sure doesn't seem like metaprogramming to a Haskell programmer any more than defmacro does to a Lisper. However, think of how a language without monad capabilities would achieve monad-like effects (consider the "before" and "after" advice of AOP). On the whole, monads seem like a sufficiently sophisticated mechanism for code transformation, regardless of the language, to qualify for "metaprogramming". Someday, when monads aren't considered fancy wizardry, maybe the "plain programming" label will be the consensus. (As an aside, one of my personal syntactical/semantic wishes is that the freedom to choose dynamic or static variable type checking becomes commonplace.)
• reddit comments: Is my only point that do-notation is a DSL, and I'm just trying to be clever about it? 1) Well, I'm always trying to be clever on this blog, as a general rule. It's one of my motivations for writing at all. But I'm sure everyone realizes that my motivation has nothing to do with whether what I'm writing is useful or not to each individual reader. Applying what you read is your responsibility. I'm not one for writing how-tos. 2) As I already admitted, calling the do-notation a DSL is shaky (the "internal" qualifier notwithstanding). That isn't at all the main point. The main point is metaprogramming. The later paragraphs are about contrasting two applications or approaches of metaprogramming, monads and DSLs. If this "Grand Unified Theory" isn't as interesting or persuasive to anyone else, that's okay. It might offer some answers for people who ask, "Can I make DSLs in Haskell?"
  
• Woo-hoo! As of right now, not only is the blog the top Google result for "rippling brainwaves", but the discussed blog entry is the top Google result for "metaprogramming monads DSLs". (Using Blogger probably helps. You Google searchers have got to ask yourselves a question: "Do I feel lucky?" Well, do ya, punks?) But I didn't write the title or the entry to "bait" readers, honest. That's partly what I was referring to in the self-deprecating remark, "This must be what happens to your mind after reading too many blogs."
• Thanks to the commentators who said they grasped what I was struggling to communicate! I appreciate the occasional reassurance that I'm being lucid, rather than gibbering...
• Update (Feb 18, 2008): I noticed that Writing a Lisp Interpreter In Haskell seems to match some of my conclusions. Nice to see that I finally caught on to what I read many months ago... First a short quote:"Haskell comes standard with a parsing library called Parsec which implements a domain specific language for parsing text." Now a long quote:

  A common misconception among Haskell beginners is that monads are an unfortunate evil necessary in a a lazy, side-effect free language. While it is true that monads solve the lazy IO problem, it is only one example of their power. As I plunged deeper and deeper into Haskell I began to realize that monads are a desirable feature that allows the programmer to build otherwise impossible abstractions. Monads, together with higher order functions, provide excellent facilities for abstracting away boilerplate code. In this sense Haskell's monads serve a similar purpose to Lisp's macros (though the concept is entirely different).