actual news about the zap2it labs shutdown

For those who care, I noticed that the Schedules Direct site has updated its page! Schedules Direct is the zap2it labs replacement service, according to the MythTV wiki.

The gist appears to be that although the "free-as-in-speech" software ride is not over (of course), the "free-as-in-beer" tv listings ride is. In return for a DVR that does whatever I want it to (for standard-definition TV, anyhow, *sigh*), I'm willing to cough up some dollars. Already have, when I assembled the computer parts...

good language for internal DSLs

It's tiny, mature, and performs well. Its syntax is minimal and straightforward, but it still supports a number of fancy features. It's embeddable. It's portable. It's extensible through a mechanism somewhat like a meta-object protocol. Its C interface is remarkably convenient.

It's Lua. You didn't think I was describing Ruby, did you?

shifting programming metaphors

Someone at work has a habit of using "Fortran" as a synonym for "simple, imperative-style programming", regardless of the specific language: "Fortran Java", for instance, means clumping all code into the main method.

What makes this amusing is that Fortran hasn't been sitting still since he last used it. Even Fortran has objects now.

callbacks within callbacks

Oh, the travails of an Ajax amateur...in the beginning the requirement was straightforward. Grab some data asynchronously by passing along an "update-display" callback function (a function with a 'data' parameter). The callback function could even be anonymous, or refer to variables in enclosing function scopes at the time of definition--nothing out of the ordinary.

Then someone has the nerve to change his mind about the interface, so now one display needs to show what is actually the patched-together result of several invocations of the same data call, all at once. At first glance, this is still easy--just fire off all the asynchronous requests required and fuhgeddaboutit, because each one has a callback to update only its part of the display.

Wrong! In this case, the parts of the display are ordered and the necessary extra data calls to fill in "missing" points aren't known until some of the data has been gathered, meaning the part of the display generated from call C must be finished after the part of the display generated from call B is finished, and so on. But all the calls are asynchronous, so there is no guarantee whatsoever of the time or order in which individual calls will return.

Flashbacks of semaphores and mutexes and Dining Philosophers dance before your eyes. No--that way lies madness. Since the display is ordered, better to just get all the data pieces, assemble them, then display the whole as if it had been from a single data call. Unfortunately, the "update-display" function can't immediately run after firing off the asynchronous calls, of course, because then it might run before all the data was gathered! So the "update-display" function must be in a callback of some kind. Yet if the calls are asynchronous, which callback should contain the overall "update-display" function? The last one to finish, of course. But which one will that be?

Puzzle it till your puzzler is sore, and one answer, not necessarily the best, may emerge. The time spent studying the techniques of functional programming may generate (yield?) a payoff. Consider the following almost-real code, in which a callback function takes its own function parameter.

function baseCallback(data,update-display-function) {
 var afterFunction = function(dataparm) {
   update-display-function(dataparm); };
 function addAnotherAfterFunction(currentAfterFunction,
                                      patching-info) {
   return function(dataparm) {
     var innerCallback = function(innerData) {
       // process data...
       // patch data using patching-info
       // (data patch location passed by reference)
       currentAfterFunction(dataparm);
     }
     dataCallInvoke(params-computed-from-patching-info,
                                       innerCallback);
   }
 }
 // process data in a loop, and within the loop,
 // when there's a need to make another data call...
 afterFunction =
   addAnotherAfterFunction(afterFunction,
                           data-patching-information);
 // after the data processing loop
 afterFunction(data);
}
function dataCallInvoke(originalParams,upd-disp-fn) {
 var normalCallback = function(dataparam) {
   baseCallback(dataparam,upd-disp-fn);
 }
 callWithCallback(originalParams,normalCallback);
}
dataCallInvoke(origArguments,overall-display-func);

The idea is that addAnotherAfterFunction repeatedly operates, as needed, on the function in the afterFunction variable. One callback becomes embedded inside another callback inside another callback. When the outermost callback (the final contents of the afterFunction variable) runs, it makes a data call and passes in the next-inner callback. The data call runs, then executes the next-inner callback that was passed to it. The next-inner callback perhaps makes its own data call...and so on, until the "true original" callback, the "update-display" function, takes the fully-constructed data and does what it will.

One downside is that the invocations of the data calls can't run in parallel (although since order explicitly matters so much in this situation, parallel execution won't work anyway). Also, tossing around so many functions with their own individual closed-over scopes surely eats up some memory (although I wonder if some "copy-on-write" implementation could be a workaround). Honestly, I'm not sure what terminology describes this technique. Um, mumble, mumble, continuation-passing-style, mumble, combinator, mumble, monadic bind, er...

stating the obvious about Microsoft and Open Source

Microsoft has a web site that serves as a "gateway" to its ventures related to Open Source. Also, Microsoft will submit its "shared source" licenses to OSI. I've read some...questionable analysis about these overtures, so here's a helpful list of obvious points. I may be a little sloppy in confusing "open source" with "free software" (Stallman essay on the topic). I think it's a given that Microsoft is more open to open source concepts than free software concepts.

• I don't have the numbers (nor do I care), but one obvious truth is that Microsoft earns obscene amounts of revenue from selling software, no matter what discounts it may offer in certain situations. It also benefits from customer inertia, because once a customer has started using particular software the cost of migrating off it can be steep. Naturally, many other software companies do the same. Now, the licensing of open source software makes this typical strategy much less viable. (Open source software can be redistributed by users, which drives down the selling-price, and open source software can be modified by users, which means they don't need to depend on the company for future releases or bug fixes). Hypothetically, if Microsoft moved to pure open-source licensing, it would lose truckloads of moola. Not a likely move. On the other hand, switching to an open source license for software which is free to acquire, like Java, makes more sense to software companies.
  
• Microsoft is mind-blowingly massive. Again, an obvious point, but there it is. Microsoft has its hands in OS, office suites, databases, development tools, Web technology, media, video games (PC and console). It even makes mice, you know. Microsoft's massiveness means several things. First, that categorical statements about the quality of Microsoft's offerings are almost always over-generalizations. Saying that one will refuse to consider the (possible) technical merits of open sourced Microsoft code solely because it's from Microsoft is foolish and uninformed. Microsoft has some incredibly smart people whom I can only assume are worth more to Microsoft as real coders than figureheads. Refusing to consider the code based on ethical concerns or fear of patent litigation, well, that's different. Second, Microsoft's massiveness means that Microsoft can pick pieces to open up to varying extents while leaving the rest closed. Microsoft gains a few favorable public-relations nods, but keeps its identity. I imagine it will primarily open software on its periphery and in contexts that have a greater chance of being appreciated (open-sourcing a standard-document converter, for instance).
  
• Executives at Microsoft have repeatedly denounced free software, on principle. Historically, Microsoft hasn't done much to grant its users additional privileges beyond, well, using. If it's taking steps in that general direction, expect those steps to be slow, rather forced, and accompanied by restatements of its stances.
• Microsoft grows by expanding into apparently-profitable new markets. From this point of view, the open source software community represents yet another opportunity. Unfortunately, as a vocal and dominant player in proprietary software, Microsoft also happens to be a defining symbol in that community for what open source stands against. Personally, I believe that defining a viewpoint in terms of its opposite ("I'm not them, I'm something else!") is lame, and there's no shortage of other targets that fit the criterion "aggressively makes closed software". And at least one knows that Microsoft isn't illegally exploiting open source software like some others, considering mere interoperability is poorly supported.
• Microsoft has a slew of competitors, some worrisome, some laughable. Those competitors have been starting to leverage open source, though perhaps more for public-relations' sake than in a hope to create better software. As product comparison checkboxes go, "connected to open source" has risen in importance. It helps to put customers a little more at ease and enhances the trustworthiness quotient of the company. The more other companies can check that box, the less desirable Microsoft seems. I think it's safe to assume Microsoft would rather compromise a tad on open source instead of losing market share.
• When Microsoft dabbles in open source, it doesn't use the same open source licenses as everyone else. To be fair, it's not alone; as is their prerogative after some legal consulting, companies will create or modify an open source license when none of the licenses is exactly right. Nevertheless, why doesn't Microsoft just use the (myriad) OSI-approved licenses, which is an easier track to "official Open Source"? To me, this is the most significant indication of an underlying lack of sincerity. In using existing open source licenses, companies gain the stamp "Open Source". My impression of companies that craft new licenses for approval is that they are trying to reshape the stamp to fit them, not acknowledging the stamp for what it is.
  

words, whether cromulent or not

I relished reading the "100 Words Every High School Graduate Should Know" (according to Houghton Mifflin Company, not blogger X). Some of the words on the list, like "nanotechnology" and "quasar" and "hemoglobin", seem to fall more into the jargon category, meaning that someone wouldn't encounter those words unless trying to. Some of the others fall into the supercilious category, like "supercilious" and "tempestuous" (I despise when people break out that word in everyday conversation!) and "loquacious", meaning that someone would most likely encounter those words as bywords intended to signal the speaker's educational level. My favorite words in the list are the words which are fun to speak and spell, like "lugubrious" and "bowdlerize" and "chicanery" and "wrought" and "incognito". The inclusion of "enfranchise" reminded me of what Krusty said on an episode of the Simpsons (a show which has bequeathed its own share of neologisms like "cromulent") after accomplishing something in Congress: "I have become enchanted and illusioned with Washington".

At the other end of the scale, while I was busy skipping ahead in thirty second bursts on MythTV, I caught the tail end of a commercial for the Big Brother show. Host what's-her-face said, "Can you say 'frenenemies'?"

Yes, I am physically capable of doing so. But why the flying freck would I?

another day, another web framework: Helma

Helma is an interesting-looking web application framework for Javascript. I haven't experimented with it at all, but its very existence amuses me after reading about Rhino on Rails. Before sneering too much about using Javascript for large-scale development, note that 1. Rhino Javascript is different (more up-to-date) from typical browser-supported Javascript and 2. Javascript can "feel" much different within a framework that performs OO prototype-modification (heard of Prototype?).

What are the choices now? There's Helma so one can use Javascript on both client and server, and there's GWT so one can use "Java" on both client and server. It's a mad, mad, mad, mad world (look under the big WWW).

C reflections

I did some work in C/C++ recently, which has left me feeling reflective. I haven't done a lot in C/C++, but I know it well enough to accomplish the goals I must. C has a special place in my memories because it was one of, if not the very first of, the truly general-purpose and professional-strength programming languages I learned about. I'm frightened at the thought of people who say they wish to study the discipline and craft of computer programming but don't know C except by reputation.

What makes that thought frightening is C's continuing vital importance. Just as human history is the invisible-but-pervasive factor that shapes present civilization and culture, so C (specifically software written in C/C++) is the invisible-but-pervasive factor underlying the present software development ecosystem. Show me an OS, a compiler, an IDE, a JVM, a device driver, and I'll show you C's influence.

C's centrality and effectiveness stem directly from its ability to map so closely onto the machine which will execute it, but still abstract the programmer from the excruciating details of the actual hardware--registers, memory segments, byte-order, opcodes. In fact, C/C++ compilers have gotten so adept at bridging the gap, an inexpert programmer who tries to do it himself may achieve a decrease in the optimization level. I know alternatives to C have come along (often not too different from the king), but C's own success has seemingly doomed it to be the de-facto, default choice in its niche.

However, to do real work in C/C++ is to be constantly aware of its balancing act between the structure of the machine and the structure of the problem domain. It's not long after the stage of thinking "I need to work closely with the machine" and therefore choosing C/C++, to the stage of thinking "geez, this is irritating to write all these steps" and therefore breaking out the standard library or one of the multitude of other libraries. I wonder to what degree just the lack of sophisticated built-in string support caused the rise of alternative languages to C for some uses. Then there's the lack of sophisticated built-in data structures such as lists and maps, which are so widely applicable it seems silly to mention it. I caught myself missing stack traces for uncaught exceptions, or even any exception-throwing at all. It's no surprise that some libraries or toolkits provide such a comprehensive cocoon of functions and macros for the programmer that the result feels like a dialect (smart pointers? vectors?).

The tradeoff, naturally, is that since none of those extra libraries or language features must be used in any given program, the overhead or complexity aren't mandatory either. And there's something satisfying in using a language that connects the programmer to the machine. The language has abundant evidence that the program is intended for a computer and not a "theoretical Turing Machine": pointers, structs which are nothing more than organizational units for groups of variables, variables that refer to memory locations rather than objects, the capability to interpret memory contents in multiple ways (hence weak-typing), void * functions for futzing with memory directly. It can certainly be tricky and even error-prone sometimes, but the "garbage in, garbage out" principle rules all. Many people are both more experienced and more skilled at it than I am (I'm more of a math/logic/language tinkerer than a hardware tinkerer).

For me, C is the emblematic programming language, because it unapologetically bridges human thought and machine computation. Someone who can write effective C is someone who can straddle both realms. Try too hard to make C code like human thought, and the program may run horribly (unnecessary recursion, for example, or number overflows?). But try too hard to tailor C code to the machine, and the program may become an inflexible and devilishly cryptic tangle. Get to know C/C++, grasshopper. Folks are fond of saying how Lisp leads to " 'aha' moments". The melding of mind and machine in C code may yield similar " 'aha' moments". At the very least, you may gain a deeper appreciation for what compilers/interpreters are actually doing for you. And if nothing else, having just reading-fluency with C/C++ is pragmatic, because it's still alive and kickin' in the "enterprise", in "legacy" code anyway. C/C++ also happens to be extremely important in the FLOSS world, thanks to this "gcc" thing you may have heard of (during my Gentoo phase, gcc may have been the largest single consumer of CPU cycles).

Oh, Yegge...

I followed one of the myriad links to Stevey's Blog Rants for the Rhino on Rails explanation (long story short: for the rest of us, JRuby on Rails or Grails probably makes much more sense, even if the project wasn't internal at this point). On the other hand, if it leads to enhancing/updating Rhino, go for it, guys!

I was a little tickled by a different entry on Rich Programmer Food (or, "Why Stevey thinks programmers should be at least familiar with how actual compilers work"). Stevey gleefully pokes fun at C++, Java, and Perl as a matter of course (especially Java--have you read about the Kingdom of Nouns?). But do you suppose he knows that part of Perl 6's design is the ability to write grammars? Moreover, the Parrot work has led to a set of tools that make implementing languages for Parrot easier. Compiler theory and DSLs, indeed!

peeve no. 249 is the casual use of 'random'

As rants go, the capriciousness of this one is up near level 11. I hardly expect anyone else to share my irritation. Nevertheless, I must tell the World.

In my opinion, the everyday use of the word "random" is nearly always inaccurate. If someone you were talking with abruptly switched the conversation's subject, neither that person nor the shift was random. It may have seemed random, but only because you weren't insightful or knowledgeable enough to fathom the connection.

To be fair, the mental leaps people are capable of making can be incredibly subtle and creative, even to the point at which the likeliness of anyone else duplicating the leap is nil, whether with or without the exact same set of information. Don't call it random. Mental leaps are essential to progress.

If someone shifts mood suddenly, the chances are good that it isn't random, although someone's true motivations may be cloudy at best. The cases that come closest to randomness are those in which the emotional imbalance is clearly linked to a chemical imbalance - and in those cases the underlying cause is definite!

Humor that consists of events happening outside of rhyme and reason is not random, either. It's a writer frantically hurling nonsensical ideas when he or she doesn't care enough to write organized material. Unfortunately, such "humor" is its own undoing, because sooner or later the audience will notice the absence of any point, then naturally wonder why they're settling for such. One should also note that even "random" humor can't, in fact, be truly random. True randomness alienates an audience. Seriously. Use some dice or something to pick a set of elements, then combine the elements. The result will not cause laughter so much as perplexity.
Humor isn't random; it must be constructed so that it works.

Sometimes the actions of large organizations, or the managers of those organizations, may be labeled as "random". If thinking that makes one feel better, fine. Specific decisions perhaps actually have random elements (if two products are so similar in price and features as to be practically indistinguishable, who cares which one will be bought?). The reasoning behind other decisions may seem shaky or hasty, at best. But you better believe that when manager 3 suddenly starts advocating wonder technology 6, the idea did not enter his head by accident.

Tragedies, or the victims of tragedies, feel "random". My genuine (discomforting) advice is to abandon the illusion that the Totality Of Reality operates according to well-defined rules at any time, tragedy or not. I have barely any control over what happens to me. Neither do you. Those in supreme positions of power don't either; they could be eliminated by microscopic organisms, an assassination, or a coup. If "random" is a synonym for "out-of-control", then "random" is just the way things are.

Ugh, now I'm depressed. Stupid Total Perspective Vortex.