Rippling Brainwaves

an unexpected bonus of my Netflix streaming plan...

2012-01-24T13:20:00.003-05:00

...is the chance to wait day after day for the surround sound option to return to all the TV series that had it before! Even better, total silence about it in Netflix's established avenues for public communication! (No, extremely vague and evasive answers from Customer Support don't count.) Sign up, and for a low monthly price you too can give your patience a thorough endurance session.

Yeah, I know, #firstworldproblems. But at least I'm not airing a "one-percenter" problem ("it's so hard to find the right personal chef!").

words that authors set on Repeat

2012-01-13T09:50:00.003-05:00

Full-length books are certain to contain repetitive words. Some of those repetitions are more noticeable than others.

In The Beginning of Infinity by David Deutsch, one of those words is "parochial".
In Darth Plagueis by James Luceno, one of those words is "rock" as a verb, which refers to a head motion, often from side to side. Are you ready to rock? Your head?

hail dpigs

2011-12-27T18:28:00.003-05:00

"dpigs" is a shell script for apt-based Linux distributions. It produces a list of the software packages which consume the most disk space. I've upgraded my installation in-place, time and time again. Hence my root partition (/) has been running out of room, especially after long uptime. Hacking away at the usual culprits was effective but still insufficient. After removing some of the unneeded yet greedy packages that showed up in dpigs, the problem is gone.

In any case, if you're in a similar situation, my first suggestion is linux-image*. Any desktop kernel version that you haven't booted in the last six months is probably not worth keeping....

this is an undefined entry

2011-11-29T10:52:00.001-05:00

Standardization is indispensable. Through independent standards, various products can work well, regardless of market competition. On the other hand, successful standards have appropriate limits. Too much constraint impairs vital product flexibility¹. To balance these concerns while remaining unambiguous, a standard might literally delineate topics which are called "undefined". It identifies and then ignores. Hence, a well-known technological gaffe is to assume too much about what's undefined by the standard.

I appreciate such indistinct confessions. In fact, one of my cynical unfulfilled wishes is to exercise that option in other contexts.

I'll finish the project by the agreed deadline. On the condition that everyone else finishes the preceding tasks promptly. Else the finish date is undefined.
I'll contribute a proportional share to the joint cost of supper. On the condition that I stopped by an ATM recently. Else I should warn you that the amount that I will find in my wallet is undefined.
I'll advocate the replacement of the software package. On the condition that the result is an unqualified improvement. Else my original level of support for the replacement is undefined.
I'll follow the specifications. On the condition that the specifications are exact matches for what I was already planning. Else my fidelity to the specifications is undefined.
I'll write a program that handles the input correctly. On the condition that all the input is correctly entered. Else the program's actions are undefined.
I'll stay alert at all times in long meetings. On the condition that I slept soundly the previous night. Else my attentiveness is undefined.
I'll continue adding more examples to the list. On the condition that I don't start to feel bored. Else the number of more examples is undefined.

¹The product can exploit the flexibility in myriad ways. It could do nothing or fail. Or it could provide additional benefit. It could violate unstated expectations. Or it could guarantee a result which the standard doesn't. It could optimize for greater efficiency by performing fewer checks. Or it could optimize for greater infallibility by performing innumerable checks.

the rational course for tiresome dollar pessimists

2011-11-22T10:24:00.000-05:00

I Am Not A Financial Advisor. Nevertheless, here's some advice for the most tiresome dollar pessimists.

I'm not referring to lowly savers who are understandably anxious about the negative real interest rates of their risk-free deposit accounts. I'm referring to the strident soothsayers who remark repeatedly that the dollar is primed to become worthless in the near future, i.e. within the next two years. They know just enough about economics to support the opinions that they hold from the beginning to the end of time, because changing your thoughts in response to objective data is considered a sign of weakness. They're enthusiastic to spread their discoveries of the various factors that are poised to sink the dollar.

If any of them are reading this, please accept the criticism that you're missing a once in a lifetime opportunity to exploit your special knowledge about the dollar's imminent path. First, you need to keep quiet. The more that the rest of the market shares your information, the lesser advantage you have. They don't know that their dollars are attached to little imaginary fuses that have almost run down. If they did, they would compete directly with your financial strategies; in any case they'd refuse to be the ignorant suckers whom you need to carry out the transactions.

Second, you need to act immediately. Your prediction is time-limited by its very nature. Once the shocking collapse springs into action, no more profit is possible. The longer that you delay, your unique prescience decreases in value. Like a stock, it's best to jump in before everyone else, at the earliest time, on the "ground floor".

Third, now that you've established the supreme unreliability and undesirability of the US dollar, you should think of a suitable alternative store of value. Since other economic agents will foolishly continue to expect that you honor their dollars and demand that you exchange dollars for their goods/services, the alternative must be relatively easy to convert. You could opt for a number of wild choices, but I suggest acting conservatively in this instance: gold pieces. Find a trustworthy local dealer who's not afraid of high volume and low margins. Don't resort to one of those huge gold piece dealers who show TV commercials; they're far too inconvenient for everyday use.

After buying gold pieces with your volatile dollars, treat the local dealer as your replacement bank. That is, periodically trade some of your gold pieces for intermediate dollars so that you can make your next batch of purchases. And when someone tries to hand you dollars, hurry to the your dealer to trade those "hot potatoes" for comforting hard gold pieces. Throughout your dealings, keep in mind that for you, dollars are like credit card balances. You use dollars purely for convenience and never hold the dollars long-term. That way, the horrible inflation rate that's right around the corner won't affect you too deeply. Going back to the stock analogy, you're closer to a day trader than a value investor. True, your earnings will be affected by the unfortunate fluctuations of the dollar, but only over very short time spans. You can also expect to lose some value due to the constant churn of conversion between dollar and gold piece, because the dealer probably expects to be compensated for his or her service and the general overhead of running the business. Think of these transaction costs as a reasonable price to pay in return for your peace of mind.

All this dependency on the gold market may make you queasy. Apart from the projected inflation of those irritating dollars, what if there are significant swings in that individual part of the economy? Those swings could erode the value of your gold pieces. To avoid that risk, you might diversify. In addition to the gold dealer, you might pursue other nonperishable assets for storing value. eBay is packed with smooth open markets for a wide variety of options. As one market goes down, you could buy using that market rather than the rest. As a second goes up, you could sell using that market rather than the rest. Call your diversified collection of markets your "basket of goods". Perhaps, at that point, you could publish your forecast of massive inflation far and wide, and then your audience would scurry to buy from your basket of goods. In this brave new world, where market participants cease to hoard dollars, you will be king.

Or you could avoid both gold pieces and diversified baskets of goods. Consider the Canadian dollar. The Loonie could be quite apropos.

Reverse Causality Illusion

2011-11-14T18:22:00.001-05:00

I debated whether to mention the Reverse Causality Illusion in my response to the book Willpower, but it was too tangential to merit space in the text and too obscure to describe in five or fewer sentences in a footnote. It's a plain idea with a short definition and lengthy implications. The Reverse Causality Illusion is a proposition about the future that "retroactively" causes changes in the present. It's much more common than it sounds. It underpins the driving narratives that motivate many human actions that are otherwise inexplicable. In a word, it's destiny¹. It's willpower's "precommitment" taken to the nth degree.

The strength of the Reverse Causality Illusion depends directly on the realistic treatment of the proposition about the future: adherence to every facet of the proposition. From a proponent's standpoint, the proposition is firmly true. Its part of the future is as irrevocable as the past². Contemplation of alternatives is fruitless. It's so certain that it repels emotional or intellectual fixation on its likelihood. Anxiety is unnecessary. Dread serves no purpose. Evidence that supports other outcomes has to be inconclusive. Decisions can't contradict it. Impulses that disagree are temporary delusions.

Is this a fatalistic outlook? Definitively. Is it restrictive? Sure, as restrictive as tying Odysseus to the boat mast so he's unable to follow the Sirens. Is it double-think? Yep, since the brain is fighting its thoughts with its thoughts (who gave me the right to order me around?). Is it nonsensical? Perhaps, unless and until someone's actions confirm its accuracy.

Some specimens achieve a still-higher notch of curious self-reinforcement. Whereas lower illusions enforce boundaries on actions and feelings, superior illusions enforce boundaries on doubts. Someone reasons, "The future is fixed but my belief in it is wavering; I should try harder to reassure myself about these facts that are yet-to-come." Essentially the "reality" of a proposition about the future takes precedence over each doubt of it. Thoughts that contradict the proposition are proof that the thinker is mistaken, not proof that the proposition is less likely! Thereby the supposed future can constrict just as completely as the past.

Nevertheless, a judicious Reverse Causality Illusion has its uses, especially for inducing calmness³ or pursuing an appropriate goal. To reiterate, in order for the illusory proposal to push through time in reverse and cause differences immediately, it cannot be weak. A working illusion is vivid and plausible. If it's not feasible or believable, it's too easily interpreted as an ignorable wish or passing whim. Necessary mental revisions or physical adjustments will increase its perceived probability⁴.

A sturdy illusion isn't enough. Perverse commitment is the second requisite. An internal mind game needs willing players. At all costs, the existence of the illusion must seem independent and indestructible. Regardless of its authorship, it transforms into an objective occurrence that happens to not be tangible. First the brain cultivates the illusion, then the brain obeys the illusion thereafter despite remembering the creative origin. Illogical or not, this strategy is actually consistent with well-known observations. Isolated parts of the brain aren't able to unilaterally sort reality from fantasy or true from false⁵. Uncritical acceptance is the default effortless response. Sections of the brain can deceive other sections by issuing prophecies.

Of course, the prophecy might be correct. Days ago, I had foretold that yesterday afternoon was the time period of tidying the exterior of my house. I dislike miscellaneous humdrum tasks, but what else could I do? I'd been expecting that event. My sense of annoyance about it had preemptively risen and subsided. I'd released my misgivings about it and shifted my angst to uncertain outcomes. Why should I continue to expend my limited animus on an impersonal and unavoidable circumstance?⁶

¹Reverse Causality Illusion is distinct from normal predictions. Predictions are far less pretentious, bossy, and insistent. Predictions are reasonable guesses which are tied to bounded confidence levels.
²Someone who's learned about Minkowski spacetime may ponder the notion that their whole worldline exists. Past, present, and future are merely coordinates. The future is "there" like the room at the end of the hallway is there.
³With the caveat that strong phobias, persistent depression, or pervasive anxiety require more drastic techniques and/or chemicals. Your mileage may vary.
⁴For example, in my communication courses I found that the cheery proposition "My next speech will go well" only felt tenable after rehearsing the speech and laying out my phrasing and gestures beforehand. Rehearsal may bother procrastinators precisely because it reminds them of the impending unpleasantness, but why not use a "safe" environment to confront and eliminate the worrisome unknowns of how the speech will go? It doesn't prohibit spontaneity at delivery time. To the contrary, close familiarity with the speech's overall delivery allows for better-informed choices of how to tweak its details freely as the situation warrants or as inspiration strikes.
⁵The same gullibility affects conscious memory. Oft-repeated or pleasing lies tend to be later recalled as truths, including when the hearer is aware of falsity at the original time of hearing. As the popular saying goes, never let the facts ruin a good story (or conventional wisdom, or common sense).
⁶I'm dramatizing. My hatred of that category of work is real but not deep.

belated comments on the Tron Legacy iso concept

2011-11-13T14:44:00.001-05:00

Far be it from me to pretend that the movie Tron Legacy is realistic or self-consistent. In an otherworldly setting, ridiculous details of many varieties are normal, if not expected. Why must the heroes do this or that? Why do objects have those peculiar shapes and colors? Just because¹.

However, the concept of the "iso" is an exception. It's as interesting to me now, when I stream the movie from Netflix, as it was when I first saw the movie in the theater. An iso is an incredibly complex intelligence that evolved from out of the movements of the information in the Tron "digital world", a highly unusual computer system that apparently is POSIX-compatible. The isoes are "bio-digital jazz, man." Supposedly the answers to every problem are achievable by them. No human or program can entirely comprehend them. They're judged too chaotic to be part of an unchanging perfection.

In my interpretation, the isoes are programs with a mind-blowing aptitude for modeling and understanding. A model is a duplication of a thing's information. Useful models enable prediction, analysis, and manipulation. To understand is to employ a working model. Of course, models may be imprecise, nonverbal, incomplete, etc.

We humans constantly brush up against limits in our ability to make models. It could be difficult to invent the right metaphor or to deduce the right mathematical expression. In any case, our brains, and therefore our mental resources for such tasks, are obviously finite. We can work around these hard realities in any number of ways, and we often do.

But imagine that the iso is a program whose computational equivalent is a brain as large as an average adult human body. With that magnitude of potential representations devoted to a model, it could be much more comprehensive and true-to-life. Although there would still be limiting factors (cf. combinatorial explosion), the model's simulations would more likely reflect the full subtlety². It would catch the higher-order, peripheral, and mutually-interacting effects, which are emergent³.

Furthermore, creative modifications of the model would have more room for exploration, so it could more easily compute how to respond to the model in order to achieve a benefit. In this sense, an iso is indeed jazz-like in its methods. It imitates, then it adds or subtracts to the original by informed trial and error. It doesn't need to resort to the human simplification of forcing something into a preconceived pattern for the sake of comprehension. Instead, it "absorbs" the thing into a mirror model, and travels the depths of that model.

An iso learns to defeat you at chess by discerning the reasoning process which you use for chess. Sooner or later it uses that knowledge to figure out which moves will circumvent the effectiveness of that process. After that point there's no real competition or game remaining; you can no longer "surprise" the iso with your decisions. You're the iso's pawn. What if the iso could similarly anticipate cancer's "moves"?

I'll grant that my interpretation might not be close to what the movie meant, assuming the movie had a specific idea in mind. Once someone presumes that artificial intelligence is possible, it's no great leap to speculate that it could be made superior to human intelligence, at least in restricted domains or according to superficial measurements.

¹To take just one example of the missed opportunities embedded in the movie's premises, the conversion of a human to and from digital data is essentially a teleporter. Or perhaps Wonkavision, when it happens over a wireless connection. Completing orders for merchandise "over the Internet" would be literal. Sharing a digitization of a person in Bittorrent would violate laws against human cloning, I suppose.
²Philosophically or theoretically, no one can ever say with absolute certainty that a sufficiently complicated simulation is perfectly accurate to an original. Yet most humans would opine that a simulation "works" (and is true) when it consistently matches all actual/empirical tests within a reasonable error threshold.
³I purposefully compared the iso to a brain rather than a supercomputer. It wouldn't be a massive array of motionless memory acted upon by unvarying procedures. The movie states that an iso is artificial life. The expectation is software layout that's inseparable from hardware layout, innumerable adjustable junctions between parts, and feedback loops galore.

Willpower and narratives

2011-11-09T14:10:00.002-05:00

I read Willpower by Roy F. Baumeister and John Tierney. According to the described research, energy is integral to self-control. Resistance to the temptation of short-term impulses is "real" work. These systematic study findings agree with anecdotal experience: tired or hungry humans are more likely to act irritable and self-indulgent, even when the fatigue is mental. Thus good sleeping and eating routines are causes or fuel for self-control as well as effects of it.

Philosophically speaking, these overall facts on willpower add to the formidable pile of compelling evidence against the mythical "disembodied mind" (soul). Humans have startling potential for decision-making and self-denial. But aside from abnormal genes and/or painstaking training, they have limits¹. They can't actually force their bodies to do whatever they wish, no matter how much perseverance stirs within their alleged supernatural "hearts". Most obviously, the underlying physicality, i.e. the brain, will shut down at some point due to simple exhaustion. Although before that happens, involuntary reactions probably will wrest the weary body away from the tyranny of conscious direction².

Whether or not someone believes in a mystical basis for willpower, the book has practical suggestions. I imagine that my initial retort is like other know-it-all readers: "That's it? If lasting behavior modification were that straightforward, why is failure so prevalent?" However, a few of the book's propositions fit snugly into my favorite personal schema for human willpower: narratives. The more effectively that someone constructs and follows a narrative of a desired plan of action to reach a goal, the greater the chance of success. Examples:

The book's preference of specific details over broad intentions is a factor in a narrative's perceived level of "reality". A vague narrative is prone to treatment as a fantasy. Mushy rules or to-dos are difficult narratives to interpret as true - or as false in the case of failure.
One of the book's foremost points is the need for frequent monitoring and short-term milestones, combined with the willingness to be flexible and forgiving in the immediate-term when inevitable complications occur. The same applies to a narrative, which must inhabit attention for it to be a tangible guide. Narratives must often enter everyday awareness in order to be measurements and signposts for actual deeds.
Long-term rewards and consequences require extra emphasis in thoughts, because the competing short-term incentives are naturally louder. Good narratives are clear on the desirability of the far-off benefit, so attentiveness to a narrative substitutes a different, imagined item for the present temptation. The narrative projects the decision-maker outside the influence of the current time and place. Taste of future victory compensates for withdrawal now. Other than abstract reinforcement, this mental operation is a helpful distraction³.
Orderly environments and personal habits aid willpower. Messes consume mental resources, leaving less room for considering proposed narratives. In contrast, tidiness of oneself is a subconscious increase in the narrative's plausibility. Little triumphs and confidence boosters bring it within reach. "I see that I can exert control. Maybe I can carry out a hard narrative too."
"Precommitment" could be the willpower technique that aligns closest with narratives. What else is the essence of a narrative, if not a vision of what future-me will do (and be)?

Perhaps the preceding are signs that I'm trying to cram the book's discoveries into a biased framework that I find attractive. I readily admit that I'm enamored with narratives as the "secret weapon" of humanity's advanced aspects. Language is a medium for narratives. Fancy plans are narratives ("Two right turns then a left"). Self-concept is a narrative. Sympathy is a narrative that features another, in which the self can be exchanged for the main character. Indeed, any interpretation of another's actions or knowledge depends on narratives⁴. Teachings on morality, character-building, and religion employ particularly dramatic narratives for manipulation.

Entrenched narratives are more than ideas, too, because humans act in response⁵. Unlike other organisms, which are dominated by fairly simple inborn drives and brains, humans incorporate surprising complexity in their decisions. Viewing themselves as part of larger narratives, their acts and roles need not exhibit complete biological/evolutionary reasonableness. The narrative mechanism enables a vertiginous third-person perspective beyond the self: "What I do here today will echo across history and inspire other trite expressions..." Angst-heavy humans are capable of envisioning the implications of a choice on the trajectory of the chooser's life story. They can be embodiments of principles. They can feel the coercion of idolizing an ideal version of themselves⁶. The grip of relentless narratives yields levels of human willpower which shouldn't be underestimated.

¹My father once commented that part of the fun of watching Survivor is to see how long the contestants manage to act normally. As normal as the typical Survivor contestant, anyway. The strain breaks/hardens people in different ways.
²Recall the common remarks, "I don't remember giving in. Eventually my attention wandered for a moment, and it happened automatically." Some commentators have said that humans more accurately have free-won't rather than free-will. Alcohol doesn't introduce strange motives. All it needs to do is suspend rational judgment. Conscious courage is the ability to override impulsive fear. Mere fearlessness could come from deficient perception or comprehension of risks.
³Distraction is underrated. Illusionists and experimenters have demonstrated repeatedly that a sufficient distraction virtually eliminates other stimuli. Stopping to ponder a questionable option is less advantageous than minimizing it by moving on. Simply put, more time spent simultaneously contemplating and yet "fighting" a motive corresponds to more moment-by-moment opportunities to surrender.
⁴Specifically, the "theory of mind" presupposes that the self's mind is an adequate model for others' minds. Their point of view is obtained by putting the self's mind in their narratives. Carl's eyes are shifty. If I were Carl, shifty eyes would indicate that I'm was hiding something. So by matching the real narrative of Carl with a hypothetical narrative about myself, I assume that Carl is hiding something.
⁵And according to what I've previously mentioned about philosophical Pragmatism, entrenched narratives play a still deeper role. The narratives that a human has judged to "work", by whatever standard, are the tools for constructing human truth from confusing/ambiguous raw data. Moreover, numerous confrontations with reality may prompt complicated revisions and additions to patchwork narratives. Otherwise the narratives no longer would "work". ("My paranoia can accommodate the new facts quite neatly...")
⁶The parenting section of the book doubts the effectiveness of training self-esteem versus training willpower. I don't dispute that. In terms of an aspirational narrative about the self, the distinction is how demanding the narrative is: not only the size of the relative gap between the ideal and the "real" self but also the extremity of the ideal. Humans who think "I've reached my apex just the way I already am" are aiming at a lower target than humans who think "I'm going to try to become elite".

to be agile is to adapt

2011-11-03T09:19:00.002-04:00

Not too long ago, I read Adapt by Tim Harford. It's an engrossing presentation of a profound idea: beyond a particular bound of complexity, logical or top-down analysis and planning is inferior to creative or bottom-up variations and feedback. Adaptation can be indispensable. Often, humans don't know enough for other approaches to really work. They oversimplify, refuse to abandon failing plans, and force the unique aspects of "fluid" situations into obsolete or inapplicable generalizations. They're too eager to disregard the possible impact of "local" conditions. Biological evolution is the prime example of adaptation, but Harford effectively explores adaptation, or non-adaptation, in economies, armies, companies, environmental regulations, research funding, and more. Although the case studies benefit from adept narration, some go on for longer than I prefer.

Software developers have their own example. Adapting is the quintessence of Agile project management¹. As explained in the book, adaptive solutions exploit 1. variation, 2. selection, and 3. survivability. Roughly speaking, variation is attempting differing answers, selection is evaluating and ranking the answers, and survivability is preventing wrong answers from inflicting fatal damage.

Agile projects have variation through refactoring and redesign while iterations proceed. Agile code is rewritten appropriately when the weaknesses of past implementations show up in real usage. Agile developers aren't "wedded" to their initial naive thoughts; they try and try again.

Agile projects have selection through frequent and raw user feedback. Unlike competing methodologies with excessive separation between developers and users, information flows freely. Directly expressed needs drive the direction of the software. The number of irrelevant or confusing features is reduced. Developers don't code whatever they wish or whatever they inaccurately guess about the users.

Agile projects have survivability through small and focused cycles. The software can't result in massive failure or waste because the cost and risk are broken up into manageable sections. Agile coaches repeat a refrain that resembles the book's statements: your analysis and design is probably at least a little bit wrong, so it's better to find out sooner and recover than to compound those inevitable flaws.

¹Of course, the priority of people over process is also quintessential.

shared links are different than RSS

2011-10-26T10:14:00.002-04:00

The point of RSS is automation. RSS is an established standard¹ so that no human needs to keep revisiting a long list of vetted topical sources that regularly publish worthwhile content. With RSS, a computer can rapidly and flawlessly determine whether another computer offers new content. If available, the content itself can be automatically retrieved, too.

Human link sharing is different. When humans share links, those links are likely intermixed with irrelevant items like unsolicited opinions. The links may be about a wide range of uninteresting topics. Although link sharing distributes the work of revisiting valuable websites and flagging fresh information, the flaws of a manual procedure continue to apply. How reliable is the crowd at performing this task? As a unit, they're more reliable than one person, but I daresay that a program is better still.

RSS is for mechanizing the syndication of "publishing sources". I'd say that humans who share links are more accurately classified as additional publishing sources than as "John Henry competitors" to efficient RSS software. I'll concede that link sharing can be a more convenient answer than RSS to the question, "How do I obtain a simple list of interesting links to visit?" But RSS is superior if the question is, "How do I, a unique individual, follow the ongoing activity of every source in a list that I control?"

Shared links won't replace RSS for my needs. To suggest otherwise is to misunderstand the purpose of RSS. When a website shuts down its feeds due to "lack of interest", I'll gladly turn my attention elsewhere.

¹Ha! RSS is a standard in the sense that you pick whichever one you like.

model dependent realism

2011-10-04T09:37:00.002-04:00

I read The Grand Design. I'm long acquainted with much of the history and physics therein, albeit at a conceptual not mathematical level. However, I was fascinated by the description of the entire universe as a Feynman path. I can't make any knowledgeable comments on that or the M-theory stuff, of course. I couldn't help wondering if the sections on renormalization and "negative energy" would've been easier to understand with the careful and hand-held inclusion of some undergraduate-level math. That's a hard balance to strike, though. Maybe I'll try some cross-referencing with the tome that's "heavy" in several senses of the word, The Road To Reality by Penrose. I doubt the two books share the same general opinions.

Since I'm monotonous, I'm obligated to compare the book's "model dependent realism" with my interpretation of philosophical Pragmatism. I noticed many similarities. In model dependent realism, humans perceive reality through the lens of a model. In Pragmatism, humans perceive reality through the lens of subjective elements like desire, focus, analysis, synthesis, theory-building, etc. In model dependent realism, humans select models for the sake of "convenience". In Pragmatism, the convenience of thoughts about reality is explicitly tied to how well the thoughts "work" for purposes. In model dependent realism, humans replace models as they compare the accuracy by experiment. In Pragmatism, humans adjust their knowledge of truth as they actively determine which individual truths are confirmed "in practice". Most infamously, in model dependent realism, an ultimate universal model of reality might simply be impossible, except as a quilted combination of an array of limited models. Just as infamously, in Pragmatism, truth isn't a standalone all-encompassing entity, except as an evolving collection of ideas whose two coauthors are the human and their whole environment.

evolution and DRY

2011-09-28T10:47:00.000-04:00

Reality may be unintuitive. One instance among many is the evolutionary march that yields complexity from chaos, harmony from cacophony, solution from error. At the timescale of a human life, and applied to everyday objects, such a progression is nonsense. In information-theoretic thinking, the introduction of greater randomness to existent information only degrades it with greater uncertainty or "noise", necessitating a communication code that can compensate by adding greater message redundancy. In thermodynamic thinking, the much larger number of disordered states overwhelm the small number of ordered states. Two gases in one box will mix simply because that's far more probable on average than all the undirected gas particles staying apart in the original groupings. In parental thinking, miscellaneous household items won't land in specific designated positions if children drop the items at randomly-chosen times and locations (a four-dimensional vector governed by a stochastic variable).

Clearly, accurate metaphors for biological evolution are lacking. Humans are justifiably amazed by the notion of entire populations of intricate organisms changing for many millions of years. And the changes are related to numerous shifts in myriad factors, including habitat and competition. It's problematic to transplant prior assumptions into that expansively complicated picture.

But a metaphor from software development might be helpful for contemplating the stunning results achieved by the unpremeditated mechanisms of evolution. No matter the particular need served by the software, a solemn rule of development is "don't repeat yourself", which is abbreviated to "DRY". The intent of the rule certainly isn't the naive claim that no repetition ever happens. To the contrary, the rule is about handling repetition correctly. Following DRY is typing a "chunk" of software just once and then somehow rerunning that solitary chunk on different data whenever necessary. The alternative to DRY is duplication. Duplication is often cheaper at the time of initial development, although it's costlier thereafter: two or more chunks are naturally laborious compared to one.

Besides the long-term savings in maintenance work, aggressive DRY has a second effect. The software is divided into chunks. These divisions and subdivisions are easier to read, understand, and analyze. Organization and interconnections take the place of a flat sequence. Appearances suggest conscientious craftsmanship, independent of any knowledge of the software's developers.

Hence, a DRY-compliant outcome has a tendency to look artificially arranged. Evolution could fall in that category. Obviously, unlike in software development DRY can't be a conscious guideline. Instead, inherent constraints "encourage" DRY to occur. Since normally DNA is strongly resistant to massive change, perhaps outright duplication of a gene across separate strand locations is improbable. Reuse of the original gene in a new context accomplishes an identical adjustment in the organism. The DRY-like modifications of the DNA trigger matching DRY-like modifications to the creature. It appears to be the product of a history that's less chaotic than it really is. Thus phenomena that human brains find comprehensible or beautiful, like symmetry or hierarchical containment, arise from frugal genes. So do displeasing phenomena, like unsightly remnants of a contemporary body part's surprising past. DRY pushes for tweaking existing genes and transforming an existing appendage of little value, rather than partial duplication of existing genes and adding more appendages.

Insistent objectors could aver that the DRY metaphor violates the common sense of the conceptual chasm between thoughtful human software developers and thoughtless genetic mutations and transcriptions. The computer programming languages of typical software are specifically designed to enable smaller chunks. Languages incorporate syntax and semantics. How can that be comparable to the simple streams of codons in DNA? Reuse can't happen without a structure to support it. Pronouns are literally meaningless without basic grammar.

Odd as it seems, the genetic code indeed has a grammar. The fundamental building blocks of grammar are symbols that modify the interpretation of other symbols. Here, "interpretation" is the translation of nucleic acids into working cell proteins. Over time, discoveries have shown how subtle the translation can be. It's affected potentially by a host of activity. Genes definitely can adjust the expression of other genes, which is why geneticists hesitate to assign premature importance to single genes. Some of the "words" in this haphazard language are likely grammatical in impact on protein replication, akin to "and", "or", "not". Some might serve both independent and regulatory functions. Incomplete human understanding doesn't cast doubt on the existence or the capacity of evolution's code. It could very well be able to encode the reuse of sections in accordance with DRY-like conservatism. Just as replacing one word in a sentence might have drastic overall implications, replacing a minimal quantity of genes might have drastic overall consequences that give off the impression of evolution acting smart or informative or experienced.

MythTV archiving nowadays

2011-09-25T13:24:00.003-04:00

I haven't mentioned MythTV in a long, long time, largely because I stopped messing around with either the hardware or software in that machine. Some months ago I attempted to upgrade the software but it kept failing before completion. Fortunately, the backup/restore feature allowed me to recover fairly easily each time. Between that difficulty and the extent by which the rest of my devices have since left the machine's hardware in the dust, I'd need to restart the entire project to do it properly. I'm not eager to expend the time, money, or energy for that (plus, the MythTV competitors are so much better now than back then...).

Regardless, it keeps working year after year, so I keep using it. When I seldom overrode the auto-delete of old recordings, my customary procedure for "archiving" was to run nuvexport to convert the source MPEG-2 file, captured/encoded by the PVR-350, into an XVid AVI. The result sometimes contained some bursts of unfortunate side effects, like small to medium "blocking", yet I thought this was a reasonable compromise for the sharply reduced storage requirements. Watching too closely yielded the impression of a video being portrayed by many tiny crawling ants. But quick and well-organized ants, I must admit, especially when seen from twice the usual viewing distance.

Recently, as I kicked off a nuvexport job and felt annoyed once again by the estimated time, I finally recognized the antiquated absurdity. The MythTV machine was put together using rather cheap parts that were current a few generations previously. My main Ubuntu desktop is a more modern computer with the corresponding increases in capability and performance. Moreover, my experiences with streaming services like Netflix or Amazon have reminded me of the advances in video compression. Time to rethink.

Instead, I've switched to transferring the source MPEG-2 from MythTV using the MythWeb interface's Direct Download, so the archiving work can exploit the newer hardware and software of the desktop. I run the h264enc script without many custom answers. The H.264 MP4 files look pretty good at around the same bitrate. And probably due to both the higher clock rate and additional specialized CPU instructions, the process really doesn't take that long to run: the stated output fps rate is faster than playback. This is despite the "nice" priority which prevents interference with all other tasks.

Of course, one "pre-processing" step remains in MythTV; I continue to employ the extremely easy interactive "Edit Recordings" feature (I've never trusted the automatic commercial detection). With the rapid "loss-less" option, the chosen edits produce just a shorter MPEG-2 file, ready for further manipulation elsewhere.

NOTE (Oct 1): Another side effect of switching to a video compression format that's conquered most of the world is that the Roku 2 supports it. But given that I have a working MythTV installation, this hardly matters...

peeve no. 265 is users blaming the computer

2011-09-11T13:58:00.001-04:00

No, user of the line-of-business program, the computer isn't the trouble-maker. It could be from time to time, if its parts are old or poorly-treated, but problems at that level tend to be much more noticeable than what you're describing. Generally, computers don't make occasional mistakes at random times. Despite what you may think, computers are dogged rather than smart. Computers do as instructed, and by "instructed" I mean nothing more than configuring the electricity to move through integrated circuits in a particular way. Computers can't reject or misunderstand instructions. No "inner presence" exists that could possibly do so.

I understand that placing blame on "the computer" can be a useful metaphor for our communication. But the distinction I'm drawing this time is substantive. To identify the precise cause of the issue that you've reported, a more complete picture is necessary. Your stated complaints about the computer's misdeeds really are complaints about something else. The reason to assign blame properly isn't to offer apologies or excuses. Figuring out the blame is the first step in correcting the issue and also in preventing similar issues.

Possibility one is a faulty discussion of the needed behavior for the program, way back before any computer played a role. Maybe the right set of people weren't consulted. Maybe the right people were involved, but they forgot to mention many important details. Maybe the analyst missed asking the relevant questions. Now, since the program was built with this blind spot, the issue that you reported is the eventual result.
Possibility two is a faulty translation of the needed behavior into ideas for the program. Maybe the analyst assumed too much instead of asking enough questions. Maybe the analyst underestimated the wide scope of one or more factors. Maybe the analyst was too reluctant to abandon an initial idea and overextended it. Maybe the analyst neglected to consider rare events that are not so rare.
Possibility three is faulty writing of the program itself. Maybe the coders overestimated their understanding of their tools and their work. Maybe the coders had comprehensive knowledgeable and didn't correctly or fully express what they intended. Maybe a fix had unfortunate side effects. Maybe the tests weren't adequate.
Possibility four is faulty data. Like blaming the computer, blaming the data is a symptom. Maybe something automated quit abruptly. Maybe manual entry was sloppy. Maybe the data is accurate and nevertheless unexpected. Maybe someone tried to force shortcuts. Maybe management is neither training nor enforcing quality control.
Possibility five is faulty usability, which faulty data might accompany. "Usable" programs ease information processing from the standpoint of the user. Maybe the program isn't clear about what the user can do next. Maybe unknown terminology is everywhere. Maybe needless repetition encourages boredom and mistakes. Maybe, in the worst cases, staff decide to replace or supplement the program with pen marks on papers or fragile spreadsheets containing baroque formulae. Downfalls in usability may disconnect excellent users from excellent programs.
Possibility six is the dreaded faulty organization, in which various units disagree or the decision-makers are ignorant. Maybe definitions are interpreted differently. Maybe the "innovators" are trying to push changes informally. Maybe the realm of each unit's authority are murky and negotiable at best. Maybe units are intentionally pulling in opposite directions. Regardless, the program probably will fail to reconcile the inherent contradictions across the organization.

Often, in the Big Picture of the blunder, the computer is the most innocent of all contributors.

software developers like punctuation

2011-09-08T11:37:00.002-04:00

Comparison of equivalent snippets in various programming languages leads to a stable conclusion about what developers like: punctuation. Namespaces/packages, object hierarchies, composing reusable pieces into the desired aggregate, and so on are relegated to the despicable category of "ceremony". Better to use built-in punctuation syntax than to type letter sequences that signify items in the standard libraries. Developers don't hate objects. They hate typing names. Hand them a method and they'll moan. Hand them a new operator that accomplishes the same purpose and they'll grin. What's the most noticeable difference between Groovy and Java syntax, in many cases? Punctuation as shortcuts. Why do some of them have trouble following Lisp-y programming languages? Containment in place of punctuation.

Oddly enough, some of the same developers also immediately switch opinions when they encounter operators overloaded with new meanings by code. Those punctuation marks are confusing, unlike the good punctuation marks that are built in to the language. Exceedingly common behavior can still be unambiguous if its method calls are replaced by punctuation, but behavior of user modules is comparatively rare so the long names are less ambiguous than overloaded operators. "By this module's definition of plus '+', the first argument is modified? Aaaaaghhhh! I wish the module writer had just used a call named 'append' instead!"

deadlocks in the economy

2011-09-07T14:37:00.003-04:00

The affliction of a specialist is the compulsion to redefine every discipline into that specialty. As someone in a software job, I see the economy exhibiting mishaps of coordination. Programs executing simultaneously, or one program whose several parts execute at once, might interfere and cause confusion. In an economy, legions of economic agents engage in transactions. Given the difference in scale, coordination issues probably are more, not less, applicable to the economy than to a computer. Some of the names, like producer-consumer, even invoke the comparison.

A fundamental topic in program coordination is the "deadlock". Put simply, a deadlock is whenever all collaborators end up waiting on counterparts to act. Say that there are two programs, each of which needs exclusive access to a pair of files to do some work (e.g. the second file might be a "summary" which needs to be updated to stay consistent with the first file after it changes). 1) The first program opens the first file. 2) The second program sees that the first file is already opened, so it naively opens the second file before the first program can. Voila! The first program waits for the second program to finish up and relinquish the second file, while the second program waits for the first program to finish up and relinquish the first file. Everything is "locked" without any way to proceed.

Back to economics. An economy is a massive set of roughly circular flows. Buyers send money (or liquid credit) to a seller, and the seller sends the desired item back. The seller then (possibly) reuses the money as a buyer, and the buyer then (possibly) reuses the item as a seller. If the buyer obtains money in the labor market, i.e. working a job to earn wages, then that's another flow which connects up to this one. These flows continually recirculate during normal functioning.

However, clearly a stoppage (or slippage) in one flow will also affect other flows. This is the economic form of a deadlock: economic agents that halt and in so doing motivate additional agents to halt. Until flows restart, or a newly created substitute flow starts, nobody progresses. No money or items are moving, so each is facing a shortage. Moreover, without the assurance of complementary flows in action, it's in an agent's selfish interest to wait rather than take risks. Therefore everyone waits for everyone to take the first step. Sounds like a deadlock condition to me.

Examined from a high level, deadlocks are clearer to spot. For instance, if the interest paid on a loan for a house is linked to a shifting rate and the rate and the interest both increase, then there could be a lack of funding to cover it. Unpaid interest implies a reduced flow in the money earned by the loan, as well as a corresponding reduced value for the loan itself (a loan without paid interest isn't worth much!). The current and projected reduction in the flow of interest disrupts "downstream" flows that otherwise would've relied on that interest. So the owner of the loan must reallocate money. That reallocated money isn't available for other lending flows. The intended recipients of the other lending flows are left unable to follow their own economic plans. And so forth. Eventually, the original cutoff may come full circle; due to the propagation of effects, larger numbers of loan-payers don't have the flow to pay their interest. The payers can't fulfill the interest payments when lenders have ceased usual risk-taking, and the lenders continue to cease usual risk-taking when payers can't fulfill interest payments. Money is in deadlock. Thus the economy's assorted flows of items (including jobs), which require the central flow of money (or liquid credit) to temporarily store and exchange value within transactions, are in deadlock too. The trillion-dollar question is which technique is most beneficial to dislodge specific deadlocks of money or to cajole activity in general.

Unlike software, humans are improvisational. Confronted with deadlock, they don't wait forever for the deadlock to break. Instead, they adjust, although it could be uncomfortable. Economic flows that were formerly wide rivers might become brooks. Flows that started out as trickles might become streams. Over a long time period, deadlocks in an evolving economy are temporary. Circumstances change, forcing humans and their trading to change.

local minima and maxima

2011-09-06T16:23:00.003-04:00

I'm sure it counts as trite to mention that humans aren't great at coping with complexity. (Computers can but only if the complexity doesn't require adaptability or comprehension.) One example is the oversimplified dichotomy between systems: 1) few pieces with highly organized interconnections and controlled variances among the pieces, 2) numerous similar pieces with little oversight that nevertheless mostly act the same and have few decisions to make. An engine is in #1. An ant colony is in #2. A projectile is in #1. A contained cloud of gas particles is in #2. In #1 systems, analysis is rather easy because all the pieces are ordered to accomplish parts or stages of a defined objective. In #2 systems, analysis is rather easy because the actions of all the pieces are generalizable into "overall/average forces". In #1 systems, statistics consist of a series of well-determined numbers. In #2 systems, variances and aggregates are tameable by modeling the population distribution.

The problem is that as useful as these two categories are, reality can often be more complicated. Loosely-connected systems could consist of many unlike pieces. Or the pieces could be alike yet affected in a nonuniform manner by an external disturbance. Or each piece could individually respond to five factors in its immediate neighbor pieces. Or pieces might have ephemeral subgroups which act in ways that loners don't. The possibilities are abundant and stymie attempts to classify the system as #1 or #2.

Consider a minimum or maximum quantity that represents a system. In a #1, that quantity is calculable directly by finding the corresponding quantities for each piece. In a #2, that quantity is an equilibrium that all the pieces yield through collective activity. Either way, the system has one minimum or maximum, and it's reached predictably.

However, this conclusion breaks down when a system is of a "more complicated" kind. Those systems contain pieces that, taken one at a time, are easily understood, but whose final effect is difficult to fathom. As a representation of that system, the minima and maxima could be messy. For instance, a specific constraint is strongest at the lower end of a range but a second constraint is strongest at the higher end. Under such circumstances, the system has more than one minimum or maximum. To the extent that the description works, the "forces" of the system then push toward the local minimum or maximum, whichever is closest.

From the viewpoint of an uninformed observer trying to cram a complex system into #1 or #2, the apparent failure to reach the absolute furthest (global) maximum could be mystifying. If it's caught in the grip of a local maximum, then the failure is more intelligible. The system "rejects" small changes that result in an immediate "worse" outcome regardless of whether or not it's on the path to an ultimately "better" outcome. In short, a wildly intricate system occasionally gets stuck in a pothole of inferiority. And for that system, that state is as natural as anything else.

Hence knowledge of local minima and maxima provides greater nuance to human interpretation. Reasoning about the national economy is a ripe area. The temptation is to reduce discussion into the relative merits of a #1 system, in which the economy is like a tightly-directed machine operated by government, compared to a #2 system, in which the economy is like a spontaneous clump of microscopic participants. This is a discussion about nonexistent options. The economy isn't solely a dangerous beast that needs strict supervision. It isn't solely a genie that showers gifts on anyone who sets it free. It's beyond these metaphors altogether.

An economy that has run aground on local minima or maxima can't be adjusted successfully by treating it as a #1 or a #2 system. "Freeing" it won't erase the slope toward the local minimum. "Ordering" it to stop misbehaving also won't. The economy doesn't always accomplish every desired purpose. On the other hand, government can't completely override the economic transactions of the entire populace (nor should it try). What government can do, potentially, is help nudge the economy out of a local minimum by bending the system. Of course, the attendant risk is that excessive bending by the government might set up a new local minimum in the economic system...

git-svn on Windows and Robocopy

2011-09-02T08:44:00.002-04:00

So...git clones of git-svn repositories aren't recommended. (Neither are fetches between git-svn clones. All collaboration should happen through the Subversion server.) Clones don't include the metadata that links the git and Subversion histories. However, unless commits in local branches are backed up elsewhere, work could be lost when catastrophe strikes the lone copy of those commits.

As decentralized version control, git's information is self-contained in the ".git" subdirectory of the repository. Thus creating a backup is straightforward: duplicate that subdirectory. But the common copy commands are wasteful and unintelligent. Must one overwrite everything every time? What about data that's no longer necessary and should be deleted in the backup location as well?

Fortunately, there has been a ready command available in Windows: Robocopy. In this case, it's executed with the /MIR switch. Between git's filesystem-based design (i.e. no database dependencies or complicated binary coding) and Robocopy's smarts, incremental changes to the git-svn repository usually result in minimal work performed during subsequent calls to Robocopy.

A developer could also mirror the entire contents of the working directory, but the pattern of making many small local commits in the course of a workday means that at any time there are few uncommitted changes in the working directory. At the time that the commits will be pushed to Subversion, an interactive rebase beforehand ensures that the "noise" of the numerous commits won't be preserved in the permanent version control record of the project.

rule of 8...

2011-09-01T13:40:00.000-04:00

...If someone discusses the budget of the federal government of the U.S.A. with fewer than 8 independent numbers or factors or categories, then chances are that the discussion is oversimplified. Figures have been skipped to present a "condensed" viewpoint that's more consistent with a partisan narrative. The 8 or more numbers are "independent" in a statistical (information-theoretical) sense: none of the numbers is a complete mathematical function of the others, such as the third always equaling the difference of the first and second.

the absent mind

2011-08-31T09:29:00.000-04:00

Recently I watched Memento again. I've never owned it but I can watch it via Netflix. No later viewing of it compares to the first, because surprise is gone. Nevertheless the progression of scenes in reverse chronological order is appropriately disorienting as ever.

This time, though, I reacted in a new way. Like an untrained reader who self-diagnoses every disease in the book, I identified with the main character. In my case, the similarity has nothing to do with the inability to form new memories. My memory isn't problematic; based on academic accomplishment, it's above average. No, the similarity is the need to leave reminders for myself. As I watched the movie, I distinctly recall thinking, "Doesn't everyone do that, but on a smaller scale?"

Age can't be the cause. I've been "forgetful" in this way since childhood. The usual driver seems to be having an "absent mind". Some people mock the truism, "Wherever you go, there you are." Those must be the people for whom it isn't a challenge. My thoughts are noisy. Emptying my mind is difficult because sometimes it churns and overflows without prodding. It's as if I'm walking through dreamlike surroundings that are far more engrossing than my real environment. External occurrences can't be guaranteed to leave a lot of impact, during a deep swim in the brainwaves. Of all activities done to excess, theorizing has to be one of the most unobtrusive to onlookers. On occasion, when the train of thought jumps tracks, I'm shocked by the time that's passed. The questions "Where am I?" and "What is the time and date?" are generally unimportant. Depending on which ideas are consuming you, the location of your body and the continual ticking of seconds are totally irrelevant.

Calling it attention deficit disorder would be inaccurate. It's closer to having too much focus or fixation than having too little. And it's also not autism. I tend to have trouble interacting naturally in casual conversation, yet the obstacle is distraction rather than disinterest (I make eye contact although some have commented that I appear to be looking "through" the speaker). My sense is that the low severity of absent mind doesn't qualify at all as a disability. It's a disposition, a personality quirk. As a child, it manifested in toothbrushes left behind after a sleepover or gloves left behind after winter travel from one building's interior into another. The tips for coping aren't too complicated:

If something needs to be remembered, but at some point after the immediate moment, make a reminder soon. Otherwise there's the possibility that an intriguing stream of thought will resurface before too long and flood everything else. Don't presume to remember to make a reminder...
Reminders don't need to be lengthy, detailed, or time-consuming. Unlike Memento's main character, the hang-up isn't that the memory is completely missing but the high probability that the intact memory won't return to mind at the proper time. All that's necessary is an unambiguous trigger. For instance, the infamous trick of tying a string around a finger is too uninformative, while a note consisting of one significant word could work. One mustn't then misplace or ignore the note, of course. It should be located within normal view.
Memento emphasizes routine and the ease of learning repeated subconscious reactions. That fits with my experience. Habits demand less concentration, which is a perpetually scarce resource. Breaking habit is healthy and invigorating, but special effort is called for. To succeed, novel actions require greater levels of deliberation and care. More present, less absent.
As in all things, simple strategies shouldn't be overlooked. Startling sensations sharply disrupt introspection. Increased alertness to the world is an automatic side effect of fresh perceptions. Simply keeping your eyes moving might be adequate. Administering pinches is a second tactic, as long as it isn't overdone.
Reminders work well for single tasks. Habits work well for routine or periodic tasks. A large list of irregular tasks, perhaps to be completed in the upcoming week, is better attacked by outlining a comprehensive schedule. Without explicit scheduling, there's considerable risk that awareness of the tasks will weaken, only to finally revitalize at an inopportune time. Or that one prominent task will monopolize available energy and mask the rest. Either way, an all-encompassing arrangement of what to do and when is, strangely enough, less of a chore than trying to repeatedly rein in a stampeding brain and urge it toward item by item. I compose the whole schedule, as a draft in progress. Then I think of it whenever I'm pondering or visualizing the time periods in the schedule. Writing or typing it is purely optional (memory isn't the problem, engulfment by my own reflections is the problem). As for an event more than two weeks away, those of us with absent minds act like everyone else. We enter it on calendars.
As illustrated by Memento, anticipation is a highly sophisticated approach. Self-knowledge enables self-manipulation. Small adjustments, like temporarily moving objects to atypical positions, can prompt corresponding adjustments in behavior. "Why is that there?...oh, to ensure I don't leave it behind." Unsubtle clues either illuminate the desired path or erect a barrier for the inadvisable path. I in the future am likely to be all right with or without the hints, but the prevention would help in the eventuality of unforeseen internal distractions (e.g. trying to recollect the name of the hidden Elvish kingdom in The Silmarillion, which was one of the last to fall to Morgoth).
Selected undertakings force full engagement, regardless of contemplation's pull. With practice, it's easier to adapt. For example, I've gradually developed my skills at driving and athletics. At work, I'm more conscientious about not drifting off during meetings, and I ensure that all my assignments are written down.

I don't want to produce the wrong impression. I'm not overly burdened by this. A bent toward abstract preoccupation is part of who I am. I realize it's not comparable to serious conditions like dementia, which I wouldn't dare to minimize.

I also recognize that a sound albeit absent mind comes in varying degrees. I've only forgotten my keys when leaving the house once. A while back someone notified me that I was wearing two unmatched shoes. I have a desk drawer that contains a stack of papers with the cryptic markings that result from pursuing a particular flash of inspiration over a few days. It could be worse. Apparently, others have been known to go out in public in partial undress, and their mysterious notes fill filing cabinets...

sure, I'll use your free storage

2011-08-30T13:53:00.000-04:00

I just noticed that my years-old unused Hotmail account has the side effect of a "Sky Drive". 25GB. The per-file limit of 50MB is adequate for most personal data files. Uh, thanks, I guess.

truth contingent on its effects

2011-08-27T13:37:00.000-04:00

It is difficult to get a man to understand something when his salary depends upon his not understanding it. -Upton Sinclair

Depressing though it may be, the above quote is consistent with many people's experiences in attempting to coax someone to a different opinion or just to convince someone to admit unlikable facts. If humans reached truth solely by accumulating and cross-checking knowledge obtained from trustworthy sources, as some philosophies presume, then the truth of a statement couldn't be so contingent on the effects of believing in the statement. Philosophical Pragmatism is singularly unsurprised, however. Since judgment is an ingredient in perception, biased judgment clouds perception. Seeing things "as they are", in practice means seeing things "as I see them". An isolated thing has no meaning. Brains compute meaning by laying isolated things side by side.

In the Pragmatist model, humans observe, reach conclusions, form plans based on those conclusions to reach goals, and then execute the plans. But this indivisible process can surely operate backward as well. Meaning, I don't want to act, therefore I doubt the conclusions that would force me to act, therefore I doubt the observations that would force those conclusions! The longer that a human clings to a "truth", the longer that a human selectively collects evidence in favor of it, and the stronger it becomes, by the human's design.

I'll end by noting a corollary of Sinclair's quote. It's difficult to get a politician to understand something when their election depends on their not understanding it. Basic accounting, biology, and climatology are notable examples.

I wish

2011-08-26T18:17:00.000-04:00

Part of acting as a mature adult in a complicated human society is to express statements that are half-true at most. Sincerity isn't a prerequisite. To avoid offense and thereby smooth social interaction, signaling an effort to be considerate is more important than unconditional agreement with the other's ideas on a divisive topic.

Nevertheless, I bristle at one kind of diplomacy that irreligious believers offer: "I wish." The words vary, but the constant is an empathetic disconnect in which emotional preferences are for religious rather than natural notions. "I wish your god existed." "I wish human souls lived forever." "I wish the universe were simple."

My irritation comes from a personal distaste for these sweet wishes. Yes, it may be true that I wish either that reality were more congenial to me or that I could more easily manipulate it to be so. The first is utopia and the second is magic. Of course these wishes are pleasing to compute; that's the whole rationale for a wish. How reassuring is it for the irreligious to admit the obvious, that they too would like human existence to not be so difficult?

A further problem is that courageous advocacy for fictional paradises will fall apart under follow-up questions. What if the hypothetical god acted like _____? What if living forever included harsh judgment for _____? What if a simple universe implied the impossibility of ______? Fully outlined proposals of alternatives might not be as enticing as the one-paragraph summary...

git's index is more than a scratchpad for new commits

2011-08-23T10:45:00.004-04:00

Someone relatively inexperienced with git could develop a mistaken impression about the index. After referring to the isolated commands on a quick-reference guide or on a "phrasebook" that shows git equivalents to other VCS commands, the learner might, with good reason, start to consider the index as a scratchpad for the next commit. The most common tasks are consistent with that concept.

However, this impression is limiting. More accurately viewed, the index is git's entire "current" view of the filesystem. Commits are just saved git views of the filesystem. Files that the user has added, removed, modified, renamed, etc. aren't included in git's view of the filesystem until the user says, with "git add" for example. With the exception of before the very first commit, the index is unlikely to ever be empty. It isn't truly a scratchpad, then. When checking out a commit, git changes its current view of the filesystem to match that commit; therefore it changes the index. Through checkouts, history can be used to populate git's view of the filesystem. Through adds, the actual filesystem can be used to populate git's view of the filesystem. Through commits, git's view of the filesystem can be stored for future reference as a descendant of the HEAD.

Without this understanding, usage of "git reset" is infamous for causing confusion. With it, the confusion is lessened. A reset command that changes the index, which happens in the default or with option --hard, is like a checkout in that it changes git's view to the passed commit. (Of course the reset also moves the branch ref and HEAD, i.e. the future parent of the next commit.) A reset command that doesn't change the index, which happens with option --soft, keeps git's "view" the same as if it remained at the old commit. A user who wanted to collapse all of the changes on a branch into a single commit could possibly checkout that branch, git reset --soft to the branch ancestor, and then commit. Depending on the desired effect, merge --squash or rebase --interactive might be more appropriate, though.

Post-Script: Since this is aimed at git newcomers, I should mention that before trying to be too fancy with resets, become close friends with "reflog" and "stash".

Post-Script The Second: Drat. The Pro Git blog addressed the same general topic, but based more directly around "reset". And with attractive pictures. And a great reference table at the bottom.

why it isn't done yet

2011-08-16T20:00:00.000-04:00

Modifying decrepit C code ain't like dusting crops, boy! Without precise calculations we could fly right through a star or bounce too close to a supernova, and that'd end your trip real quick, wouldn't it.

omniduction break down II

2011-08-13T19:30:00.001-04:00

To lampoon the use of omniduction...

Inviolable Proposition 1: The Constitution is a sublime and superb document, and if interpreted in a particular way (not necessarily the same way that judges have), it would cure society's ills.
Inviolable Proposition 2: Political compromise is a despicable, cowardly act that yields terrible results. Staring contests are better.
Historical Fact 1: The Constitution is packed with political compromises.

SYSTEM ERROR!

the dash hole principle

2011-08-13T09:59:00.000-04:00

The cigarette lighter receptacle has an amusing name. In my automobile and many others that I've seen, the present form isn't actually usable for lighting cigarettes. Now it's a round hole in the dashboard with a cover that's labeled as a power outlet. Over time, cigarette lighter receptacles turned into dash holes. The users of an object emphasized the secondary applications of it until the object itself dropped its primary application. It changed meaning through inventive usage.

Software users can be expected to act the same. Software developers should accept that the users, acting like humans, will adapt by introducing their own concepts and assumptions to a "finished" project. As DDD advises, the key is their language. When they speak about the software, and therefore the underlying design or data model, their words throw attention onto their interpretation of the "problem domain". They might describe data groups/categories and store their evolving understanding with rigid entries, like attaching "special" semantics to product identifiers that start with "Q". They might take several hours to run a series of automated preexisting reports, stuff the conglomerated figures into a spreadsheet, and then generate a chart - additional work which could all be accomplished by a computer in a tenth of the time.

The point is, software in the hands (and brains) of users can easily become a dash hole: an original design that came to be viewed much differently in practice. Developers who don't meet the needs of users will be bypassed manually as time goes on. In some cases, this may be a good approach. Some changes in usage just don't justify substantial software modifications. However, to state the obvious, not everyone is a good software analyst. Ad hoc solutions, enforced not by the software but by scores of unwritten rules, are prone to causing data duplication due to no normalization, chaos due to employee turnover or mere human frailty, and tediousness due to not thinking thoroughly about the whole process.

Dash holes function as adequate power outlets. But imagine if irritating dash holes could've been replaced with something designed to serve that purpose.

irreligosity does not imply apathy

2011-08-11T14:12:00.000-04:00

A typical response to anything said or done by the irreligious is, "Why? If by your own admission religious ideas are irrelevant to you, then what are you accomplishing?" Let me list a few...

Public expression and inclusion of irreligious beliefs deserves as much protection and accommodation as religious beliefs. Irreliogiosity is of course really a broad classification, so the specific form and content of it varies greatly. Just as there cannot be a single universally recognized symbol for religiosity, there cannot be one for irreligiosity. I like a stylized atomic diagram as a symbol to indicate philosophical materialism ("disciples of Democritus!"), but presumably others of less strict views might prefer alternative irreligious symbols. (A critic of irreligosity would probably make the facetious suggestion of a "No symbol".) In any case, public exposure of irreligous beliefs does matter. Acknowledgment and tolerance of presentations of a belief reinforce the freedom for it to exist in a society. Disqualifying the mere exposure of a belief has a corresponding intimidation factor against potential believers. Removing a belief from discussion also gives the false impression that it doesn't exist or barely exists. Religious believers may expect to receive divine rewards for their publicity efforts, but secondarily they feel mundane emotional satisfaction simply for displaying the "truth" to society. Irreligious believers have purely the latter as motivation; nevertheless, their self-esteem too is boosted by feelings of belief "validation" regardless of how perfunctory it may be.
Irreligious encouragement of anti-conversions may prompt the taunt, "What are you trying to do, save the lost from going to heaven?" The taunt is well-aimed in that self-consistent irreligious definitely can't be motivated by impossible consequences in a non-existent afterlife. And it's also true that they can't be logically motivated by spiteful envy of a religious individual's bus ticket to heaven. Still, they may wish sincerely and unselfishly for anti-conversions in order that more humans spend lesser proportions of their limited lifespans in accordance with fallacies. Unlike the concept of humans who continue on in some shape for all time, humans composed of atoms have finite time which shouldn't be wasted or predicated on false hopes. Pascal's Wager contains the idea that a religious life ultimately disproved is no loss relative to an irreligious life ultimately proved. Wrong. Religiosity exacts non-zero tangible and intangible costs, which vary widely by belief system, and a singular life implies that the decision to pay those costs can never be recouped. Anti-conversions enable someone to expend the rest of their time as they choose. No guarantees apply to the amount of time left, either.
Some may opine that irreligious notions are too ill-defined, or too defined by negatives, to have effects like religious notions do: "Acting on behalf of 'no religion' make no sense; there's nothing of substance to debate or achieve!" In some situations, that could be correct. Apart from the definition "uncommitted to a religion", general irreligiosity doesn't have unifying creeds and ideals. It does have a unifying cause, though: blunting or reversing the detrimental outcomes of religiosity on society. I don't intend to claim that religiosity is uniformly awful. I mean that the irreligious are likely to try to counteract destructive repercussions that come to their attention. Assuming that religious notions provide plans of action, irreligious notions could possibly provide corresponding plans of opposition. For instance, attempting to insinuate dogma will provoke strong irreligious reaction. Attempting to relieve poverty won't. (To the contrary, the irreligious who care just as much might successfully ally toward this common enemy.)

loopier and loopier

2011-08-10T14:03:00.000-04:00

Aware that truth doesn't arrive gift-wrapped on the doorstep, a pragmatic thinker should be willing to consider ideas from any source. In particular, sources of false ideas can be more useful than anticipated. The source could have true ideas intermixed, if not often. False ideas themselves could be stepping-stones or flawed clues to the real truth. Still more subtly, ideas could possibly be literally false and yet "true" through hidden correspondences to nuggets of truth.

For instance, the concept of the human mind, of a nebulous decision-maker that mysteriously controls the brain/body (not vice-versa), is a handy fiction. The self's "mind" provides the setting for an otherwise gaping hole in the categorization of human experience: the locale of subjective mental phenomena. These are actually real in the sense of being real effects of the real activity of the self's real brain. But from the self's perspective, which naturally doesn't include moment-by-moment observation of the originating brain, orienting ethereal thoughts in a dual mind domain is a tidy solution; when one "sees" things without eyes or "hears" sounds without ears, some explanation is warranted, no matter how far-fetched!

At a more advanced level of abstraction, a deepest ghostly "soul", which is distinct from the mind, scrutinizes and directs the gamut of human existence. The mind contemplates and manipulates objective items, but the theorized soul in turn contemplates and manipulates the mind. It thinks about thoughts. Events in the mind, as well as objective reality, are the raw material for the soul. Just as the idea of a mind is constructed to answer the fundamental question, "What and where are subjective mental phenomena?", the idea of the soul is constructed to answer the fundamental question, "What observes subjective mental phenomena?"

Besides having explanatory value, souls are instrumental in behavior modification. Specifically, all the many procedures for "transformation" recommend the triumph of the soul over the mind, although "soul" may not be explicit in the procedures' language. The sign is the treatment of the mind as an object rather than the subject: "calm your mind", "analyze your motivations", "recognize your negatively-charged aura". According to their accounts, transformed humans purposely change their minds. Afterward they cease their past actions and habits, because their new minds are incompatible. They will say that the old inclinations return occasionally, but each one is rejected and ignored not long after inception.

Such tales pose no problem for someone who believes minds and souls to be factual (my position less than six years ago). This isn't an option within the context of the belief that minds and souls are convenient human inventions for meeting some needs. Regardless of the deceptiveness of the impressions described in the stories, something must be behind the honest storytellers' impressions. Since everything occurs in the confines of one brain, the clear indication is that the illusion of hypothetical entity Q observing hypothetical entity M arises from out of a internal brain loop. That is, brain network Q interacts with brain network M. Activation of network M somewhat indirectly activates network Q.

Therefore, in effect human culture's encouragement of the dominance of the "soul" is encouragement of more sophisticated brain loops. Deliberate introspection ("meditation") is a loop in which thoughts appear, enter short-term memory, and then undergo processing. It requires training precisely due to its essential parallelism. Paying too much attention to momentary thoughts interrupts the arrival of new thoughts. Paying too little attention to momentary thoughts fails to accomplish the overall goal of "detachment", of treating thoughts subsequently as independent mental objects. Unsurprisingly, practitioners who are working to set up the brain loop benefit from periodically returning their attention to their breathing. Breathing is a sensation whose inherent rhythm aids in coordinating the brain network that acts as the "observer soul" and the brain networks that act as the "observed mind". (Reminiscent of the synchronizing "clock ticks" of a computer processor.)

Similar loops constitute the enhanced impulse control of the "transformed" human. Someone whose consciousness frequently returns to a moral standard (and/or an omniscient judge) will incidentally discover whatever shortcomings were in progress immediately before the return. Or, after spending extended time associating unpleasant reactions with "evil" impulses, a conditioned response of hesitation starts to accompany the impulses in the future. Alternatively an association could be built up between a strong desire for a positive aim and impulses consistent with that aim. At the ultimate extent, loops could be so ingrained, sensitive, and efficient that some parts of the brain virtually prevent other parts from intruding into decisions under normal circumstances. In any case, the human brain is amazing in its loopy ability to employ its incredible capacity to react to nerve signals that originate from "inside" as well as "outside".

source code comments and evolution

2011-08-05T09:36:00.000-04:00

In judging whether the source code of a computer program has been copied, similarity in the two programs' effects isn't a conclusive exhibit. As long as the programs were both written to accomplish similar purposes or solve similar problems, it's reasonable for the two to behave similarly. The base strategies used by the programs could very well turn out to be not that different at all. When a particular mathematical formula yields the right quantity, other formulas would probably need to be alike in order to also yield the right quantity. (This is why some argue that patents on software, as opposed to copyright of the source code, are questionable and/or unenforceable.)

Although it's not hard to believe that two programs could be created without copying and yet have a resemblance, it stretches credibility to allege that the source code of the two is the same. Human-friendly source code allows for a variety of choices that fall under the amorphous category of "style". Names, extra spacing, units of organization, and so forth distinguish the source code and therefore authorship of two independently-written programs. Still more idiosyncratic are the comments in the source code, which are completely extraneous lines of text that explain the code to readers - including the writer, who will undoubtedly need to return to the code later. It's improbable to see source code with identical details, especially when those details serve no function in the operation of the actual program! One program writer could have used the name "item_count" and the other "number_of_items". One could have included the source code comment, "prevent division by zero", and the other, "avoid a zero denominator". The great freedom that's possible for nonfunctional features of the source code also makes coincidences too unlikely to consider. If these features match, the confident assurance is that the source code is a copy.

DNA is the metaphorical source code of organisms. And like its metaphor, DNA has well-known nonfunctional ("noncoding", perhaps "junk") elements, too. Observations of such elements match across a huge range of organisms. Just as these matches strongly indicate common origins of program source code, the DNA must have common origins. Evolution, in which species spring from other species, is consistent with the observations. Copying is rampant, but at least there aren't any violations of copyright. There can't be; the material is copying itself!

descriptive or prescriptive Pragmatism

2011-08-04T14:11:00.000-04:00

As I understand Pragmatist philosophy (and repeat endlessly), truth is a holistic enterprise which a human performs using the available mental tools: sensation, experimentation, imagination, reason, assumption, intuition, inclination, and so on. If a truth survives this gauntlet, it "works".

The all-too-obvious problem (for philosophers) of this rather realistic definition of truth is that it doesn't take a theoretical "position". Essentially, the complaint is that this Pragmatist account of truth is descriptive, not prescriptive. It seems patently evasive to answer the fundamental question, "Is X true?", with the reply, "X is true when the human evaluating X determines it." The questioner can hardly be blamed for making the rejoinder, "By thrusting the evaluation of truth onto individual humans, you can't avoid the conclusion that 'truth' differs depending on who's evaluating. I'm so sorry, but in my estimation Pragmatist 'truth' doesn't work, therefore it's false. In your estimation, it may work and be true, but my results happen to diverge from yours, unfortunately."

Sly remarks aside, acknowledgment of the pivotal role played by the perceiver/interpreter/thinker is both inescapable and valuable. Two uncontroversial corollaries follow. First, there's no excuse for a single human to pretend to have an egocentric total grasp of all truth. Since humans have differing actions and abilities and outlooks, the truth they discover can be different in highly illuminating ways. Pooling the truth among a group is the best strategy. Advanced cooperation and language are two of the strengths that enabled the predominance of the human species. Second, conceding the importance of the subjective contribution leads to a personal "time-based humility". Humans change and adapt. An unrelenting and honest seeker notes that the present collection of truth isn't identical to the collection five years ago. Considered separately, the present self and the prior self are a pair who don't agree about everything. "They" could be equally certain of having superior knowledge of the truth. At any one time, the self could bloviate and/or blog about many things undeniable, and then deny the same at other times. If truth has no element of subjectivity, then how can you say that you know the truth better now than before, or be sure that you will never know the truth better than now? Even declarations of timeless truth happen in time.

No matter the downsides or upsides of the descriptive aspect of the Pragmatist concept, I believe it's prescriptive too. Given no standalone ideas can be inherently true, we are spurred to carry out human verification, i.e. whichever physical or mental actions are sufficient to produce proof. We aren't obligated to categorize an idea as true when it lacks proof convincing to us. Alternative claims on truth aren't satisfactory. Explanations and justifications, of why a statement is concretely true, defeat unfounded assertions.

Furthermore, I believe the Pragmatist "prescription" is to judge the strength of a proof in proportion to the strengths of the proofs on which it depends. As much as possible, proofs and truths shouldn't be treated as independent. Large chunks of evidence are mutual supporters. Data substantiates other data. Chains or networks of proof are the Pragmatist reality. Unconnected pillars or islands of proof are prone to suspicion; "Why is the truth value of this one statement excluded from the standards and substance of the rest?"

a la carte TV has arrived

2011-07-31T12:29:00.000-04:00

...but probably not in the form or at the price that consumers wish. A recurring complaint has always been, "Why must I pay for 'packages' full of channels that I don't want? I'd rather pay for only the stuff I want."

With iTunes or Amazon, that's possible. Customers truly pay for only stuff of their choosing: no subscription to a package, channel, or even a program. Rather, they buy episode by episode, like buying a periodical off the rack every week or month. This degree of granularity and inconvenience is the opposite extreme of a channel package. It might be just right for some, yet others likely would prefer a moderate option; they'd be willing to pay more in return for a larger conglomerate of programs. Already, specialized "channels" both produce and aggregate programs according to specific interests of the audience. Is there a reason why they couldn't fill similar roles in the domain of digital on-demand entertainment?

Some object to the episodic prices, at two dollars or more, for fresh and in high definition. Of course, the bytes that constitute the episodes are undeniably cheap to transmit/copy, so the "per-unit production" cost is far lesser than the retail price. But the per-unit production cost is always a baseline and never the entirety of the cost to produce and sell. Producers and sellers must cover miscellaneous other costs, e.g. fixed bills like the company payroll or investments like R&D. (Although some products sell at a "loss" in order to stimulate future purchases.) The final price is what the "market will bear"; sellers and buyers settle on an amount acceptable to both. It seems to me that the price level could have a few reasons:

The most obvious is that the price should be greater than zero to compensate everyone involved in the creation and distribution of the episode. If professionals are to create episodes, then they can expect to trade their work for money. If professionals are to create and manage technology to store and send the bytes, then they can expect to trade their work and fixed-costs for money. Since advertisers aren't contributing (as they typically do during "free" Web streaming), the revenue must come directly from the audience. As with any product, the gross revenue is in effect spread throughout the full "supply chain".
Prices for fresh and well-encoded digital musical works, the experience of which lasts between three and six minutes, range from $0.69 to $1.25. At its cheapest this is $0.11 per minute of enjoyment ($0.69 divided by six minutes). At that rate, a 40-minute TV episode would be $4.40. This calculation ignores the fact that the TV episode is both aural and visual and therefore each minute is really superior to the song, whether in terms of raw quantity of data or in terms of the subjective impression.
The episode is bought, not rented. Presumably the price to see an episode only during a limited time frame would be cheaper, which is the rule for the movie "rentals" on these stores. What's being sold is the permission to stream (or download to a scrambled file) the episode for all time; in any case, certainly more than one viewing. While this is different than the traditional meaning of "ownership" (what happens when the seller/provider goes out of business?), it's not too far off. Generally speaking, the usual market alternative that's closer to "true" ownership, a DVD or BD stored on a home shelf, is relatively more cumbersome and expensive (per episode) and also not as timely.
Massive bundling in TV achieves large-scale economic efficiencies that wouldn't be possible if everything were unbundled. Hence the actual price per channel or program or episode will tend to be correspondingly greater. The supply and demand price curves for each item will vary. Items with greater production costs will cost more. Highly-desired items could cost more (because buyers are willing to pay more). Market competition will be more important than ever, assuming programs can act as substitutes for one another.

omniduction formality

2011-07-30T09:59:00.000-04:00

Formally, what I called "omniduction" seems to be universal instantiation running amok. The root cause is hastily asserting the universal qualifier over too large a set, thereby mistakenly asserting that one or more instance(s) are inside the set (and applicable to universal instantiation). Cramming the world into too limited of a logical domain.

omniduction breakdown

2011-07-30T09:24:00.001-04:00

Apologies for flashing some political stripes, but here goes... Recently I read an angry online comment about the "threat of raising taxes by gradually reducing the mortgage interest deduction". 1) On one hand, the comment's writer believes in limited government that doesn't intervene in private markets. 2) On the other, the comment's writer believes that all rises in the effective tax rate are by definition terrible. So removing a tax cut, which is evil according to #2, would reduce government intervention in the housing market, which is good according to #1.

The attempt to derive an opinion by omniduction has produced a "SYSTEM ERROR". Perhaps policies in the real world have trade-offs and case-by-case factors to consider, and individual policies can't be attacked in isolation from the rest?

peeve no. 264 is omniduction

2011-07-29T11:27:00.000-04:00

Two thinking processes generally receive attention: deduction and induction. Logical consequences and generalizations. Debaters construct their arguments from these basic blocks. Participants who rely on other tactics are subject to being systematically dismantled by opponents. (There's also abduction, but I'm ignoring it here.)

Au contraire! This is an incomplete account of actual reasoning in human society. Besides deduction and induction, I propose a third process: "omniduction". Omniduction is the creation of many small details from out of unassailable preconceptions. Its rallying cry is, "I don't need the data! I know what the data is already because I know that ___(preconception)___ could never ever be false!" The miracle of omniduction is akin to spinning gold from straw. Reality can be so simple just by producing facts through grandiose assumptions, not vice-versa.

Unfortunately, omniduction tends to be problematic when practitioners try to converse. Unless everyone happens to be applying omniduction identically, the produced universes could be in conflict. When facts follow directly from overall assumptions, discussion of evidence is futile.

"My theory is correct. Your information must be wrong."
"The complex specifics of this situation must be irrelevant. My flawless system of beliefs doesn't require such minutiae to render a verdict."
"Truths cannot be complicated. If you'd only consider the issue from my perspective, you'd realize that a small set of self-contained opinions can explain anything."

Omniduction is so freeing and economical. Forget the collection and analysis of heaps of data. Acquisition of knowledge is for nerds. Omniductive reasoning provides ample justifications for taking whatever action is most intuitively appealing. Some can't see the forest for the trees. Omniduction is still better; there's no need at all to see trees when you're absolutely sure about the forest.

drinking game for Choices of One by Timothy Zahn

2011-07-27T08:36:00.000-04:00

Take a swig whenever you read:

a form of the verb "grimace"
"said grimly"
"mentally _____"
a form of the verb "growl"
a form of the verb "twitch"
the reply "Point", but that's actually very rare
Mara Jade deflecting a blaster shot back at the shooter
Luke thinking about the fact that he isn't a "real" Jedi or that he's not very good at doing _____
Han thinking about the fact that Leia's interested
"Firekiln"
"Threepio" or "Artoo"...psych! Those two are missing!

Review? Oh, right. I liked it a lot. It feels like a character reunion hosted by Timothy Zahn. Like almost all Star Wars books, it isn't as consistently entertaining as the Hand of Thrawn duology.

placing the Roku 2 upgrade expense in perspective

2011-07-24T11:25:00.000-04:00

I belong to the group of Roku customers who bought a Roku device in 2011 before the arrival of the Roku 2. According to this statement, the better Netflix streams cannot be made available to "older" Roku, despite those Roku having 1080p/surround sound hardware support. Therefore we get to buy Roku 2 if we feel like viewing Netflix with the full capabilities of our entertainment devices. Since I detest negativity, I'm choosing to consider this situation through Happy Goggles, which to my knowledge do more than nothing...

Roku 2 are better appliances. I'm someone who connects it by HDMI and wireless, so no problems there for me. The new gaming features have no appeal. Technology changes; I for one never expected Roku's product line to stay the same as its competitors exploited advances in hardware. Or my Roku to have sufficient horsepower to handle every upcoming enhancement to the Roku "channels".
Older Roku will continue to work normally. In fact, future free software updates will ensure that it shall improve. If the older Roku was worth the cost before, then surely that remains the case. There's no loss. Nothing is being discontinued or crippled retroactively.
Prices for Roku 2 are hardly huge burdens. For HD video with comparable diversity and DVR-like controls to pause/rewind/skip, the dominant companies charge more per month than buying both two Roku in one year. $80 + $80, divided by 12 months, is $13.33. Add in the $7.99 monthly cost of a Netflix subscription and it's $21.32. I'll grant that a complete comparison is more complicated than this and not really "fair" due to differences in offered content, e.g. ESPN 8 "The Ocho" and the side-boob pay channels. Internet cost affects the overall calculation for a household budget.
My less-than-$400 netbook has HDMI-out. Yes, it sends audio along with the video. I merely mention this to remind everyone again that, technologically speaking, it's not too difficult getting Internet content or indeed any computerized content into a typical entertainment setup. (That doesn't change my opinion that, for several reasons, the Roku is worth buying.)

a story about a dynamic programming language

2011-07-22T09:47:00.000-04:00

Every once in a while, I like to pretend that I can reload past mindsets like Smalltalk images. Today's story is about a dynamic programming language.

It loaded libraries at run-time. It sat at a level of abstraction from the actual computing hardware: there were no pointers. Memory management happened during execution in an automatic mechanism. Characters weren't single bytes. Checks of all kinds took place as code ran, affording a minimum of "protection". For smaller programs, start-up time itself was a performance constraint. Method dispatch wasn't fully resolved until the time of the call. Commentators often complained about the costs of all this dynamism.

Naturally, I'm describing Java as it was viewed a handful of years ago. This is why it's so amusing to hear that Java has somehow turned into the static language standard-bearer. At least where I was working, the original competitor to Java Servlets in the Web domain was Perl CGI, not C or C++. It was a battle between two dynamic options, in which greater simplicity of string manipulation and memory management trumped other concerns. Java was the company-pushed "compromise" solution that had better threading, Unicode, fairly easy C-like syntax, and so on, yet without some of the traditional downsides of the static languages. In retrospect, other languages could have filled that niche quite well (especially with a tweak or two), but lacked comparable levels of publicity, support, education, and engineering. Regardless of accuracy, the negative perceptions of Java's potential opposition were sufficient to leave an opening. As much as academic and research programmers would prefer it to be the case, programming languages aren't chosen based solely on the sophistication and self-consistency of syntax and semantics.

I'm not seriously proposing that the possibilities for dynamism in Java are comparable to the languages usually labeled as "dynamic". I'm only reiterating the truism that all the aspects of a language and/or its execution "platform", including and beyond the type system of its variables, are on a continuum between static and dynamic.

"feels dynamic" is a ridiculous expression

2011-07-20T10:29:00.000-04:00

I should start out by saying that most of the actual information content in Scala: The Static Language that Feels Dynamic is great and worth your time. Scala deserves any attention it receives. And obviously any complaints I advance about what Eckel writes are gadflies to his cow.

Regardless, my reaction to "feels dynamic" is extreme annoyance. Dynamic typing has no feel. Dynamic typing is merely one of many aspects to a language. I might agree that languages have a "feel", although it still sounds risible in a serious technological discussion. Go ahead and mention "feel", but at least trot out numerous examples that partially convey your meaning. Fortunately, Eckel does go on to do this in the article, which as a communicator puts him above the typical Rubyist/Pythonista/whatever-stupid-name-they-enjoy.

I'd suggest an alternative expression "feels Pythonic". Still better, "is as succinct as Python syntax". Based on the text, that's almost exactly what Eckel intends. Distinguishing a "dynamic feel" from a "static feel" just sounds like someone has an astounding lack of programming language breadth. For instance, ML-family or Lisp-family languages, not to mention Prolog or Forth or J, probably have different "feels" than that simplistic two-state viewpoint. For frak's sake, a hypothetical Java "replacement" more or less identical to Java, but with a type-inferring compiler and a much more convenient standard library, would "feel dynamic" according to some language cheerleaders. Get thee out to read about not only Scala but Groovy++.

reconciliation of frames of reference by hindsight bias

2011-07-10T15:59:00.000-04:00

I missed an aspect of free will in wildly varying frames of reference. Namely, the confusion fades. As time passes, manifestations of free will, which were formerly judged as harsh failures of the decision-maker's character, begin to be judged as moderate mistakes, challenges, catalysts, or bridges. For the religious, outrage could have been the contemporary reaction to an outrageous act or abstention, but feelings change as someone is said to "grow wiser", "see the larger picture", "accept it", "learn the lesson". Through these transforming sentiments, the present rationalizations for the past confounding decision are assumed to be part of the original frame of reference responsible for it. It seemed like nonsensical or malicious evil at the time, but now it's an enriching albeit formidable episode in the saga. There were unseen reasons and side-effects that become retroactive justifications.

However, the mechanism is suspect. Healthy humans have resiliency and emotions that are ever-changing. Dramatic inner screams that the world is ending have been wrong many times. The impact of an old memory is less than the impact of immediate sensation, so it's natural to underestimate the severity of the recalled event. Intellectually, the "hindsight bias" is probably active during remembrance. Since it's pretty easy to invent a likely narrative once all the facts are already known, hindsight fools someone into thinking that random occurrences with little warning were "obviously" inevitable and predictable. History defuses complexity.

In the grip of bias, and all too eager to tame and exploit troublesome data, storytellers turn the lead of negative tests of a god's principles into golden "insights" that teach the surprising nuances of the god's frame of reference. Looming over the past as they remember it, humans are thought to be that much closer to approximating the transcendental view. Not so. A decision-maker who excludes nothing from the balance-scale is nevertheless inscrutable. The human frame of reference for any decision may be a simple "good" balanced against a simple "evil". Yet the sweeping global frame of reference for that decision may have mile-high stacks of evil implications intermixed with the "good", and therefore the simple "evil" choice is the cosmic better. Pity the foolish human who tries to accurately score experiments of an impenetrable viewpoint, or gauge the illusory value of a decision through the hubris of hindsight.

free will in frames of reference

2011-07-09T20:50:00.000-04:00

One of my irritating habits is to flip around ideas and probe for holes. It's not always necessary for someone else to reexamine my arguments for weaknesses, because I do it before them. Combine this propensity with the blog format of publishing frequent tidbits, and expect a stream of entries on a theme.

"Free will" is worthy of extended treatment. Whether a sentient being is a human or god or fictional robot, I last asserted that free will is an inadequate excuse for failing to demonstrate its qualities through definitive actions. (The truth of having a quality is verified pragmatically.) I continue in this belief, but I've noticed a counterargument which needs to be addressed.

The potential problem with any real test or experiment of anything is an unavoidable entanglement with external facts. Interference should be minimized. Output data should be tied as closely as possible to the tested thing and as loosely as possible to all untested things. Setup should aim for isolation. Overall, tests should be carefully designed to confirm a single supposition without introducing extraneous factors.

In the case of a decision, the entangling external facts take the form of a "frame of reference". The decision-maker evaluates each choice by referring to the frame around the decision. Information, prediction, and inclination are all included. The frame of reference is how the decision reduces to a solvable puzzle. Apart from it, the decision-maker is lost and bewildered.

Its importance is felt in the common regret, "If I'd only known then what I know now." A decision evaluated through a disparate frame produces a disparate selection. Hence the frame is a crucial ingredient in judging what someone decides and reconstructing the rationale. It's indispensable in the assignment of blame or praise. An observer could commit grave mistakes without knowing more about the decision-maker's frame.

During a test decision, the frame of reference of the sentient is one of the test's complicating external factors. Simply put, the sentient may have greater or lesser knowledge than the tester! Then the disparity in knowledge leads to a disparity in frame and, crucially, a disparity in checking the result. Perhaps the tested sentient had the quality X all along but a divergent frame overrode/concealed quality X or demanded a convoluted expression of it. In order to dodge a test failure and continue to maintain that the tested sentient has tested quality X, an objector may propose that the sentient decides through a poorly-understood frame of reference. Checkmate!

As I hinted earlier, I think the "frame of reference counterargument" is shrewd but not devastating to my main point about testing sentients who have free will. Frames of reference certainly can be distinct, so I agree that there can't be an absolute guarantee that tester and tested concur in basic analysis of the test decision. My disagreement centers on how insignificant a problem it is.

To interpret an apparent "bad" decision as "secretly good" requires an alarming mismatch between frames of reference. If preserving the link between the sentient and quality X involves radically disconnecting the sentient's frame from the tester's, then I believe that even this solution throws quality X into question. For example, I'm stymied by the very meaning of "mercy" when a sentient's frame considers the killing of an unbeliever to be "merciful".

But not every gap needs to be that wide. As a matter of procedure, a tinier and less consequential test presumably avoids the frame-based inaccuracy that plagues weighty and thorny tests. A sentient with genuine quality X should show it in tests, no matter the size. Arguably, the most trivial decisions are the most revealing and the least able to afford excuses. Someone fixated on punctuality will arrive early to unimportant events. Yet there's a ready explanation too for failed tiny tests, which have barely any scope for frame ambiguity: the tiny "doesn't matter enough" to the sentient, who only bothers to act on the big stuff.

Finally, I'm dumbfounded by the apogee of the counterargument by frame of reference. The most extreme discrepancy is between the tester and a sentient whose frame is all-inclusive of space and time. A sentient with consciousness of everything is inherently not testable by decisions. The sentient's decisions are not understandable by the tester. Nobody can estimate a decision made by reference to everything in every time. Once again, instead of saying, "The god with quality X chose not to pass the test due to a frame of reference incomprehensible to us", the more honest summary is "The god with a frame of reference incomprehensible to us chose not to pass the test due to the utter worthlessness of characterizing decisions in that frame as having 'quality X'." Faith is trusting a god when a god's decision-making is irresistibly opaque. It may be true that the god isn't "beyond" good and evil, but how would anyone know which choices are which, when measured from that perspective?

some more about free will

2011-07-08T15:36:00.000-04:00

The question that once haunted my being has been answered. The future is not fixed, and my choices are my own. And yet how ironic...for I now find that I have no choice at all. (Dinobot in "Code of Hero", Beast Wars: Transformers)

I mentioned in the previous entry on "god tests" that the religious can explain the gods' failing score as a manifestation of godly free will. Mystifying privileged knowledge, not included in the test, led to an apparent not genuine failure. In essence, the test was flawed, not the god(s).

Of course, this alternative interpretation hinges on the concept of free will. Humans have free will; why not the god(s)? Coincidentally, the opening quote suggests that the same is true for transforming fighting robots. The speaker, Dinobot (not the most creative of names), had been shaken by the possession of a future object containing historical information. Does a history already-written imply that individual actions aren't free at all? Not too many episodes later, Dinobot finds out two things: the future is indeed mutable, and the villain of the story plans to exploit the information in the "future history" to efficiently commit genocide through the murder of ancestors. Ergo Dinobot comments that although he believes in personal freedom again, only one path is in fact morally allowable: stopping the villain by any means necessary.

As Dinobot discovered, a free will is both more and less complicated than simple unpredictability; roulette wheels, shuffled cards, and dice rolls are poor models of free will. Free will is more complicated because the decision depends on numerous factors and constraints. Free will is less complicated because the tendency is toward very few options. Due to the latter, a single flip of a coin is sufficient to resolve a moral dilemma, but due to the former, deciding by coin flip is called insane precisely because the coin is independent of the dilemma's facts. A "hypothetical" Dinobot whose "decision processor" is a randomly-moving switch based on emissions of a radioactive isotope would on average opt for inaction half the time. The "actual" Dinobot opts to intervene with zero uncertainty.

Therefore, the idea of free will is shaky. Before a decision-maker executes the decision, multiple mutually-exclusive possibilities lie ahead. After, only one could happen. More possibilities beforehand correspond to a greater impression of free will. Then the key question is, who's listing the possibilities? A disinterested observer, with a vague mental model of the decision-maker, could build an unassuming list. A socially-invested observer, with a broad and proven mental model of the decision-maker, could imagine a differing list. Finally, the decision-maker could report a surprisingly lengthy list. All three of these lists are flawed since all contain excess, unrealized, misleading items. This experience of modeling error is "free will".

Regardless of the errors, humans attempt to model behavior because they live in complex groups. Some of the attempts work fine. Especially in a common culture, individuals have sufficient resemblances to the rest. They can use themselves as rough templates for their counterparts. They learn which differences to expect and which presumptions to accept. With the aid of language they further develop an advanced "third-person" viewpoint in which an abstract human, or more specifically a "mind", interacts with its environment in accordance with the found examples of others and self.

It's unsurprising that humans have difficulty calculating the actions of their companions, because they aren't able to analyze external brains' activity moment by moment. They cope somewhat through instinctual comprehension of facial expression and body language. The more telling contributor to the construction of "free will" is the purely subjective piece. Based on introspection, indeterminism in free will is supposedly more profound than an incomplete awareness of external "minds". For not even extremely intimate details of the self's "mind" are enough to absolutely foresee the eventual outcome when the time to act arrives. Meditative attention to the ongoing maelstrom often reveals an assortment of proposals in play. A decision is when one wins out and the losers are cut loose (the metaphorical origin of the word "decide" is from cutting). Free will is the result of the brain's unsurpassed parallelism! Ambivalence, or to be "of two minds", is a coarse description of competing firing patterns in one brain. Moreover, experiments have illustrated that the well-known battle between short-term/emotive/impulsive and long-term/logical/judgmental decision-making is a battle between brain regions. And the temporary suppression of either region, e.g. via alcohol, produces different behavior.

However, like most dichotomies it's a simplification. Evenly-divided plans, the sort that ethical and political philosophers study, tend to each have facets which appeal to emotion and reason. Principled compromise is symptomatic of creativity not indecisiveness. Synthesis is arguably a more impressive feat than unbending commitments that purposely disregard situational distinctions. "Free will" is a highly unhelpful construct when applied to a human who refuses to attack problems in dull fashion. "You have the free will to select A or B." "But I select Q because A and B are inadequate."

Thus, from a variety of angles, the essential connecting thread underlying free will is the ignorance of the observer, not inherent unpredictability of the actor. Brains certainly reach motivated and nonrandom decisions ("halfhearted" decisions may naturally cause a creature to move in fits and starts or trip over itself). To know free will is simply to not know the full contents and workings of others' brains or of one's own brain.

From this viewpoint, the prospect of applying free will to any god raises suitable questions. How similar is the god to the "mind"? Which motivations does it share? Does it experience something akin to pride? Envy? Lust? Love? Anger? Anxiety? What is its thought and decision-making process? Does it hesitate? Is it capricious? How many courses of action does it contemplate? If it's an "embodiment" of a particular set of abstract characteristics, then can it act contrary? Which factors does it seek to maximize by its decisions? Which constraints does it honor? There might not be complete answers to all these questions, but available data pertaining to the god should shed at least a little light.

Once the god's precise similarity to "mind" is established, no matter how tenuous, the boundaries of the god's free will should be clearer. And in a limited sense the god should be predictable and - here is the upshot - testable. Given a specific slate of choices, the god can be expected to fulfill the one within those identified bounds of free will.

Is this not a reasonable test? If the god doesn't pass, then there are two categorical opinions: 1) the god failed, 2) the god used free will to be unpredictable. Opinions in category 1 imply that the god's qualities are different than previously available data indicated. Opinions in category 2 imply that the range of the god's free will is wider than previously available data indicated. But the domain of action that's included in free will is itself restrained by the god's qualities. Either category leads to the conclusion that the god cannot be consistent with the assumptions behind the "test". An accurate follow-up assessment of the failed test is that nobody really knows for sure what the god is like, i.e. how to compare it to "mind".

Examples are easier, so back to Dinobot. If Dinobot is a hero, then when Dinobot is put to the test, Dinobot acts heroically. Otherwise, Dinobot isn't a hero, or Dinobot is a "hero" who just freely chose not to act heroically. The latter implies that Dinobot is a "hero" by a definition that, bizarrely, doesn't encompass choosing to act heroically. There's a third rather metaphysical avenue related to the famous Euthyphro dilemma: the figure of Dinobot is perfectly synonymous with "hero" and perceived conflicts are the observer's fault for trying to think of heroism apart from the single exemplar, Dinobot.

Free will is a self-defeating escape hatch from failing god tests. To say that a god has quality X is equivalent to saying that a god chooses ("free" or not) to act or abstain in ways that match quality X. To say that a god has quality X, and also that a god can and does choose to act or abstain in ways that contradict quality X, throws doubt on either the god or quality X. The religious may concede that they don't know for certain that god has quality X. Or they may concede that they don't know for certain what exactly quality X is, i.e. how a god with quality X would act. Ultimately, their likely response is faith, which is invincible immunity from evidence.

god tests

2011-07-02T15:15:00.000-04:00

In the previous entry, on the point of explanation behind why I accept the classification of "atheist", I tried to establish that disproof of specific gods is indeed possible, but only for carefully considered definitions of "disproof" and "gods". Namely, when the god(s) result in real indicators, the absence of those indicators is a valid chain of reasoning to the statement that the god(s) must not be present. By straightforward logic known as the "contrapositive", p implies q is equivalent to not-q implies not-p.

An instructive example is the famous failed theory of the luminiferous aether, the hypothesized medium and absolute reference frame (for the uniform constant speed c) of light. Wikipedia has thorough coverage, but the quick overview is that theoretical predictions weren't confirmed by experiments and competing theories with greater generality arose, i.e. relativity and wave-particle duality. Could the luminiferous aether exist? Maybe, but if it does then nowadays nobody has any pragmatic rationale to use it in mental or physical tasks. They can think and act on the basis that it isn't there. (Ethernet doesn't count.)

A god(s) "theory" is subject to similar pragmatic evaluation. All that's needed is to figure out and run the corresponding tests of reality. Unfortunately, this approach is typically not applicable by the religious. How come?

Foremost is a taboo against questioning religious propositions. Of course, the emphasis on the taboo can vary, and novices of the religion may ask questions as a normal component of instruction. Befuddlement is permitted. "Attacks" and/or suggested alternatives to the core essentials, especially by members who should "know better", is prone to responses ranging from insistent re-education to ejection. Not all groups are heavy-handed and hierarchical but most will take steps to forcefully prevent the spread of "corruptive lies". Admittedly, this feature is not peculiar to religion. Groups united by common beliefs, i.e. pretty much every lasting group there is, probably will remove naysayers for the sake of survival, akin to an immune system. Otherwise, the group wouldn't in fact be united by common beliefs!
After a god test, the attendant religious measurement and interpretation is equivocal. Given that posited gods have freedom to make decisions, each test must be unsure. Either the test passes or fails, and either the god decided to show up for the test or not. And due to godly free will, the god and the test are statistically independent, more or less. That is, the correlation is zero between the test's actual outcome and the god's existence. For instance, as long as the god can possibly choose to not participate, all failed tests might mean nothing more than that. Thereby the god tests are inconclusive. According to the popular definitions, free will cannot be predictable because a decision known beforehand isn't "free". The more that an observer knows about the decision-maker, the more predictable the decision appears, and the less "free will" could be involved. A human who's known to be compassionate tends to predictably perform compassionate actions when the chance arises. Is this free will at work? What if this human, unknown to the observer, underwent tragic personality-changing brain trauma in the recent past (there have been well-publicized cases)? The ignorant observer could watch the human decide to act contrary to compassion and then exclaim "I see again that I can't predict what someone will do! That's free will at work!" What's perceived as free will could be a lack of knowledge. The nice occasionally are mean on their worst days, but who knows whether today qualifies? In like fashion, when a god test fails, and so the interpretation must be the god's exercise of free will, the "scope" of free will is customarily enlarged through greater and greater presumptions of a lack of knowledge about the god's decisions: "It doesn't make sense for my god to make that decision; my understanding of my god's qualities must be much more incomplete than I estimated". In the face of many failing tests for a god, the believer claims less and less to be attuned to the god's utterly mysterious and unlimited free will.
Mystery is far from a problem in religion. It's more likely to be honored, for without mystery there is no opportunity for faith. The religious scoff at the proposal of god tests, which serve no purpose for the faithful. Their god(s) simply exist, regardless of the experience or reasoning of lowly humans. Religion is concerned with great celestial realities, hence present human reality need not be entirely consistent with it. Temptation is an enemy, yet doubt is the most fatal. Letting doubt roam free potentially endangers the great panacea, faith. Intellectual discontinuities shall be overpowered by emotional flurries of trust and hope. Take a deep breath and let your troublesome mental conflicts gooooooooooo...

Postscript: the primordial god test

I don't wish to be seen as negligent for not addressing the primordial god test: isn't a universe an adequate clue that a god must be around? My answer is no. The problem with a god-creator theory is its startling incompleteness. It's really closer to being a sketchy conjecture. There's a solitary data point, which is the universe. That's not much to work with in order to distinguish between theories. What would be different if a god started the universe, as compared to, um, something else? To what can I compare the purported act of creation?

Moreover, this idea is very vague about the god. "This picture was painted." "By whom?" "The Painter, naturally." The creator-god is a riddle wrapped in a mystery inside an enigma. Were there many creator-gods working together or at cross-purposes? Is the creator-god in the same form as at the time of universe creation or did it empty itself into its creation? Is it what humans call intelligent or a clumsy oaf committing accidents? Is it needy and/or accomplishing a goal? Is creating an effortless hobby? These are valid albeit mischievous questions. Remember, the metaphorical Painter could have been a hyperactive dog whose paws had firmly-attached rectangular paint sponges.

I suppose that I could be persuaded to hypothesize that 1) the universe started through an inscrutable process, whose placeholder label is "creation", and 2) the process was initiated and overseen by an inscrutable thing, whose placeholder label is "god". But after accepting the hypothesis merely for the sake of debate, it amounts to bupkis. It certainly doesn't follow that "universe-creation god" is all-powerful or all-knowing or immortal or a close personal friend of a proud species that scrambled to the undisputed top of the food chain on planet Earth orbiting around star Sol somewhere in the Milky Way galaxy (sincere apologies for the terribly sloppy address).

locating myself in the irreligious taxonomy

2011-07-01T22:15:00.000-04:00

I chose to use "irreligiosity" as a broad category for four reasons. First, "areligious" is too easily confused with "religious" and it doesn't seem like a popular coinage. Second, I felt that the "non-" prefix on "nonreligious" sounds too, um, noncommittal. Third, on the other hand, "antireligious" insinuates excessive and active opposition. Fourth, "irreligious" shares the "ir-" prefix with "irreverent", which is appropriate. Irreverence merely indicates that, unlike others, someone doesn't revere something. As I understand it, irreverence of religion is an adequately-drawn yet far-flung border for irreligiosity. For irreverence could take many forms of thoughts or actions.

Nevertheless, after wading through Wikipedia, starting at the Irreligion page, I've learned of finer distinctions communicated by other terminology.

I hadn't considered the possibility of honoring or obeying aspects of a religion, integrating it into personal identity, but not believing its ideas. Religion may be a cultural/ethnic institution as opposed to a belief system. Therefore someone who doesn't claim the existence of any gods could deny being "irreligious" because religion retains importance in their lives. In my opinion, "irreligiosity" applies to the status of held conclusions rather than circumstances of birth or upbringing. I would say that irreligious individuals sometimes perform religious acts. For example, I've ceased personal religious behavior, but that won't stop me from participation in societal holidays of religious origin. Meaningless customs can still be enjoyable. I don't feel a need to associate myself with specific religion, but I comprehend that my situation is not universal.
I agree to the philosophical stance of naturalism or materialism. This is a stricter judgment than general irreligiosity. I'm dismissive of all supernatural speculation. But this need not be true of everyone who doesn't assent to religious notions. They might reject every religion they've ever heard and opine the positive existence of unknown or loosely-defined gods. Perhaps they're ambiguous on the "god question" and maintain that human spirits can exit bodily confines. Maybe they're fond of trusting that "someone" is always watching or they simply assume that there can't be a universe without a creative Force to jump-start it. Anyway, none of these beliefs in supernaturally-based things are sufficiently convincing to me.
Along the same line, I'm not impressed by the idea that the supernatural is unprovable. I understand that the absence of proof isn't a proof of absence. However, in my interpretation of Pragmatic philosophy, absolute proof isn't a realistic or necessary standard. Coming to the point, the dubiousness of supernatural hypotheses doesn't justify a provisional judgment in favor and/or changes to future actions. I can invent a multitude of unverified explanations, but I'm not obligated to incorporate the fictions into the working assumptions that affect my actual existence. Believable theories display Pragmatic truth through implications that are susceptible to human actions like experimentation and calculation and observation. Under these guidelines, supernatural "stuff" that has no effect detectable by human means, not even in theory, is...not..."true" like everything else is "true". Anything supernatural that supposedly has such effects has the possibility of being true. In short, imagined supernatural objects that are completely ignorable could only be "true" in a pitifully weak sense. Since almost all proposed supernatural objects impinge on reality, almost all proposed supernatural objects can be judged as true or false via the usual methods and evaluations. I can't prove that irrelevant gods cannot exist, but I think it's reasonable to pragmatically determine, by induction, that theoretical gods whose alleged traces are missing or indefinite do not exist. By and large, "atheist" seems like a label that closely approximates my perspective.
As I keep repeating, I don't wish to annihilate religion. I'm eager for breaking links between religion and political power. Religion shouldn't affect a citizen's status. Government shouldn't endorse or fund religions. Religious and irreligious can exercise their democratic freedoms. Furthermore, religion is beneficial insofar as it provides motivation for living with more considerateness and deliberation. I don't mind attempts at non-coercive conversion (I'll grant that all too many cases veer too close to emotional exploitation and manipulation). These opinions set me apart from "antitheism".
At this time, I'm unsure whether "Humanist" is an accurate description of me. I support happiness and freedom and not being egocentric, but surely Humanistic ethics has more substance than what I'm seen so far?

irreligious practice

2011-06-29T16:03:00.000-04:00

I realize that "irreligious" is a broad and loose description so I can't speak with comprehensive authority. Regardless, I offer some personal opinions on commendable irreligious practice.

Ritual. First, disregard suggestions to indulge in ironic, satiric, or parodic versions of religious rituals; the mere negativity of scornful attacks is a sadly hollow method to assert self-identity. Instead, consider the two truths that humans are entirely corporeal and all activity ceases at death. They shall encounter an inevitable end. In response, it's prudent for humans to nurture their long-term health. One precaution that's almost universally recommended is regular moderate exercise, which is achievable without onerous costs of time or pain. It's a ritual, observed on most days of the week for a minimum of a half-hour (depending on intensity/ability), to the central importance of preserving the body to forestall its eventual decay. Unlike many religious rituals, the participant can choose how to do it. A second irreligious ritual is any action that's unreservedly approved by conscience but happens to be forbidden by a religion. Detestable and/or stupid actions don't qualify. The point isn't to celebrate anarchic freedom from behavioral restrictions by acting like a villain. The point is to ignore unnecessary religious prohibitions and thereby improve the lives of self and others. Show that there's a better way. Truth be told, this ritual isn't strictly irreligious. Religious adherents themselves often dump all the parts of their nominal religion that they sense to be unconscionable. Or just perplexing.
Study. My advice to the irreligious is to cultivate wide-ranging curiosity. We have finite opportunities to learn, explore, and experience. There aren't any additional "realms". There isn't a preferred path of narrow study that produces god-pleasing mental states and therefore unlocks a superior afterlife. All we can have is what's in front of us. Experimental, expansive learning is more mathematically probable to find the pleasing and the good (and greater insight into the displeasing and the evil). Of course, "study" could be secondhand, as in reading worthy books, or firsthand, as in developing a skill. And religion is an interesting and varied object of study for the irreligious, who aren't concerned about the "danger" of exposure to competing knowledge.
Activism/Values. I'm not convinced that rapid elimination of religions is possible. Nor do I suppose that a sudden absence of religion would guarantee a widespread, uniformly-positive change in well-being. Bitter assault on the religious, through techniques of shock and disrespect, is counterproductive. However, I'm in favor of lawful democratic efforts to equalize the rights and privileges of the irreligious with the religious. Depending on the case, the best approach could be removing discrimination based on personal beliefs altogether. Or gaining peer recognition of "irreligiosity" as an included category (no matter the name). Generally, in public forums of all kinds, counterbalancing irreligious perspectives should be stated when appropriate, and the form of expression should amount to much more than indignant insults or aggressive attempts to de-convert. It should go without saying that some quixotic fights are a waste of time and so is taking offense at minuscule unintentional oversights. Irreligious practice is at its least persuasive when it's characterized by irrational hatred directed at a nonexistent god. Rather the route to mutual tolerance should consist of demonstrations that irreligiosity isn't synonymous with amorality. I like the Christmas (look, I didn't type "x-mas"!) slogan "be good for goodness' sake". It's simplistic almost to the degree of tautology, but it does communicate the basic idea that ethical concepts and sentiments and decisions aren't contingent on thunderbolts or hellfire. When the irreligious promote values and causes that fit their ideals of less oppressive suffering in human existence, perhaps cooperating with the religious on selected projects, prejudice appears unfounded. Charitable contributions and volunteerism contradict the pernicious presumption that humans without otherworldly judges are purely selfish. Actually, I suspect that many of the religious outwit themselves, morally-speaking: they're either so confident about their acceptability before god, or so obsessed with perfection of irrelevant private minutiae, they don't bother to be a constructive influence.

some opinions on dialogue with the religious

2011-06-27T16:04:00.000-04:00

No matter how personally gratifying, calling anyone "insane" or "stupid" for holding different beliefs is likely to end the dialogue. Of course, the same applies to calling the irreligious "heartless" or "lawless".
Discussion about particular proofs may not be as illuminating as discussion about general principles and standards that affirm or disqualify propositions. For instance, is it permissible to detect the supernatural through intuitions that manifest as shivering? And in regard to some major world religions, is the truth of claims within the sacrosanct text(s) utterly beyond questioning? Depending on differences on central points such as these, delving too soon into peripheral arguments could be futile. Intact philosophical roots are unshakable by indirect blows.
Technique is also important for better credibility. Responding to a proof with a proof from another domain shows an apparent lack of confidence and forthrightness. Whenever someone advances a metaphysical objection, the relevant approach is to undermine the objection metaphysically. Whenever someone refers to a testimonial, an opposite testimonial is appropriate. Flatly ignoring a statement seems evasive and inflexible. Even if the assertion is ludicrous, it needs to be patiently corrected in detail rather than contemptuously dismissed.
A few exhibits are so old and well-known to not be worth mentioning. Every thinking adult who has an opinion about religion already has a well-established viewpoint on the topic, so raising it is fruitless. Examples: 1) Human existence is suboptimal in myriad ways, so supernatural beings cannot be both benevolent and powerful.. 2) The universe must have an origin, so there must be a creator god. 3) People have ethics, so there must be a judgment god. 4) It's impossible to absolutely prove nonexistence, so the supernatural could exist. 5) The number of contradictions among religions is innumerable, so no possible god is sufficiently universal to be credible.

the defining data structure

2011-06-25T21:32:00.000-04:00

The human brain is a body organ of countless cells and regions, whose chemical/electrical work consumes considerable energy to affect microscopic matter and thereby think. But pictured within a more abstract realm of information, its efforts accomplish the arrangement of data into a workable structure. It redirects, recombines, and generally tames a storm of data in the form of nerve impulses. Crudely put, brains are the coin-sorters that efficiently separate the "coins" of data by biological value. Given that a datum is never alone, i.e. information can only be informative in concert with other information, then the brain acts as the defining data structure.

Moreover, the data structure has several parts and processing steps. First, each bit of data is accompanied by its contemporary bits, into a structure of context. Second, this larger hunk of data is classified by arousing similarities to stored past data, into a structure of remembrance. Third, the data is correlated with goals and desires, into a structure of significance. Fourth, it becomes part of plans and scenarios, into a structure of proposed actions.

Since each of these tasks of definition are distinct to some extent, and also segregated by type of sensory stimuli, it's not too surprising that the brain's structure is closer to a conglomeration of specialists than a single thinly-spread generalist. When a creature develops a sense, it's cost-effective to gain narrow advantages from it in the service of common situations, e.g. fight or flight. On a limited food budget, brain cells that are well adapted to extracting information from sense data are preferable. By contrast, unshaped cells that are supremely flexible and unassuming are also not immediately helpful. Evolution prioritizes richer data structures of perception. It's more vital to absorb and distinguish raw data than to perform an orderly sequence of advanced manipulations and transformations.

Therefore, as a mirror of the brain's layout, general or abstract data structures are likely built from collections of specialty data structures. Thoughts about visual sensations, even if subconscious, happen via the brain sections that evolved to handle vision. Singers who recognize that a note's pitch is too low are using auditory specializations. Episodic memory, encompassing several senses at once, employs a wide-ranging slice of the brain. Words like "determination" or "below" or "nation-state" have connections that are more diffuse but nevertheless existent. Concepts rely on foundations, although the path can be long and twisting.

As complex as the entire mixture is, further complications apply. Instant initial sensations are not the sole data source that demands structure. For instance, incoming data can be persistent after the time of introduction, recurring by displacing recent items or fading to an activation level near sub-conscious. Either way, the long-lasting data structures have the possibility of meshing with seemingly unrelated structures to form unanticipated syntheses.

Another less traceable data source is unprovoked cell activity. Fully awake brains encountering sedate environments are nevertheless busy. Resting brains during a REM cycle are quite energetic, too. Clearly a measure of unpredictability and autonomy is built into the human organism.

Fortunately, mechanisms counteract the inherent chaos and competitiveness. Under normal circumstances data structures require consensus in order to continue having effect. No separate all-knowing judge is necessary to arbitrate. Matching data structures mutually reinforce and silence deviants, just like fitter organisms multiply and crowd out the less-fit in the absence of an overseer. A reasonable structure of data is strengthened through like-minded alliances with the expectations that it meets. A nonsensical structure of data is weakened because of its isolation.

Overall this bottom-up strategy is a gradient for progressively pushing data in the direction of verisimilitude. Inappropriate answers are often a mere starting point. Some variations on that start will link up with strong data structures in the brain, but some will not. Useful creativity is a trial-and-error enterprise that strains known constraints without breaking. Repetitive feedback establishes which prospective differences are a step forward or backward.

Hence the momentary restructuring experiments are anchored by relatively pivotal and dominant data structures. One category is urgent external reality. The freshest streams of experience are naturally vivid. Related data structures propagate by channeling the vibrancy. Remote data structures, like reveries, tend to dissipate quickly. Abnormal breaks in the division, triggered particularly by interference of ingested chemicals, can certainly reverse the usual hierarchy and produce disorienting confusion of whether data is "imaginary".

Besides urgent external reality, data structures may take a controlling role by belonging to the broad category of organism objectives. The power of basic evolutionary drives is sufficient to train a multitude of secondary aims and behaviors, perhaps resetting the original relative priorities. Emotive attraction or revulsion enriches corresponding data structures. Data that's been tied to such reactions is more likely to retain its structure. Marketers, politicians, and public-relations experts confirm the effectiveness of the tactic.

Thus the human brain begins with preexisting data structures for survival. Initially other creatures' brains are set up differently in as many ways as the creatures differ: less social creatures have lower proportions of data structures about empathy, creatures with advanced noses have more data structures devoted to recognizing smells, e.g. to discover claimed territory. Compared to the prehistoric time during which evolution encoded blunt instructions into the brain structures, not only into the human species but into its forebears, large-scale organized civilizations are the latecomers. While there isn't a biological way to transmit specific memories to offspring, inborn behavioral adaptations of general value in survival or reproduction are excellent traits for natural selection.

Sophisticated data structures produced and transmitted in human society are much more intricate and fine-grained than evolution's. However, one doesn't replace the other. The primal data structures are indispensable raw material and tools. Abstract structures can reuse earthy structures. In order to "understand" an abstaction, a concrete metaphor allows confident contemplation. Human brains are quite astute at extrapolating movement of an object; once the sequential levels of a quantity are visualized, the next level is the position the quantity would be if it were a moving object. AI data structures start from nothing. Defining data structures in the human brain start from an array of hard-won knowledge which meets a minimum standard of relevancy. Computers must be told exactly how to perform a task. Humans can be told a problem, and then they can twist their abundant data structures to define the problem and identify linked solutions.

"energy" misuse

2011-05-30T09:28:00.001-04:00

The prevailing ideas of a culture change over time. Not terribly long ago, and less than an eye-blink in geological time, my ancestors would've observed reality through the eyes of spirits. Demons and angels hindering and assisting humans. God willing the sun to rise each day. And perhaps the most persistent idea of all: a soul operating the body like so many levers.

Around the time in history that I live, humans have moved on to naturalistic theories supported by reproducible experiments and communicated by mathematical relationships. Atoms aren't considered nearly as controversial. The inferred forces are impersonal; no empathy is necessary to calculate the operations. Public education has taught (hopefully...) the essentials of a scientific interpretation of the universe.

With these differences in belief, common vocabulary has shifted also. Presently it's much more expected for someone to use the word "energy" for mysterious substances to explain everything in moral and egocentric terms. "Bad energy" in place of "demon", "good energy" in place of "angel", "weak energy" in place of "phlegmatic humour". As we've all learned, energy is everywhere and in everything, so why couldn't there be energy of "spirit" in the same way? Energy is seen by its effects. Confounding effects could just be psi energy. Ghosts and souls are sentient clouds of energy.

My complaint is that the scientific definition of "energy" isn't applicable to those contexts. There's no compatible theory to back up the use of that highly technical word. Central to the notion of energy is that it's convertible and measurable. Energy does stuff. Mystical energy isn't "energy" as understood by scientists unless it's clear how to turn other forms of energy into it or turn it into other forms of energy. Even the most "intangible" of energy can be rated by what precisely happens when the energy is converted. How does it affect temperature and motion of matter? In the scientific understanding, one can't posit as many new types of energy as one wishes. There's a detectable balance of energetic quantities (subject to quantum indeterminacy at the truly tiny scales). If the entire energy equation has no missing "terms" within a reasonable bound of error, no mystery energy is there.

Therefore, to avoid giving the wrong impression, I propose that spiritual hypotheses change their terminology from "energy", which has firm physical definitions, to "mana". Speaking in good faith, "mana" is a more effective reflection of the level of discourse. Say "naughty mana" rather than "negative energy". "Think-y mana" rather than "mental energy". "Ghoulish mana" rather than "disembodied energy". "Clogged mana" rather than "blocked energy flow".

Simply by adjusting word usage to match the distinction, the mana experts can more effectively move on to research the open question of how mana relates to energy. For instance, how high in meters can the mana in the average human soul lift an object of mass 102 grams?

peeve no. 263 is offhand Java bashing

2011-05-27T13:59:00.000-04:00

It's a sign of immature writing to insist on inserting opinions and extra comments. Within opinionated persuasive essays (or blog rants), this is an expected practice because it's the whole point. Within a tutorial, it's an irritating distraction. If you're writing disinterested and objective prose about a technological topic, then it's...not...about...you!

When someone's stated goal is to educate, as opposed to spreading a point of view, side-swipes at programming languages should be left out. There's certainly a time and place to "fight the good fight" for your faction, but not by dumping stray sentences in the middle of a how-to on neutral ground. I understand, Advanced Technical Writer, that you're eager to give off the impression that you're an independent thinker of exquisite taste. Why not fully express your sophisticated perspective into a separate long-form document, rather than confining it to a cryptic handful of sentences or footnotes in everything you write? Tossing off some quick one-sided insults doesn't do justice to yourself or the reader. Advocate in your advocacy articles. Propagandize in your propaganda.

I would've written this example shorter, but Java is awful, am-I-right? Now watch me apply the words "noise" and "clutter" to perfectly understandable and correct syntax. Isn't a shame that I used an anonymous class here, when other languages (wink-wink, nudge-nudge) offer different possibilities? I apologize for writing loops and not overriding operators; my example language ties my hands so tight that it burns. Sorry about these checked exceptions ruining my 'sunny-scenario-only-dear-bob-don't-copy-it-as-is' code sample. Please don't laugh at the parentheses in my 'internal DSL', I did the best I could with the terrible tools available. Oh, how I long to have avoided that well-known design pattern through the deep meta-programming magicks that you naive mortals cannot comprehend. And the explicit data types torture me so...

symbol divergence

2011-05-25T15:07:00.000-04:00

Often, the inner workings of something are clearest when it breaks. When it's functioning as expected, the details mesh into a sublime unit of successful cooperation. A big happy result overshadows the individual contributions; a jumbled unsettling result redirects attention to whatever broke.

Perhaps surprisingly, that holds for symbols. To determine what information corresponds to a symbol, consider all the ways to prove that differing symbols are mismatches in some fashion. For a symbol to have value, alternatives to it are to some extent inappropriate and inadequate. Which typical messages are judged "true" when the symbol is in place but judged "false" when other symbols are? Or "false" and "true" respectively, or "false" for either, or "true" for either? And veracity isn't the only message outcome to judge and compare. Within messages otherwise identical, does a symbol communicate relatively greater or lesser specificity, formality, emotionality, neutrality? Subtext is still information, as demonstrated by the difference in the recipient's potential reaction.

Then the divergence in message outcomes is a basis for inferring the information for the new symbol via the indirect route of other symbols, which act as points of reference. The true message "X is a telephone" narrows down the information that X could symbolize. Another true message "X is wireless" eliminates further possibilities. A symbol's information is a synthesis whose parts and connections themselves can be symbolically communicated. Divergence outlines the real boundaries of the synthesis, like how parallax bounds objects' real positions. For instance, "X is a telephone" implies a fairly restrictive range for the mass and volume information included in the synthesis that X symbolizes. All one's prior examples of telephones, being sufficiently alike to X to merit the symbol "telephone", are like lines of sight directed in the metaphorical vicinity of X's synthesis of information. X's particular synthesis of size and price and brand and usability, as judged from messages in which X appears, is different than those prior examples but is nevertheless metaphorically closer than the synthesis symbolized by "sasquatch".

At the extreme, symbols could more or less fail to exhibit divergence. When that happens, the superficiality of symbolism is most clear. Generally, humans are too easily impressed and preoccupied by this occurrence. "What if someone abruptly started using the word 'green' for the perception of the color red?" Given the above discussion of pragmatic symbol divergence, it's not that difficult to guess what would happen. If the person never expressed any messages containing that symbol, the mistaken substitution wouldn't matter since symbols are used in messages. He or she could still reach accurate unexpressed conclusions about colors. However, after communicating a message containing the symbol, the recipients would respond by rapidly pinpointing the divergence. Sender: "I got a wonderful green balloon whose color is as vibrant as my car." Recipient: "But your car is red." Sender: "No, my car is plainly green. Green like the round fake nose on the clown who handed me the balloon." Recipient: "Hmm. It sounds like you're calling 'green' what the rest of us call 'red'."

Moreover, the proposed thought experiment is in reality a quotidian event. Scholars of language evolution have tracked gradual divergence of the elements of language. Words replace or combine. Innovative distinctions emerge and outdated distinctions fade. Separate languages cross-pollinate. Creative experiments, intentional or not, sometimes spread and sometimes die. Many times "incorrect" popular usage tramples correctness over the long run. Old-fashioned yet memorable phrases live on in isolated contexts. Idioms break the usual procedures for interpretation. Throughout these changes, communicators employ diverging symbols constantly. Simply by not introducing or modifying every symbol simultaneously, the messages that either use or explain the modified symbol remain understandable. Symbols are the bridges that symbols cross to reach other symbols.

Symbols are the fiat currency of information. Like local price changes, adjustments to the meaning of a few symbols are healthy responses to outside shifts. Like widespread hyperinflation, too many adjustments to too many symbols are detrimental to comprehending the smallest piece.

how long has Netflix Streaming been like this?

2011-05-24T08:48:00.000-04:00

After seeing a few relatives' satisfaction with Netflix Streaming, I recently signed up. I'd heard that Netflix signed a deal for Star Trek TV series (almost all of them) starting later this year, but I assumed that there would be enough to watch until then. Of course, a lot of TV shows have also finished their runs for a few months, so that was another motivator.

After poking around, I have to say I'm amazed. Even pre-Star Trek, Netflix is streaming...

Farscape
Doctor Who
Science documentaries, which are freely available on the PBS websites but not with the same playback possibilities
Various Joss Whedon signature shows
Lost
Merlin, a fun but simple show that hooked me during its US showing
Battlestar Galactica (2004)
Futurama, as if I didn't have all the DVDs
Heroes, if I ever feel like seeing the latter seasons
Mythbusters
Stargate SG-1
Eureka

I concede that some of these episodes are, er, aged, because I assume that new episodes don't become available on Netflix until long after original airing. Still, especially for the price, this is pretty incredible for someone of my sensibilities...

Pragmatist afterword

2011-05-17T16:16:00.000-04:00

Some of the points of philosophical Pragmatism, or at least my personal digestion of William James' views, correspond nicely to the numerous instances of a datum never being alone:

"Meaningful communication occurs in a purposeful context". Pragmatically, interpretation is poorly-defined for messages like double-talking philosophical statements that appear to assert nothing in particular or to argue about relations among items whose existence is questionable at best. These statements correspond to messages composed of symbols that lack thorough and steady connections to information. Murky ground for the symbols in a communication implies that regardless of the fidelity of the message's bits, the message's intent is uncertain.
"Meaning involves constructive actions". Pragmatists see concepts as works in progress that, appearances to the contrary, came about for the sake of specific goals. This approach is a stark contrast to the alternative, which is that some "special" concepts exist regardless of humans. The mental constructive actions correspond to the procedure that defines a new set of symbols from other information. Symbols neither preexist nor are always obvious.
"Relations of concepts are also concepts". In other words, the same analysis and status of concepts applies also to relations. During one "mode", i.e. for the sake of some relevant goal, the human's raw experience might be treated like an undivided whole. During a second mode, i.e. for the sake of a different goal perhaps of equal importance, the human's raw experience might be treated like a convoluted structure of concepts separated by supporting relations. Whether concepts or relations among concepts, the contents of mentality mutually depend on each other and more importantly on an external rationale. Similarly, the general steps in procedures for sets of symbols are also information and therefore also communicable by symbols. "The color 'burgundy' is a dark and purplish shade of red" is a procedure that presumes "dark and purplish shade" is a valid symbol for a detailed informative manipulation of "red" information. And naturally that symbol itself depends on the symbols "dark" and "purple".
"Reality is evaluated according to human interest". Some critics attack this proposition by intentionally underestimating the depth and breadth of actual human interest. They pretend that motivation is simple-minded, when experience has shown it to have endless variations. Infants and household pets admittedly have primitive opinions on what's interesting and why, but it's ludicrous to restrict human interest to this limited model. Anthropologists know better. Besides, the salient factor isn't the interest's characteristics but its effect, which is the human's manipulation of raw information in pursuit of ends. One sees a good automobile and labels it as a tool of transportation. Another sees the same automobile and labels it as an intriguing hobby project. The total raw physical information of the car's existence is objective. Human perceptions are translations of that raw information, augmented with the observer's interest. These interpreted perceptions, not the original physical facts, are the pragmatic segments at the bottom of symbols.
"The essence of truth is piecemeal and it progresses by lurching, courageous leaps". To the extent that a philosophy has a disposition, Pragmatism is earthy. It's prone to discourage promotion of human thoughts into ethereal and unsullied domains of theoretical purity. It learns toward induction more than deduction. Its advice is to discover unifying truths via patterns in details, not via cramming details into remote and calming abstractions. Feasible approximations are acceptable substitutes in place of brittle logical categories. Hence having fewer symbols in a set is easier to contemplate, at the cost of fewer distinctions and less information. Cutting the world in two symbolic halves has a poetic simplicity, but it would be an awful coordinate system for navigation. Profundity requires few symbols because it has so little to say.

a datum is never alone

2011-05-15T18:54:00.000-04:00

The proposition that meaning is isomorphism has a partner: fundamentally, information is only informative in collaboration with other information. Thus, the importance of context shouldn't be underestimated. Examples abound. Pronouns need referents. Sentences need dictionaries, mental or physical, stated or inferred. Amounts need units of measurement. Hobbit heights need people nearby for visual comparison. No datum (singular of data) is useful in isolation. No datum is a "loner".

a bit is "this not that"

In a primal sense, communication is "meta-", with connections outside itself. Since direct telepathy neither is desirable nor is physically possible (the mapping from one brain to another is too complicated because of many differing neural connections), thoughts cannot be sent "raw". Every element of a message can only be symbolic. For the message's symbols to be understood, the sender and recipient must share common knowledge. They must already be aware of both the symbols actually in the message and the symbols' source set (i.e. vocabulary). The symbol cannot be independent of the participants' prior knowledge.

Furthermore, a comprehensible message is a series of selections from the symbols in the set. To choose one symbol is to reject other symbols. Part of a chosen symbol's meaning resides in its distinctions from the rest of the symbols in the set. It's vital that the communication of the symbol be unambiguous, to prevent confusing one symbol for another, and conventional, to avoid needing exotic media or equipment.

The most obvious tactic is to count the symbols in the set. The first symbol counted will be represented by communicating "1", the second "2", and so forth. In the resulting number sequence, it's always clear which number represents which symbol and the message itself doesn't require unusual communication capabilities. The set of symbols is in the form of a number line (like a ruler) with each symbol as a numbered point.

However, there's a better option. The set could be in the form of a "road" in which the symbols are "houses". To identify/reach a symbol is to travel down this road according to the directions, and since there's only one road the directions are no more than many answers to the sole repetitive question "Stop at the next house?" Instead of a unique number, each symbol's representation is a unique progression of yes/no answers.

This technique has considerable advantages. First, the directions for symbols have distinct stops. No spacing is needed between the adjacent symbols in a message. The last answer in a symbol's directions is always "yes", so the answer after it is part of the next symbol's directions. Second, since the "closer houses" have shorter directions, putting first the most probable symbols or "houses" makes it more likely that the whole message, i.e. a sample of symbols, will be shorter. Third, the media needs are even less demanding than sending (base 10) numbers. Now just "0" and "1" are enough. Two possibilities, repeated. Bits.

Does a bit have meaning? On its own, not much. But along with many companion bits, enough to meet any need.

a set is "what you get by following procedure"

Communicators can send piles of bits. Taken together, those bits in turn signify a specific chain of symbols. Those symbols in turn signify atomic members of a source set (or alphabet). Everything depends on something else, with one remaining exception at the end of the dangling thread. Doesn't it appear that the source set is now the true wellspring of meaning that needs nothing else?

Not remotely! Being a collection of symbols, the source set itself is symbolic. No matter how self-defining it seems, nevertheless it's about still other information. For example, a digital spin on the ever-popular philosophical discussion of color words.

Over the years, digital images have represented color through a range of strategies depending on the relative priority of conserving space. (This detail admittedly feels more and more quaint to the oblivious user who at the present time routinely dumps a multitude of "RAW" exposures from the camera into the computer's permanent storage.) One of the trade-offs is to reduce the number of distinct colors to employ in the image, called the palette. By this reduction fewer bits are necessary to identify a color; a smaller range requires smaller numbers or indexes for the colors. Especially for an untrained eye and images like a logo or icon, 256 colors often works well enough and may be overkill.

The digital image's color palette is certainly a close analogy for a symbol source set, and really a direct application of the concept. Assuming the raw image had a much wider array of colors, the palette is a purposely limited sample of that original array. It's an abstraction of it, an approximation, a pigeon-holing, a filtering. Moreover, for exceedingly colorful and detailed images that contain a lot of grades, the choice of palette becomes uncertain and debatable. In general, symbol sets aren't unmistakably derivative from the symbolized information. Symbolization starts with categorization, and categorization is an action.

For a set of symbols to be reproducible accurately, the steps to produce it should be spelled out with clarity and precision. Consequently the definition of a set is a procedure. It has the two sections previously listed, categorization and then symbol-assignment to the categories. Sometimes the procedure could be straightforward (by referencing another set), e.g. "integers greater than 10 and less than 30, each integer symbolized by the typical numerals". Sometimes the procedure could be complicated and even surprising in its results, e.g. the Mandelbrot set. Sometimes the procedure could be an exhaustive (finite) listing of all symbols with a per-symbol definition. At the extreme, a stream of symbols continuously produced through a "procedure" of pure randomness (not pseudo-random!) could be the basis for a perfectly mysterious and unbreakable representation of a message; what makes it usually impractical is that the stream has to be as long as the message and the recipient requires full copies of the message and the stream to be assured of correct decoding...

Indeed, the random set of symbols is an ideal illustration of a valid objection to the last few paragraphs: "It's fine that communication depends on bits, and bits depend on symbols, and symbols depend on a predefined set, but how can the set depend on a 'procedure'? How do you communicate a procedure? By your own reasoning, how can it be possible to communicate anything, let alone a procedure, before the set of communication symbols is known?"

Of course, in keeping with the theme "a datum is never alone", the predictable reply to that objection is "Communication of the set-defining procedure happens by reliance on still other symbols". Those other symbols are individual processing instructions. By assumption, the receiver already knows and can perform a set of elemental and easy processing actions, each of which has a fixed symbol. The procedure to construct the new set of symbols is one more message whose symbols happen to represent requests for action by the receiver.

For symbols for integers greater than 10 and less than 30, some of the steps are checking that an integer is greater-than and checking that an integer is less-than. For the Mandelbrot set, feeding the result of a mathematical function back into it. For the image color palette, storing the 256 colors one-by-one. In a computerized procedure, each action is literally minuscule: store information, load information, add two numbers, check if a number is less than zero. These actions and therefore symbols (i.e. processor instructions) seem quite pointless out of context. In the wake of a huge organized group of actions, a highly meaningful procedure takes shape and yields products.

a segment is "interest in raw experience"

Strictly speaking, a symbolic list of generalized actions isn't sufficient to communicate and obey an actual procedure. Like other information, actions aren't "loners". An action is only significant insofar as external information has effects on it and it has effects on external information. The exchange of effects may often be slight and subtle, producing a temporary result in preparation for succeeding actions. In any case, ultimately actions are changes to information: copies, creations, modifications, extensions, adaptations.

Such manipulations are the essence of the meaning of the generated symbol set. From a surface perspective, symbolism is substitution, i.e. convenient shorthand names of potentially lengthy manipulations of information. "Let me introduce 'Martin', whose parents are ___, who lives in ___..." Decoding the symbols is reversing the procedure. "I'm sure that you remember Martin." "The Martin whose parents are ____, who lives in ____?"

In practice, procedures for frequent symbols are rarely repeated in full. Instead, the excellent pattern-matching powers of the human brain notice rapidly that a symbol's procedure always terminates at the same external information. Once the pattern is seen, the brain devises a shortcut of the procedure by forming a pair out of the start and end, a direct line, a conditioned "reflex" of information. It happens with little effort. Eventually, a human can have trouble consciously recalling the procedure at all. Profound intertwining of a symbol and its external information leaves a familiar whole, with the illusion that the symbol isn't separate.

Symbols denote procedural bridges between information. The procedure probably manipulates information via previously known symbols, leading to a novel "macro" set of symbols whose meaning becomes a procedure of procedures of procedures. Presumably, no matter the heights of the overall edifice of labor that underpins all the symbols and the interconnections, there is irreducible information far down below, unmediated and unprocessed. Else the complex of symbols would be circular. The massive loop could still be intricate and interesting, perhaps also grammatically correct based on some standard, but to humans it wouldn't necessarily be any more informative than frimjun tollywobbits bersing shugbovs. Irreducible information that precedes and fuels symbols shall be named "segments".

Segments seem like the most raw and standalone information possible, but that impression too is mistaken. As suggested by the name, segments are selective portions of "raw experience". A human's raw experience is the sum of everything that affects the human, causing shifts in activity in nervous system cells, which propagate across axons and synapses. It's everything heard, everything seen, everything felt, everything smelt. A camera or microphone picks up some of the same physical phenomena, albeit through differing mechanisms.

Physical information of raw experience is the least subjective, the least egocentric. Well-known measurement and calculation limitations notwithstanding (e.g. intrinsic relativity and quantum uncertainty), information about the positions and movements of particles is the bottom line. Mythical complete information of a set of particles is the Axiom of that set; there's nothing further to speak of. Laying aside the theoretical and practical impossibilities, other messages about that set are derivatives of that Axiom. Hypothetically, "cold glass of water" is backed by untold quantities of "physic"-al information on the movements of molecules of dihydrogen monoxide. And an irrepressibly boring communicator who sends more than that many bits of (present-tense) information directly about just the water is being redundant.

Given the horde of physical information, arguably a brain's ingenuity is what it elects to discard. Focus is indispensable. Survival is closely tied to a creature's ability to attend and respond to information of interest. Brains transform raw experience into segments because a lot of information is quite irrelevant and monotonous. It's "background" that's worthy of "peripheral" monitoring alone. Segmentation is a primary component of perception.

Visual points, of similar color, moving in the same direction and speed at once, are a segment of raw experience that draws the attention of most creatures. So do segments of loud and sudden noises or tasty odors. Pains discourage and pleasures encourage. The common factor again is interest. Human brains are eminently trainable but inborn interests offer the first opinion and initiate the first constraints on the innocent tide of information of raw experience. Perceptual unity is useless to answer the foremost questions of a competitive, feasible organism. Segments simplify reactions that craft beneficial answers.

Thereafter, years of human social interaction and mimicry build on the "natural" segments. They learn verbal symbols for the most prominent segments: kin, food, body. Simultaneously they learn to learn and to seek feedback by trying out symbols. To start, the teaching proceeds by gestures and demonstrations for simple segments such as nouns, verbs, adjectives, prepositions in immediate shared perception. The actions that define this set of symbols are literal acts: the motions of the teachers enacted on segments that already have interest to the learner. Sooner or later, the repetitious well-established actions allow for teaching the symbols for segments that have weak to nonexistent natural interest; at that point, humans' untiring curiosity and desire to please are great motivational aids. The ability to identify with the speaker's perspective, to imagine what the speaker intends to say, is a vital skill.

After a "critical mass" of symbols and grammar, the actions for learning new sets of symbols become the previously-mentioned mental manipulations of information. Description, contrast, and metaphor are typical symbol-defining actions. Explosive growth of information ensues. Symbols trigger computations and then symbolize the outcomes. The cycle continues.

Art's Razorette

2011-05-10T19:44:00.000-04:00

Art's Razorette: When "Occam's razor" is mentioned by name in a casual conversation, the simplest explanation is that the speaker is trying far too hard to impress listeners with his or her purported intelligence and/or learning.

(Sometimes it is appropriate to name-drop the razor, particularly in the midst of formal or academic argumentation, but only with due discretion.)

I started reading an information theory textbook, and...

2011-04-22T22:33:00.002-04:00

...I've concluded that in an optimal, entropy-approaching code for the textbook, the bits for sigma Σ would be minimal. Ba-dum, ching!

peeve no. 262 is verification by bluster

2011-04-15T15:56:00.000-04:00

In some ways, human nature is flawed. Complaining about it won't help. But here I go anyway...

Many arguments in debates are intellectually wrong or flimsy at best. This is well known and oft mentioned by pedants. In my opinion, these transparently fallacious arguments are less insidious than their emotional counterpart, "verification by bluster". With a sufficient combination of affirmation, bravado, machismo, etc., humans appear to forget about the concept of proof.

Confidence levels may be out of proportion to the supporting data. In a card game this would be called a bluff. If anyone asks "Are you sure?", then the answer is always "100%", in a firm voice and accompanied by a steady gaze. Do it right and humans just won't bother with whether the steely assurance is grounded in reality. This is particularly important in sales or management; complete detailed honesty about actual evidence would merely demonstrate "lack of belief".
Naive arguers sometimes act as if winning means discrediting the other side's facts. Not necessarily true. It might be enough to invent a sufficiently nasty name for a part of the opposing argument, then repeat it endlessly.
It's similarly naive to act as if an individual data point "speaks for itself". It's a comparatively simple matter of weighting. Take the contradictory data point, juxtapose it against a supportive data point, state assertively that you're providing "full context". Humans are unlikely to request entire data sets in any case, because analysis might involve, y'know, math.
In general, complexity is a bad tack. Messy corner cases and ambiguous conclusions aren't gratifying. Exceptions to beloved rules are downright unsettling to some personality types. Hence the more profitable route of argumentation is to accept simplistic thinking, not challenge it. It won't work to spout admonitions to replace laughably course-grained prejudices with complicated observations. Confronted with the world as it is, right now, humans instead prefer a personalized/sanitized perspective in which solutions are easy, relationships are causal, and heroes and villains are well-defined.

I don't doubt the levels of sincerity or complacency in a human who's attempting verification by bluster. What I doubt is the level of substantiation. Put frankly, any human can vomit propositions about this or that, and experience has shown me that his or her stirring faith in those words doesn't count as verification. Pragmatism's approach is to check precisely what the human's generalizations symbolize and imply. "I know ____ because of ____. Therefore you can observe it for yourself." I find this so much more satisfactory than when someone says "I know ______ because I SAID SO. My force of personality in some unspecified way justifies ignoring my lack of concrete data and/or my refusal to allow others to examine my statements objectively." At least explain how, in principle, an unbiased agent would confirm the hypothesis. Even better, explain how that agent could falsify it.

package delivery tracking can make you sadder

2011-03-24T12:01:00.000-04:00

One of the more striking points in How We Decide by Jonah Lehrer is the strong and sophisticated reward/punishment feedback systems of the brain, which people experience as strong emotions. A vital factor incorporated into these responses is expectation. Expected rewards or losses have less effect than the unexpected. Gaps between expectations and reality make the difference. An unexpected windfall of $20 feels good. A lackluster gain of $20 for an expectation of $40 feels bad.

Hence compulsive usage of online package delivery tracking is unlikely to leave the user happy about being well-informed. Each time the user checks the tracking, the expectation is "delivery has progressed since the last time I checked". If the result matches that expectation, the user feels a burst of reward stimulus; in fact, that experience may "hook" the user into compulsively checking again a short time later. If the result doesn't match that expectation, the user feels a loss despite the delivery happening precisely on schedule! The tracking falls into the category of long-term planning with intermittent milestones. If I'm interpreting the book's advice correctly, the better way to make decisions in this category is with reason, the prefrontal cortex. Logistics is more of a math problem ("A delivery truck embarks at 5:00 AM on a 200 mile trip going the posted speed limit, when should it arrive?").

Also, the tracking creates a subconscious illusion of control. Like the sports fan who cheers the home team playing on TV, mere observation is clearly powerless to affect the outcome. However, each time an observation yields a favorable outcome, the brain "learns" that observation is a good tactic. Of course, we know that watching the pot of water won't make it boil, and reloading the tracking Web page won't make a sophisticated transportation network run according to design. This is why someone should resist the temptation to check the package delivery more than twice per day. That's a setup for disappointment and less happiness over time. For some people, a three-week delivery with no tracking at all may actually be more pleasing than a three-day delivery with tracking. ("I forgot this package was on the way. What a nice surprise!")

cognitive load reduction

2011-03-23T15:15:00.000-04:00

Debates about how to write better code (i.e. fewer bugs) revolve around increasing maintainability, but not too long ago I recognized a related and perhaps fundamental criterion: cognitive load reduction. The fewer disparate items that a developer must contemplate simultaneously, 1) the lower the chance that a mistake will slip in unnoticed, 2) the greater the amount of attention left for the details of the problem/domain rather than the twists and turns of the code. When code is confusing and demanding to comprehend, the cognitive load is greater, and therefore it's more difficult to write, trace, debug, and modify.

Awareness of impact on cognitive load should change the choices that someone makes. Sure, the first task is to produce code that meets the known requirements. Yet developers shouldn't then neglect the second task of refining the code until it's sensible. Code has two audiences, machine and human. This is a lens for perceiving the usual code debates.

Units of code organization with hard boundaries reduce cognitive load by freeing the reader from looking through many peripheral lines to trace execution.
Good names reduce cognitive load by freeing the reader from inferring what a variable is for.
An easier build process reduces cognitive load by freeing the builder from rehearsing and reciting a series of error-prone manual steps.
Version control that meets the team's needs reduces cognitive load by freeing the team from devising complicated workarounds.
Domain models that match the way that everyone thinks (according to common agreement) reduce cognitive load by freeing them from continual lossy translation of one another's statements.
Frameworks reduce cognitive load by freeing the reader from examining custom-made immature solutions to ordinary incidental problems, e.g. templating, MVC, protocols. On the other hand, obtrusive frameworks may increase cognitive load by overshadowing and complexifying the base code without marginal benefit.

Effective writing in natural human language doesn't place an excessive burden on the reader, who's trying to interpret the message. Similarly, effective writing in programming language doesn't place an excessive burden on the maintainer, who's trying to interpret the code's intent.

calling a truce on sprocs

2011-03-20T22:57:00.000-04:00

For a while, I've mostly been dismissive of database stored procedures or sprocs. The rationale is that databases are for storage ("Really, Capt. Obvious?"). By contrast, calculations, conditions, and data processing in general belong in a separate, dedicated tier; the clear benefit is a much more flexible, capable, reusable, and interoperable platform/language than the typical sproc. In this middle tier the intelligence resides in neatly divided objects that could potentially exploit different "persistence strategies" than the default database of choice. These objects presumably act as better models of the domain than collections of rows and columns. Application development happens on top of this middle tier rather than the database.

The opposite path is integration at the database level. Differing software all use the same "master" database. There may be a recurring import script that populates one or more tables with external data, entry interfaces that quite clearly manipulate rows and columns, canned reports whose queries become increasingly complicated. Knowledge of which tables to join or which column values to exclude spreads out through everything that performs a similar task. Analysts speak of the database as if it were the domain. Their first implementation question on new projects is "What tables do we need to add?"

Consequently, integration at the master database level can result in fragmentation and duplication. Enter sprocs. Essentially, a thoughtful agglomeration of limited and self-contained sprocs could take the place of a nonexistent middle/domain tier for some purposes. If everyone needs to run the same query all the time, at least putting it in a sproc will consolidate it. A complex calculation that everyone repeatedly makes could be computed in a single sproc. Ugly warts of the database model could have workarounds specified in sprocs.

Storage technology independence is lost with sprocs, but ongoing integration at the database level already makes that impossible. Sproc writing requires some learning but is offset by the considerable advantage of not having to rewrite the code in multiple clients. IDE support is less than ideal but a sproc shouldn't be too large anyway. Names and calls of sprocs are also rough but are likely to require less extra documentation than the alternative of laboriously touring table relations.

Sprocs: better than nothing.

generalized distributed communication of data sequence graphs

2011-03-16T22:19:00.000-04:00

https://github.com/ArtV/DataSequenceGraph. This is an experimental release whose quality is not assured. Data Sequence Graph represents a collection of data value sequences (IEnumerable<T>), called "chunks", as possibly overlapping one-way routes in a single graph. More introductory details.

First some up-front definitions and essential points so the jargon isn't too opaque:

Graph: * a complete somewhat large set of nodes and edges that encodes chunks as full routes
Apply: * the overall action of adding new nodes/edges to a graph, * the source data may be a chunk or a delta
Delta: * a possibly fragmentary and somewhat small set of new nodes and/or edges that, when applied to a destination graph, will enable it to store the same chunks as the source graph (e.g. the CLI program produces a delta for the parameters "-c", "-m", "-l"), * a delta is not a "graph" so its sole usage is sending/receiving updates
Base: * the cumulative effect of applying prior deltas, * the base is a graph identified by the last delta applied, * past communication establishes a base graph as a shared baseline for future communication, * a delta has one base for which it is valid, * a peer has a "base" that includes all deltas it has received and sent
Local graph: * a graph that possibly includes chunks and nodes/edges that haven't been communicated
Peer: * a particular instance/site of a graph that communicates with other peers for the graph, - a peer has deltas and bases and a local graph, * peers communicate with one other peer at once
Theory: * unique ordering of deltas in which the base for each delta is the preceding delta (except for the first delta, whose base is nothing/null/empty graph)

Continuing communication of deltas to a data sequence graph shared by two peers had a straightforward conflict resolution protocol: the first peer to send a delta obliged the second to honor it. If the recipient's local graph encoded additional chunks that hadn't been sent, then the recipient "rebased" them by applying the delta to the base and then reapplying them afterward.

To generalize to more than two peers, the "publish or perish" rule requires more nuance. Just as the local graphs of two peers can diverge and force subsequent rebasing when one communicates a new delta, entire subgroups or partnerships of peers could potentially diverge from the rest. In this situation, it doesn't make sense for the sender of a delta to always take precedence. A peer stuck in the middle between two others could hypothetically end up flip-flopping between the conflicting deltas they send or arbitrarily ignoring one indefinitely.

There must be a unique universal set of deltas, and a unique universal ordering of those deltas, that takes superiority. So I've extended the publish or perish rule to include time. For each base, the delta that "wins", and officially becomes applied to create the next base, is the delta that was sent and generated first in time. Hence the naming scheme for a delta is 4 digits for the number of chunks/routes in the delta's base, then "-delta-", then a UTC RFC 3339 timestamp that substitutes dashes for colons for the sake of file-system friendliness. It looks like "0023-delta-2011-03-13T05-33-55Z". The scheme follows the RFC 3339 lead in allowing alphanumeric order to match chronological order. # of chunks in the base must come first because a delta to a base with greater chunks always comes after a delta to a base with fewer chunks. No matter which peer created or received or sent a delta, its status and position in relation to other deltas is fixed.

This strategy has two implications: 1) there is a definitive evolution of the graph that emerges from the actions of all peers, 2) at any given time a peer probably doesn't have sufficient information to be completely accurate. A peer can only use the deltas it has received and sent to form its own series of deltas/bases. As communication happens, this "theory" will go through stages of expanding, rewriting, and even partial abandoning. The peer uses the base that results from the theory when it sends and responds to delta requests, and at that time the other peer may send contradictory deltas to replace the erroneous latter part of the "current" theory. Peers are eventually consistent. To facilitate this give-and-take, a peer keeps past delta files in a dedicated directory (specified by "-d" in the CLI). Listing the delta files in order yields the most recent theory and the means to reconstitute a base by repeated applying of deltas (think incremental backups onto the last full backup).

Deltas are sent on demand, i.e. the familiar stateless client/server, request/response, pull interaction. Of course, a peer could switch roles from interaction to interaction. A delta request consists of a ".base"-extension text file. The contents are delta identifiers in reverse order and delimited by pipe "|". The first/recent delta identifier is also the name of the file. The quantity of deltas in the file can vary, presuming no deltas in the requester's theory are skipped over. Right now the implementation sends 5 (if available). Delta requests ask, "Do you have a graph delta intended to be applied directly after this delta? If not, how about the one before? Or the one before?..." Since the last delta applied to a base is a practical identifier for the base, these delta identifiers are really a list of candidate bases for new deltas.

Indeed, the first task of the response logic is to locate the latest match against the bases in the request. If there's no match, the response looks like a counter-request .base file, containing bases of the responder that are older than the oldest base in the request. Note that the response isn't really a request for deltas but suggestions or notifications for the requester. The responder would request deltas in a different transaction in which it acts as the client.

If there's a match, the response is instead a .tar.gz flat archive of deltas against the matching base. The archive also includes a .base file. The prime rationale for .tar.gz is increased convenience, and most certainly not saving space through compression; delta files use the binary+text graph format and offer scant opportunities for compression aside from similarities among differing data values. Some delta files may be smaller than the file header information that .tar keeps about the file. Whenever the responder has zero deltas to send despite the match, the response is no more than a .base file that echoes the request.

The requester's reaction to the response mirrors the actions taken by the responder. If the responder sent back a plain .base response whose first entry matches the requester's base, then the responder had nothing to send and nothing need be done. Otherwise, there's yet another search through the list of suggestions for a match, and then generation of a new delta request. If there's a match, the new request is for it. Else, the new request is for the next batch of candidates: bases that are older than the oldest suggested base in the response.

An archive response, as the end goal of the interaction, is more involved. When the requestor has no deltas applied to the corresponding base yet, the deltas are extracted to the delta directory and the requestor's local graph chunks are reapplied to the resulting new base. However, preexisting deltas against the base will force a comparison. In the event that the preexisting deltas have primacy, the archive is ignored. Alternatively, preexisting deltas without primacy are "trashed" by appending an "x" to the file extensions. Then as in the simpler case the archive deltas are extracted, the new base is created, and the local graph chunks are finally reapplied.

The CLI has parameters for these operations. "-d" for the delta directory to use, "-n" to initialize the delta directory with the passed primary graph, "-R" to create a brand-new delta request, "-i" to pass and handle an incoming delta request, "-b" to pass and handle a ".base" response to a delta request, "-a" to pass and handle a ".tar.gz" response to a delta request.

continuing communication of data sequence graphs

2011-03-09T11:03:00.000-05:00

function prompt { ">>" }
>>echo "Knock knock. Who's there? Closure. Closure who? Closure mouth when you eat." > closure.txt
>>echo "Knock knock. Who's there? Hugh. Hugh who? You, that's who!" > hugh.txt
>>echo "Knock knock. Who's there? Eliza. Eliza who? Eliza lot. Don't trust em." > eliza.txt 

Next, store the words of closure.txt in a data sequence graph, sent to files "clos.dat" and "clos.txt". Designate the creator as communicator 1, who then sends the graph to communicator 2. So this initial graph is the common base graph. Later, communicator 1 adds the words of hugh.txt to the initial graph to produce a local graph sent to files "closhugh.dat" and "closhugh.txt". Being talkative, communicator 1 figures out the delta between the local graph and the base graph and sends the delta to communicator 2. Since the base is the initial graph, the secondary graph (to run the delta against) is in clos.dat/clos.txt. The delta goes into files "hughdelta.dat" and "hughdelta.txt". These files are sent to communicator 2. Since the delta is published against the old base graph, the delta applied to the old base graph becomes the new base graph. Communicator 1's local graph in closhugh.dat/closhugh.txt is now the base graph.

>>.\DataSequenceGraphCLI.exe -s closure.txt -E clos.dat -T clos.txt
>>.\DataSequenceGraphCLI.exe -e clos.dat -t clos.txt -s hugh.txt -E closhugh.dat -T closhugh.txt
>>.\DataSequenceGraphCLI.exe -m -e closhugh.dat -t closhugh.txt -f clos.dat -u clos.txt  -E hughdelta.dat -T hughdelta.txt 

Sometime after communicator 1 sent the initial graph, communicator 2 was unlucky enough to stumble on the eliza.txt knock-knock joke, which then got stored in communnicator 2's local graph as "closeliza.dat" and "closeliza.txt". Communicator 2 made sure to keep around the last-communicated graph, clos.dat/clos.txt. Not being talkative, communicator 2 opted to delay producing and sending a delta to the last base graph.

>>.\DataSequenceGraphCLI.exe -e clos.dat -t clos.txt -s eliza.txt -E closeliza.dat -T closeliza.txt

When communicator 2 receives the delta from communicator 1, communicator 2 first uses the delta on the old base graph to create a new base graph for future communication, stored as "closhugh2.txt" and "closhugh2.dat". Second, communicator 2 creates a new local graph using the old local graph, the delta, and the old base. The new local graph goes into files "closhugheliza.dat" and "closhugheliza.txt". Neither of these communicators have much flair for file names. After the rather involved command, communicator 2 ends up with a graph that contains the sentence chunks that are in 1) closure.txt, 2) hugh.txt, and 3) eliza.txt.

>>.\DataSequenceGraphCLI.exe -e clos.dat -t clos.txt -f hughdelta.dat -u hughdelta.txt -E closhugh2.dat -T closhugh2.txt
>>.\DataSequenceGraphCLI.exe -e closeliza.dat -t closeliza.txt -f hughdelta.dat -u hughdelta.txt -g clos.dat -w clos.txt -E closhugheliza.dat -T closhugheliza.txt

Eventually, communicator 2 gets around to computing a delta against the last base graph stored in files closhugh2.dat/closhugh2.txt, and then sends it to communicator 1 in files "elizadelta.dat" and "elizadelta.txt". Communicator 1 has no local chunks, so nothing more need be done than updating communicator 1's copy of the old base, which communicator 1 stores in "closhugheliza1.dat" and "closhugheliza1.txt".

>>.\DataSequenceGraphCLI.exe -m -e closhugheliza.dat -t closhugheliza.txt -f closhugh2.dat -u closhugh2.txt -E elizadelta.dat -T elizadelta.txt
>>.\DataSequenceGraphCLI.exe -e closhugh.dat -t closhugh.txt -f elizadelta.dat -u elizadelta.txt -E closhugheliza1.dat -T closhugheliza1.txt

Communicator 1 and communicator 2 have resynchronized their data sequence graphs. The graph in communicator 1's most recent base, closhugheliza1.dat/closhugheliza1.txt, is the same as the graph in communicator 2's most recent base, closhugheliza.dat/closhugheliza.txt .

telling knock-knock jokes to a data sequence graph

2011-03-08T09:54:00.000-05:00

function prompt { ">>" }
>>echo "Knock knock. Who's there? Closure. Closure who? Closure mouth when you eat." > closure.txt
>>.\DataSequenceGraphCLI.exe --verbose --splittext closure.txt --outedges edges.dat --outvalues values.txt

These are the long-form parameters; one-character parameters are also available. The line splits the contents of closure.txt into sentence chunks with word values, sends the resulting graph in binary+text format into two output files, and because of "verbose" also echoes back the entire graph contents, first node by node then chunk (route) by chunk...

Added sentence starting with Knock at start node 0
Added sentence starting with Who's at start node 3
Added sentence starting with Closure at start node 6
Added sentence starting with Closure at start node 8
Added sentence starting with Closure at start node 10

0 Gate
   ..1 if already -1..-1
1 Value: Knock
   ..2 if already 0..1
2 Value: knock
   ..0 if already 0..1
3 Gate
   ..4 if already -1..-1
4 Value: Who's
   ..5 if already 3..4
5 Value: there
   ..3 if already 3..4
6 Gate
   ..7 if already -1..-1
7 Value: Closure
   ..6 if already 6..7
   ..9 if already 8..7
   ..11 if already 10..7
8 Gate
   ..7 if already -1..-1
9 Value: who
   ..8 if already 8..7
10 Gate
   ..7 if already -1..-1
11 Value: mouth
   ..12 if already 7..11
12 Value: when
   ..13 if already 11..12
13 Value: you
   ..14 if already 12..13
14 Value: eat
   ..10 if already 10..7

------ Chunks in graph:
#1:
0..1:Knock..2:knock
#2:
3..4:Who's..5:there
#3:
6..7:Closure
#4:
8..7:Closure..9:who
#5:
10..7:Closure..11:mouth..12:when..13:you..14:eat

Now a second knock-knock joke. The previously generated graph for closure.txt is loaded by specifying the input files and the "missing" option requests output of the difference rather than showing the end result after adding...

>>echo "Knock knock. Who's there? Hugh. Hugh who? You, that's who!" > hugh.txt
>>.\DataSequenceGraphCLI.exe --loadedges edges.dat --loadvalues values.txt --splittext hugh.txt --missing --outedges hughdelta.dat --outvalues hughdelta.txt
Added sentence starting with Knock at start node 15
Added sentence starting with Who's at start node 16
Added sentence starting with Hugh at start node 17
Added sentence starting with Hugh at start node 19
Added sentence starting with You at start node 20

15 Gate (new) no requisite to next required
1 Value "Knock" requisite is route edge #0
2 Value "knock" no implied edge to next node
16 Gate (new) no requisite to next required
4 Value "Who's" requisite is route edge #0
5 Value "there" no implied edge to next node
17 Gate (new) no requisite to next required
18 Value "Hugh" (new) no implied edge to next node
19 Gate (new) no requisite to next required
18 Value "Hugh" requisite is route edge #0
9 Value "who" no implied edge to next node
20 Gate (new) no requisite to next required
21 Value "You" (new) requisite is route edge #0
22 Value "that's" (new) requisite is route edge #1
9 Value "who" no implied edge to next node

hughdelta.txt contains the string "Hugh|You|that's" (15 characters or 15 UTF-8 bytes). That end list of nodes/requisite information is what goes into hughdelta.dat, in the binary form that I described here. hughdelta.dat is 36 bytes. Each line in the list is a node/requisite record. There are 5 new gate nodes, one for each sentence/chunk/route, which consume 5 * 2 = 10 bytes. There are 7 lines that use preexisting value nodes, which consume 7 * 2 = 14 bytes, for a running total of 24 bytes. Finally, there are 3 lines of new value nodes, which consume 3 * 4 = 12 bytes (2 bytes for the usual sequence number and guide bits and 2 bytes for the index for the new values), to reach the final total of 36. Notice that since all requisites fall into one of "no requisite or no implied edge", "starting requisite" (lines with route edge #0), "the previous edge is the requisite" (lines with the route edge #1), the reader of the file can reconstruct the requisites without the requisites consuming actual file space as stated data values. One last knock-knock joke, this time just adding the new joke to the first graph and showing the result...

>>echo "Knock knock. Who's there? Eliza. Eliza who? Eliza lot. Don't trust em." > eliza.txt
>>.\DataSequenceGraphCLI.exe --loadedges edges.dat --loadvalues values.txt --splittext eliza.txt
Added sentence starting with Knock at start node 15
Added sentence starting with Who's at start node 16
Added sentence starting with Eliza at start node 17
Added sentence starting with Eliza at start node 19
Added sentence starting with Eliza at start node 20
Added sentence starting with Don't at start node 22

0 Gate
   ..1 if already -1..-1
1 Value: Knock
   ..2 if already 0..1
   ..2 if already 15..1
2 Value: knock
   ..0 if already 0..1
   ..15 if already 15..1
3 Gate
   ..4 if already -1..-1
4 Value: Who's
   ..5 if already 3..4
   ..5 if already 16..4
5 Value: there
   ..3 if already 3..4
   ..16 if already 16..4
6 Gate
   ..7 if already -1..-1
7 Value: Closure
   ..6 if already 6..7
   ..9 if already 8..7
   ..11 if already 10..7
8 Gate
   ..7 if already -1..-1
9 Value: who
   ..8 if already 8..7
   ..19 if already 19..18
10 Gate
   ..7 if already -1..-1
11 Value: mouth
   ..12 if already 7..11
12 Value: when
   ..13 if already 11..12
13 Value: you
   ..14 if already 12..13
14 Value: eat
   ..10 if already 10..7
15 Gate
   ..1 if already -1..-1
16 Gate
   ..4 if already -1..-1
17 Gate
   ..18 if already -1..-1
18 Value: Eliza
   ..17 if already 17..18
   ..9 if already 19..18
   ..21 if already 20..18
19 Gate
   ..18 if already -1..-1
20 Gate
   ..18 if already -1..-1
21 Value: lot
   ..20 if already 20..18
22 Gate
   ..23 if already -1..-1
23 Value: Don't
   ..24 if already 22..23
24 Value: trust
   ..25 if already 23..24
25 Value: em
   ..22 if already 22..23

------ Chunks in graph:
#1:
0..1:Knock..2:knock
#2:
3..4:Who's..5:there
#3:
6..7:Closure
#4:
8..7:Closure..9:who
#5:
10..7:Closure..11:mouth..12:when..13:you..14:eat
#6:
15..1:Knock..2:knock
#7:
16..4:Who's..5:there
#8:
17..18:Eliza
#9:
19..18:Eliza..9:who
#10:
20..18:Eliza..21:lot
#11:
22..23:Don't..24:trust..25:em

binary format of data sequence graph

2011-03-07T17:03:00.000-05:00

https://github.com/ArtV/DataSequenceGraph. This is an experimental release whose quality is not assured. Data sequence graph represents a set of data value sequences (IEnumerable<>), called "chunks", as nodes and directed edges in a single graph. More introductory details.

The binary format of a data sequence graph has steadily become more complicated and abbreviated. Some demystifying is warranted. These details are accurate presently but could be partially or even fully obsolete at any date in the near or far future, since the code has no guarantees of format stability or compatibility.

Its overall structure is a chunk's route, containing node/requisite records in visitation order. Nodes that follow one another (in the route and consequently in the format) imply an edge in-between. The first item for each node is an unsigned short whose lower 12 bits are the actual unique node number. Unsigned numbers are almost perfect because valid node indexes are nonnegative. To make room for the official invalid node index, -1, all node numbers are incremented before writing and decremented after reading. 12 bits are a compromise between piddly 8-bit numbers and hulking 16-bit numbers. More on this here.

The leftover 4 bits in the first unsigned short are complete situational directions for interpretation and subsequent writes/reads of the record. These are the "guide bits". All-zero guide bits encode a gate node and therefore moving on to the next node in the route, i.e. the next record in the file.

When the guide bits indicate a value node, there may be other items for the node. If the value node is preexisting in the destination graph, the data value index is skipped. Alternatively, if the value node is new, then the index for its value, another unsigned short, comes next. There are two possibilities: 1) if the destination graph contains a different node that represents the same value, its index is used and the guide bits reflect this; 2) otherwise the value itself is new so it will go in the complementary delimited text file of string-converted values, and the guide bits indicate that the value index applies to the text file instead. A subcategory of #2 is a new node whose value isn't in the destination graph but is already in the text file, so that text file value index is reused for additional new nodes with an identical value.

Lastly comes the vital missing information for the implied edge between the node and its successor: the requisite edge. Given that a requisite edge occurs previously in the same route, this is an 8-bit index into the component edges of the route.

As it turns out, still greater space efficiency is possible for requisite edges in some simpler cases. First, if there isn't an implied edge between successive value nodes in the binary file, which is possible whenever the transmitted route can happily reuse preexisting nodes and edges in the destination graph and therefore omit one or more intermediate edges, then the guide bits can indicate that fact and completely skip the writing/reading of the requisite edge information.

Second, if a requisite edge happens unfortunately to be the one completely unique edge of the route that trumps all other requisites, i.e. the edge from its gate node to the very first value node, then once again the guide bits can convey this without more space.

Third, a more difficult yet straightforward and common pattern for requisite edges is the immediate predecessor edge. New edges tend to be added in clumps, so avoidance of (most) explicit requisite edges for the entire clump can be a significant savings indeed.

In summation, it's clear that the guide bits, the top 4 bits of the first unsigned short of the record, are overloaded with implications. The four node varieties are gate node, existing value node, new value node whose value duplicates an existing value node, new value node whose novel string-formatted value is in the complementary text file. These node types determine what to do with the node's recorded value index or whether to skip it altogether. For each of the three value node record types, either the general case of a stated requisite edge or one of the three special-case strategies for skipping the requisite edge data may be applicable. The total is 3 types of value handling for value nodes multiplied by 4 options for requisite handling plus 1 independent gate node possibility, or 13 numbered "instructions" that fit in 4 bits.

git graphs compared to data sequence graphs

2011-03-04T11:04:00.000-05:00

A git graph has no cycles/circles (a commit cannot be its own grandpa, for instance). A data sequence graph may, but never within a particular data chunk's route. The routes use the cycle like a traffic roundabout, jumping into the circle and then leaving before completing a circuit. This is possible because of the rules that an edge's requisite edge must be met and the edge to be passively followed is the one edge whose requisite occurs first/earliest in the route's history.
Nodes in a git graph aren't centrally numbered like in a data sequence graph. It couldn't use that strategy and still be thoroughly decentralized. Instead, the renowned "content addressable filesystem" of git names the commit with a hash of its information including its immediate ancestors. Git avoids global name collisions, but at the unavoidable expense of the required space for the hash. Although in practice a short prefix of the hash/name suffices to be unambiguous. Since a global or decentralized data sequence graph would probably be useless (but it's an intriguing idea...), the same constraint isn't applicable.
Git also requires immutable commits or nodes. Not in the sense that rewriting or amending is impossible, but that a commit's identifier must also change whenever the commit's information changes (well, if you have the know-how you can bend this a bit, but normally...). The state of the revision-tracked content must reflect its ancestors definitively at the point of any commit. A selected commit represents one and only one content state. Whereas a node in a data sequence graph non-uniquely represents an individual value, and many data chunks/routes could possibly include that value/node. This difference also implies that while the identifier of a git commit would need to be different if it were a destination of greater or fewer edges, the simplistic numeric identifier of a node in a data sequence graph must remain the same no matter how other edges and nodes change.

musings on information in data sequence graph

2011-03-03T17:10:00.000-05:00

Github link: https://github.com/ArtV/DataSequenceGraph. This is an experimental release whose quality is not assured. Data sequence graph represents a set of data value sequences (IEnumerable<>), called "chunks", as nodes and directed edges in a single graph.

The idea behind a data sequence graph is to store information not only about each chunk and its data values but also about the order of the values. By storing information about order in the graph as edges, hopefully chunk routes can share edges as well as value nodes. For when routes share edges and nodes, the shared edges and nodes are an actualized model of mutual information between routes and by extension their represented chunks. If the routes are in two different data sequence graphs whose nodes/edges are subsets, then there's mutual information between the graphs. And if the proper-superset graph is the sender and the proper-subset graph is the receiver, the communication channel for route nodes/edges may exploit the mutual information.

Specifically, given a secondary/destination/subset graph (i.e. MasterNodeList<>), a chunk's route (i.e. DataChunkRoute<>) can figure out just what needs to be added for the chunk route to function normally there [i.e. the "*missingComponents*" method(s)]. For example, when chunk X already has a route in the data sequence graph, and chunk Q is equivalent to chunk X with one additional data value appended, sending a route for Q merely requires the communication of a new gate node, an additional node for the appended value (but only if a node with that value isn't already in the graph!), and a few edges to connect the new nodes to the existing route for chunk X. Using the space-conscious binary+text format, the necessary bytes could possibly be a quarter or less of the bytes for the original entire Q chunk.

Naturally, as easy as it is to think of situations in which a data sequence graph leads to efficiencies, it's easy to think of inefficient situations. Since the purpose of all the many edges is to encode sequencing information, chunks of data values that occur more or less independently and randomly result in edges that are fruitless and wasteful. Placing data with no pattern into a graph yields a graph packed with huge quantities of both nodes and edges. The counterpart to the previous example would be an infinitesimal chance of known route X having a profitably-similar resemblance to new route Q. In the worst case, Q has nothing in common with its predecessor chunks, neither the order of its values nor any of the values themselves, and as a result the overhead of the node/edge definitions makes the route larger in space than the data it represents. This pathological case of a data sequence graph route is a cousin of the familiar pathological cases for data compression algorithms ("compression" enlargement). True randomness won't be denied or masked or neatly summarized; it can only be confronted in its full glorious ugliness.

But it's a broad domain between the perfect and worst sets of data chunks to encode as routes in a data sequence graph. The choice of how to break up the data into chunks is vital. Similarity within a chunk results in excessive numbers of nodes to recapture the identical repetitious values. Similarity between chunks results in effective reuse of each other's nodes and edges. Thus, for data that's in some way periodic or recurring, the period boundaries should be the chunk boundaries. Chunks that are too large or too small will fail to maximize the benefits of sharing redundant pieces of the overall graph.

That's the goal: to encode one route in terms of another route and thereby one chunk in terms of another chunk. By considering a route as an aggregate of data value nodes and the edges as relations, two routes in the graph with significant overlap are a partial "isomorphism" in the mathematically-unrigorous sense of the rambling post that Metamagical Themas inspired me to write: mapping one aggregate (route) onto the other "preserves" certain relations (edges). Furthermore, a data sequence graph whose nodes/edges form a proper superset of a second's is a partial isomorphism. As I outlined in the rambling post's "communication" section, when an isomorphism exists among communicators, the knowledge about correspondence of aggregates/parts/relations reduces uncertainty about related messages and therefore reduces the absolute necessary message length for the recipient to infer the whole. The *missingComponents* methods of the DataChunkRoute<> demonstrate this truism.

introduction to data sequence graph

2011-03-02T15:56:00.000-05:00

In a data sequence graph, a sequence of data values, called a "chunk", corresponds to one of many possible node-to-node routes over the one-way edges.
Most nodes in the graph are "data value" nodes that represent one and only one data value. A value node in the route represents the value in the chunk. Many value nodes may represent the same data value.
A route starts and ends at exactly one route-specific special node called a "gate" node. No route contains more than one gate node, and no route includes gate nodes of other routes. The gate node functions as an entry/exit point into the graph. It doesn't represent a data value.
Other than the gate node, a route never includes a node more than once. Data values that occur more than once in the chunk must be represented by separate value nodes in the route.
Nodes are numbered for unique identification, whether gate or value nodes.
A value node may have any number of connecting graph edges to other nodes. An edge can only be traveled in one direction, although there may be a separate edge between the same two nodes that goes in the reverse direction.
Due to its function, a gate node has exactly one outgoing edge to a node that represents the first data value of the chunk, and exactly one incoming edge from a node that represents the last data value of the chunk.
An edge has exactly one explicit requisite edge. A route must have already traversed the requisite edge in order to traverse the edge. The exception is the one edge from a gate node to a route's first value node.
If a route is being passively followed (i.e. reproducing a chunk from a route) rather than actively created (i.e. creating a route to represent a chunk), the outgoing edge to follow next is the edge whose requisite occurs first/earliest in the route.

Implementation

Platform is .Net 4 in C#. The implementation is generic on the data value type, which must have a correct Equals implementation and otherwise work as a Dictionary key. The most important classes are

the MasterNodeList<>, which holds all graph nodes and in effect the entire graph, is the gateway for most client requests and queries
the DataChunkRouteBlazer<>, which is responsible for translating a chunk into a new route through the graph, has a computeFullRoute() method to ensure a functioning graph route for its chunk from start to finish
the DataChunkRoute<>, which models a chunk route in the graph, has a followToEnd() method to walk the route (starting/finishing at the gate node) to regenerate the chunk

External Formats

The XML format is a relatively straightforward verbose listing of all the nodes and edges in a graph. The binary+text format uses a number of strategies to concisely store a graph as a binary file of nodes/edges and a delimited text file of node data values. Each format relies on two interfaces to translate the node values back and forth to strings: NodeValueExporter<> and NodeValueParser<>. There are implementations for string node values to be treated as-is: ToStringNodeValueExporter<> and StringNodeValueParser. (ToStringNodeValueExporter<> is directly reusable by any type for which ToString() is a working storage translation.) As written, the binary+text format exploits several hard assumptions for efficiency: 1) the number of nodes and therefore their unique serial numbers must be no greater than 4095, 2) the string translation of any node value must be no greater than 120 characters, 3) the string translation of any node value must not contain "|".

Command Line Interface

The included program uses the above code to manipulate data sequence graphs of sentence chunks and individual word values (i.e. each sentence is a route and each word is a value node). It has options for displaying, reading, writing, and comparing data sequence graphs in either format, as well as the option to restrict operation to a passed graph chunk/route.

reactions to unit test failures

2011-02-24T13:27:00.000-05:00

I suspect that deepest dedication to a suite of automated unit tests isn't proven by having a test that touches each public class. Nor by a test for each public method. Nor by a test coverage tool that reports a proportion as close to 100% as is reasonable for that codebase. Nor even by strict test-first methodology.

The deepest dedication is only proven later in the code's life, when changes happen. Code changes of significance should lead to test failure or stark test compilation (or interpreter-parse) failure. That moment is when dedication takes effect. What will be the programmer's reaction? If it's a tiny test that broke, then a rapid and focused adjustment to the test or the code is likely to be the happy outcome. (Sidebar: this argues for not skipping trivial tests that probably are redundant to big tests. A failing trivial test is easier to interpret than a failing big test!) If it's a complex test or a large subset of tests that failed, then the reaction might not be as, er, placid.

The worst reaction is to impulsively eliminate the failing test(s). Better but not by much is to turn the test(s) into comments or otherwise force a skip. A disabled/skipped test is always a temporary measure of compromise to reduce mental clutter during the thick of an intense task. It carries an implicit promise to enable the test at the next possible opportunity. Excessive distracting nagging is awful but permanently removing a safety net clearly falls into the "cure worse than disease" category.
Assuming the motivation for the code change was a real change in requirements rather than code refactoring and improvement, then direct elimination may be correct. Before doing so, remember that unit tests act like executable specifications for that unit, and ask yourself "Does this test correspond to a code specification that still applies to the changed code but in a different form?" When the answer is "yes", the test should be replaced with a corresponding test for the transformed specification. Consider previous tests that caught corner cases and boundary conditions. If an object previously contained a singular member, but due to changes in the problem domain it now contains a collection, then the test for handling a NullObject singular member might correspond to a replacement test for an empty member collection.
On the other hand, whenever the change's purpose is to improve the code while leaving intact all existing functions/interfaces of importance, elimination or fundamental rewrites aren't the right course. The test stays, regardless of its inconvenience in pointing out the shortcomings of the redesign. The right answer may be to rethink part of the code redesign or in a pinch to add on to it in some small way with some unfortunate adapter code until other modules finish migrating. Sometimes a big fat legitimate test failure is the endpoint and "smoking gun" of an evolutionary mistake of the code, and the professional reaction is to disregard personal/emotional attachment by cutting off or reshaping the naive changes. Never forget that to users the code is a semi-mysterious black box that fills specific needs. Sacrificing its essential features (rather than unused feature bloat) is too high a price for code that's more gorgeous to programmers. Granted, skillful negotiators can counter by pledging sophisticated future features that the redesigned code will support, in which case the pledges must turn out to be more than vaporware for the trick to ever work again.
With any luck, the ramifications are not so dire. A confusing unit test failure may not be a subtle lesson for the design; it may be nothing more than a lesson to write more (small) tests and/or test assertions. It seems counterintuitive to throw tests at failing tests, yet it makes a lot of sense given that tests are coded expectations. In effect, confront the failing test by asking "What did I expect?" immediately followed by "Why did I expect that?" Expectations build on simpler expectations. Attack the expectation in top-down step-wise analysis. The expected final outcome was 108, because the expected penultimate outcome was 23, because the expected count was 69, etc. Write tests for those other, lesser expectations. Now the tests narrow down the problem for you at the earliest point of error, as if being an automatic debugger with predefined breakpoints and watch-expressions.
It's a well-known recommendation to write an additional unit test for a bug discovered "in the wild". This test confirms that the bug is fixed and then reconfirms that the bug doesn't resurface, assuming frequent runs of the entire suite. After a few unsuccessful tries at passing this novel test, don't be too rigid in your thought habits to ponder the possibility that the untested test is buggy! In the prior items my encouragement was to not react by blaming the tests, since an unmodified test that passed before a code change and fails afterward logically indicates that what changed, i.e. the code, must be to blame. Philosophically man is the measure of all things and a unit's tests are the measure of the unit. Not so during the introduction of a test. At this special time, the test isn't a fixed ruler for measuring code errors. It's its own work in progress in a co-dependent relationship with the code it measures. Initially the code and the test are at danger of dragging the other down through bugs. A buggy test is a false premise that can lead to a false conclusion of fine code that appears to be buggy or worse buggy code that appears to be fine. Be careful to write tests as minimal, unassuming, and straightforward as is practical. Complex tests that check for complex behavior are acceptable (and hugely important!). Complex tests that are intended to check for simple behavior are less justifiable and trustworthy. Tests are miniature software projects. The more convoluted and intricate and lengthy a test becomes, the greater opportunity for bugs to sneak in and set up shop.

when 255 is too little and 65535 is too much

2011-02-23T11:27:00.002-05:00

Before I begin, yes, I do realize that low-level bit twiddling is often not worth the time any more. When storage and memory is so cheap, and the sheer scale of processing has progressed so far even on a "smart" phone, little micro-optimizations may...just...not....matter. Switching to an alternative high-level algorithm/strategy better suited to the specific scenario is generally a much more promising route. However, network transmission and/or mass data record layout are two prominent exceptions to this rule. The fewer bytes that must be sent between computers on potentially high-latency and congested networks, the better, and the fewer average bytes per record that is multiplied by a potential quantity of thousands up to millions of records, the better. So there still is some life yet for bit twiddling.

One unsigned byte represents nonnegative integers up to 255 (including 0). Two unsigned bytes represent nonnegative integers up to 65535. The storage doubles, but the representational range squares. In basic-arithmetic-speak the range is "256 times its original amount". In science-speak the difference might be called "an approximate difference of 2.40824 on a logarithmic (lg) scale". In marketing-speak the difference might be called "an increase of 25,500%". The upshot is that this exponential jump could very well be overkill in particular cases. For instance, if data values never are less than zero or greater than 4000, 12 bits or one-and-half bytes is sufficient (2^12 -1 = 4095).

Yet reading and writing an arbitrary number of bits like 12 isn't well-supported, for valid performance reasons. Manipulating 12 bits in practice implies manipulating the next-greatest number of bytes, 2 or 16 bits. And that implies 4 wasted bits for "spacing" or "alignment". Why not use these bits? Two of these 12-bit unsigned integers sum to 24 bits or 3 bytes. When each of the two 12-bit number is stored in 16 bits, the fourth overall byte, consisting of the top 4 unused bits from the first number and the top 4 unused bits from the second, are waste. Avoidance of that fourth byte reduces total storage needs by 25%. Whether this substantial relative space savings amounts to a substantial absolute/raw space savings depends on the application and other factors. I'd venture that in the majority of the time it doesn't. But when it does...

Each 12-bit integer is a byte-and-half. For two, just combine their two half-bytes (4 bits) into byte number three (8 bits). The conversion procedure from two nonnegative 12-bit integers, originally stored as two 16-bit integers (i.e. unsigned short) or 4 total bytes originally:

1) Get the first output byte by truncating/coercing/casting the first 16-bit integer. This grabs the 8 low bits of the first number, leaving the remaining 4 high bits to be stored separately.
2) In the same way as #1, get the second output byte by truncating/coercing/casting the second 16-bit integer.
3) Get the remaining 4 high bits of the first integer by right-shifting it by 8 and then coercing to a byte. The result is the top 4 bits of the first integer stored in a byte as-is. All the lower 8 bits of the first integer, that were previously stored in #1 and became the first output byte, are right-shifted out completely into oblivion and zeroes take the place of the shifted bits. This byte is now the "top byte" of the two bytes in the original 16-bit storage of the integer.
4) In the same way as #3, get the remaining 4 high bits of the second integer by right-shifting by 8 and then coercing to a byte.
5) The third and last output byte will be a concatenation of the two integers' top 4 bits, which after #3 and #4 are now themselves stored as bytes in which the top 4 bits are waste. The lower 4 bits of the output byte will come from the #3 byte, in which the correct bits are already at the correct lower position. But the higher 4 bits of the output byte will come from the #4 byte, in which the correct bits are also in the lower position, which is incorrect for those 4 bits. Therefore, left-shift #4 byte by 4 to get the bits in final position.
6) Finally the two parts of the third byte, the #3 byte and #5 byte, are ready, with correct bits in correct positions and zeroes everywhere else. Thus each bit in each byte must "override" the zeroed-out bit in the other. If corresponding bits are both 0, then 0. If corresponding bits are 0 and 1, then 1. Since this is the definition of a bitwise OR (|), perform that operation on the #3 byte and #5 byte to get the third byte.

The conversion from three unsigned bytes back to two 12-bit integers, stored individually in two unsigned 16-bit variables:

1) The lower 8 bits of the first integer is the first input byte. Expand/coerce/cast the first input byte to 16-bit storage.
2) The lower 8 bits of the second integer is the second input byte. Also expand/coerce/cast to 16 bits.
3) The third byte has the top 4 bits of the first integer in lower position, and the top 4 bits of the second integer in higher position. To get the top 4 bits of the first integer as a byte all to itself, just the top 4 bits of the third byte must be replaced with zeroes. For each of those top 4 bits, 0 must result in 0 and 1 must result in 0. For each of the lower bits, the bits that must remain undisturbed, 0 must result in 0 and 1 must result in 1. A bitwise operation that sometimes turns the first bit into 0 and sometimes into 1, depending on the second bit, is AND (&). Any time the second bit of an AND is 0, the result will be 0. Any time the second bit of an AND is 1, the result will equal the first bit. Bitwise AND fits the purpose, provided all the bits in the second byte are set accordingly. This second byte for the AND operation, known as a bitmask because the zeroes in it will "mask" or hide bits while the ones in it will "pass through" bits, must simply have zeroes in the 4 higher bits and ones in the 4 lower bits. Expressed as a byte, the bitmask is "0000 1111" in binary or "0F" in hexadecimal or "15" in decimal. In any case, the result of the AND operation between the third input byte and the bitmask is the upper 4 bits of the original first 12-bit integer.
4) The higher 4 bits of the third input byte are the top 4 bits of the original second 12-bit integer. To get these 4 bits as a byte all to itself, right-shift the third input byte by 4. This pushes out the lower 4 bits of the third input byte, that was stored in #3, and leaves the higher 4 bits (with replacement zeroes in higher position).
5) The #3 byte, in which the higher 4 bits of the original first12-bit integer are in lower position within one byte, equals the original higher byte of the 16-bit storage of the original first 12-bit integer. All that's necessary is to put the #4 byte into correct 16-bit position, which is precisely one byte or 8 bits higher. Expand/coerce/cast the #3 byte into 16-bit storage and then left shift by 8.
6) In the same way as #5, expand/coerce/cast the #4 byte into 16-bit storage and then left shift by 8.
7) #1 and #5 are 16-bit unsigned integers whose bits are all in the right locations for the original first 12-bit integer stored in two bytes. And any bits in #1 and #5 that don't match are zero. This means that a bitwise OR between the two will "override" the placeholder zeroes with the real bits in the other and get the original two bytes of the first 12-bit integer.
8) In the same way as #7, #2 and #6 after bitwise OR will be the original two bytes of the second 12-bit integer.

information hiding applied to generic types

2011-02-17T13:34:00.001-05:00

Over time, I have a growing preference for the term information hiding over encapsulation. Encapsulation communicates the idea of wrapping data in an object or protecting privileged access through managed methods. But design analysis shouldn't stop there. Information and/or knowledge can leak and spread through a design in more subtle ways. For instance, I previously wrote that the HTML page's DOM is an overlooked example of shared state in Javascript. When many code pieces, each meant to serve separate and independent goals, have accidental dependencies outside themselves, both code reuse and intelligibility suffer. A part can no longer be understood without understanding the whole.

The purpose of information hiding, interpreted in a more general sense than encapsulation, is to thwart the natural tendency for code to not only collaborate but conjoin into the dreaded big ball of mud. Information hiding preserves metaphorical distance between expert objects with well-defined responsibilities. Then whenever a question arises, there's one answer and that one answer comes from the expert. The opposite outcome is several answers, found by wandering through the system, and one can never be quite certain that all the right generalists have been duly consulted and cross-checked.

Generic types may be susceptible to violations of information hiding. For in order to concretely consume a class with a generic type parameter, the type for the parameter must be specified (of course). But the conscientious designer should frankly ask whether the type parameter information belongs in the consuming class. By including it, the consuming class must either be coupled to a type-specific instance of the generic class or itself become a generic class that requires and passes on the same type parameter. The first option makes the consuming class less reusable/flexible while the second option leads to greater implementation complexity and further propagation of the type parameter information.

The third option is to hide the generic type information from the consuming class altogether. Some possibilities:

Have the generic class implement a minimal non-generic interface. Then couple the consumer class to this interface. Creating/obtaining new instances of the generic class for the interface would happen through a separate factory/service locator/dependency injector.
If the generic class is part of an inheritance hierarchy, then move the non-generic portions into a superclass. It's permissible for a non-generic superclass to have generic subclasses. Now the consuming class can work with instances of the generic subclasses by typing them at the non-generic superclass level.
Assuming the generic class doesn't have generically typed state, consider making the class non-generic with some methods that are generic only when necessary. In such situations the compiler probably can infer the methods' type parameter based on which types the consuming class passes to the methods, so not even calls to the remaining generic methods need to have the complexity of "looking generic".

As with any usage of generic types, consider the trade-offs carefully. Generic types sacrifice some clarity and usability. In particular, when code isn't getting anything back out of a class, the information of how to fill in the generic type parameter tends to be irrelevant, distracting, and potentially too restrictive. Generic types in one part of a project shouldn't cause the entire project to take on needless generic types everywhere.

quick tip for using gitextensions with git svn

2011-01-13T08:28:00.000-05:00

I imagine some others have already noticed this, but git svn commands can be added as "Scripts" that are available from the history context menu. For instance, for "git svn rebase": Go to Settings > tab Scripts, click Add, enter a Name and be sure to click "Add to revision grid context menu", enter "C:\Program Files\Git\bin\git.exe" for the Command, "svn rebase" for the Arguments, then click Save. Now, with a right-click on the history graph in gitextensions, the new command should show up down at the bottom with the entered Name. Choosing it will bring up the expected output window. Similar steps apply for "git svn dcommit --dry-run" and "git svn dcommit".

It's a small change, yes. And for the full glory of git you still must click on the little terminal icon on the gitextensions toolbar and use the command line. But repetitive tasks should be made as rapid and unobtrusive as possible to conserve the programmer's cognitive load (which is why running one's unit test suite should also be extremely easy and painless). Of course, even the tiniest enhancement to workflow adds up over many times thereafter. Just as a mostly-positive monthly cash flow is the key to long-term financial sustainability, a mostly-frictionless development flow allows programmers to expend their valuable time on stuff that matters, like design.

git is a VCS for the imperfect programmer

2011-01-11T09:07:00.000-05:00

I just recently had a workday in which I realized how appropriate git can be for imperfect programmers. My department had released a project to the users for the first time, so I was working through the inevitable few tickets and/or change requests that ensue when software leaves a controlled development phase and collides with people.

I'd completed several commits on top of the release when I got a call about a bug related to a highly exceptional set of data. "No problem," I thought. "I can work it in with all these other commits and it will go out with them on the next scheduled full redeployment to the website." So I started working, but after designing a fix I discovered that it had so few systemic dependencies that I could easily push it out on its own without disruption, and enable the user who called to continue her work on the problematic data set. In another VCS this might have been unwieldy, but not with git. I stashed my work, created and checked out a branch at the last-released commit, popped the fix off the stash, and committed it.

Unfortunately it was only after that point that I noticed a possible weakness of the fix (and also the original code). Once again, with git this wasn't cause for alarm. I corrected the fix by amending the commit I'd just made. Finally, I had a file whose only difference from the last release was the fix. I deployed the file.

I should mention that an older edition of Subversion is the official VCS of my team, and we simply don't use branches or tags (it's not my decision). As a small team in a small organization with lots of informal communication, it's mostly sufficient for our needs, although I imagine that we'll need to become more sophisticated in the future. Thus, my fresh branch for the isolated bug-fix had to be a local git branch only. In order to ensure that the next deployment included that commit, I had to incorporate it into the Subversion trunk. With git, a rebase of my branch onto the HEAD of master was easy, and of course the actual merge of it into master was then a fast-forward. After a quick delete of the merged local branch and "git svn dcommit", everything matched up again.

A number of things about the procedure were imperfect. I'm an imperfect individual who made missteps. And I work in an imperfect setup with a decidedly imperfect team VCS. But it turns out that git fits these conditions just fine.

hidden conceptual connectors

2010-12-31T19:28:00.000-05:00

The least controversial statement possible about the brain's functioning is that it's enigmatic on large scales. Despite knowledge of input and output in any given situation, the complex path in-between can't be interpolated. It's a fuzzy tangle of inherent unpredictability as it continually adapts with each new environmental shift in the flood of external information. Small wonder that some scientists have insisted instead on restricting analysis to the most predictable of tangible, measurable behaviors.

But a statement that contradicts this one is also uncontroversial: from a first-person perspective, the brain's work isn't quite as enigmatic. It yields ephemeral subjective/mental/semantic "sensations" under the catch-all name, the conceptual stream of consciousness. Examination of the stream leads to psychological speculation about its supposed causes, e.g. the dream proves that you have a death-wish. Extraction and conversion of the stream to the form of an ordered chain of logical reasoning is a vital step to publishing polished arguments for others to appreciate and check.

It seems to me that a persistent frailty of these post-hoc explanations of the first-person experience is omission of essential steps. Sometimes it's crystal-clear that something must be missing from the person's fragmentary or abbreviated account. Sometimes the person may not have realized that they skipped a step in the telling; when questioned they say it was too "obvious" to be noticed. Sometimes one or more steps are purposely excised to avert dull boredom of the audience. Hence, although the underlying physical motions in the brain that produced the output are undeniably inscrutable, the perceived conceptual motions in the first-person consciousness or "mind" are also filled with gaps! Pure reason, the foremost ally of philosophers, is reliant on hidden connectors between the manifest thoughts.

Inventive parts of my brain must be whirring away without my express order or consent. How else can I explain the noteworthy clue that my consciousness can play its own soundtrack in the absence of corresponding sound waves? The first guess is that the music filling the interstices of my attention is randomized firings, but the selection is too often apropos to be due to coincidence. The meaning of the lyrics aligns with my focus. More than a few times, I had an anticipatory song. I read toward the top of the page, heard a song in the "mind's ears", and further down I read a textual header whose content matched up more or less directly to the already-in-progress song (most obviously when the header's a pun on well-known song lyrics). As my eyes searched the page as I read, my brain subliminally picked up the text of the below header in passing and that snippet then seeded a highly meaningful background tune. I mention the "smart jukebox in the head" as a prosaic illustration of the currents which are active yet not consciously initiated.
The next indication is visual. Abstract topics under extended consideration by me may become entangled in a particular unrelated location from my memories. When I return to the topic, a view (always from the same approximate "orientation") of the associated location "shows up". But unlike the inner jukebox music, the choice of stage has no connection to the topic; it's "just there" as the main sequence of my thought is engrossed in the topic. My theory is that when I'm deeply sunk ("transported") into the topic, my usual contextual awareness of my actual location loses force and allows an arbitrary remembered location to emerge. As I ponder the topic, it intertwines with the secondary setting, and from then on the two are joined. Like the apt background music, this occurrence isn't perceivably helpful to the main process, but it suggests that more is happening than the stream of consciousness (would we call the rest of the activity "brooks"?).
By contrast, heuristics can be wonderful aids. A few good mental shortcuts reduce the time and effort to obtain an answer, as long as the error rate is tolerable. What's striking is the frequency at which heuristics develop without deliberation. When a person performs repetitious tasks, generalizations crystallize. If a computer program always asks for confirmation and "Yes" is always clicked in response, there's no need to read the words every time thereafter. Sooner or later users may find it so routine and mindless that they barely can recall the clicking, either. Long exposure and training build an intricate web of heuristic reactions. People who perform identical tasks at their jobs for years appear "dead" to onlookers but they're really more like robots. They do A for case X, B for case Y, C for case Z. Eventually, solely the most exceptional cases manage to evade the heuristics. "In all my years working here, I've never seen..." The ability to leap instantly to the correct course of action isn't as remarkable when the solver is hastily reapplying a hidden, practiced heuristic.
While sneaky street-smart heuristics receive less credit than forthright book-smart education, the latter depends on unrevealed links too. Verbal learning happens as teachers lay new definitions upon previous definitions. As they become better acquainted with the new ideas, the learners likely won't need to refer to the various mediators that first facilitated their initial understanding. After the information is embedded in the learners, the comparative "glue" employed by the teachers during the lessons is unnecessary. Still, the educated person must acknowledge that these clumsy intermediaries were formerly invaluable for achieving minimal comprehension. The teacher's formula "___ is like a ___" was a bridge that could've been forgotten afterward. Thinking of atoms or other particles as microscopic balls or points is deficient to the reality in several respects, but the image is close enough for an introduction.
A skeptic could cast doubt on the necessity of the stepping stones laid during learning. Couldn't the instructor plainly state the base facts rather than fumbling with imperfect comparisons? Perhaps, but ultimately it's unavoidable because all communication is the learning process in miniature. Each word shapes the interpretation of its successors. The importance of context shouldn't be underestimated. When someone proclaims a love for "bouncing", the information intended by the proclamation is unlearnt until the arrival of more details. If according to grammar the object of the bouncing is "pogo stick", then the listener's uncertainty decreases, provided he or she knows about pogo sticks. Similar illumination would happen were the object of the bouncing "rubber ball", "check", "unwelcome guests". The point is that the word "bounce" is isomorphic to multiple stuff, a word like "check" that constrains "bounce" could also be isomorphic to multiple stuff, and the final overall sentence isomorphism is an intersection or conjunction of the possible/candidate isomorphisms for the individual words. It's a fundamentally parallel process that harnesses the ambiguity of isomorphisms to permit efficient and creative messages; word choices need not be exact, and continual recombinations use limited word sets to cover an astronomical range of thoughts real and unreal.
The sheer power of language for logical reasoning, fused to the uncanny human capacity and motivation for inventing explanations, can give the misleading impression that the form of nonverbal reasoning is no more than unverbalized logic. However, countless counterexamples are everywhere. A familiar category is visual estimation and evaluation such as whether a floor lamp can fit inside a box. Skill at such questions is independent of skill at logical/mathematical questions. Moreover, translating a question into a different format, viz. visualizations, can greatly affect its perceived difficulty. Solutions can arise through subtle analogies. Many anecdotes show a startling similarity in the structure of major breakthroughs: the discoverers were confounded until they adjusted their underlying approach or otherwise became inspired by knowledge from seemingly unrelated domains. In the more mysterious accounts, they arrived at the answer partly by interpreting material from dreams or dream-like meditative states in which frothy free association ended up mentally attaching and attacking the problem in the forefront. Metaphors may be simultaneously ridiculous and indispensable. Asking "what if?" might lead to a fruitless destination. On the other hand, it might lead to the key insight. A nonsensical whole can act as a temporary frame between important contradictions before the thinker determines the plausible interconnection.
At another level of remove, seasoned experts of a specific mental domain describe ideas about the ideas and assumptions about the assumptions. This "feel" is largely what qualifies the expert as an expert. A novice would pursue tempting and irrelevant side details or surface illusions where an expert would employ the well-honed knack for striking at the most beneficial edge. They speak of a "nose" for the right path or a "sense" for the likeliest "shape" of the right conclusion. For them the domain approximates a familiar environment in which even abstract concepts are almost like manipulable objects. Upon prompting they try to describe the domain's "aesthetics", which probably involve symmetry and complementarity. Their patterns of analysis enable them to quickly decompose a challenge and make it appear easy by a lack of wrong turns. Once again the end result doesn't fully expose the hidden concepts that somehow participated in its birth.
Lastly, and most mystifying of all, are the opaque reports. "I don't know." No matter how they are pestered, they claim to be oblivious to the origin of their highly-confident outcome. In short they know but not how. Naturally, they could actually be wrong, but for a time at least they express certainty. "It just is." No work is remembered. No argument left memorable traces. It's as if verification is possible but not reverse-engineering. There's no clearer proof for the uninformed nature of our reckoning of our own reckonings.

bow to the gitextensions cow

2010-12-18T11:19:00.000-05:00

Recently I tried out gitextensions. A rhapsodic blog post seems to be in order.

There's an installer that includes msysgit and kdiff3. This means I haven't needed to download anything else to get started. The installer asked, up-front, how to handle the *nix/Windows line-ending issue and what to use for my name and email address. The GUI contains an easy way to edit .gitignore entries and it comes with default entries that are relevant to almost all Visual Studio development. It suggests and directly supports integration with the PuTTY tools for SSH authentication. This means I haven't needed to find and edit configuration files or go online to research recommended entries. As someone who considers himself to be at least minimally competent, I'm not phobic of manual configuration or command line usage, but why shouldn't the easy and predictable modifications be even easier?

My intense appreciation continued as I started using it. All the typical functions and their typical options are available. (Long-time git users doubtless prefer to perform the same tasks by rapid rote typing; there's an icon to pop open a "git bash" at any time, which is good to keep in mind.) Creating a branch is just a matter of entering a name when prompted, with a checkbox if you want to also immediately check it out.

The view includes the annotated history graph, the current working directory, and the current branch. Clicking on the branch name brings up a drop-down list of other branches. Choose one, and you check it out. Clicking on a commit in the graph brings up information about it in the bottom part of the screen, such as full commit details and the diff and the file hierarchy (each directory expandable and each file right-button-clickable for file-level commands like individual history). Clicking one commit then CTRL-clicking a second brings up the diff below.

Remember how git newbs tend to have trouble navigating the movements of files between the index and the working directory, especially before git became more friendly and talky? In gitextensions, the commit window simply has separate panes with buttons to move added/modified/deleted files in-between. There's also a button for amending. After the commit, or any other moderately-complicated operations, the git output pops up in a window for review.

Of course, pull, push, merge, rebase, cherry-pick, branch deletion are present, too. All are fairly straightforward assuming the user can follow the on-screen instructions and isn't completely ignorant about git. gitextensions has a manual that contains abundant screen captures, yet I imagine it's more useful as a reference for figuring out where/how in the GUI to accomplish a specific task than as a tutorial. I was pleasantly surprised by the smoothness of my first series of gitextensions conflict resolutions. kdiff3 came up, I chose the chunks and saved, then I clicked a continue button. Despite my later realization that I could've accomplished my goal through a more streamlined procedure, the end result was nevertheless perfect in the sense that I didn't need to apply a "fix-it" commit afterward (the credit likely should be split among git and kdiff3 and gitextensions).

My praise keeps going. gitextensions offers fine interfaces for "gc" and "recover lost objects", although thus far I haven't strictly needed either in my short usage span. It adds right-click items to the Windows file explorer. It adds both a toolbar and a menu to Visual Studio. If it isn't obvious, my personal preference is to keep the gitextensions GUI open all the time, supplemented by git-bash. On occasion, when I'm otherwise manipulating a file in explorer, I might invoke file operations right from there.

The remaining question is: are gitextension upgrades frictionless? Sooner or later the cow will tire of wearing that Santa hat...

Postlude: Farewell, Mercurial

Uh, this is uncomfortable. I'm sure you've heard this before, but it's not you, it's me. The cause definitely isn't something awful you did. You're still a great VCS that could make other developers very, very happy. I'm just looking for something else. My horizons have broadened a bit since we first met, and we don't think as alike as we did then. There are other options and considerations that I need to take into account. If I stuck with you forever, I worry that I'd become regretful or resentful. Some day, as we both change over time, I may come back to visit. Until then, I genuinely wish you well.

impossible flawless imitation

2010-12-05T19:03:00.000-05:00

The inherent limitations to the analysis of a Turing Machine (TM) include a consequence that deserves more attention. A TM cannot be imitated perfectly through a generalized algorithm. Stated differently, a TM's full "description" cannot be inferred from an arbitrary amount of its output ("tape"). No empirical method can be all-sufficient for determining the exact source TM. You can't deduce a TM by its "footprints".

The reasoning is similar to other TM arguments, e.g. the inability to compute when/if a TM will halt. Suppose we know that some TM "S" produces a specific sequence of finite bits, and thereby we contrive our own TM "I" that produces the same sequence. Well, what about the next bit after that sequence (or the next erasure, etc.)? "I", according to its present configuration, produces either a 1 or 0. But given that the output is only assured of matching up to the last bit, there's no theoretical reason why "S" couldn't be identical to "I" in every way except for the configuration for the next bit, in which case our otherwise flawless imitation is ruined. For example, even if "S" has steadily produced alternating bits, it still might produce "00" at position 30, or 35, or 1,462.

Moreover, the situation could be quite possibly worse in a multitude of creative ways. Perhaps "S" is really a complex specimen of Universal Turing Machine with two "source" TM that are much different (analogous to a "piece-wise" mathematical function). "S" executes source TM "A" and then increments a count. When the count exceeds a predetermined value, "S" branches to a configuration that executes the other source TM "B". One may elaborate on "S" with further counts, branching, and source TMs. The point is to reiterate that although we can invent an "I" to imitate "S", we can never conclude that "I" and "S" are exact matches, and in fact we can't figure the ultimate similarity at all as execution stretches to infinity. Failure of "I" is due to missing information about the "S" TM, but worse is the more subtle problem: there's no way to know how much information is missing at any time!

So...

Generally speaking, clean-room or black-box re-implementations of software can't be guaranteed to succeed in every detail. For instance, exceptional conditions or highly specific combinations of input could trigger diverging outcomes. The imitation of the previous software's bugs can be particularly troublesome.
Whenever we compute/emulate anything that requires TM-level processing, we can't be absolutely sure whether we ever achieve total fidelity. This doesn't imply that a conceptual TM is the most convenient theoretical model of a phenomenon (it usually isn't!), but merely that the chosen model can somehow run on a "computer". Or in more formal terms, the model outlines an explicit procedure that yields predictions of one or more modeled quantities. In some open-and-shut cases, the model's error is persistently trivial under all experiments and we therefore have no justification for doubting its effectiveness. Yet history is filled with times in which a serviceable first model (circular planetary orbits with epicycles) is later replaced by one with still greater nuance and realism (elliptical).
Third-person human brains (and behavior) undoubtedly fall under these conclusions. That is, any verbalized or intuitive model of a specific brain cannot be completely understood. Sympathy has its limits. While careful observation combined with genuine insight yields an impressive array of knowledge about "what makes his or her brain tick", especially among species evolved to be highly social, the application of the knowledge often proves how superficial it truly is. Biographers can detect what they call formative experiences, but tasking the biographer to reliably predict what a living subject shall do next will illustrate that the analysis works best retroactively. Of course, if someone theorizes that the human brain performs computations beyond the power of a TM, then the argument is correspondingly strengthened. Two such brains surely cannot be synchronized when two lowly TMs cannot. The proposed mechanism of brain-based super-computation, e.g. quantum effects of exotic pieces/states of matter, would increase not decrease the difficulty.
More surprisingly, first-person human brains (and behavior) are also prone. There's no "imitation" of oneself, but there's a self-concept. The very process of answering the self-evident question, "Why did I think or do that?", necessitates the mental model of self. The brain attempts to develop a "TM" whose workings imitate the actual and mysterious workings of itself. Unfortunately, it's sabotaged by bias toward a pleasing answer! Transparency and self-knowledge are notoriously plagued by mistakes. Countless fictional works have relied on the humor if not the tragedy of self-delusion.

escaping infinity

2010-12-02T13:02:00.000-05:00

I've previously mentioned a chain of reasoning similar to this: 1) everything that is real is material, i.e. subject to the usual scientific "laws"; 2) whenever a substance "means" another substance, that relationship is an isomorphism recognized/superimposed by a material brain; 3) there is no ultimate theoretical barrier to the possibility of inventing a sufficiently advanced computer/program that could process "meaningful content" (i.e. isomorphisms) as well as a brain.

Many varying objections apply. One category pertains to an insinuated reduction and comparison of exceedingly-complicated consciousness to the current generation of inflexible, literal-minded computers and programs. And although it's likelier than not that the computers and programs that would successfully imitate the brain would be laid-out much differently than ours, these objections still make some valid points: a computer with more or faster components is "merely" another computer and a program with rather chaotic and evolutionary qualities, broken into segments running in parallel, is "merely" another program.

For instance, assume (if you're an objector, purely for argument's sake) that in principle a brain could be adequately simulated by the appropriate computer and program named Q. Brains, at least after suitable training, can simulate Turing Machines, so Q can also. We know that a sufficiently clever/defective program can end up running for infinite time (nonterminating loops) regardless of how "smart" its host is. Despite its sophistication, Q remains susceptible to this threat. But if Q is susceptible, then by assumption the brain that it adequately simulates is susceptible. How absurd! Everyone knows that people don't suffer from infinite thought repetitions. Thus, the original assumption has led to a false conclusion and there must not be a Q.

My list of responses fit a standard akin to natural selection. The brain's, and by extension Q's, safeguards against infinity aren't perfect (that's impossible), but are good enough.

Distraction. Brains are exquisitely tuned to react to change. Whether an alarming signal arrives through the senses or a long-gestating answer to a dilemma bursts into flower, the typical course of thought is continuous transformation in unanticipated directions. In the face of the avalanche, perhaps the better question is how there can ever be a mental impression of a unitary and uninterrupted flow of logic. Healthy and normal brains display a moderate weakness, if not desire, for distraction. Meditation is difficult. For some, stopping their feet from tapping and their knees from bouncing is difficult!
Memoization. The importance of context shouldn't be underestimated. It's fueled by short-term memory, the "scratchpad", which contains a temporary record of recent brain work. Moreover, since retrieval strengthens a memory, any cycle in the brain work will tend to self-promote. Hence the brain is primed to store previous trips through the loop. The other ingredient is pattern-matching. Each trip, something in the end leads back directly to the start. It's not much of a leap for the brain to construct an isomorphism among these remembered trips, thereby detecting the similarity and extracting the common pieces. Finally, these pieces allow for a short-circuit or "unrolling" of the loop because the brain now knows that the start always leads eventually to the start once more. There's no more need to carry out the (infinite) loop; staying at the start has the same ultimate effect. The execution of the loop has been optimized out of existence or "memoized". Clearly, memoization works best for short or easily-generalized loops. Lengthy or subtle loops could evade notice perhaps indefinitely. Consider cases of asymptotic progress, in which someone is fooled into the belief that the destination is reachable because it grows closer.
Simulation. The power of simulation or imagination allows a brain to contend instead with a comparatively toothless and "imaginary" version of the infinite loop. At all times, the brain can maintain its metaphorical distance, its suspension of disbelief. Through simulation, the loop is closer to an object under manipulation than a dictator reciting irresistible commands. The loop can be stopped, resumed, replayed backward, etc. The brain halts momentarily after every operation to review and reconsider the result, so the danger of runaway computation is nonexistent. If the whole enterprise is carried out with tools such as writing implements or another suitable device (e.g. smart phone?), then the halt is accomplished by nothing more intricate than resting hands. In short, the vital difference in simulation is that the brain may treat the loop "code" as data. Strictly speaking, it never really switches between modes of perception and simulation. It perceives and simulates and perceives and simulates. Dynamic feedback is built-in.
Ruination. Undeniably, real instances of infinite loops don't run forever on devices. Assuming the loop accomplishes something cumulative, its effects are likely to start causing noticeable trouble, even to the point of a forced halt when memory or disk is full. Alternatively, the physical parts of the system will quit functioning eventually since that is what continuous wear and tear does. Earthy considerations also affect brains. Moreover, brains are notoriously prone to failure. Fatigue, hunger, boredom, illness, etc. will cause a loop-stuck brain to produce mistakes in processing the loop, and the mistakes could incorrectly terminate it (e.g. your ComSci classmates who played Halo all night fail to find the right outcome during a "paper trace" question on the final exam). Once again, the greater the complexity of the loop, the more this factor kicks in. As a system shaped by evolutionary forces, a brain is better at survival than flawless mental calculation. Robust error recovery and correction is a more realistic strategy.

It may appear silly to ponder why biological beings don't end up in infinite algorithmic loops. However, infinite behavioral loops surely aren't nearly as ridiculous. We usually call them habits.

shared state is not hard to find in Javascript

2010-11-07T18:42:00.000-05:00

...and it's called "DOM" (the webpage Document Object Model). State is shared between pieces of code whenever all the pieces may read and write to it, and that applies to the DOM. No declarations are necessary; neither reads nor writes require express permission or coordination. In this freewheeling land known as the "document", anything is up for grabs.

Yet why does it matter, given that dynamic changes to the DOM are a huge part of Javascript's appeal? It matters because any shared state, by its nature, can lead to problems for the unwary, especially as size and complexity grow.

If various Javascript functions touch the DOM, then those functions may cease working properly whenever the page changes structure. One change, no matter how trivial, has the potential to mess up code in several places at once. For instance, say that two tabs switch order...
Similarly, whenever there's an algorithmic adjustment, all the functions that affect relevant individual parts of the DOM must be spotted and redone. Rounding and displaying four decimal places, but only in inputs for numerical data of a particular category, means code changes for everywhere that those input values are set or read.
My impression from some blogs is that people are feeling skeptical about the actual prospect of code reuse in many circumstances, but it's still a worthy ideal. As always shared state is a hindrance to reusability simply because it isn't parameterized. A function that includes an instruction to remove a style class from the element of id "last_name" is pretty difficult to reuse elsewhere.
On the other hand, shared state opens up the possibility of no-hassle collaboration. Function A (assuming a better name!) can take on the responsibility of setting up the shared state in some way. Then function B can do some other task to the shared state, any time after A has recently run. But function C can run directly after either A or B. So function A must leave the shared state in acceptable configurations for either B or C, and B must also account for C. Of course, if there's a new special data value in the shared state that affects what C must do, one must be careful to modify both A and B to set it accordingly. Hence, although shared state makes it highly convenient to intertwine the operation of many pieces of code, the intertwining also greatly reduces readability! It's much easier to analyze separate pieces of code with well-defined connection points.
Furthermore, implicit shared-state dependencies don't combine well with asynchronous execution. Javascript doesn't have concurrent execution (i.e. multithreading), but it's certainly possible for separate pieces of code, such as callbacks for clicks and timers and network requests, to execute in an unpredictable order. So while there won't be deadlocks in the traditional sense, unwitting callbacks that fire in an unintended sequence could leave the DOM in a useless or false form after the dust clears.

Many techniques could apply to mitigation of the shared state known as DOM. For information storage, rely on variables residing in purposeful scopes rather than DOM elements and/or attributes. Treat the DOM as an end, instead of intermediate, data format. Isolate the code that handles the DOM from the code that processes data. DOM modifications that always happen together should be collected into a single function with an appropriate abstract/semantic name.

Not all of the shared state in an application consists of variables. Whether log file, database, or DOM, access logic should be carefully considered to avoid maintenance headaches.

a bad statistics joke

2010-11-01T19:29:00.000-04:00

Since I recently posted something with a statistical lean...

I recently encountered the statement: "Why be normal? No one else is."

And all I could think in response was: "Why be within one standard deviation of the mean? Only 68% of the population is, assuming it to be normal."

statistical inference and racism

2010-10-30T17:38:00.000-04:00

Up-front disclaimer: None of the following should be construed as supporting racism. If anything, my point is that racism is as shaky intellectually as it is morally. It's simply wrong to unfairly treat an individual based on indirect inferences from their similarities to a particular race (or ethnicity, gender, religion, etc.). I also apologize up-front for offending statisticians.

It occurred to me, as it doubtless has to others, that racism has some connections to faulty statistical inferences. Of course, I don't mean that the typical racist is computing actual statistical regressions, but that part of their error consists of an "intuitive" misunderstanding that would appear unsound if it was quantified and written out. And naturally, the inability to estimate correct conclusions about probability and statistics is present in any untrained person, not just the racist one.

the instrumental variable

I've heard people aver that, since racial discrimination has continued to decrease, race should no longer be considered a determining factor in people's lives. While this is a happy thought, it naively misses a crucial aspect: (direct) discrimination is not the only way that racism acts.

Assume a statistical model or equation in which race and discrimination aren't present. It could be for any personal outcome like educational attainment, as long as race isn't included in the model's variables. Essentially, the mathematical equation expresses a complete lack of obvious relevance and causation between race and the outcome. Discrimination doesn't even enter the equation, as we posited.

Now run a thought experiment on our model. Suppose we apply the model to randomly selected people, i.e. we grab the info for each person and then calculate (not measure) the outcome with the model. Further suppose that we also record each person's race. Then we find, as in reality, that when we partition the calculated outcomes by race, the distribution is drastically and consistently different. How can this be? Race wasn't one of our variables, because without outright discrimination it had no mechanism of causation.

The answer is that, in this hypothetical model and thought experiment, race could still act as an instrumental variable. Roughly speaking, race is correlated with the outcome through the model variables. For instance, in a model of personal income, it's highly reasonable to have the parents' income as a variable. Although the model may leave out "race", race can nevertheless affect personal income (the outcome) as long as it affects the parents' income (the modeled variables). Thus race can be a cause of an outcome despite not being a "cause".

Moreover, race is likely to be an instrumental variable for an array of models, and all the models are likely to be related too. Bad (or missing) education tends to lead to low-paying employment or unemployment, which tends to lead to poverty, which tends to lead to crime, which tends to lead to low property values and taxes, which tends to lead to low school funding, which tends to lead to bad education. Regardless of whether there's a blatant racial barrier presently at work, the long-lasting "fingerprints" of racial differences may be active (e.g., systematic segregation sometime in the past that restricted access to the same set of opportunities).

the overeager Bayesian

Stereotypes are simplifications by nature. Sometimes a stereotype can be a helpful mental shortcut, but often it doesn't reflect the most accurate concept of a "typical" sample from a population. This is notoriously true of racial stereotypes that highlight the worst representatives.

For compared to a proper Bayesian analysis, "reasoning" by a stereotype is overeager to apply shaky conditional probabilities. Consider a very hypothetical population with only two races that are each 50% of the population. 10% of the entire population has an undesirable characteristic (left unnamed). Of the people in the entire population who have the undesirable characteristic, 75% are of race A. Whenever someone in this population is of race A, what is the probability of having the undesirable characteristic?

When it comes to a stereotype for the undesirable characteristic, chances are the stereotype is of race A due to prevalence within that subgroup. So the stereotype would tend to override the right Bayesian answer, 7.5%, with a much greater probability that overlooks the facts that 1) only 10% of the entire population has the undesirable characteristic at all and 2) therefore 85% of race A does not have it.

peeve no. 261 is Obi and programming language mentality

2010-10-29T10:33:00.000-04:00

The Force can have a strong influence on the weak-minded. (--Obi-Wan)

Similarly, a programming language can have a strong influence on weak-minded programmers. If your programming language "changed your life", "made you a better coder", "expanded your mind", etc., then I suggest that you may be going about this programming thing the wrong way. If a lack of language support for OO results in you producing a tangled web of dependencies among procedural functions, or easy access to global/file scope results in you spraying program state everywhere, then the programming language is your crutch. It's the guide-rail for your brain.

I'm glad that you found a programming language that you enjoy, but consider me unimpressed by rather impractical claims about the ways that it alters your consciousness.

neuronal pragmatists

2010-09-29T11:54:00.000-04:00

Two ways in which neurons act like fans of pragmatist philosophy:

In the pragmatist viewpoint (interpreted by me), the significance of knowledge is closely tied to its value in practice, assuming highly elastic definitions of "value" and "practice". Similarly, the strength and variety of connections between neurons are closely tied to the impact of neurotransmitters. And the release of neurotransmitters is clearly correlated with emotions, broadly categorized as "pleasure" or "pain", and emotions in turn are a large part of subjective value. Research has also shown that once this link is forged between knowledge and emotions, mere anticipation and contemplation can trigger the experience, albeit at a less vivid level. Hence, anatomy and physiology support the pragmatic notion that creatures naturally evaluate knowledge based on its effects. There's a feedback loop between knowledge and its short-term and long-term results. Of course, the mechanism can operate independently of high-level abstract insights about danger or desirability, like in cases of phobia and mental trauma. Or when someone has difficulty memorizing knowledge that's important yet provokes no reaction in the person besides indifference.
More mysteriously, the judgment of knowledge commonly has a nebulous and inarticulate component of "reasonableness" that's integral to pragmatic truth but neglected by other philosophies. When this component is verbalized, people describe it as intuition, the "ring of truth", "feeling right", "clicking into place", "harmony", "seeing the whole picture", "jumping levels of meaning", maybe even related to the "oceanic feeling". (Software developers probably refer to it when they babble on and on about "beauty" and "elegance" in their computer code. Mathematicians say that they felt "guided" and "sensed the right track" to a solution.) My speculation is that this brain sensation corresponds to instances of mental economy. In other words, one "item" of knowledge connects up to another item of knowledge to form some kind of unity, even if it takes the form of opposites on a continuum. Data or concepts were in chaos, but now are in order. Exceptions turn into new, subtler rules. Furthermore, I speculate that this mental economy corresponds in turn to synchronization of the firing neurons that store those items. It's well-known nowadays that the brain is filled with activity in most of its states, and that the unconscious/subconscious/non-conscious takes up a rather startling percentage of that activity. Thus it's noteworthy whenever there's an unusual amount of synchronization of neurons in consciousness (does this help explain the human enjoyment of music?). Any knowledge that acts as a bridge or otherwise "performs double duty" causes a synchronization and a new network-of-networks of the participating neurons. So when we speak of resonant ideas, the description approaches literal truth, for the ideas' neuron firings resonate. Perhaps this is a feasible physical hypothesis for what happens when pragmatists claim that people value an idea based in part on its "reasonableness".

working truth

2010-08-07T10:45:00.000-04:00

Whether or not my overall standpoint qualifies as bona fide pragmatic philosophy, I appreciate and appropriate some of its postulates . Unfortunately, one in particular has attracted both informed and ignorant criticism. Expressed in its shortest and most recognizable form: "the truth is what works." If this doesn't mean "anything that can make me rich must be right", as some allege, then what does it mean?

As I understand it, the intended meaning of "the truth is what works" is simply that the bestowal of the label "true" is the product of physical or mental action. True is reached, earned, assigned, verified, measured. People set criteria, and then they employ those criteria to divide true from false. True (or false) is a conclusion.

And the vagueness of this definition of truth is intentional, in order to cover an unbounded expanse. The operation of determining when something "works" is tightly coupled to the object and the goal, each of which could vary. What also varies is the threshold for delineating "does work" and "doesn't work" in the operation's results.

Clearly, some cases are self-evident. For the proposition "The battery is missing", the object is the battery, the goal is to confirm that it's missing (but that goal is probably a sub-goal to something larger like "force my camera to start responding again when I press the power button"), and the operation is removing the cover to the battery chamber to look inside. The threshold of the operation is seeing the uncovered battery chamber with or without batteries; "The battery is missing" works and is therefore true on the condition that one observes a chamber without batteries.

But other cases are more difficult, like the proposition "Person X is trustworthy." Eyeballs aren't sufficient to detect trustworthiness, the object. Which operation and how stringent its threshold depend greatly on the importance of the goal. Is Person X a candidate to house-sit for me, or just someone whom I'm asking for directions to the nearest location of engine fuel? Someone will likely use not only their senses but their reason, creativity, emotion, intuition, and so on, in conscious and subconscious double-checks upon double-checks.

No matter what they tell you, people frequently "feel" the truth, although it may act as a starting point rather than the entirety of the procedure. Such factors illuminate the muddiness of separating objective and subjective truth in many routine instances. Thus all the ways in which a truth "works" form a unity, fused together by the person, within the context of the person's other truths, whether those be sensations or reasonings or any variety of "mind-stuff".

However, these personalized unities-in-contexts don't imply solipsism or relativism, at least not in crude terms, because of two manifest and natural meta-truths that are normally left unstated. 1) It works to assume that some contents of experience have independent (albeit malleable) existence apart from one's thoughts. When someone writes a reminder, he or she is explicitly relying on the reminder to not be dependent on, or as flighty as, human memory. 2) It works to assume that, based on communications and empathy, other people interact with the same thought-independent contents of experience. To take an early example, a group hunting a mammoth could hardly coordinate the attack without this working assumption.

Put together, what one could call "the two meta-truths of objectivity" outline a common area of endeavor for people, a ground for discussion. And the recognition of its predominance characterizes a mature perspective. A person's uniqueness lies in the combination of "ingredients" that are individually unremarkable and pervasive. Other people have lived on the same planet and nation and culture and city, have experienced similar personalities and feelings and upbringing and education and traumas, and according to an evolutionary/genetic analysis are not much different (though the differences can have vital implications in society and health). Hence, the humble person, aware of his of her startling non-uniqueness, will exploit all the many applicable truths found and transmitted by other people, after winnowing out irrelevancies and mistakes. Of course, the secondhand truths are subject to the rule: do these reports work, evaluated by whichever mental/physical operations? If direct verification isn't possible, as it often isn't, then the operation of trust is unavoidable.

Trust is doubly (or triply) necessary for truths that are universal. By definition, a universal truth is intended to hold true not solely for example Q or group X but for all actual and hypothetical N that meet the truth's conditions. Universal truths are special in that no (finite) number of confirmations is enough. To invoke a universal truth in the execution of an activity is to engage in a pre-sumptive, pre-emptive "confirmation" of it. For universal truths aren't exceptions: truthfulness isn't inherent but confirmed time after time. In fact, each one owes its very existence to how well it works in acting as a tool or shortcut in summarizing, predicting, etc. in the midst of the unities-within-contexts previously explained.

Admittedly, the confirmation can be a perfectly unsurprising formality, especially for inarguable formulas or other patterns undergirded by rigorous "airtight" experiment/proof. The proof's effect is to demonstrate impersonally that the end is substitutable for the start, without doubts or gaps between. Afterward, nobody needs to rework or review the "guts" of the proof to use it, as long as a person trusts the correctness of the proof like any other secondhand truth. (To be sure, people could and do rework or review proofs for a wide range of valid reasons.)

On the other hand, there is a category of universal truths whose correctness isn't quite provable but elicits strong commitment regardless: ideals. An ideal is zealous desire embodied by a universal idea. It "ought to be". In some sense, it's a fiction made real via its connection to a person's irresistible sentiments. It could be elaborate or fragmentary. It could be solitary or part of a complex. It could be structured or incoherent. In any case, the ideal's impact on behavior is precisely how it works; that's its level of truth. People who pursue an ideal are witnesses to it.

Therefore people who ignore an ideal thereby mark its truth as flawed or incomplete. But they could nevertheless use the ideal by feigning their devotion. Even ideals claimed by no one to be "true" might still work as interesting fodder for conversation or as imagined adversaries to rally around. Just as ideals come in varieties, so do reactions to ideals. Objections to "the truth is what works" might be objections to uncreative or minimalist definitions of works. I don't think it's always synonymous with "profitable". If people seek to "maximize a payoff", then realistically speaking the payoff, options, and factors must be churning like a turbulent flow in endlessly intricate directions (e.g. affected by psychological priming). Part of the reason for the unpredictability is that ideals may conflict and the ideal that "wins" in one situation can "lose" in another. Altruism and self-denial are unprofitable, but all that's necessary to uncover the "mysterious" rationale is to ask the participant which ideals were the motivation. A lack of obvious profit surely doesn't prevent the ideals from working for the participant in some way.

Truths are real, not ethereal. And a truth is known by its real consequences on real people who perform real actions.

I am a Cylon, and candid opinions thereof

2010-07-27T23:15:00.000-04:00

I'm aware that I'm not a Cylon. Cylons are fictional, first of all.

But it's an excellent metaphor for how I felt when it began to dawn on me that, despite the deeply religious tenets that I formerly cultivated throughout my life, I underwent a "gradual intellectual anti-conversion" which culminated several (less than five) years ago. "Intellectual" because emotion didn't participate in the process; there was no instance of shaking my fist at clouds and also no desperate clinging to my fading assurances. "Anti-conversion" because I view the change as a reversal/erasure of specific beliefs rather than switching to a new "side" as such. Like the fictional characters who lived as humans only to arrive at the breathtaking realization that they were unmistakable examples of the hated enemy Cylons, I discovered to my surprise that my assumptions had changed sufficiently that the most accurate label for me now is "atheist".

It was "gradual" because while it was happening I sometimes noticed the mental conflicts, but I couldn't recall a definitive moment of selection or rejection. There was no single choice, but at the end there was a belated acknowledgment that some of my beliefs had simply deposed some others in the manner of a silent coup. In fact, further mental struggle had long ago become pointless since one side now occupied all the "territory". Further battles could only be as "asymmetric" as guerrilla warfare.

Before the point of mental consolidation, I considered myself "religious with doubts and reservations". My thinking seemed unsteady but regardless I continued to act the same. After the crystallization point, the prior motivations and habits were present yet ceased to exert authority; I saw through. I could select my level of compliance without compulsion, like adjusting the volume on the playback of a speech. The majestic and imposing scenery shrunk down to matte background paintings. However, my new-found godlessness didn't lead to unrestrained lawlessness or carelessness (just as not all Cylons who lived as humans chose immediately to quit behaving like humans). I suppose that as an immigrant to atheism, by default I naturally cherish the customs of the land of my previous patriotism.

Actually, this phenomenon was another jolt to my perspective. The religious are accustomed to categories such as 1) young people who rebel passionately by doing exactly what is forbidden, perhaps when they start higher education at a faraway location, 2) wishy-washy attendees who "backslide" progressively worse in their actions until they can't bear to even pretend to be devotees any longer, 3) people who deny the existence of any god and therefore act as chaotic and selfish manipulators, iconoclasts, and nihilists (I can very well envision someone commenting "Say what you like about paganism, at least it's an ethos"). My gradual intellectual anti-conversion doesn't fit into this tidy taxonomy.

For regardless of my unbelief, I'm surely not one of the bogeymen in the scary third category. Sure, some of my politics and causes are different now, but I'm not aiming to overthrow the right to religious expression and culture, presuming there's no entanglement among important social institutions that should be neutral toward such beliefs. I'm also not against religious evangelism and conversions as long as there's no abusive coercion or exploitation. Frankly, my interest in which religion dominates dropped tremendously when I self-identified as atheist, for whom afterlife and divine judgment are nonexistent consequences of incorrectness. I don't even care about convincing other people of my atheistic point of view, as much as I care that atheists not be stereotyped or persecuted in the larger society.

Furthermore, my present sentiments regarding religion go beyond a lack of competitive zeal against it. I have a lingering appreciation for it. Although not all effects of religion are good, to say the least, I know many people for whom religion is an essential pillar of the mindsets that calm their psyches and motivate them to accomplish amazing progress in themselves and their surroundings. And I think it's baldly inaccurate to accuse the religious of widespread stupidity or weakness. Religion can at times demand either courage or intelligence. Besides, evangelistic atheists should keep in mind that shaming people into following your example is not a highly effective technique anyway, especially if the goal is willing long-term commitment. That tactic certainly played no part in convincing me.

Moreover, these conceded emotional comforts of religion tempt me as well, whenever I personally contemplate death. Given that life is an exceptional configuration of matter and not a separate substance that "inhabits" it, death is many other configurations of that matter. My consciousness requires a blood flow of necessary materials to operate; when this flow stop working normally, my consciousness will also halt. Life's fragility is staggering. Without the promise of an afterlife independent of the vagaries of complex biological systems which can forestall entropy solely for a limited period, the inestimable value of healthy living should be obvious. Responding to death's finality by living dangerously is nonsensical. I should think it obvious that the saying "you only live once" doesn't imply "risk and/or waste the only life you have". To an atheist, "you" are your body.

But if every human is a body and no more, then the fact of the terrifying impermanence of one's own life comes with discomforting companions: every dead human, in all of his or her uniqueness, must be irretrievably gone. The precise configuration of bodily matter that constituted him or her shall never arise again. It's no more likely, and to the contrary far, far less, than the chance of a thorough shuffle of a pack of 52 playing cards producing the exact same order as before the shuffle. Of course biological reproduction perpetuates a lot of the mere genetic bases, but a perfect clone would still develop differently than the source. Indeed, without the clone retaining an identical set of lifelong memories, it couldn't fool any observers. This is why the question of "legacy", i.e. the lasting consequences "left behind" by a human's actions after death, is extremely pertinent to atheists. In a highly literal sense, someone's legacy is the only "part" that can justifiably be termed eternal (although "his" or "hers" separated particles of matter and energy are conserved, but that's scant consolation at the human scale).

I understand if people choose to question my unflinching claim that bodily death entails total death. How can I certainly pronounce that all of the deceased, whether family, friends, or martyrs, haven't moved on to an unexperienced "plane", since I clearly haven't gone through it? Without direct evidence, isn't it more fair and conciliatory to postpone discussion? Well, my first, curt reply is the suggestion that everybody else postpone discussion, too. If they aren't, then I won't. My second, earnest reply is the blunt admission that evidence, as commonly defined, isn't the solitary foundation of my thoughts about reality. Evidence has a strong tendency to be patchy and/or conflicting. Therefore judgment is indispensable, and based on my judgment of the overall array of evidence, including the pieces of evidence that appear to be absent, death is the end of humans. This statement is the honest reflection of my outlook, as opposed to something comparatively half-hearted like "due to lack of positive evidence, I don't know". I profess a materialistic universe; so there's simply no room for anything supernatural, much less the person-centered afterlife usually described. I readily affirm the incompleteness and uncertainty embedded in my knowledge, but I don't waver on the central axioms.

Odds are, the open mention of the role of judgment/selection provokes objections from two diverging groups: 1) from people who see themselves as pure empiricists, because they say that evidence always "speaks" for itself and any evidence that isn't perfectly unambiguous is invalid, 2) from people who subscribe to a particular revelation of a supporting supernatural realm, because they say that people who use individual and error-prone ability to define ultimate truth will instead substitute their own relative, subjective, and quick-changing preferences. But neither objection adequately captures a holistic and authentic viewpoint of actual human activity. Perception, interpretation, goals, experiences, and actions, etc., feed and affect one another. Personal desires and idiosyncratic mental processing are intermingled throughout mental events. No matter which concepts people use to anchor their thoughts, the coronation of those concepts is not passive but active. Regardless of what they say in debates, the pure empiricist decides how to categorize and evaluate his or her valuable evidence, and the devotee of a supernatural dogma decides how to extrapolate and apply it to the modern situations thrust upon him or her. And the confident atheist is no different...

Except during an interval in which transforming thoughts rise in power sneakily like an unfamiliar tune one can't shake off, and the final remaining task is to put it into words: I am a Cylon.

persistence of private by the Nucleus pattern

2010-07-14T16:50:00.000-04:00

The encapsulation of data by an object's methods is one of the foremost goals of effective OOP. Restricting exposure of the object's private information prevents other code from accessing it. The inaccessibility ensures that the other code can't depend upon or otherwise share responsibility for the private information. Each object has a single responsibility: a sovereign private realm of information and expertise.

However, this ideal conflicts with the reality of the need to give objects persistence because most programs require data storage in some form. And the required interaction with the storage mechanism clearly isn't the responsibility of the objects that happen to correspond to the data. Yet how can the objects responsible, often known as repositories or data mappers, mediate between external storage and other objects while obeying encapsulation? How can information be both private and persistent without the object itself assuming data storage responsibility?

The "Nucleus" design pattern, very similar to an Active Record, addresses this issue. According to the pattern, a persistent object, similar to a eukaryotic cell, contains a private inner object that acts as its "nucleus". The nucleus object's responsibilities are to hold and facilitate access to the persistent data of the object. Therefore its methods likely consist of nothing more than public "getters and setters" for the data properties (and possibly other methods that merely make the getters and setters more convenient), and one of its constructors has no parameters. It's a DTO or VO. It isn't normally present outside of its containing object since it has no meaningful behavior. Since the nucleus object is private, outside objects affect it only indirectly through the execution of the containing object's set of appropriate information-encapsulating methods. The containing object essentially uses the nucleus object as its own data storage mechanism. The nucleus is the "seed" of the object that contains no more and no less than all the data necessary to exactly replicate the object.

Naturally, this increase in complexity affects the factory object responsible for assembly. It must initialize the nucleus object, whether based on defaults in the case of a new entity, or an external query performed by the storage-handling object in the case of a continuing entity. Then it must pass the nucleus object to the containing object's constructor. Finally, it takes a pair of weak references to the containing object and nucleus object and "registers" them with the relevant stateful storage-handling object that's embedded in the execution context.

The object pair registration is important. Later, when any code requests the storage-handling object to transfer the state of the containing object to external storage, the storage-handling object can refer to the registration list to match the containing object up to the nucleus object and call the public property methods on the nucleus object to determine what data values to really transfer.

Pro:

The containing object doesn't contain public methods to get or set any private data of its responsibility.
The containing object has no responsibility for interactions with external storage. It only handles the nucleus object.
Since the nucleus object's responsibility is a bridge between external storage and the containing object, design compromises for the sake of the external storage implementation (e.g. a specific superclass?) are easier to accommodate without muddying the design and publicly-accessible "face" of the containing object.

Con:

The nucleus object is one additional object/class for each persistent original object/class that uses the pattern. It's closely tied to the containing object, its factory object, and its storage-handling object.
The original object must replace persistent data variable members with a private nucleus object member, and the containing object's methods must instead access persistent data values through the nucleus object's properties.
The containing object's constructors must have a nucleus object parameter.
The factory must construct the nucleus object, pass it to the containing object's constructor, and pass along weak references to the storage-handling object.
The storage-handling object must maintain one or more lists of pairs of weak references to containing objects and nucleus objects. It also must use these lists whenever any code requests a storage task.
The code in the storage-handling object must change to handle the nucleus object instead of the original object.

explicit is better than implicit: in favor of static typing

2010-07-07T15:14:00.000-04:00

Right now, static not dynamic typing more closely fits my aesthetic preferences and intellectual biases. And the best expression of the reason is "explicit is better than implicit". The primary problem I have with dynamic typing, i.e. checking types solely as the program executes, is that the type must be left implicit/unknown despite its vital importance to correct reasoning about the code's operation. The crux is whether the upsides of a type being mutable and/or loosely-checked outweigh the downside of it being implicit.

Dynamism. I'm inclined to guess that most of the time most developers don't in fact require or exploit the admittedly-vast possibilities that dynamic typing enables. The effectiveness of tracing compilers and run-time call-site optimizations confirms this. My experiences with C#'s "var" have demonstrated that, for mere avoidance of type declarations for strictly-local variables, type inference almost always works as well as a pure dynamic type. Stylistically speaking, rebinding a name to multiple data types probably has few fans. The binding of a name to different data types is more handy for parameters and returns...

Ultimate API Generality. As for the undeniable generality of dynamically-typed APIs, I'm convinced that most of the time utterly dynamic parameters are less accurate and precise than simple parameters of high abstraction. This is seen in LINQ's impressive range of applicability to generic "IEnumerable<T>" and in how rarely everyday Objective-C code needs to use the dynamic type id. With few exceptions, application code needs to implicitly or explicitly assume something, albeit very little, about a variable's contents in order to meaningfully manipulate it. In languages with dynamic typing, this truth can be concealed by high-level data types built-in to the syntax, which may share many operators and have implicit coercion rules. Of course, in actuality the API may not react reasonably to every data type passed to it...

"Informal Interfaces". According to this design pattern, as long as a group of objects happen to support the same set of methods, the same code can function on the entire group. In essence, the code's actions define an expected interface. The set of required methods might differ by the code's execution paths! This pattern is plainly superior for adapting code and objects in ways that cut across inheritance hierarchies. Yet once more I question whether, most of the time, the benefits are worth the downside in transparency. Every time the code changes, its informal interface could change. If someone wants to pass a new type to the code, the informal interface must either be inferred by perusing the source or by consulting documentation that may be incomplete or obsolete. If an object passed to the code changes, the object could in effect violate the code's informal interface and lead to a bug that surprises users and developers alike. "I replaced a method on the object over here, why did code over there abruptly stop working?" I sympathize with complaints about possible exponential quantities of static interface types, but to me it still seems preferable to the effort that's required to manually track lots of informal interfaces. But in cases of high code churn, developers must expend effort just to update static interface types as requirements and objects iterate...

Evolutionary Design. There's something appealing about the plea to stop scribbling UML and prod the project forward by pushing out working code regardless of an anemic model of the problem domain. In the earliest phases, the presentation of functioning prototypes is a prime tool for provoking the responses that inform the model. As the model evolves, types and type members come and go at a parallel pace. Isn't it bothersome to explicitly record all these modifications? Well, sure, but there are no shortcuts around the mess of broken abstractions. When the ground underneath drops away, the stuff on top should complain as soon as possible, rather than levitating like a cartoon figure until it looks down, notices the missing ground, and dramatically plummets. Part of the value of explicit types lies precisely in making code dependencies not only discoverable but obvious. This is still more important whenever separate individuals or teams develop the cooperating codebases. The other codebase has a stake in the ongoing evolution of its partner. A "handshake" agreement could be enough to keep everyone carefully synchronized, but it's more error-prone compared to an enforced interface to which everyone can refer. During rapid evolution, automated type-checking is an aid (although not a panacea!) to the task of reconciling and integrating small transforming chunks of data and code to the overall design. Types that match offer at least a minimum of assurance that contradictory interpretations of the domain model haven't slipped in. On the other hand, unrestricted typing allows for a wider range of modeling approaches...

Traits/Advanced Object Construction. No disagreement from me. I wish static type schemes would represent more exotic ideas about objects. Still, most of the time, applied ingenuity, e.g. OO design patterns, can accomplish a lot through existing features like composition, delegation, generics, inheritance.

I want to emphasize that my lean toward static typing for the sake of explicitness isn't my ideal. I direct my top level of respect at languages and platforms that leave the strictness of the types up to the developer. I like a functioning "escape hatch" from static types, to be employed in dire situations. Or the option to mix up languages as I choose for each layer of the project. I judge types to be helpful more often than not, but I reserve the right to toss 'em out when needs demand.

foolish usage of static and dynamic typing

2010-07-07T09:03:00.000-04:00

Whenever I read the eternal sniping between the cheerleaders of static and dynamic typing, I'm often struck by the similarity of the arguments for each side (including identical words like "maintainable"; let's not even get started on the multifarious definitions of "scalability"). Today's example is "I admit that your complaints about [static, dynamic] typing are valid...but only if someone is doing it incorrectly. Don't do that." Assuming that sometimes the criticisms of a typing strategy are really targeted at specimens of its foolish usage, I've therefore paired common criticisms with preventive usage guidelines. The point is to neither tie yourself to the mast when you're static-typing nor wander overboard when you're dynamic-typing.

Static

"I detest subjecting my fingers to the burden of type declarations." Use an IDE with type-completion and/or a language with type inference.
"The types aren't helping to document the code." Stop reflexively using the "root object" type or "magic-value" strings as parameter and return types. Y'know, these days the space/speed/development cost of a short semantically-named class is not that much, and the benefit in flexibility and code centralization might pay off big later on - when all code that uses an object is doing the same "boilerplate" tasks on it, do you suppose that just maybe those tasks belong in methods? If a class isn't appropriate, consider an enumeration (or even a struct in C#).
"Branches on run-time types (and the corresponding casts in each branch) are bloating my code." Usually, subtypes should be directly substitutable without changing code, so the type hierarchy probably isn't right. Other possibilities include switching to an interface type or breaking the type-dependent code into separate methods of identical name and distinct signatures so the runtime can handle the type-dispatch.
"I can't reuse code when it's tightly coupled to different types than mine." Write code to a minimal interface type, perhaps a type that's in the standard library already. Use factories, factory patterns, and dependency injection to obtain fresh instances of objects.
"I need to change an interface." Consider an interface subtype or phasing in a separate interface altogether. Interfaces should be small and largely unchanging.
"When the types in the code all match, people don't realize that the code can still be wrong." Write and run automated unit tests to verify behavior. Mocks and proxies are easier than ever.

Dynamic

"It's impossible to figure out a function's expected parameters, return values, and error signals without scanning the source." In comments and documentation, be clear about what the function needs and how it responds to the range of possible data. In lieu of this, furnish abundant examples for code monkeys to ape.
"Sometimes I don't reread stuff after I type it. What if a typo leads to the allocation of a new variable in place of a reassignment?" This error is so well-known and typical that the language runtime likely can warn of or optionally forbid assignments to undeclared/uninitialized variables.
"Without types, no one can automatically verify that all the pieces of code fit together right." Write and run automated unit tests that match the specific scenarios of object interactions.
"Beyond a particular threshold scripts suck, so why would I wish to implement a large-scale project in that manner?" A language might allow the absence of namespaces/modules and OO, but developers who long to remain sane will nevertheless divide their code into comprehensible sections. Long-lived proverbs like "don't repeat yourself" and "beware global state" belong as much on the cutting-edge as in the "enterprise".
"My head spins in response to the capability for multiple inheritance and for classes to modify one another and to redirect messages, etc." Too much of this can indeed produce a bewildering tangle of dependencies and overrides-of-overrides, which is why the prudent path is often boring and normal rather than intricate and exceptional. Mixin classes should be well-defined and not overreach.
"In order to read the code I must mentally track the current implied types of variables' values and the available methods on objects." Descriptive names and comments greatly reduce the odds that the code reader would misinterpret the intended variable contents. Meanwhile, in normal circumstances a developer should avoid changes to an object's significant behavior after the time of creation/class definition, thus ensuring that the relevant class definitions (and dynamic initialization code) are sufficient clues for deducing the object's abilities.

peeve no. 260 is factional prejudice in IT

2010-06-30T13:54:00.003-04:00

Faction: a group or clique within a larger group, party, government, organization, or the like

I don't know what it is about IT that results in professionals forming factions. Use text editor X! Align your punctuation thusly! Avoid all software from company Q! Programming language R is your cool friend that all the popular kids like! Every problem should be flattened via solution U! Your VCS is DOA!

I understand that people develop attachments to their favorite tools and passion for their work; that's fine. What drives me nuts is the all-too-common over-exaggeration of these feelings. Acting as a cheerleader for your chosen IT factions can be fun, but if you ground your personal identity in it then you've officially gone mental. An IT professional should make technological decisions based on actual requirements and well-understood trade-offs, not based on which option fits your personal "style". Of course, usability and even nebulous subjective appeal are nevertheless legitimate factors in the decision simply because a difficult and hated option is detrimental to productivity and morale. The difference is a decision-maker who critically weighs all applicable criteria instead of just always "voting with the faction". Emotional investment is present without overpowering every other fact. Someone doesn't say "I'm a BLUB programmer". Rather, "I know the most pertinent portions of the syntax and semantics of BLUB, and I like BLUB because of BLURB".(I've complained before about grouping programmers by language.)

Although I'm ranting against factional prejudice, I caution readers to not interpret my opinion as a case of "binary" thinking (i.e. thinking characterized by a simplistic division into only two categories). Some factional prejudices are quite beneficial and practical, when kept within reasonable and specific bounds. For example, past behaviors of a company or open source project are good reasons to be wary of current behavior. A platform's reputation for lackluster efficiency is a worthwhile justification to avoid it unless the platform demonstrates improvement (or the efficiency is sufficient for the task). Evaluate factional prejudice according to its motivation and relevance and realism.

For businesses at least, technology isn't an accessory or an avenue of personal expression. It serves a purpose beyond the satisfaction of one's ego. The One True Way might not be what's right for the situation and the client.

watch out for a rebase of a merge

2010-06-25T13:32:00.000-04:00

So, a while back I tried to publish some detailed foolproof instructions for using named branches and hgsubversion. I've been using Mercurial as a Subversion client for a little while now, and I figured that if I posted my discoveries and practices then others with an Internet connection could benefit as well. After revising the post multiple times, I finally decided to pull it (HA!) from the blog because it clearly wasn't as foolproof as anticipated - as proven by the mishaps of this fool, anyway. At this point I redirect you to the Stack Overflow question "How to handle merges with hgsubversion?" and parrot the advice there: don't bother trying to reconcile named branches to Subversion! To fork development, clone the repository instead.

(Normally, my personal inclination is toward branches since clones feel like a hack solution. Given that I'm doing the right thing by using a VCS, why should I need to have separate directories for separate lines of development? It's true that environment variables or symbolic links make switching between the two directories fairly easy, but having to do those manipulations outside the VCS then feels like a hack of a hack. More and more frequently I wonder whether I'll settle on git, which has its own well-known set of warts, in the end.)

One of the primary shatter-points of my confidence was a rebase of a merge. This is the situation in which the default branch was merged into the other branch in the past, and the time arrives to rebase the other branch onto the default branch. (During work on a branch that will be ended by a rebase, the better course of action would have been to update it by a rebase onto default with --keepbranches rather than merging in the default branch.) The root problem is that a merge duplicates changes on the default branch by re-applying to the other branch, so to rebase this merge is to attempt to apply the default branch's past state to its present state, although the default branch may have changed in numerous ways after that merge!

As you might expect, a rebase of a merge often produces conflicts, and the likely desired resolution is to choose the version in the tip of the destination branch, "local" or "parent1". I can only write "likely" because, as usual with conflict resolution, only the developer's judgment can determine the right mixture whenever significant feature changes are involved in the conflict too.

While all these conflicts can be irritating, hair-pulling may be a more proper response to the instances of no conflict; Mercurial dutifully sends one or more files in the default branch backward in time without comment. I know of two causes of this behavior. 1. After the merge from default into other, a later change in default undoes one of the merged changes, and through the rebase of the merge the undoing is undone (reviving the original mistaken change). 2. After the merge from default into other adds a new file, a later change in default modifies the added file, and through the rebase of the merge the added file's later modifications vanish.

Therefore, after a rebase, it's a good idea to run a diff --stat in the range of the default's tip before the rebase and default's tip after the rebase. And for any of the unexpected changes that have no relevance to the feature just rebased, revert those files to the version of the default tip before the rebase and check in. Constant vigilance!

my turn to explain type covariance and contravariance

2010-06-21T22:41:00.000-04:00

The Web has a wealth of resources about type covariance and contravariance, but I often find the approach unsatisfying: either code snippets or abstract-ish theoretical axioms. I prefer an additional, intermediate level of understanding that may be called pragmatic. Pragmatically understood, any pattern or concept isn't solely "standalone" snippets or logical axioms. A learner should be able to connect the concept to his or her prior knowledge and also to the question "When would I use this?" I'll attempt to remedy this shortcoming.

For code reuse, a pivotal issue is which data type substitutions in the code will work. Can code written for type Y also function properly for data of type Z? Well, if the definition of data type Z is strictly "Y's definition plus some other stuff", then yes, since the code can treat Z data as if it were Y data. Z is said to be derived from Y. Z is a stricter data type relative to Y. We then say that Z is covariant to Y but Y is contravariant to Z. Code that acts on Y can also act on Z because Z encompasses Y, but code that acts on Z may not be able to act on Y because the code may rely on the "other stuff" in Z beyond Y's definition. In languages that support it, the creation of a data type that's covariant to an existing type can be easily accomplished through inheritance of the existing type's language artifacts. (When someone uses the language feature of inheritance to create a type whose data can't be treated by code as if it were of the original type, confident code reuse is lost, which is the goal behind type covariance/contravariance!)

Thus far I've only mentioned the reuse of one piece of code over any data type covariant to the code's original type. The data varies but the code doesn't. A different yet still eminently practical situation is the need for the code to vary and nevertheless join up with other pieces of code without any of the code failing to work (e.g. a callback). What relationship must there be between the data types of the code pieces in order for the whole to work?

All that's necessary to approach this question is to reapply the same principle: a piece of code that works with data type Y will also work with data type Z provided that Z is covariant to Y. Assume an example gap to be filled by varying candidates of code. In this gap, the incoming data is of original type G and the outgoing data is of original type P.

Consider the code that receives the outgoing data. If the candidate code sends data of type P, the receiving code will work simply because it's written for that original type. If the candidate code sends data of a type covariant to (derived or inherited from) P, the receiving code will work because, as stated before, covariant types can "stand in" for the original type. If the candidate code sends data of a type contravariant to P, the receiving code won't work because it may rely on the parts of P's definition that are included in P alone. (In fact, this should always be true. Whenever it isn't, the receiving code should have instead targeted either a less-strict type or even just an interface. A "snug" data-type yields greater flexibility.) So the rule for the candidate code is that it must be written to send data of type P or covariant to P.

Meanwhile, the code that sends the gap's incoming data has committed to sending it as type G. If G is the original type that the candidate code is written to receive, it will work simply because it's written for that type. If a type covariant to G is the original type that the candidate code is written to receive, it won't work because it may rely on the parts of that type that aren't in G. If a type contravariant to G is the original type that the candidate code is written to receive, it will work because G is covariant to it, and once again covariant types are substitutable. So the rule for the candidate code is that it must be written to receive data of type G or contravariant to G.

Hence, the rules for covariance and contravariance in various programming languages (delegates in C#, generics in Java or C#) are neither arbitrarily set nor needlessly complex. The point is to pursue greater code generality through careful variance among related data types. Static data types are "promises" that quite literally bind code together. Pieces of code may exceed those promises (legal covariance of return); however, the code may not violate those promises (illegal contravariance of return). Pieces of code may gladly accept what is seen as excessively-generous promises (legal contravariance of paramaters); however, no code should expect anything that hasn't been promised (illegal covariance of parameters).

agile's effect on productivity

2010-06-10T09:45:00.000-04:00

Someone needs to state the obvious for those who miss it. Agile software development is not a silver bullet for productivity. Agile is not something you "mix in" and yield the same result faster. Applying agile practices to the humongous project won't result in massive software at lightning speed. That's not the aim of agile.

Agile's main effect is a decrease in project latency, not an increase in throughput. Agile development means delivering minimal but user-confirmed business value sooner, rather than delivering maximal but doubtful business value once at the end. The hope is that by going to the trouble of more frequent releases, the software will grow gradually but surely, as actual usage not guessing motivates what the software needs to be. The economy of agile doesn't result in a greater quantity of software features per development time period, but it's certainly intended to result in a lesser number of wasteful software features per development time period.

This shift in perspective affects everybody's expectations, so agile development is more than a set of new habits for developers. Project managers need to break their world-changing ambitions into much smaller chunks. Users need to become more involved. Analysts need to laser-focus their requirement-gathering. Architects and modelers (may) need to recalibrate the balance between resource costs and the pursuit of perfection.

If a plane flight is like a software project, then agile development won't decrease the time it takes to go from Seattle to Boston non-stop. But it will change the non-stop flight into a series of short connecting flights, and by the time you reach Chicago you might realize that you didn't really need to go to Boston in the first place.

DDD and intrinsic vs. extrinsic identity

2010-06-09T13:59:00.000-04:00

Ask someone for an introduction to domain-driven design, and it will likely include descriptions of "the suggested building blocks". Two of the blocks are known as entities and value objects. The distinction between these two may appear to be pointless and obscure at first, but it's invaluable to a clearheaded and thoughtful model.

The difference turns on the meaning of identity, which philosophers have long pondered and debated, especially in regard to individual humans (i.e. the question "Who am I really?"). The answers fall into two rough categories: intrinsic and extrinsic. Unsurprisingly, the dividing-line can be fuzzy.

In the extrinsic category, a thing's identity is tied to sensations and ideas about that thing. This would be like identifying my phone by its shape, color, model, and so on. Based on extrinsic identity, if any of these "extrinsic" propositions about my phone ceased to be true, my phone would therefore no longer have the same identity. The phone would not only appear but be a different phone. Its identity is dependent. When someone meets a new acquaintance, his or her identity is generally rather extrinsic. "I don't know this person, but his name is Fred and he works in Human Resources."
The opposite of extrinsic identity is intrinsic identity, which is independent of sensations and ideas about a thing. A thing with intrinsic identity "wears" sensations and ideas but is "itself" regardless. This would be like disassembling my precious phone piece by piece and then continuing to call the pile of pieces my phone. Transformations of all kinds don't change intrinsic identity. When someone has maintained a long-term connection to another person, his or her identity is generally rather intrinsic. "She's changed her residence, weight, politics, and career over the last several years, but she's still Betty to me."

Neither category of philosophical answers is "right", of course, but merely more or less appropriate depending on situational factors like the thing to identify and the goal to meet by distinguishing identity. In DDD, the choice takes on added importance when modeling a domain object's identity.

If the object's identity consists of a particular combination of attributes, then its identity is extrinsic. Two objects of the class with identical attribute values are indistinguishable (for the purpose of the domain). These are known as value objects. Each instance is a "value" similar to a number or a string of characters. Since extrinsic identity is by definition fully dependent on a thing's parts, a value object never changes. But someone often creates a new value object using the attribute values of preexisting value objects. This is just like adding two numbers to yield a third number. Modelers and developers can exploit the unalterable nature of value objects for various advantages, such as avoiding the need for storage. Possibly, there could be a highly reusable "mathematics" of the value object in which all methods are "operations" that 1) operate on and return value objects and 2) are completely stateless (free of side effects).
Instead, a domain object's identity could be persistent throughout arbitrary transformations, which indicates intrinsic identity. Instances of the class probably have vital "history" and might not represent the same physical domain item despite identical or near-identical attribute values. These are known as entities. Normally a nontrivial entity will aggregate one or more value objects but along with a mechanism of unique identification. That mechanism could be a GUID or something else as long as no two instances can contain the same identifier by accident. Unlike value objects, (long-lived) entities change and correspond more directly to physical domain items, so a storage strategy for the entity and its associated value objects is necessary. (The advice of DDD is to keep the entities' code as purely domain-driven as possible by abstracting the storage details in a repository object.)