-
Vijay Fafat
- Published on
The literature is quite rich in the exploration of harmful memes which can take over the mind through the body’s sensory apparatus, effectively seizing up the brain into a coma or an endless loop. In 1978, Christopher Cherniak wrote a very smart, tightly written story-in-the-form-of-a-report which explored this theme, where a “Riddle” - a piece of computer code - causes the reader with comprehension of the code to go into an irreversible catatonic state, a “Riddle Coma”. The story builds on precept that the mind is a software which can get tangled up by loops of circular logic or paradox or deeply intricate webs of logical propositions and deductions, a state of fugue from which it cannot extricate itself. The mind’s eye get locked in a basilisk’s stare, for it is “a mind-arresting proposition, one that throws any mind into a sort of paradoxical trance, perhaps even the ultimate Zen state of satori.”. This paradigm was thought particularly plausible in the early days of computer logic in the 70s (note: a potential weapon to bring to down the entire hive-mind of the Borg in Star Trek was a paradoxical puzzle program with exactly that logic, and a strategy employed in MC Pease’s story from 1949, “The Devious Weapon”). This is also quite reminiscent of the thought back in the days that thinking about transfinite numbers (a la Cantor) or deep logical propositions (a la Godel) induced some form of madness.
Cherniak’s story starts with the case of
“C. Dizzard, a research fellow with the Autotomy Group at M.I.U., who had previously worked for several small firms specializing in the development of artificial intelligence software for commercial applications. Dizzard’s current project involved the use of computers in theorem-proving, on the model of the proof in the 1970s of the four-color theorem.”.
He is found one day in a totally non-responsive state in his lab, and after many days of chasing down all potential biological sources of such a state and another person suffering from the same affliction, authorities realize that some part of a large program Dizzard had been working on induces the coma by an unknown mechanism. Quickly dubbed, “The Riddle”, the question on how to identify the source and an antidote takes center-stage. As the report succinctly puts it:
“The central question, “What is the Riddle?” obviously has to be approached very cautiously. The Riddle is sometimes described as “the Godel sentence for the human Turing machine,” which causes the mind to jam; traditional doctrines of the unsayable and unthinkable are cited. […] If the computational theory of mind is at all correct, there is some program, some huge word, that can be written into a machine, transforming the machine into a thinking thing; why shouldn’t there be a terrible word, the Riddle, that would negate the first one? […] the Riddle seems an idea whose time has come —like the many self-referential paradoxes (of the pattern “This sentence is false”) discovered in the early part of this century.”
A natural question arises whether the phenomenon is restricted to compuetr code or mathematical or logical propositions, a question which takes more urgency when:
“A topologist in Paris lapsed into a coma similar in some respects to Dizzard’s. No computer was involved in this case. The mathematician’s papers were impounded by the French, but we believe that, although this mathematician was not familiar with Dizzard’s work, she had become interested in similar areas of artificial intelligence.”
[…]
“The question arose of whether the Riddle had in fact been discovered earlier by hand and then immediately lost. A literature search would have been of limited value, so a biographical survey was undertaken of logicians, philosophers, and mathematicians working since the rise of modern logic. It has been hampered by precautions to protect the researchers from exposure to the Riddle. At present, at least ten suspect cases have been discovered, the earliest almost 100 years ago.”
The report is also concerned about a very logical implication of any public policy response to this state: can someone stumble into the Riddle without even thinking about it just based on warnings given about The Riddle, in a manner similar to the proof of a lemma which leads to the final proof of a major Theorem?
“The principal objective of our report was at least to decrease further coma outbreaks. Public demand for a role in setting research policy has emphasized the dilemma we confront: how can we warn against the Riddle, or even discuss it, without spreading its infection? The more specific the warning, the greater the danger. The reader might accidentally reach the stage at which he sees “If p then q” and p, and so cannot stop himself from concluding q, where q is the Riddle. Identifying the hazardous areas would be like the children’s joke “I’ll give you a dollar if you’re not thinking of pink rats ten seconds from now.”
And finally, the age-old dilemma of the double-edged sword which is scientific research, leading to the final paragraph and a somewhat funny and ironic conclusion:
“A question of ethics as well as of policy remains; is the devastating risk of the Riddle outweighed by the benefits of continued research in an ill-defined but crucial set of fields? In particular, the authors of this report have been unable to resolve the issue of whether the possible benefit of any report itself can outweigh its danger to the reader. Indeed, during preparation of our final draft one of us tragically succumbed.”
I highly recommend reading this story in Hoftstadter’s book. Hofstadter provides a very in-depth, scholarly commentary on the associated issues in his typical style, in a long “Reflection” section after the story.