(Part 2 of a multi-part post. See Part 1 here.)

Words weave through sentences, submerge from view, resurface, resubmerge, warp and weft of weave or whatever metaphorical model we can throw at it. A word indexes many situations; a situation indexes many words. Can we see words and narrative, i.e. semantics and episodics, as orthogonal to one another, at right angles? Say, “story” is the X axis, “wording” the Y. Cognitive psychologists for a long time have entertained the distinction of semantic from episodic memory, and sought to discern whether they are, as hypothesized, truly independent from one another. Steven Prince, at al. (2007) report in Psychological Science that “the neural correlates of EE [episodic encoding] and SR [semantic retrieval] are dissociable but interact in specific brain regions.” In an example of this kind of interaction, semantic associations enhance the retrieval of episodic memories (Menon, et al., 2002). One site of interaction between the semantic and the episodic happens at the very instant a word is semantically integrated into a sentence, the very instance, in effect, when it enters into the telling of an episode.

While Swinney’s and others’ interest is pitched at an understanding of how we “compose” an interpretation out of a sentence’s building blocks, from the words and the grammatical relations within it, it does not seem to me to be too great a stretch to map this fine-scale event to the wider action of narrative within which it happens. Those ephemeral moments interest me when irrelevant meanings are sloughed away and only meanings fit to the flow of narrative within which they are embedded are left. This is the moment I have called “reading between the words.”

For every word we read, our mind opens and closes on that word’s potentialities, like the gate of a movie projector opening and closing the light on frame after frame of a film that slips a pull-down of filmstrip though, click-click-click in those interstices between openings, a sentence operating on our minds like a filmstrip operating on our eyes and visual cortices. By persistence of sense just as to the eyes, the film works by persistence of vision. Open, closed, open, closed—but it all comes coherent in a single stream, illusory yes, but to the mind, real. In fluent reading, whole gobs of text go in before consciousness opens its aperture. Flick-flick-flick. But it seems continuous.

William Nestrick, citing Walter Benjamin’s essay “The Work of Art in the Age of Mechanical Reproduction,” states that “the film is the animation of the machine, a continuous life created by the persistence of vision in combination with a machine casting light through individual photographs flashed separately upon the screen.” Narrative, an older technology, is an animation of another machine. Our deeply naturalized habits of reading elide the composite, discontinuous nature of our construction of meaning. Like the animation of the machine, it invites comparison to the monster from Frankenstein—and hypertextual disruption, what Shelley Jackson encodes in Patchwork Girl.

In the prevalent “invisible style,” films build seamless scenes from successions of discrete shots much as the mind builds seamless sentence understanding out of successions of words. What is left out is simply not noticed. Maybe it is repressed or abjected—but to waking, normal cognition, something must be, because the entirety of the worlds we build in our minds arises from selectively attending to some things and not others. Repression in one sense is the inescapable flip side of being in the world with a mind. It is the flip side of the selective attention that gets knitted together by wetware magic into scenes and stories we assemble, recall, reform, tell, and retell. No word drawn from the batch of “non sequitur,” “relevant,” “irrelevant,” “emphasis,” etc. could bear on our way of writing and transacting communication if it were not in a context of narratives, stories, and episodes, with their characteristic features: topics, action, motivation…. And none of these concepts would have any utility to us if construction of stories in our minds were not inherently selective, if thought itself were not inherently so.

The process of selection and of assembly follows a pattern something like what Katherine Hayles describes in “Flickering Connectivities in Shelley Jackson’s Patchwork Girl” (2000). More than the signifier, though, it is signification that flickers, the association of signifier with signified, on a pattern of flickering consciousness, in which access to the results of the automatic, unconscious process of lexical access flickers in and out, in fluent reading seldom noticed. This kind of processing, highly automatized by skill, can be disrupted by divergence from conventional expectation. Since 1917,  when Russian “Formalist” Viktor Shklovsky described it, this disruption of the automatic has been prominent in aesthetics, theorized as “strange-making” (ostranenie to transliterate the Russian), or “defamiliarization.” The practice has its precedent in the work of William Wordsworth. Explained by Samuel Taylor Coleridge in his Biographia Literaria:

“Mr. Wordsworth, on the other hand, was to propose to himself as his object, to give the charm of novelty to things of every day, and to excite a feeling analogous to the supernatural, by awakening the mind’s attention to the lethargy of custom…”

In the frame of the present post, comparing the cognitive integration of text with the sensory integration of motion pictures, this would be akin to slowing the film down below the critical flicker frequency, so as to break the illusion. In the technoromantic frame, the mechanism of novelty’s action in the brain is the “orienting reflex,” first described by Ivan Petrovich Pavlov as the “shto eto takoi” (“what is it?”) reflex. The orienting reflex is the mechanism that breaks the automatic.

Hayles:

“In Patchwork Girl, one of the important metaphoric connections expressing this flickering connectivity is the play between sewing and writing. Within the narrative fiction of Frankenstein, the monster’s body is created when Frankenstein patches the body parts together; at the metafictional level, Mary Shelley creates this patching through her writing.”

Shelley Jackson’s work strategically disrupts narrative at several levels, from fine-grained lexical interactions to collisions between textual elements arising in the restructuring of narrative (as hypertext) and exploitation of a medium (the computer) that restructures the reading interface. Significant to this is Donna Haraway’s “Cyborg Manifesto,” which treats the reconfiguration of the human form itself. Hypertext operates, for now, on the evolved biological capacity for cognitive reconfiguration, hacking the machine via its present interface.

What lies at the asymptote of narrative restructuring by hypertext? Where does it lead? Again sketching: At the outer extreme of hypertext is the “chaotic novel” described in “The Garden of Forking Paths” by Jorge Luis Borges. Any (coherent) narrative we read picks out senses of the words within it, biases us to read words one way and not another. So, if a narrative like the Garden of Forking Paths were to be realized—in which we visit every possible line of a story that unfolds within it—must we then also visit every possible sense of every word within it? Not just as we encounter them, in the way Swinney shows us is so often unconscious, but as we integrate them into their contexts? What happens if word senses themselves mutate with context, with the events that happen around them in the stories they enter in? What does representation of this level of complexity represent?

——————————————————————————–

Caveat:

I have assumed in this series of blog posts that what is happening at the lexical level in sentences is a microscopic instance of a larger sweep of events in our cognitive construction of narrative meaning—that the sentence in some sense works like the scene, and so on up. This theoretical portrayal is adumbrated, sketchy: conjectural. Reflexively, though not necessarily destructively, it is subject to the same form of critique, of illusory continuity. Delve into the psycholinguistic findings, and extend them, and expect the results to richen or complicate the present picture. A side-effect of reliance on Swinney’s work may be acceptance of a modular model of word representation in the brain, something others have inferred from it, with fully independent representations of word meaning. This may contradict “interactionist” models of sentence interpretation, which in my view are essential to understanding how we handle figurative language. For one form of experimental challenge to Swinney et al., see “Early Integration of Context During Lexical Access of Homonym Meanings,” by Janet Lee Jones in Current Psychology 10, no. 3 (Fall 1991): 163-181. For a theoretical consideration of lexical interaction in sentences, see “Should Natural-Language Definitions be Insulated from, or Interactive with, One Another in Sentence Composition?” by L. Jonathan Cohen, in Philosophical Studies: An International Journal for Philosophy in the Analytic Tradition 72, no. 2/3 (Dec. 1993): 177-197.

—————————————————————————————

Works Cited:

Borges, Jorge Luis. 1964. “The Garden of Forking Paths.” In Labyrinths: Selected Stories & Other Writings, 19-29. New York: New Directions Publishing Corporation.

Coleridge, Samuel Taylor. 1983. Biographia Literaria. Ed. James Engell and W. Jackson Bate. Princeton, New Jersey: Princeton University Press.

Haraway, Donna. 1991. “A Cyborg Manifesto: Science, Technology, and Socialist-Feminism in the Late Twentieth Century.” In Simians, Cyborgs and Women: The Reinvention of Nature, 149-181. New York: Routledge. (Available online at: http://www.stanford.edu/dept/HPS/Haraway/CyborgManifesto.html)

Hayles, N. Katherine. 2000. “Flickering Connectivities in Shelley Jackson’s Patchwork Girl: The Importance of Media-Specific Analysis.” Postmodern Culture 10, No. 2.

Jackson, Shelley. n.d. “Stitch Bitch: The Patchwork Girl.” MIT Communications Forum. http://web.mit.edu/comm-forum/papers/jackson.html

Jackson, Shelley. 1995. Patchwork girl, or, A modern monster by Mary/Shelley, & herself: a graveyard, a journal, a quilt, a story & broken accents. Watertown, Massachusetts: Eastgate Systems, Inc.

Jones, Janet Lee. 1991. “Early integration of context during lexical access of homonym meanings.” Current Psychology, 10 (3).

Menon, Vinod, et al. 2002. “Relating semantic and episodic memory systems.” Cognitive Brain Research, 13:261–265.

Onifer, William, and Swinney, David A. 1981. “Accessing lexical ambiguities during sentence comprehension: Effects of frequency of meaning and contextual bias.” Memory & Cognition, 9(3): 225-236.

Prince, Steven E. et al. 2007. “Distinguishing the Neural Correlates of Episodic Memory Encoding and Semantic Memory Retrieval.”  Psychological Science, 18 (2): 144-151.

Swinney, David A. 1979. “Lexical Access during Sentence Comprehension: (Re)Consideration of Context Effects.” Journal of Verbal Learning and Verbal Behavior, 18:645-659.

(first of a multi-part post)

“Thinking is conducted by entities we don’t know, wouldn’t recognize on the street.” –Shelley Jackson, “Stitch Bitch: the Patchwork Girl”

In “Stitch Bitch,” an essay about her hypertext novel Patchwork Girl, Shelley Jackson treats us to a succession of metaphors for embodiment and the way we read: The body is a “statue,” a “hard kernel.” Hypertext is “the banished body”; it sets up “rendezvous between words never before seen in company”; when it diverges at a choice-point it dissolves as “a Cheshire aftercat.”

Hypertext brings into action properties that lie dormant in conventional linear narrative. Jackson enumerates these properties in the section entitled “Collage.”

“We don’t say what we mean to say. The sentence is not one, but a cluster of contrary tendencies…. But nobody can domesticate the sentence completely. Some questionable material always clings to its members. Diligent readers can glean filth from a squeaky-clean one. Sentences always say more than they mean, so writers always write more than they know, even the laziest of them.”

Hypertext through its strategy of design activates those dormant meanings:

“It was not difficult, for example, to pry quotes from their sources, and mate them with other quotes in the ’quilt‘ section of Patchwork Girl, where they take on a meaning that is not native to the originals.”

To understand what Jackson hypothesizes to be happening in hypertext, it helps to understand the workings of the narrative style it would subvert. Conventional linear narrative, which Jackson describes as “fated slalom,” is configured so as to shepherd readers away from divergent threads: “Plot chaperones understanding, cuts off errant interpretations.” How narratives repress the penumbral interpretations that could emerge from them, how they tame the many voices of inherent allusion and come to sound like one voice or one story, how they filter their rich harmonies down to singular melodies, is far from entirely understood. At the sentence level, this resolves to a question of how a sentence’s prevalent meaning is composed out of words that are in themselves inherently polyvalent and ambiguous. Arriving at sentence’s end, we usually have an unequivocal idea of what we have just read. But how does the brain make meaning, word by word, as it reads a sentence? When—at what moment in the reading—have we dispensed with alternative interpretations, including the senses of words that don’t fit?

In a series of experiments beginning in the late 1970’s, psycholinguist David A. Swinney developed an innovative way to pinpoint the moment of  word disambiguation in sentences. Swinney asked: As we are reading, do we access all of a word’s senses at once, and only then disambiguate them to fit the prior context, the flow of the sentence they occur in? Or, does the sentence we are reading, and our understanding of it, preselect what senses of a multivalent word’s meaning we perceive, so that we never even entertain irrelevant ones? At stake was an understanding of how verbal memory is organized, and whether word senses are accessed independently from sentence interpretation. Do we really have access to the “contrary tendencies” made possible by the breadth of allusion, and semantic potential, that its constituent words carry within them? We disambiguate words at some point; but when?

Swinney’s questions concern what psycholinguists call “lexical access.” To answer them, his group developed the ingenious “cross-modal priming task,” where one mode was auditory, the other visual, where listening to a soundtrack and reading from a screen could together be used to tease out the timing of disambiguation for words within sentences. In one experiment (1981), the auditory track plays a set of strategically constructed sentences, each with a carefully chosen word—let’s call it a target word–placed somewhere within it. That target word is in itself ambiguous, but the sentence is designed to support just one interpretation of it.   At the same time, via the visual track, other carefully chosen words are flashed on a screen.  The flashed words are, in fact, prompts that are semantically related to one or another meaning of the ambiguous target word from the auditory track. And all this takes place while the experimental subject engages in a psycholinguistic “lexical decision task” (LDT).

The experiment uses a series of sentences and target words, but as an example, let’s say the ambiguous target word is “scale,” and that two among its available meanings are: “a weighing device” and  “a protective plating on a fish or reptile.” We can’t be sure which meaning is relevant until we hear it used in a sentence: “The postal clerk put the package on a scale to see if it had enough postage” vs. “The dinner guests enjoyed the specially prepared river bass, although one did get a scale caught in his throat.” You (the experimental subject) are asked to listen to one of those sentences read aloud as a soundtrack over headphones, but at the same time you have a task to perform: Watch a monitor screen and when a string of letters appears, just as quickly as you can, press a button telling whether it is a word or not . This is the “lexical decision task.” Ideally, when you see “Glmople” or “~!@#$%^&” you press the “no, not a word” button and when you see “breakfast” you press  “yes, that is a word.”

For a long time psycholinguists have known that performance on the LDT can be facilitated, speeded up, if you have read semantically related words just beforehand. Swinney’s addition of the auditory track to the visually presented LDT enabled his uniquely time-sensitive measure of lexical access. Thus, using the ambiguous target word (e.g. “scale”) Swinney could flash related words on the screen and measure the time it took for people listening to that audio track to press the button for “yes” or “no,” “word” or “not a word.” He could probe with those visually flashed words to see which senses of “scale” were active in the listeners’ minds. All he had to do was to choose probe words to flash on the screen that were related more to one sense than the other, at any given moment. A probe word related to “scale” as “a device to measure weight” could simply be “weight,” then; and a word related to “a protective plating on a fish or reptile” could simply be “fish.” If there is priming—if hearing “scale” helps us to answer the LDT more quickly—then we can measure it.

Swinney found—contrary to earlier experiments by other researchers who had probed for lexical access only after the sentence was over—that at the moment the ambiguous word (“scale” again) was played on the soundtrack, multiple senses of that word were accessed.

That’s at the moment we read a word. How long, though, do all senses of a word remain available to us, after we read it? Swinney knew from others’ experiments that alternative meanings were unavailable by the time a sentence had been read. How long during sentence reading were alternative word senses available? Swinney’s second experiment was designed to provide an answer. Probing with the same sets of visually flashed words a second-and-a-half later, a second-and-a-half after the onset of the ambiguous word in the soundtrack, Swinney found the priming effect was gone. Only the probe words related to relevant senses of “scale” gave rise to a faster LDT while irrelevant senses that had been primed in the previous experiment were gone–knocked right out of the sentence interpretation. As Onifer and Swinney (1981) conclude, “In the absence of any strongly biasing context, it appears as though all meanings of a lexical ambiguity are accessed, at least momentarily. Such access is not available to conscious introspection, and the listener eventually becomes aware of only one of the meanings accessed for the ambiguity.”

Swinney has illuminated the cognitive processing that goes into reading within a small unit of narrative, the sentence. On this fine scale, his group’s results bear out Shelley Jackson’s descriptions of a conventionally linear “slalom” narrative, a narrative that does not invite attention to its inherent ambiguities. Each word arises in turn with its full multiplicity of meaning, only to be delivered to consciousness in a tightly narrowed sense that fits the sentence’s unitary whole. There’s a verbal sleight of hand in an unequivocal narrative that sluices the course of consciousness: It’s a magic trick. To disperse attention is to disrupt what happens between the words, before reading resolves into an unequivocal interpretation.

None of this is to say that we never access multiplicities of meaning within the “slalom” linear narrative, or that we cannot. It is simply a typical case, and subject to habits of reading as much as it is to the formulae of writing. What Swinney’s work shows is a particular instance where, as Jackson declares, “We don’t say what we mean to say. The sentence is not one, but a cluster of contrary tendencies.” Lexically, it inescapably is. Hypertext works to subvert and call attention to a multiplicity of meaning inherent in any text. Its point of intervention is that interstitial moment when we are at the business of unconsciously sloughing away meanings that fail to fit a larger frame of narrative, where we are reading, literally, between the words.

(Part 2 is here.)

I.

I’ve talked in part about my experience coding the Frankenstein manuscripts, at least from a practical, project-oriented viewpoint, but there seems to me something lost in only talking about that side of things. While step-by-step instruction for a coding-based project like ours is certainly useful information in one sphere, it inevitably leaves out the impetus, the drive, the sheer imaginative hook, that got us roped into such a thing in the first place.

Something there is that doesn’t love the non-canonical: the marginalized, the deleted, the abjected, the notes and writings that never quite make into a final manuscript. This accounts for our fascination with celebrity interviews and bonus footage on DVDs, with directors’ cuts and ‘never-before-seen’ acting spots–and in the literary world (though much less glamorous than the film industry) it accounts for our desire to read the biographies of our favorite writers, to mine their drafts, letters, manuscripts, and notebooks, for glimmering bits of data that might, on one level, satiate our personal fan-boy appetites, and, on another level, serve as keys to unlock, inform, or explain our scholarly impulses and queries.

It’s a curious thing, this love of outside material, one we often pretend to not have (“death of the author,” ad nauseam) but, in the end, wear like chocolate on our faces.

(more…)