Machines That Read Minds

Gary Selden

Science Digest, October 1981

Electrodes pressed to your scalp, you sit down while scientists watch your thoughts as waves on a screen.

When I was a kid dreaming about time warps and antigrav belts, one of the gadgets I wanted most was a tape recorder I could plug into my brain. The idea was to "write down" somehow the orchestrated chemoelectromagnetic music of mentation, to photograph the evanescent threads of thought so they could be played later at a convenient time. With such a machine we could save forever those rushes of joy that bring coherence to life; we could listen abstractedly to the Muse, knowing that we could remember all that she said.

I still don't own a thought recorder, but in scores of laboratories throughout the world researchers *are* using a new computer technology to read and record portions of the brain's vast internal hubbub. From this electronic mind reading they are beginning to learn which brainwaves appear consistently with which sights, sounds and other stimuli.

The waveform that the brain characteristically emits after absorbing an external event is called an *evoked potential* or an *event-related potential*. (Following current scientific language, I will use the former term and the latter's abbreviation -- ERP -- interchangeably.) Evoked potentials may constitute one of the most complex languages humans have ever tried to decipher, but even the limited vocabulary we already have is a versatile diagnostic tool and a guide to formerly uncharted aspects of the brain's activity. No one knows where a complete dictionary of the mind could take us.

An English physiologist, Richard Caton, first observed the brain's electrical field -- in lower animals -- in 1875. It was not until 50 years later, however, that Hans Berger, a German psychiatrist, recorded the first human electroencephalogram from platinum wires he had pushed into his young son's scalp. Berger thought the whole brain emitted only one wave, but soon it was found that when electrodes were placed at several points on the scalp, they recorded different patterns, indicating various waves. Today as many as 32 separate leads and channels are used to record and trace brain waves; the tracing is called an electroencephalogram, or EEG.

By reading an EEG, we can know when large parts of the brain are actively working. Smooth, evenly rounded waves appearing in long "trains" or short "bursts" indicate that the cells recorded are all firing rhythmically in unison -- in effect, "idling." This phenomenon is called coherence or synchrony. When part of the brain takes up some chore, the pattern becomes choppy, the waves irregular and smaller, indicating that a particular cell or group of cells is going about its business relatively independently, no longer synchronized with the others. Alpha waves (8-14 cycles per second) are most often synchronous when your eyes are closed but you are awake; they go their own way, at least briefly, when you open your eyes. The regular idling pattern of alpha waves in synchrony is a rough measure of activation or involvement in a task. So consistent are these and other patterns that they have become the basis of the science of biofeedback.


But the EEG is a crude tool. Nearly all of the waves are composites from many different structures, whose functions at any given moment may be unrelated to each other. Computers have recently made it possible, however, to draw out specific wave components, in response to stimuli whose meanings may differ among individuals, from the "noise" of the EEG's composite line -- noise being whatever the researcher is not interested in at the moment. When a researcher presents a single, rigorously controlled stimulus -- one flash of light, one click, one letter of the alphabet, one word -- the segment of EEG roughly half a second thereafter will contain a wave in response to that stimulus, although it is invisible because of the noise. However, when the same stimulus is repeated 50 or 100 times and those EEG segments are averaged, the noise, being so complex as to be almost random, cancels itself out, leaving the investigator with a specific waveform that has been *evoked* by that particular stimulus.

Responses to some stimuli will be straightforward. A click evokes from the auditory cortex a wave that first descends far below the cortical baseline, rises steeply to just above it, then slowly dies out. A tap on the hand produces two downward peaks, the second deeper than the first. ERPs can be roughly classified by the length of time they take to appear -- their *latency*. Sensory stimuli (lights, sounds) get to the cortex by simple pathways with few synapses (gaps between neurons) and generally evoke their responses within two-tenths of a second. Many short-latency ERPs are fairly easy to interpret and can be used as diagnostic tests to see whether sensory pathways are working properly.

Complex processes that use many synapses produce longer-latency waves, and therefore are harder to interpret. Some long-latency waves *have* been interpreted, however. In 1964 W. Grey Walter and associates at the Burden Neurological Institute in Bristol, England, described the contingent negative variation, or CNV. It is now becoming known as the "expectancy wave" because it occurs whenever a subject anticipates something pleasurable or at least nonthreatening. The CNV definitely indicates immediate anticipation, and it becomes larger the more pleasant the expectation. A small wave appears when a subject has learned to expect a light after a buzzer. Large CNVs appeared in an experiment when heterosexual men were about to see photos of nude women or when homosexual men were about to see photos of nude men. The CNVs disappeared or got very small when the slides went against the subjects' sexual preferences.

The P300 (named, as all ERPs are, by its polarity -- in this case, P for positive -- and its latency -- in this case 300 milliseconds) is sometimes called the "surprise wave." Discovered in 1964 by Samuel Sutton, of the New York State Psychiatric Institute, the P300 occurs in response to any stimulus that is important and must be dealt with. Though it is more complicated than this, basically you put out a P300 in everyday life when, for instance, the doorbell rings; in the taking of an EEG, a P300 would result when the experimenter suddenly inserted a buzz into a series of clicks.

Since the brain is organized to pay most attention to surprising stimuli (they might be dangerous), P300 is probably related to the process of deciding what the stimulus is and how seriously to take it. If an uninteresting stimulus is repeated, the evoked P300 gets smaller. On the other hand, if a flying saucer landed in your backyard, it would produce a very large P300 in your brain, and the subsequent waves would not be much smaller if a few more happened to land.

Even the absence of an expected stimulus will elicit a P300 wave. For example, Dr. Sutton found that the P300 appears 300 milliseconds after the point in time that subjects listening to a series of regularly spaced clicks noticed that one of the clicks was missing -- as if a subject were saying, "I distinctly heard that clock not strike."

The N100, also called the "cocktail party wave" or, more properly, the selective attention wave, shows up whenever you pay attention to one stimulus out of many. Steven Hillyard of the University of California, San Diego, who did much of the original work on this wave, expects N100 research to yield important clues to several types of mental illness and disorder. It has long been postulated that schizophrenics and autistic persons, for example, are overwhelmed by a cascade of stimuli gushing through a mental filter that doesn't work right.

Recording from subjects who were willing to take a few stiff drinks in the name of science, Helen Neville of the Salk Institute for Biological Studies in La Jolla, California, found that the N100 diminishes -- and that there is an associated loss of attention -- as the level of alcohol in the blood increases. Diminution of N100 when people get drunk explains why it's so much harder to hear what someone is saying at a booze bash than on an equally noisy subway platform. Neville also found that the P300 wave was markedly smaller in anyone who had imbibed alcohol and who also had an alcoholic in the immediate family. When the implications of that finding are understood, she believes, we may be close to having a test that can predict who is in metabolic danger of becoming an alcoholic.


An intriguing recent discovery, the N400 brain wave, shows how ERP analysis can focus on one kind of thought at a time. The N400, for example, seems to mean "Huh? What was that again?" according to Marta Kutas, of the University of California, San Diego, and Steven Hillyard, who discovered this "double-take wave" last year. Their subjects saw a series of seven-word sentences flashed on screens at one-second intervals; the ERPs were recorded after each word. A normal recognition pattern of brain waves followed each word in such ordinary sentences as "It was his first day at school." But when the experimenters threw in a zinger such as "She took a drink from the radio," they saw a big negative peak after the last word. Kutas and Hillyard think they have caught the brain in the act of reprocessing, trying to make sense out of nonsense. N400 does not seem to appear in response to grammatical mistakes, but it does seem to show up when an incongruous mental picture of a radio substitutes for the word *radio*, suggesting the wave's relationship to the *meaning* rather than the sound or form of the word.

The double-take wave promises, among other uses, to improve the teaching of reading. "After all," Hillyard reasons, "when you're first learning to read, nearly all of the words look out of place and the the mind is continually backtracking to reanalyze the meaning.

Every thought, like any outside stimulus, also triggers a complex series of brain waves. For example, in the early 1970s, John Hanley, of the Brain Research Institute of UCLA, taught a chimpanzee to play ticktacktoe. While recording from the chimp's brain, Hanley noticed a wave that regularly appeared just as the ape was about to make a winning move. The wave was entirely different from waves that appeared before moves that lost the game.

A human subject's decision to move a given set of muscles also produces a wave in the region of the brain that controls those muscles. This wave can be averaged out of the EEG record just *before* the movement. A decision to wiggle all the fingers, for instance, makes waves in the supplemental motor area where this "simple" movement seems to be programmed and sent to the finger-control area for execution.


Many scientists have long assumed that the speaking of a word must be preceded by a program, not unlike a computer program, as reflected by brain waves that are word-specific. They have spent fruitless hours looking for such language waves, which would be the heart of any putative thought scanner. Now, at the University of Missouri, two researchers say they've found "motor template waves" associated with about 20 different syllables, although just how they are associated remains unknown. Neurophysiologist Donald York and speech pathologist Thomas Jensen are now trying to separate a sound-forming component from a meaning component by comparing the ERP differences between homonyms (words that sound the same but have different meanings -- for example *ate* and *eight*).

A Russian scientist has reportedly isolated specific waves for specific meanings, claiming to have found, for example, that waves for concepts such as *chair*, *desk*, and *table* are all overlapped by another wave that corresponds to the word *furniture*. Most Western scientists, however, remain skeptical about this work.

York and Jensen are interested not in building a thought recorder but in trying to help brain-damaged people speak and walk again. By comparing brain waves from injured brains to normal ones, they think they can give a patient feedback as his brain gets closer to or farther from the proper pattern during rehabilitation.

With a device called a phase-lock loop, paraplegics may one day be able to use their own brain waves to walk again. A phase-lock loop permits a satellite tracking antenna to lock into an orbiter's signal and pull telemetered information down to earth. Using a complex computer simulation, John Hanley has demonstrated the possibility of similarly locking into and tracking specific human brain waves. If it works, the tracking procedure would open the way to artificial joints, each with its own phase-lock loop tuned to the specific brain waves related to the movement of that joint. Using the signals received from the brain, the device would then either stimulate the appropriate muscles to perform the appropriate movements or activate an electric motor that would make a pulley perform the movements. Either method would bypass the severed nerve connections and enable a paralyzed person to move a limb, perhaps even to walk down the street, as other people do -- by willing it. Hanley remains hopeful in the pursuit of this dream, even though he has not yet been able to get funding for it.

Uninjured people might also benefit if Hanley is ever able to make his idea work. One of the problems in operating many complicated machines is work overload on the brain. In flying a modern jet plane, for example, the number of dials to be watched and interpreted, the number of controls to be moved and the number of factors to be balanced in mind comes close to the mind's limit for simultaneous processing. Many investigators of plane crashes think that accidents happen when this limit is exceeded.

The P300 surprise wave could be the key to connecting mind and machine, because changes in its latency shape and size may reflect how efficiently the brain is evaluating stimuli and making decisions. Emanuel Donchin and his co-workers at the University of Illinois at Champaign-Urbana have developed methods using the ERP for measuring mental workload in pilots. And several ERP researchers believe it might eventually be possible for an overworked pilot to fly a plane with only his brain.


Evoked-potential research is at the exciting, chaotic stage that comes when new instruments yield new discoveries almost every time they're switched on. The result is great confusion. While some short-latency ERPs translate into reliable clinical results, the waves that accompany more complex mental acts have raised more questions than they have answered.

Take the matter of lateralization. Hardly anyone doubts that the brain's two hemispheres process information differently. Using newly developed pattern-recognition programs, however, Alan Gevins, of the EEG Systems Laboratory at UCSF Medical School, recently discovered that instead of being confined exclusively to the left or right hemisphere, very simple numeric and spatial judgments actually involve many areas on both sides of the brain. Complex patterns of brain electricity associated with these judgments changed very rapidly; each sixth of a second, a totally different set of complex patterns was seen. These previously unseen patterns were not apparent in the average evoked potential.

Under the increasingly close scrutiny of brain scientists, some of the familiar waves, longtime guideposts in the murk, are beginning to dissolve. Good old P300, the surprise wave, often appears to overlap with one or more positive waves that probably have different functions. N100 may also be a composite wave, reflecting other mental processes besides selective attention, according to recent work both at UCLA and in Finland. The expectancy wave and several others have also been resolved into several components.

It is now unclear just what waves were evoked in many earlier experiments. "We are in a hell of a mess," says P300's discoverer Samuel Sutton. "We have opened up several cans of worms, and I think we had better watch them wriggle for a while." On the other hand, he adds, "The new findings really are telling us that things were always more complicated than we thought they were. We now have more adequate directions ... so we can begin to move toward a greater clarity."

To produce that clarity, several baffling questions must be answered:

*Where are the waves coming from?* The skull and scalp are a "smoky window," as Robert Thatcher, of the University of Maryland, Eastern Shore, put it. It's tempting to assume that the current is generated underneath the point where the signal is strongest, but this isn't always the case, and there's seldom any way of knowing how far down the current originates. Now two mathematical techniques -- current density source analysis and equipotential mapping analysis -- are being developed to try to "triangulate" the brain-wave generators from a rosette pattern of ten or more electrodes.

*What are the other cells saying?* Most of the brain's recordable electricity comes from 3 to 7 percent of its cells. These are the large pyramid-shaped cells with long dendrites and axons (nerve fibers) that make up most of the cortex and the pathways between many other structures. But most of the brain is composed of small cells with spherical dendritic fields whose contribution to brain waves is not yet fully understood.

*Where are the emotions?* No waveform reflections of the feelings or drives that color all sensory input have yet been identified by researchers.

*What is the role of magnetism in the brain?* Wherever electrons flow, whether through a copper wire or through a nerve cell, an electromagnetic field is born. According to W. Ross Adey, professor of physiology and surgery at Loma Linda University in California, electromagnetic fields applied to the brain can alter reaction times.

Among ERP researchers the hope of a "thought dictionary" has faded before the unforeseen complexities they have found in our heads. "I'd never say it's impossible," muses Helen Neville, "but I don't think we'll know nearly enough for at least twenty years." Alan Gevins is more skeptical: "I think you can have the same electrical pattern a dozen times without necessarily having the same thought twice."

The data on electromagnetism are grounds for speculation, though. Based on Adey's work, Robert Thatcher suggested placing around the skull a set of microwave generators that would transmit at an energy low enough not to cook the brain. The interference patterns produced as these beams interact with the brain's electromagnetic activity could then be built up by computer into a three-dimensional moving picture of mental processes. If we could learn to interpret that picture, we'd have a true thought scanner.

With remote monitors, such an instrument would be a spy's dream. Indeed, CIA spokespeople have admitted "following" ERP research, perhaps the way the agency followed LSD research in the 1950s. It's all too easy to imagine CIA-KGB brain-picking capers, scanning of Cabinet member's minds after closed-door meetings and "internal surveillance" of dissidents.

Of course, quite a bit can be done right now. A foolproof truth detector, for example, could be built today -- if it hasn't already been. Helen Neville demonstrated a few years ago that when you show a person a photo of someone he or she knows, an ERP called the "recognition wave" is markedly enhanced. This raises the possibility that if a prisoner you were interrogating wouldn't say whether Joe Blow was his accomplice, all you'd have to do is show him a picture of Joe, and the brain wave would tell you.


"We were amazed when we found the implications of our results," Neville remembers. "But then, the same wave is useful in studying preverbal infants, stroke patients, memory loss in Korsakoff's syndrome and so on." The CIA is also well aware of the implications and has funded scholarly work on the recognition wave.

Of course, the bright side of the thought scanner is just as plausible as the dark side. By allowing the faintest glimmer of intuition to be played back before it faded, a thought recorder might accelerate the process of problem solving in all endeavors, even brain research. And you might no longer need to rely on a daisy to tell you whether your beloved loves you or loves you not.

In *The Natural History of the Mind*, Gordon Rattray Taylor elegantly compared monitoring the EEG to listening to the noise from a party from outside the house. We know only that there is a gathering, he said; we don't know who's there or what kind of games they're playing. With the help of ERP technology, the door to our minds has begun to open. No matter how far the research takes us toward good or evil, its greatest reward will be the knowledge it gives us about that magnificently complex party in our heads, knowledge that will remain with us when the monitor is switched off.