Local or Global Coherence?

In my last post I wrote about Oliver Sacks’ essay, “Stereo Sue” in his book The Minds Eye where he told of a woman who first developed stereoscopic vision in her 40s. In the following essay, “The Persistence of Vision”, he talks about his own loss of stereopsis as a result of the growth of a melanoma behind his right eye. First, as a result of the tumor, and then as a result of the treatment to kill the tumor, he lost vision in the eye – slowly, over a period of four years.

Besides this being scary (Who knew you could get melanoma behind your eyeball?), it turned out to be fantastically informative about the complex way in which vision works. Sacks, ever the scientist, kept a careful journal, drawing what he saw and describing how his brain interpreted the changing data stream. I think that there’s a lot to learn from his experience, but two points struck me as telling us something interesting about how the brain works: filling in, and reinterpreting, both saying something about the way our brain build the virtual world we live in. (See my earlier post, “Axioms”.)

At one point in his treatment, he had lost vision in the foveal part of the eye – the highly accurate cluster of retinal cells that we do most of our direct looking at things with. He retained peripheral vision (and has interesting things to say about that), but developed a scotoma – a large blind spot in the middle of the vision of his right eye. It’s well known that the brain “fills in” the blind spot – up to a point. (You can try this yourself with your own smaller and normal blind spot.) I read about this in a psych book a couple of decades ago. I recall a story of a man with a scotoma looking at a person sitting on a couch. When he turned to put the person’s head in the hole in his vision the person’s head disappeared – but an image of the patterned wallpaper behind the couch filled in the space where the head should have been. Sacks explored what filled in and what didn’t.

“It was easy to fill in a simple repetitive pattern – I started with the carpet in my office – though a pattern took a bit longer than a color, perhaps needing ten or fifteen second to duplicate. It would fill in from the edges, like ice crystallizing on a pond…I found that movement could be also filled in to some extent. If I looked at the Hudson River, slowly swirling or rippling with small waves, these too were reproduced…But there were strict limits. I could not simulate a face, a person, a complex object.”

Interestingly, he found that if he cut off his foot with his blind spot but wiggled it,

“the stump seemed to grow a sort of translucent pink extension with a ghostly protoplasmic halo around it. As I continued wiggling my toes, this took on a more definite form, until, after a minute or so, I had a complete phantom foot…which seemed to move with the movements I was making.”

This, and other examples he cites, seem to indicate that the brain “recruits resources” – hunting for things that will bring together the various bits of information it has to make a coherent sense.

But a second observation leads in a different direction. Once he lost peripheral vision in his right eye he lost the ability to see anything to the right side of his nose and developed what neurologists call “unilateral neglect.”

“…whatever comes into my visual field from that side is unexpected and startling. I cannot overcome the sense of bewilderment, even shock, when people or objects appear suddenly from my right. A massive slice of space no longer exists for me…”

Particularly surprising is that his brain seems to treat objects that move into that space as vanishing – even people.

“Kate [his wife] and I finished our walk and headed back to my office, I walked ahead and got into the elevator – but Kate had vanished. I presumed she was talking to the doorman or checking the mail, and waited for her to catch up. Then a voice to my right – her voice – said, ‘What are we waiting for?’ I was dumbfounded – not just that I had failed to see her to my right, but that I had even failed to imagine here being there, because ‘there’ did not exist for me.”

This (and other such examples) are fascinating because he knows at some level that he cannot see on the right. Yet the changing in accustomed patterns of sensory data seems to undo a component of his sensory coherence. He cites a colleague (M.-M. Mesulam) as saying, “Patients with unilateral neglect behave not only as if nothing were actually happening in the [blind side] hemisphere, but also as if nothing of importance could be expected to occur there.”

This, and other examples seems to suggest that the brain builds a local coherence, but can neglect large blocks of data that it knows perfectly well in responding to an immediate environment.

One of the most controversial theoretical issues in science education today is the appropriate scale at which to expect coherence in student thinking. I remember a conversation at an international science education conference nearly two decades ago with a friend and colleague about this issue. He took the point of view, “People live in the world. They have to have a coherent view of it. If they don’t, they would go crazy. When students give answers that seem bizarre to us, it’s just that they have a different coherence from ours. As science educators, it’s our job to find out how to one might look at the world coherently and see it the way they do.”

This is an extreme view (and he may have moderated it by now), but many education researchers assume – sometimes tacitly – that students are fully coherent in their thinking about something, even if we, from our perspective, don’t see what the coherence is that they are seeing.

If something as fundamental and as evolutionarily vital to survival as our interpretation of the spatial world around us through vision shows only local but limited coherence, I suggest that we can’t take coherence of brain function for granted. Rather, we have to treat it as an empirical issue, to be explored under varying contexts and conditions.

The Joy of Sacks -- Recruiting Resources

While trolling the library the other day for a stack of new trashy mystery novels (my relaxation of choice these days), I was delighted to find a new case study collection by Oliver Sacks, The Mind’s Eye (Borzoi, 2010). Sacks is a neurologist who first became famous with the publication of a collection similar to this, The Man Who Mistook his Wife for a Hat (Touchstone, 1998). If you have any interest at all in how the mind works, I strongly recommend Sack’s work. These case studies tell stories of individuals who suffer some brain injury and as a result lose a piece of what they usually do automatically. These dissociations of functions we normally see as unitary gives real insight the functioning of the brain. The title case study of the 1998 book is about a music professor who lost the ability to pull together bits of visual data into recognized objects. He really did, at one time, reach over to grab his wife’s head because the curve of her scalp matched the curve of his hat.

What I find particularly meaningful – and hopeful – about Sacks’ case studies is that he often picks ones where the people who have suffered a brain injury and a resulting loss nonetheless do not fall into despair, as many must, but find a way to go forward with their lives. He tells of an athletic young woman in TMWMHWFAH who lost her ability to interpret the signals from her muscles telling her about the orientation of her body and limbs (her proprioception) and almost became effectively paralyzed as a result – she would flail around knocking into things. But she worked to learn to replace her muscular perceptions by visual ones and regained much of her ability to function normally. A painter (in TMW…) lost his color vision and, instead of giving up painting, developed a new (and, it turns out, more popular) style. A novelist (in ME) had a stroke and lost his ability to read – but not to write (alexia sine agraphia)! He could write but couldn’t read what he had written to revise or build coherency. He learned to dictate and have people read to him and he has continued to maintain his output (of mystery novels).

I know that these are selected stories and that many people’s illnesses and injuries are too severe to give them a path back to a life of satisfaction and joy, but as I get older and the probability of something bad happening increases, I get satisfaction in knowing that at least some people manage to get over even severe bumps in the road.

But I digress. (Hey! I’m an academic! That’s what I do. Get over it.) The point of this screed is not for me to get maudlin about aging; it’s to see what points these stories make about how the brain works. I’ve got two for now.

The story of the novelist makes me appreciate the role of external objects and symbology as components of our everyday cognition. (cf. some parts of activity theory)  My best example of this is building a talk in PowerPoint. I know how to do lots of things with PowerPoint – but it’s me and PowerPoint that know it together; I don’t know it by myself.  Occasionally someone will ask me how to do something in PPT. I might know, but not be able to tell them. What I know is which chain of items to choose from a series of menus when they are presented to me – but I don’t have the menus themselves stored in memory. When I am deriving something in physics or math that requires far more steps than the 7 ± 2 that fits easily in my working memory, it looks like I have much more in play since I write my equations down and by scanning up and down the page, I can quickly swap in and out what I need without having to reconstruct those that I can’t keep active. Writing a novel (or an academic paper) certainly must be dramatically simplified by our ability to have written stuff down and quickly look back at it.

But the case study in The Mind’s Eye that really excited my and is responsible for my pulling out the computer on a Saturday night is “Stereo Sue”. In this, Sacks tells of a woman born with strabismus (cross eyes). Despite a number of operations when she was a child, she never achieved true stereoscopic vision. Now lots of animals get along without stereoscopic vision. In fact most prey animals such as antelopes and deer (also horses) have eyes that are on the side of their head so they can scan closer to a full circle more quickly. Presumably they have much more awareness of what’s to the side and behind them, but we seem to be descended from tree-living animals that had much more need of depth perception to negotiate a complex 3-D environment.

Now Sue was able to get along just fine using the other parts of her vision that allows us to place ourselves in space, particularly motional parallax – how things appear to change when you move. She could drive and even play softball. But in her 40’s she began to develop problems with her vision. She found a careful and flexibly thinking doctor who discovered that not only had she been cross-eyed, but the vertical alignment of her eyes was off. By adding prisms to her glasses, the doctor was able to correct this. And, with a series of exercises, Sue was able to initiate stereoscopic vision.

I wish I could repeat the whole story here, but this is too long and I want to (eventually) make a point. Go read it for yourself. I actually have two more points to make, one about the practice of science, one about the brain.

First, for decades, Sue’s doctors told her that because she had missed the “critical period” as an infant, her brain would never be able to learn to interpret the data from her two eyes stereoscopically. Of course that’s the result that Hubel & Wiesel found with kittens (and won the Nobel prize for). It’s a great result, but it does not obviously carry over to other species (ferrets, for example) and the human brain seems way more plastic – at least potentially – than we often assume. There are stabilities, but we may be making a serious error when we ignore or under-appreciate the possibility of dynamic reorganization.

My second point about the brain is this. After Sue had pretty fully developed her stereoscopic vision, Sacks tested her with an interesting example. This was text presented in a stereoscopic viewer – strings of unrelated words. When viewed monocularly, it looked just like text on a page – flat. But viewed stereoscopically, it became obvious that the words were on different levels – as if they were printed on stacked panes of glass.

When Sue first looked at this she saw the text as flat. When Sacks pointed out the 3-D aspect, she said, “Oh, now I see” and was able to see it just fine. Sacks says

“Given enough time, Sue might have been able to see all seven levels on her own, but such “top-down” factors – knowing or having and idea of what one should see – are crucial in many aspects of perception.”

Later he says

“If a stereo photograph is flashed on a screen for as little as twenty milliseconds, a person with normal stereoscopy can perceive some stereo depth straightaway.  But what one sees in a flash is not the full depth; the perception of this requires several seconds, even minutes, in which the picture seems to deepen as one continues to gaze as it – it is as if the stereo system takes a certain time to warm up…. The underlying cause for this is unknown, though it has been suggested that it entails the recruitment of additional binocular cells in the visual cortex.”

This just strikes me as so similar to what I see with my students operating at a much higher level of brain activity and complexity. I find it delightful to see framing and the need to recruit other resources before something clicks into making sense occurring at the direct perceptual level.