A research team from the Hebrew University of Jerusalem recently demonstrated that the same part of the visual cortex activated in sighted individuals when reading is also activated in blind patients who use sounds to “read”. The specific area of the brain in question is a patch of left ventral visual cortex located lateral to the mid-portion of the left fusiform gyrus, referred to as the “visual word form area” (VWFA). Significant prior research has shown the VWFA to be specialized for the visual representation of letters, in addition to demonstrating a selective preference for letters over other visual stimuli. The Israeli-based research team showed that eight subjects, blind from birth, specifically and selectively activated the VWFA during the processing of letter “soundscapes” using a visual-to-auditory sensory substitution device (SSD) (see www.seeingwithsound.com for description of device).There's lots of research like this. People are excited by mirror neurons because they are cells in motor cortex that are activated by both motor activity and perception of that motor activity. It's incredible, people cry - cells in a part of the brain that we said 30 years ago does one thing seem to also do another thing. How could this be??
I would like to propose a simple hypothesis to explain these incredible results and that is that we have been labeling the brain incorrectly for a long time. The data telling us this has been around for a long time too and continues to roll in, but for some reason we still think the old labels are important enough to hold onto. It's time to let go.
I'm going to go out on a limb and say it's a bit more complicated than this |
An informational hypothesis
I don't know exactly how this would play out, but I start this the place I start everything: by thinking about information. Sabrina has developed a draft taxonomy of information types which we are currently elaborating for some upcoming papers. The thing that drives the taxonomy is what kind of behaviour the information is capable of supporting. For the online control of action, the information probably needs to be specifying information of the kind Gibson identified and studied. Specifying information is information the organism uses so as to organise it's behaviour with respect to the thing that created that information (so moving so as to catch a fly ball uses information specific to your ongoing attempt to intercept it). But we can also use information to organise our behaviour in ways that have nothing to do with what created the information. The motor act of speaking creates information, but we don't use that information to perceive the details of the motor act of someone speaking; we use it to engage a person in conversation.
Information is all the same kind of thing. It's all just structure in energy arrays. But these variables vary in their stability and what we use them for, and presumably the brain is a major player in connecting information to behaviour. One way to think about the organisation of the brain is therefore to consider the nature of the information that it is working with. Why is visual cortex organised the way it is? Not because it's visual cortex, but because the kind of information that comes via vision has certain properties and is therefore used to support some but not other kinds of behaviour. We are not our brains - our brains reflect what we do.
From this perspective, the fact that 'visual' cortex can be involved in a reading task even when the reading happens via sound is not magical or all that surprising. One of Gibson's insights is that modalities are irrelevant; information is primary and you can often present information about the same thing through different systems (see my last post on the size-weight illusion induced via echlocation, or this paper by me on the haptic perception of the information for relative phase). If you present information that has certain characteristics, then brain regions organised to best handle those characteristics will presumably get involved. The 'visual word form area' is therefore nothing of the sort and needs to be relabeled with respect to the form of the information involved in the tasks this area is involved in.
This is, of course, very speculative and very early on. One obvious counter here is that the people in this study were blind since birth; why did the occipital lobe still get specialised for handling this kind of information? I have no idea, although I would hazard a guess there are genetic and developmental pressures that bias the development of the brain, and reconsidering those as biases for handling information rather than modalities is well worth doing.
Regardless, our ecological, embodied cognition has many implications for neuroscience (hence our research topic at Frontiers) and for what we think the brain is doing with all that energy it's using. We're already proposed (with Eric Charles) that the next big job for embodied cognition is to provide a new language for talking about the brain so that neuroscience can start asking better questions. Results like this current study and all the many many other studies like it prove that we need this new language soon, so we can all stop being amazed by the fact that nature doesn't know anything about our current labels for all the bits of the brain. Instead, we might end up with labels that are more closely aligned to the the system actually works.
Great post - I couldn't agree more.
ReplyDeleteI wanted to point out a highly relevant recent open access paper that people interested in this topic might want to check out. It is a great inspiration for me in thinking about structure-function relationships in the brain and how we might use ecological principles to understand them: http://onlinelibrary.wiley.com/doi/10.1111/phc3.12155/full
Cheers,
Matthieu
matthieumdewit.wordpress.com
we don't use [the information content of speech] to perceive the details of the motor act of someone speaking, we use it to engage a person in conversation
ReplyDeleteThe purpose of communication in any mode presumably is to affect the behavior - current or future - of the recipient. Stretching the concept of "communication" a bit (although arguably no more than stretching the concept of "information"), one can view a surface as communicating via the structure of reflected light. In linguistic communication, the information conveyed at the level of the sensory stimulation itself is clearly not "about" the entity that caused the stimulation (although at a higher level it may be; eg, an utterance can refer to the vocal chords). But it isn't clear to me that the same can't be said about catching a fly-ball. Context (eg, the game is baseball rather than football) provides the would-be catcher information about the ball independently of information communicated via the structure of light reflected by its surface. The immediate perceptual information seems mostly - if not entirely - about the ball's trajectory rather than about the ball's surface.
You and Sabrina emphasize that language is conventional rather than lawful. However, an argument can be made that behavior in response to linguistic input is much the same as behavior in response to other types of sensory stimulation. In either case, via a learning process an organism develops context-dependent behavioral dispositions each of which is triggered by a sensory stimulation/context pair that is similar to the one in which the disposition was learned. Then behavioral inconsistency is due to variations in stimulation and/or context rather than whimsy (admittedly the perspective of a strict determinist). Which is to say that behavior due to linguistic input is lawful notwithstanding that we can't formulate those laws.
This view suggests that another organizational principle for the brain might be action types. This fits well with linguistic communication in which mode presumably plays at most a contextual role (eg, tone of voice). In a given context, the response to "Let's meet at noon" presumably would trigger the same disposition whether conveyed via voice-mail or e-mail. In fact, this organizational principle is implicit in this quote from the post:
the kind of information that comes via vision has certain properties and is therefore used to support some but not other kinds of behaviour
I would agree with
DeleteYou and Sabrina emphasize that language is conventional rather than lawful. However, an argument can be made that behavior in response to linguistic input is much the same as behavior in response to other types of sensory stimulation. In either case, via a learning process an organism develops context-dependent behavioral dispositions each of which is triggered by a sensory stimulation/context pair that is similar to the one in which the disposition was learned.
but not that language might still be lawful. To the organism, interacting with information is all just coupling structure to behaviour. Some structure is law based and can therefore support things like action control. Some structure is conventionally used and can therefore only support things like language. This will be ok, I think :)
It appears that our disconnect has to do with what we think follows from the structure of sensory input being "law based".
ReplyDeleteIn general, an organism presumably doesn't know the laws that govern the formation of such structures. So, I assume that what follows from a structure's being law based is only that in a sufficiently similar environment the structure will be sufficiently similar so that a learned responsive behavior will still be appropriate to the organisms objectives. Eg, if one has learned from previous experience that a certain structure (and hence, a certain pattern of neural activity) has been compatible with grasping an object, then that structure (or neural activity pattern) is a reliable indicator that a graspable object is present. As you say in your reply, it's "just coupling structure to behavior", and doing so in a way that is reliably repeatable in a given context.
I see responding to language as working the same way. Clearly, the correspondence between the linguistic features of an utterance - the particular sounds that correspond to word meanings, the grammatical structure of sentences formed from those words, the meaning of a sentence, etc - are conventional. But again, from the conversational participants' perspectives, the issue is reliable repeatability of response. If a speaker utters a sentence in a context in which the speaker assumes the hearer has learned a response to that utterance, will the hearer reliably produce that response? And I claim that if the speaker's assumption is correct, the answer must be "yes". I assume that the mechanism by which the response is generated is a context-dependent behavioral disposition, a neural structure the operation of which is deterministic. And that is the sense in which I mean that the response to language is law based. Although again, neither speaker nor hearer (nor currently anyone else) knows what the laws are.)
You say you agree with the passage quoted in your reply- which I see as merely an abbreviated version of the preceding paragraphs. So, is there more to the claim that language can't be law based than a misunderstanding of what I mean by that phrase?
In her example of a person receiving directions but ignoring them, Sabrina attributes the deviant behavior to whimsey. But that ignores a component of what I mean by "context", viz, the hearer's overall biological state. There are numerous reasons the person might not follow the directions: misunderstanding them, feeling that the direction giver isn't trustworthy, suddenly remembering a forgotten appointment - all resulting in contextual differences. And contextual differences can also alter the behavior of the would-be fly ball catcher. But in either case, for a given stimulation in a given context, mustn't the response be the one learned, ie, lawfully determined? If not, how do you account for the difference?