Expectations and the Size-Weight Illusion

The size-weight illusion (SWI) occurs when people are asked to judge the weights of two different sized but identically weighted objects. The smaller object is judged to be heavier. There are a variety of explanations for this illusion (see Buckingham, 2014 for a review). I'm going to be reviewing some papers on it as I develop some experiments connected to my throwing research.

One set of explanations is 'bottom up', i.e. perceptual. Amazeen & Turvey, 1996 suggested that people do not perceive weight but inertia (this is the dynamic touch hypothesis about the inertia tensor) and Zhu & Bingham (2011) have proposed the illusion is not the misperception of weight but the correct perception of throwability (I obviously quite like this one, and have discussed it here). Interestingly Zhu et al (2013) have since shown that the inertia tensor does not explain the throwing related SWI!

The second set of explanations is 'top down'. The basic hypothesis is that the sensorimotor system expects larger things to weigh more than smaller things, within a class of 'things'. This expectation has been learned over time via experience of the real world in which this is basically true. Large mugs weight more than small mugs, even if large mugs weigh less than small anvils.

There are two interesting papers that have looked at the top-down hypothesis.

Affordance-based control (Fajen 2005, 2007)

The most commonly studied tasks in the ecological approach involve the perceptual control of actions such as interception and steering. These models all involve perceiving some variable and moving so as to null the discrepancy between a current value and an ideal value. However, none of these approaches involve the perception of affordances; specifically, none of them address how people work to keep the required corrections possible, given their action capabilities. Fajen (2005, 2007) proposes affordance-based control, an ecological research framework that brings these questions to the fore and leads to the discovery of new, affordance based control strategies that account well for the data and solve the problems of simple information-based control models. 

My current sense is that Fajen is absolutely correct in his assessment of the problems and has done sterling work developing an ecological solution. What follows is a brief description of the problems and his solutions; in the future I will blog some thoughts as I work to align my throwing affordance work with this framework.

Your hand is not a perceptual ruler

Visual perception has a problem; it doesn't come with a ruler. 

Visual information is angular, and the main consequence of this is that the apparent size of something varies with how far away it is. This means you can't tell how big something actually is without more information. For example, the Sun and the Moon are radically different actual sizes, but because of the huge difference in how far away they are, they are almost exactly the same angular size; this is why solar eclipses work. (Iain Banks suggested in 'Transition' that solar eclipses on Earth would be a great time to look for aliens among us, because it's a staggering coincidence that they work out and they would make for great tourism :) 

This lack of absolute size information is a problem because we need to know how big things actually are in order to interact with them. When I reach to grasp my coffee cup, I need to open my hand up enough so that I don't hit it and knock it over. Now, I can actually do this; as my reach unfolds over time, my hand opens to a maximum aperture that's wide enough to go round the object I'm reaching for (e.g. Mon-Williams & Bingham, 2011). The system therefore does have access to some additional information it can use to convert the angular size to a metric size; this process is called calibration and people who study calibration are interested in what that extra information is.

The ecological approach to calibration (see anything on the topic by Geoff Bingham) doesn't treat this as a process of 'detect angular size, detect distance, combine and scale', of course. Calibration uses some information to tune up the perception of other information so that the latter is detected in the calibrated unit. The unit chosen will be task specific because calibration needs information and tasks only offer information about themselves. A commonly discussed unit (used for scaling the perception of long distances) is eye height, because there is information in the optic array for it and it provides a fairly functional ruler for measuring distances out beyond reach space. 

Linkenauger et al (2014) take a slightly different approach. They suggest that what the system needs is something it carries with it and that remains constant (not just constantly specified, as with eye height). They present some evidence that the dominant hand is perceived to be a fairly constant length when magnified, and suggest that this length is stored and used by the system to calibrate size perception in reach space. There are, let's say, a few problems with this paper. 

What are the units that perception measures the world in? Firestone vs Proffitt

Perception is an act of measurement, and, like all acts of measurement, it needs a scale in order to be useful. Think about placing something on your kitchen scales; all that actually happens is that the object presses on the scale and the scale registers that something has changed by some amount in response (the location of a tray, for example). In order to know what that change means, the change is presented to us on a calibrated scale (by moving a needle around to point at some number, for example). The needle always moves the same amount for a given weight but the resulting number can vary (you might have an imperial rather than metric kitchen scale, for example). Without the scale, you can say that one thing is heavier than another by noting that it moves the scale more (this is an ordinal evaluation) but you need the scale in order to say what the weight difference is (the metric evaluation).

Visual perception measures the world in terms of angles; objects subtend a certain number of visual angles that depends on their size, distance, etc. Your thumbnail held at arm's length is about 1° of visual angle. You can get ordinal information directly from angles (the fact that one thing is closer/bigger/etc) but you need a scale to get the metric information required to use vision to control action. For example, you need to perceive how big something actually is in useful units in order to scale your hand size appropriately when grasping it; relative size doesn't help. One of the fundamental questions in (visual) perception research is, therefore, what are the metric units that the perceptual systems use to scale their measurements?

Dennis Proffitt has been studying this question for a long time and is in favour of task-specific, body-scaled units. His evidence comes from studies in which people perceive their environments differently as a function of their ability to act on that environment. Probably the most well-known example is the study that showed people judge hills to be steeper when they are wearing a heavy backpack (Bhalla & Proffitt, 1999). The idea is that the backpack will make traversing that hill more difficult, and when the visual system measures the slope, it scales its measurement in line with this perceived effort. The hypothesis is that this is functional; it's a feature of the visual system that helps us plan appropriate actions. 

Perspectives on Psychological Science recently hosted a point-counterpoint debate on this topic. Firestone (2013) reviewed the literature on this type of action-scaling in perception and concluded that not only do the data not really support Proffitt's account, but that this account couldn't work even in principle. Proffitt (2013) rebutted Firestone's arguments and defended his view. I'm interested in this because Proffitt is at least a little ecological, and the basic idea he defends is one I would defend as well (although not in the form that he proposes). So who won?

Bojana Danilovic, the woman who sees the world upside down

I came across an utterly fascinating case study on Twitter the other day (via Mo Costandi; see this video too):

Rare brain condition leaves woman seeing world upside down
Bojana Danilovic has what you might call a unique worldview. Due to a rare condition, she sees everything upside down, all the time.

The 28-year-old Serbian council employee uses an upside down monitor at work and relaxes at home in front of an upside down television stacked on top of the normal one that the rest of her family watches.

"It may look incredible to other people but to me it's completely normal," Danilovic told local newspaper Blic.

"I was born that way. It's just the way I see the world."

Experts from Harvard University and the Massachusetts Institute of Technology have been consulted after local doctors were flummoxed by the extremely unusual condition.

They say she is suffering from a neurological syndrome called "spatial orientation phenomenon," Blic reports.

"They say my eyes see the images the right way up but my brain changes them," Danilovic said.

"But they don't really seem to know exactly how it happens, just that it does and where it happens in my brain.

"They told me they've seen the case histories of some people who write the way I see, but never someone quite like me."

Learning the affordances for maximum distance throwing

Over the last couple of posts, I have reviewed data that shows people can perceive which object they can, in fact, throw the farthest ahead of time by hefting the object. Both the size and the weight of the object affect people's judgements and the distance thrown; however, only weight affects the dynamics of throwing (release angle and velocity are unaffected by changes in size). This rules out the smart perceptual mechanism proposed by Bingham et al (1989), which proposed that both size and weight changes affect hefting and throwing the same way. So how are people perceiving this affordance?

Patient DF uses haptics, not intact visual perception-for-action to reach for objects

Before functional neuroimaging techniques like PET and fMRI became common, what we knew about which parts of the brain did what came from neuropsychology. This is the study of patients with specific injuries to the brain, and the basic logic of the field is that if you have a patient with a lesion in area A who can't do task 1, then area A is involved in performing task 1. It gets a little more complicated than this, as you search for double dissociations, etc, but this is essentially it.

A surprising amount of what we think we know about the brain comes from neuropsychology; famous case studies such as HM have informed theories of memory so that they include short and long term storage, which are separable, and so on. These case studies can have a profound effect on research; my favourite story, though, was about a memory researcher who had a skiing accident and temporarily developed retrograde amnesia - he couldn't remember anything except that there was this guy in Connecticut (HM) who couldn't remember things either!

I always enjoyed classes in neuropsychology; the case studies are always fascinating. But they are deeply limited in what they can actually tell us about the brain. First, they are typically single patient case studies, which restricts how general the conclusions are. Second, they are data from damaged brains; the fairly linear assumption that some localised function has been subtracted out is simply not true, and the damage will have had complex effects on distributed functional networks.Third, the damage is never straight-forward, because these almost all come from accidents or strokes (HM's surgery being a rare example of more detail being known). This has not stopped the field being very excited by these cases, though, and from basing a lot of theory on these patterns of deficits.

In movement research, the most famous neuropsychology case study is Patient DF She suffered bilateral damage along the ventral stream of visual processing (James et al, 2003). The effect was visual form agnosia: she is able to control her actions with respect to objects, but cannot describe or recognise these objects verbally. Crucially, her accident did not damage her parietal lobe; specifically, the dorsal stream of visual processing was left intact. These two streams are well defined anatomical pathways leading out of primary visual cortex, and were first described by Ungerleider & Mishkin, 1982). DF's pattern of deficits led Mel Goodale and David Milner (Goodale & Milner, 1992) to suggest functional roles for these streams. The ventral stream, they suggested, was for perception - things like object and scene recognition. The dorsal stream, in contrast, was for perception-for-action, and used visual information for the online control of action. This perception-action hypothesis has been hugely dominant in the field, and the theory rests heavily on DF's shoulders.

Recently, Thomas Schenk (2012a) published some data which claims to show that DF's visually guided reaching is not normal if she doesn't have access to haptic feedback about the object. His data suggests that the only reason she succeeds at reaching while failing judgment tasks is that haptic information is only normally available in the former case. If correct, this is actually quite a shot across the bow of the perception vs perception-for-action work; naturally Goodale and Milner don't buy it, and have published a reply to which Schenk has then replied.

An invitation
I like seeing these arguments happen in the literature; but to be honest, the time scale is too slow. Schenk publishes, then Milner et al get to reply and Schenk gets right of reply to that. They may or may not iterate again and it's always left as 'we agree to disagree'. But these critiques have answers, and I think a blog comment feed might be the right place to work through the various cycles of suggestions and rebuttals until the obviously wrong things have been weeded out. It would also provide a place for other interested parties to weigh in. So if Schenk, Milner and Goodale (and anyone else!) feel like using the comments for this post or another made to purpose to bang around ideas until an obvious experiment or analysis pops out, please feel free!

There's More to Us Than Our Brains - So What Does The Brain Do?

I'm not that interested in the brain.

It's hard to be this way in modern psychology. Cognitive neuroscience is where it's at, and I think I come off as  a bit of a Luddite when I try to convince people fMRI is a bit of a waste of time. Not caring much about the brain is certainly a sociological reason why ecological psychology doesn't get taken very seriously; we're just the crazy people who don't think there are mental representations, based on some work from the 50s-70s. Surely modern imaging has shown us the activity of mental representations? Clearly, the brain is the source of all behavior! Popular science writing on psychology is all cognitive and representational; most of the psychology blogging I come across is neuroscientific. What else could it be?

I've certainly spent a lot of time waving the flag against the infiltration of neuro-talk into places it doesn't yet belong; but to be honest, as I get older, I've begun to worry that I'm trying to be 'fair and balanced' in the sense Fox News is fair and balanced: relentlessly playing up one side to offset a perceived imbalance elsewhere. What I actually want to do is be actually fair and balanced: I want my own discussions about these issues to be internally balanced and coherent, giving credit where credit is actually due. I want to start teasing apart a few issues I've conflated over the years, so that my strong concerns about the relevance of fMRI  and cognitive neuroscience work stop getting swallowed up in a general dismissal of the brain's role in our lives. The brain is clearly interesting, but it's not representing, and if not that, what is it doing?

This post is therefore a first swing at integrating a lot of the things I've been blogging about for a while and doing so in a way that leaves a sensible role for the brain. I'm going to need some neuroscientists to talk to, though; I'd appreciate it if people could spread the word on this a little, because there are just some things I want to go a few rounds on with people who know what they're talking about. 

On being (briefly) unimanual - and worse, right handed!

Six weeks ago I broke my left wrist playing soccer. For the first two weeks I was in the temporary cast from the Accident & Emergency ward. It was only supposed to be a couple of days, but I had to travel to the US before my appointment to get a proper cast. I therefore had this unwieldy cast on while everything was sore, rendering me effectively one handed. The real cast was lighter and gave more support, and enabled me to use my arm more; the wrist was still constrained, though, and so I still had to rely on my (non-preferred) right hand for many tasks. I'm out of the cast as of Tuesday, although I need to rehabilitate the muscles back before I'll be 100%.

The experience has been quite interesting (in between being very boring). I've observed transfer of learning, recalibration of my arm as effector, and adaptation of numerous actions to the point where I can't really remember what it felt like to do them prior to the injury. I'm going to have to spend some time going back again now the cast has come off, and these things all touch on the topics I'm interested in covering on this blog.

A caveat: the plural of anecdote is not data. I'm not trying to convince anyone of my interpretations here, just thinking about my experience through the lens of my theoretical understanding of perception/action.