One set of explanations is 'bottom up', i.e. perceptual. Amazeen & Turvey, 1996 suggested that people do not perceive weight but inertia (this is the dynamic touch hypothesis about the inertia tensor) and Zhu & Bingham (2011) have proposed the illusion is not the misperception of weight but the correct perception of throwability (I obviously quite like this one, and have discussed it here). Interestingly Zhu et al (2013) have since shown that the inertia tensor does not explain the throwing related SWI!
The second set of explanations is 'top down'. The basic hypothesis is that the sensorimotor system expects larger things to weigh more than smaller things, within a class of 'things'. This expectation has been learned over time via experience of the real world in which this is basically true. Large mugs weight more than small mugs, even if large mugs weigh less than small anvils.
There are two interesting papers that have looked at the top-down hypothesis.
Flanagan and Beltzner (2000) showed that when people lift identically weighted large and small objects 20 times each, their fingertip forces at lift-off rapidly adapt to the true weight. Specifically, people initially apply the wrong amount of force and have to make some quick corrections, but after experience go into the lift applying the correct force for the weight independent of size. The size-weight illusion persists, however! People still judge the smaller object to be heavier, even though their fingers seem to know the truth.
Flanagan, Bittner and Johansson (2008) then experimentally manipulated the expectation that weight increases with size. They trained participants extensively with a set of objects in which weight decreased as size increased. People again showed rapid sensorimotor learning (i.e. they learned to lift with the correct forces for the actual weight); again, there was a dissociation between perceptual experience and the action system's behaviour. More interestingly, the SWI eventually reversed! People began to experience larger objects from the training set as heavier and a smaller object as lighter, matching a violation of the expected weight (just not a sensorimotor expectation). They did not test for changes in the SWI on untrained objects, though, so it's not clear to what extent experience with this set of weird objects might generalise.
Taken together, we can see a few things:
- The SWI does not change when you repeatedly (20x each) lift two differently sized objects of equal weight. The expectation of 'larger = heavier' is repeatedly violated but this does not alter experience.
- The SWI does change when you repeatedly lift three differently sized objects of different weight, when the weight increases as size decreases. The expectation of 'larger = heavier' is repeatedly violated (in fact, reversed) and this does alter experience. (1050 lifts in a single session zeroed the illusion; 3320 over four days zeroed the illusion; 2640 lifts over 11 days produced the peak reversed SWI - no change after 11 days and it peaked at half the magnitude of the pre-training SWI).
- Fingertip lifting forces calibrate to the actual weight relatively quickly over repeated lifting (5-10 trials for the two same weight objects; 240 trials gets you most of the way adapted for the reversed size/weight objects). The expectation of 'larger = heavier' is repeatedly violated and this does alter lifting behaviour. Note that while this timescale is different than that of the illusion experience, it is still quite slow learning!
- The SWI is therefore not caused by a violation of a sensorimotor expectation, because they seem to vary independently and on different time scales.
Some thoughts
Another way to frame the SWI result is that two differently sized objects need to be different actual weights to have the same felt heaviness. There is a size-weight relation that maps equal felt heaviness (actually there are probably a bunch making up a surface; I don't know if anyone has mapped this out though).
For an object of a given size and weight, there is a set of objects of different sizes and different weights that feel equally heavy. This set defines the specifics of the illusion effect you show (i.e. which objects you pick in a SWI task with a given object). In the context of throwing, this set feels equally throwable (Zhu & Bingham, 2011).
So with respect to the points above; people come in with some size-weight relations in place - the expectations Flanagan talks about. We aren't perceiving weight, or size, but size-weight (or possibly inertia).
- When people repeatedly lift two differently sized but identically weighted object, these do not live in the same sets and so are judged as different. Constantly interacting with objects from different sets and judging them to be different does not make you start judging them as the same (even though fingertip forces calibrate to actual weight).
- When people repeatedly lift differently sized and differently weighted objects that do not fit an existing size-weight relation expectation (e.g. larger objects are lighter) this can drive a change; a recalibration of the relevant size-weight relation, or perhaps the creation of a new size-weight relation that is somehow specific to the objects that led to it.
I don't yet know how to ecologically characterise these size-weight relation expectations. If it's not an expectation but a calibrated perception of the throwing affordance, then this might be a useful framing to drive some experiments and predictions. I do know that novice throwers start out as poor judges of the throwing affordance (they choose objects on the wrong size-weight relation curve as throwable to a maximum distance) but after training this perception becomes calibrated and they select objects from the right relation (Zhu & Bingham, 2010). This accords with the reversed size-weight object study; experience can calibrate which size-weight relation curve you are on.
Experiments
I'm still thinking about how to progress on this, and this post is just me spending some time with some data and concepts.
I'd like to map out the size-weight equal heaviness space for a bunch of people. Give people objects that vary in size and weight and ask them to find the equally heavy one from a series of sizes, and do this for as much of the size-weight possibilities of the reference object as you can get away with, participant time wise. I'd expect to see individual variation; it might also be fun to test the same people multiple times to see how stable it all is.
If you found useful individual variation, you might be able to map out some size-weight combinations that live in the same or in different equal heaviness sets for some people and not others. Then you try training studies to see under what conditions the equal heaviness judgements could be made to change.
The other thing would be to connect this to throwing. Zhu & Bingham (2010) showed that people trained to throw also got better at perceiving the affordance, so long as they threw with vision and thus had access to all the necessary information about which objects went to maximum distance. What if you gave people manipulated feedback about distance using VR? Can you get people onto any old size-weight relation or are there limits? (Felice Bedford has a lot of nice work on calibrating reach spaces probing the limits of what you can make people do and why).
Summary
These are some very initial thoughts and I'd love feedback on them!
References
Nice overview – here’s a few things to add which may be of interest to the work you’re building toward
ReplyDelete1. In a paper which goes nicely with Flanagan’s 2008 one, I’ve shown that you can manipulate expectations to affect weight perception from one trial to the next. So give people a peek of something large before lifting, and they will judge it as feel less-heavy than when they get a peek of something small before lifting (size-weight illusion in a single object - Buckingham, G. & Goodale, M.A. (2010). Lifting without seeing: The role of vision in perceiving and acting upon the size-weight illusion. PLoS ONE, 5: e9709. doi: 10.1371/journal.pone.0009709.). Worth noting though that it’s only about half as strong as the full-strength size-weight illusion – perhaps isolating the contribution of expectations to this effect?
2. “…with respect to the points above; people come in with some size-weight relations in place - the expectations Flanagan talks about. We aren't perceiving weight, or size, but size-weight…” This seems intuitive but here is where my view breaks with that of Flanagan because it leads to two natural conclusions: (1) the SWI should vary in strength depending on the expected weight difference of the subsequently-lifted stimuli and (2) the SWI should only be felt within categories of objects (i.e., one of these ‘size-weight mappings’ that Flanagan talks about). But it’s been shown that you have equivalent SWI for big and small chunks of metal (which the lifter should expect to be v. different weights’ as you do for big and small chunks of polystyrene (which the lifter should expect to be very similar in weight) [Buckingham, G., & Goodale, M.A. (2013). Size matters: A single representation underlies our perceptions of heaviness in the size-weight illusion. PLoS ONE, 8: e54709. doi:10.1371/journal.pone.0054709.]. You also have a whopping big SWI across object categories – golf balls feel heavier than identically-weighted beach balls, even though you would expect the former to be heavier than the latter [Buckingham, G., & MacDonald, A. (2016). Weight of expectation: A size-weight illusion without contrasting prior expectations of heaviness. Quarterly Journal of Experimental Psychology, 69, 1831-1841. ]. My thought laid out in the review article is that the SWI reflects a single (presumably the mean) size-weight mapping of all stuff you interact with (your forces, by contrast, get access to all these nice discrete ‘smart’ categories). But other than saying this is a perceptual shortcut, it really muddies the ‘why’ of the SWI. Any ideas…?
3. There’s a third view about the cause of the SWI which is that it’s something to do with density. This has always been a bit vague, but a few nice recent papers have formalized this a bit. Megan Peters makes a pretty compelling case that the illusion is a consequence of how we combine expected density with size [Peters, M.A.K., Ma, W.J., & Shams, L. (2016). The Size-Weight Illusion is not anti-Bayesian after all: A unifying quantitative Bayesian account. PeerJ 4:e2124 doi:10.7717/peerj.2124.], based on some nice environmental statistics work showing that, on average, smaller things are actually denser than larger things in our environment [Peters, M.A.K., Balzer, J., & Shams, L. (2015). Smaller = denser, and the brain knows it: Natural statistics of object density shape weight expectations. PLoS ONE 10(3), e0119794.]. I’ve seen similar work at a conference showing that this sort of Bayesian combination of size information and density can explain the 2008 inverted size-weight illusion result as well. But it’s completely agnostic to the material weight illusion etc and also isn’t obviously action driven.
OK, one at a time (I'm trying not to rush responses while I chew on this :)
Delete1. Lifting without Seeing
3 identical weight but different sized objects.
Condition 1: Lift all objects by a handle under full vision. Visual information about size; haptic information about weight; no haptic information about size.
Result: standard SWI + standard adaptation of grip forces to the actual weight.
Condition 1: Lift middle object only by a handle after brief view of objects of different size. Visual information about size; haptic information about weight; no haptic information about size.
Result: SWI about half strength + NO adaptation of grip forces to the actual weight.
Thoughts
1. I'm not surprised you got the SWI in both conditions. Size information was only available visually and you had vision in both cases. Perhaps you could vary the strength of the condition 2 effect by varying the viewing time (either amount or proximity to the reach).
2. The intriguing result is the lack of fingertip force adaptation. This, as you note, means fingertip forces are not being corrected simply by haptics. I agree with your paper that it would interesting to track the dynamics with an extended study, more trials, see if it ever happened. As with (1) it's clear "amount/timing of visual information" is playing an interesting role.
One way to read Figure 1 is that relative to the full vision condition, the no-vision lifts kept a higher lift force and a slower rate of force change. Both are conservative strategic moves, as you note. What is the 'correct' lift force and lift force rate? The assumption seems to be the full vision lifts of the middle sized object are the correct ones. But where should they be, in order to safely lift the object?
Interesting study!
I also did a follow up study on this weird effect. Not quite the study you suggest (which would also be good to do) - more a attempt to figure out which visual cues were driving the info
ReplyDeletehttps://docs.google.com/viewer?a=v&pid=sites&srcid=ZGVmYXVsdGRvbWFpbnxvYmludGxhYnxneDoxMjZjNjg1NjE3ZTAyY2Q2
Got a few more unpublished ones along this lines as well, which mostly point toward the idea that when vision is gone we shift into a safe operating mode where we weight prior experience more heavily.