An (Draft) Ecological Approach to Hallucinating

Sabrina and I are planning our next papers, and in typical style she's been thinking about how to tackle a hard problem - this time, hallucinations. These are one of those go-to topics for representational people, because hallucinations by definition are not based in the detection of perceptual information. They are a kind of perceptual experience, however, and so seem to be a good candidate for identifying how perceptual experience is constructed internally. 

We've never let a little thing like a topic being hard stop us before, so it looks like this is next on our list. The goal is to lay out an ecological analysis and see where we end up. We are going to build on the work we did in the Ecological Representations paper, in which we considered how to understand (at least some) neural activity as the selection of consequent neural actions (pg 243 and on). This is the first of a few papers we have in mind where we apply our ecological analyses as worked examples to interesting topics (verbal instruction in coaching is on my mind too, as are cells making blood vessels). 

In this post, I'm going to do my usual thinking-out-loud about my notes from our first chat; all conclusions are works in progress! At this point, I am just assembling the resources our ecological approach provides us, and lining them up in their proper places so we can use them rigorously.

1. Verb Your Nouns

Taking an ecological approach to a topic that has already been framed representationally means starting from the beginning and reframing the topic. The first step is to verb your nouns, because we study verbs. In this context, that means people do not have hallucinations, nor are we studying hallucinations. People hallucinate, and we study hallucinating. 

This immediately stops us from asking 'where is the hallucination?', which immediately gets you into representational territory. Instead, we now ask 'how does hallucinating play out?'. What brings it on? What sustains it? What disrupts it? What ends it? What are the contexts in which hallucinating occurs, and what are the effectivities required of a person to hallucinate in those contexts (vs not)?

In terms of how hallucinating plays out, one obvious aspect is their perceptual nature. One can hallucinate visually, auditorily, or some combination. Auditory hallucinating is most common, around twice as common as visual. Other modalities are then less common, as far as I know. Smell hallucinations can be a symptom post-stroke, and opioid use can create tactile hallucinations (e.g. ants crawling on your skin; note that this has a physiological mechanism involving release of histamines so it may not count as hallucinating). Phantom limb syndrome might count as kinaesthetic hallucinating, although I don't know that anyone talks about it as such. So one of the sources of constraint on how hallucinating unfolds will be how these perceptual experiences typically play out, and thus how our nervous systems work to support that perception. This in turn is dictated by things such as the nature of the perceptual medium, how densely packed it is, etc. 

Importantly, however, hallucinating is not perception. Perception requires information and by definition, hallucinating is what happens when you have an experience that isn't caused by information. However, understanding information is still going to be an important part of this analysis, because the ecological analysis of the nervous system is explicitly grounded in information use. In addition, hallucinating happens in the context of normal perceiving, and so there is a critical role of that perception to address. So we need to take a small detour to lay out some terms that will come in handy later.

2. Get Your Information Story Right

We have been working within an information framework we developed over several papers (specifically, Golonka, 2015; Golonka & Wilson, 2019). This framework proceeds as follows.

All information is specific to the dynamical property that is being lawfully projected into an ambient energy array. However, organisms are not always constrained to use that information as information for that property. This is because organisms have to learn how to use information, and we learn what works. Sometimes we learn to use that information as information for something defined by a sufficiently stable convention, rather than a law. This analysis gives us information, and two classes of use for that information: law-based use of information and the convention-based use of information. 

The law-based use of information can support both action selection and action control. For example, optical information specific to the catchability of a fly ball can be used to create the details of the actual catch, or to identify whether the ball is catchable or not. The evidence for the law-based use of information for action control is that the spatiotemporal structure of the information shows up in the spatiotemporal structure of the movement as it unfolds over space and time. 

The convention-based use of information can only support action selection. For example, if I say 'pick up the red cup' to an English speaker, they can use the acoustic information in that speech to choose the red cup over other possible targets, but cannot use it to actually perform the reach-to-grasp action. 

Hallucinating cannot, by definition, entail the law-based use of information, because there is nothing real to create that information. This means that regardless of how realistic these are, people will not be able to couple their actions to the dynamics of the hallucination. For example, if they are hallucinating something coming at them, they will be unable to time their evasion via time-to-contact information. 

Hallucinating also cannot, by definition, entail the convention-based use of information, because again, there is nothing real creating that information. However, we (Golonka & Wilson, 2019) used the possibility of convention-based use of information to ground an ecological approach to 'representation-hungry cognition' by identifying how that use might play out neurally. This is where we need to go next, to explain how a person would be able to select actions while hallucinating; for example, they could choose to duck an incoming object. The details of that duck would effectively be miming, however.

(Notice here how verbing helps our analysis. To a noun researcher, the fact that someone ducked in response to a hallucination would be evidence of the apparent reality of that hallucination to the person. To a verb researcher, the emphasis on process allows us to predict that ducking while perceiving will proceed differently than ducking while hallucinating. We can measure behaviour and learn about the structure of hallucinating! 

I will also, at this point, just make a note that the information-based perception of the context in which hallucinating happens will be something we will need to get into later. What is the relationship between something you are hallucinating vs something you are perceiving at the same time? Can they influence each other?). 

3. Get the Neural Story Right

Both the law-based and convention-based use of information are supported by the nervous system. Organisms learn to connect information to behaviour, and the nervous system is the sub-system of the perceiving-acting system that changes on the right timescale to support forming/implementing these connections. However, we expect different types of activity from the nervous system in the two cases, with important consequences for the behaviours that activity can support. Golonka & Wilson (2019) worked this out in more detail.

The law-based use of information to control actions requires the nervous system to designate the information to the action systems, and so it should preserve the spatiotemporal structure of the information. This is what van der Meer et al find when they study how nervous systems respond during the perception of a looming stimulus and the development of the avoidance response. This kind of activity is stable and robust for as long as the information is present and detected (+/- some calibration) but we predict it cannot be generated to this quality in the absence of the information. 

The convention-based use of information, in contrast, will not create nor require activity of this kind. The reason is that for this use, the spatiotemporal details of the information are not what's important; rather, it's whether the information is present of absent. This has two consequences. First, neural activity supporting convention-based use of information cannot support action control (which requires access to the spatiotemporal details). More positively, however, at least some of this type of activity should be able to be generated without the information being present, because the details of the information are not the point and because the activity will have shorter timescales. 

Most relevantly here, we noted that while there is typically a precipitating external event for this type of neural activity, there could then easily be precipitated neural activity of this type between the perceived information and the resulting action selection. Neural activity might select neural activity as readily as it can select behavioural activity. We also speculated that this kind of neural activity, once sufficiently practiced, could become relatively decoupled from precipitating perceptual events and be used as components for 'representation-hungry' cognition. 

The implications for hallucinating are that it is supported by neural activity that can only support action selection. This makes a lot of predictions about what hallucinating can be like and what it cannot be like, and I think there are many ecological questions to ask about how people can and cannot interact with hallucinations. They all require a different kind of testing though, focused on movement kinematic, etc. There is also a lot of existing work on the phenomenology of hallucinating, although of course a lot of it only had access to a non-ecological vocabulary. I think this domain will be a really interesting testing ground to find the limits of our ecological neural representations. 

Interim Thoughts

A few predictions and questions pop out, which I'll list here for reference

Predictions

We would expect auditory hallucinating to be the most common (which they are) because of the fact auditory information is primarily only used for action selection, vs action control. Auditory looming is a thing, of course, and people have done sensory substitution studies providing auditory information with a more continuous structure. But in general, audition supports action selection and so we are really good at action selection, convention-based activity involving sound and the neural activity that comes with it. We would also expect auditory hallucinating to be more coherent and robust that visual hallucinating, so the same reason. 

Questions

So many questions! Many of these are of course already lively topics; I do think that our ecological framing will change the details of how we ask these, though. 

• Is there individual variation in inner speech quality, and does this track with individual variation in auditory hallucinating? What about imagery variation and visual hallucinating?
• Is hallucinating precipitated by anything? Is it instead abnormal attention directed to default mode style neural activity? 
• How does hallucinating vary across the various 'modalities'? What can and can't you do with respect to various kinds of hallucinations? At what point does the ambiguity that is a hallucination get revealed by it's relationship to the rest of the perceptual array? 
• What about developmental/age differences? Cross-cultural differences? Given the fact it has to all be based in learned, convention based neural activity, there must be intriguing differences. 

The next thing to is to do a bunch of reading of the relevant literatures, scoping out what's known so far.