Saturday 20 December 2014

The visual field and ideas about consciousness

First post for a while and it's a long one. Starts off talking about a recent New Scientist article, its implications for the nature of perception and consciousness and then goes off into waffle about politics and protest.

There's a particularly interesting article by Laura Spinney in an October edition of New Scientist which I've only just got around to reading (as I lay in my freezing bed this morning, bemoaning how little time and energy I currently have for the intellectual pursuits I love). On the website the article is behind a pay wall, so I've copied it below in the hope that if readers find it really fascinating they'll take out a subscription, or maybe just buy a copy of NS every so often.

The article is nominally about "inversion-goggle" experiments, and how we construct our visual field. I'd read about these experiments while studying psychology as a teenager. In the classical studies, an experimenter donned goggles which inverted his (yes, his) vision - both horizontally and vertically. After several days of confusion and fraught interaction with the world, the experimenter typically would report a flipping of the visual field such that suddenly the world was the right way up again. All's well (until he removes the googles, and it takes his brain another few days to reinvert his vision).

Contemporary versions of this experiment, informed by developments in the philosophy of mind and cognition, have provoked different interpretations. The experimenters (still he, as far as I could tell from this article) now find that a single flip of the visual field never occurs; rather, there is a gradual but incremental adaptation to the new way that one must interact with the world the eyes describe. Certain functions adapt quicker than others and there is a fragmenting of vision such that some objects and processes appear "right-way" up at the same time as others are inverted.

This has provoked some commentary about the nature of consciousness and the problems with the Cartesian model of a single internal representation of the world which the subject uses to navigate their way around the real world outside them.

Someone unmentioned in the article but who immediately sprang to mind for me, is Daniel Dennett who developed a fragmentary, process-based model of consciousness in Consciousness Explained. In this text Dennett uses many scientific studies to build a theory that consciousness is effectively textual (that's my wording, he never says that); our interpretation of the world is a construct based around what we expect to be the case based on the best evidence to hand at the time - and one which is constantly being rewritten. For Dennett there is no Cartesian theatre of conscious awareness (and no corresponding subconscious) - rather we are constantly in the process of adding tiny fragments of information into a bigger picture which is never complete. In one example, he cites a study where two consecutively blinking lights centimetres apart appear, to the observer, to be one light moving between different points; such movement never happens, but the brain constructs it because it expects such a pattern of information to imply movement. Dennett asks, what would happen if we paused the subject's brain inbetween the two flashes? Would they be experiencing the light half-way between the two points, as they later remember having done? No: that movement was only retrospectively written into the experience, there was never a qualia of that movement. Dozens of other examples bolster his (very convincing) argument for such a revisionary textual model of consciousness over the traditional panoramic one where the subjective world is presented as a photograph in which the subject can wonder, where facts and detail remain the same regardless of the focus of attention.*

*It is common knowledge in memory studies now that memory is not a matter of accessing stable recordings of events encoded in our brains, but rather a reimagining every time the memory is accessed; we effectively rewrite the memory each time we think it, reconstructing it along with the associations and interpretations it comes to have as we grow and learn more about the world and ourselves.

Spinney goes on to describe a theory oc perception called "enactivism". This holds that "thinking and feeling arise in the dynamic interaction between an organism and its environment" and therefore "your subjective experience of being is created by your awareness of the myriad different ways your self interacts with the world as you move around an explore it." This has striking implications for consciousness because it means that the senses, and the kind of body, through which we interact with the world, determine the structures and patterns of our thought. This is important because it suggests - in fact requires - that thought has to be located subjectively within the parameters of particularity. Most interestingly, it means that other lifeforms will have different structures of thought which are to varying degrees unlike our own. Of course, life on earth is as far as we know all variations on a single theme which developed millions of years ago, sharing an environment who's nature has certain strict boundaries; but life, or thought, is not necessarily constrained to earth or the kinds of environment it provides; again, not mentioned in the article is one of my recurrent ponderings, that any kind of computer-based intelligence, even though designed by us, would evolve ways of thinking that were unimaginable to us; based on their experience, their specific modes of embodiment, and their relationship with the world (which included their relationship with humans, whatever format that might take). But the main conclusion, that consciousness cannot be reduced to a single process or substance, is well-supported by the argument and evidence and one I in particular find easy to accept.

Before I post the article, there is something related to all this which I think is important to bring forward. In these times of tension, violence, repression and intrastate conflict, there has very much been a marking of lines and separation in sides: who supports the police in their actions to "protect" society, even though they may sometimes make mistakes; who supports the protestors in places like Ferguson, and chant All Cops Are Bastards; who supports Israel's use of force to protect its civilian population and who supports the Palestinian struggle attempting to release itself from this grip. These two (four?) situations, extremely contentious and cause of many arguments, are, to me, part of a bigger issue which will long continue to blight our world - that social roles are effectively determined, once a dynamic begins to take shape then behaviour becomes almost impossible to vary. The famous Stanford Prison Experiment demonstrated that a single group of students, arbitrarily divided into prisoners and guards, would very quickly develop the kind of behaviours which we see in the real world and likewise relationships to each other: the guards become authoritarian and abusive, the prisoners become either passive victims and collaborators, or aggressively rebellious. There is a dynamic where if you give someone a uniform and a weapon and tell them it is their job to protect society, they will dehumanise anyone they see as acting against them: these are the troublemakers, the ones who are threatening peaceful civilisation (even if they are just selling cigarettes); for those victimised by the police, it is rather the police who are dehumanised in their eyes, becoming tools of repression, essentially violent and stupid. No one is right in their views - in the process of dehumanisation we see not the complexity of human judgment, the structures which determine our thoguht processes and conclusions, we see only the end-result, the us and them and, working back from that, we project some monstrous kind of beliefs, some un-empathisable wickedness which has led "them" to act in such a way, a way which to us is inconceivable because our own experiences - our own environment which helps to structure our thought - cannot possibly lead to that view of reality.

I have often heard it said (and this says more about the people I listen to than anything else) that white society doesn't understand the struggles of that black (or other minority) groups have to go through; likewise that men don't understand how much pressure there is on women in our culture. This is true; so is the opposite, however (and this isn't a prelude to some horrific kind of meninism). It's very difficult to step outside our own shoes. In these examples the oppressed groups are focussed on, and rightly so. But it's also the case that in debates about things like Ferguson, in questions about police violence, in discussions about Israel-Palestine, we have to see that there are two sides both of which are inhabited by human beings, but in which there are clear and distinct social roles which play out regardless of the individuals in those places. "How can people do this?" It is often asked - but only by people who've never been in that situation. How can police attack an unarmed protestor? How can kids riot instead of going to school? How can the IDF bomb schools? How can Hamas hide weapons in schools? How can Bin Laden attack the WTC? How can America and Russia use Afghanistan as a battlefield for their proxy war? How can other people make the choices they do? These very one-sided narratives always serve to dehumanise the other, to not recognise that through different life experiences different conclusions about what is to be prioritised are reached; that in the power dynamics of the human world, where you sit in the scale of entitlement has serious consequences for your understanding of the world and its social structures as liberating or oppressive; to be celebrated or fought. Whether the protestor is a menace or a freedom fighter depends already on your own location in regard to them.

The point I'm getting at in this last section is that it's very easy to see politics in monolithic terms - that there is a factual answer about how the world should be, and about right and wrong. But if we can stem our arrogance and begin to believe that other alternative viewpoints are just as logical, just as human, then we go a long way towards resolving disputes. If protestors and police can both understand that the others are human beings making difficult decisions based around their experiences, maybe there's a way for empathy and compassion to enter their actions too. Admitting that the internal world is constructed and not an objective representation is a step towards that.


A MAN walks confidently towards an open gate but instead of going straight through he raises his knee very high as if he were stepping over a low wall. He strides forward, reaching out to shake a friend's hand. But again he misjudges, and his friend draws back in alarm to avoid being punched in the nose.

This is Innsbruck, Austria, in the 1950s, and no, the man hasn't been drinking too much schnapps. He is psychologist Ivo Kohler, and he is wearing a pair of goggles with a built-in mirror that turns his world upside down. In a grainy black-and-white film that records his stumblings, the eternally surprised Kohler dives to catch a child's balloon drifting skywards and turns a teacup upside down against a stream of water being poured from above.

Kohler is just one in a long line of researchers who have used inverting goggles to try to understand how we see. The latest to pass through the looking glass is a young philosopher called Jan Degenaar. For him, however, the experiment is not simply an exploration of vision. By stepping outside his normal perception of the world and seeing it in a different way, he thinks he has gained an insight into the so-called hard problem of consciousness – how to explain the feeling of sensation. How do our brains turn a set of signals into the redness of a rose, the softness of velvet, the pungency of raw onion, and all the rest? His experience supports a new theory about consciousness – that it is not merely in the mind, but extends beyond the boundary of the body. The idea is not just weird and esoteric, if correct it has ramifications in fields ranging from animal consciousness to robotics.

Degenaar's foray into the hard problem of consciousness began with an interest in visual perception. Orthodox understanding of how this works dates back to the 16th century and French philosopher René Descartes, who suggested that our brains construct an internal model of the world, which we then view like a cinema playing inside our heads. Degenaar is among a growing number of researchers who question this interpretation. In 2011, while studying for a PhD at the University of Groningen in the Netherlands, he was reading descriptions of experiments with inverting goggles when he noticed something interesting. While some experimenters described the mental image of the world flipping, others related how they learned to adapt their behaviour to the inverted image. Intrigued by the discrepancy, he decided to try the experiment for himself.

Degenaar's goggles flipped the left and right sides of space by placing a right-angled prism in front of each eye. He wore them for an average of 4 hours a day for 31 days – earlier experiments having shown that you can adapt without wearing them all the time. With objects on his left now appearing on his right and vice versa, he immediately experienced a major conflict between the feedback from his visual system and other sensory input, especially touch. He became as clumsy as Kohler. Initially, however, the most disturbing aspect of the experience was his sense of visual instability. Each time he moved his head, the scene rushed past him and he couldn't track anything in it. On the first day, the nausea this induced was so intense that he vomited.

The visual instability gradually eased and had vanished entirely by day 13. At that point, Degenaar could move his head while keeping his gaze fixed and see objects where he expected to see them. If he kept his head still, however, he had to think hard about which way to move his eyes to bring an object into the centre of his vision from the periphery. Other skills returned at different rates. Unable to orient a knife correctly with respect to a tomato on day 1, for example, he managed to cook a simple meal three days later. He developed strategies for walking that involved turning his head in the direction he wanted to go in. At first his path zigzagged but it gradually straightened out, and on day 15 he was able to walk home from the university, armed with a white stick – though it took him an hour rather than the usual 30 minutes (Phenomenology and the Cognitive Sciences, vol 13, p 373).

This piecemeal adaptation has been reported by others. American psychologist George Stratton was a pioneer of inverting glasses in the late 19th century. With one eye covered, he strapped a contraption over the other, inverting the world left-right and up-down. He reported that different elements of the scene "righted" themselves at different times and in different contexts. In the 1960s, a volunteer working with psychologist James Taylor at the University of Cape Town, South Africa, got quite good at riding a bike around the campus wearing left-right inverting glasses. However, even when he could easily navigate between buildings, writing on signs on those buildings still appeared reversed, only becoming legible after he had practised reading with the goggles.

Illusion of reality

Everyone agrees that describing inversion effects to people who have never experienced them is extremely difficult, and researchers argue over the meaning of past accounts. In Stratton's case, for example, what was "righted" could have been either his visual experience, or his behavioural response to it. What is consistent in most reports, though, is the incremental nature of the adaptation. For Degenaar and his former mentor, Erik Myin at the University of Antwerp in Belgium, it represents a nail in the coffin of the Cartesian model of vision.

"There is no internal image in the brain, and nothing flips," says Degenaar. The real nature of visual perception is quite different, he says. At any given time we see only a tiny portion of the visual scene – the part our eyes are actively exploring. The impression we have of gazing out on a unified visual world is mere illusion, he believes, arising from the knowledge that we would see another portion of the scene if we were to move our eyes there. It's our active, if partial, sampling of the scene that gives it the quality of reality. In his view, adaptation to inverting glasses involves learning a new set of relationships between our movements and the changes in sensory input they now generate. It therefore depends on how much a person has practised a certain action, which could explain the staged return of consistent, accurate visual judgement. "You start to see vision not as one capacity, but as a set of interrelated capacities," says Myin.

One person who agrees with this interpretation is Kevin O'Regan, who is based, ironically, at Paris Descartes University in France, and in whose lab Degenaar works. "Seeing involves actively interacting with the world," he says. There is no Cartesian cinema playing inside our heads, just a mass of different interactions between our senses and our environment. "Saying that we have the impression of a coherent visual field is simply an abbreviated way of saying that we are comfortable with all the ways that we visually interact with the world." He gives an example the rest of us might just be able to relate to: shaving or putting on make-up in a mirror isn't easy the first time you do it, but with practice you get better. You reach out with your razor or mascara wand to the right place on your face, and you do so automatically, without telling yourself to do the opposite of what feels right.

O'Regan's model is not simply about vision, it encompasses all forms of perception. A decade ago, when he began talking about his "sensorimotor theory of perception", it was highly controversial. Today, enactivism – as variants of it are collectively called – is gaining in popularity. Enactivists believe that thinking and feeling arise in the dynamic interaction between an organism and its environment. Thus an organism "enacts" a world. And this insight might help crack one of the biggest mysteries of all – the hard problem of consciousness. In O'Regan's model, your subjective experience of being is created by your awareness of the myriad different ways your self interacts with the world as you move around and explore it.

If O'Regan is correct, the particular senses with which you explore the world shape your subjective feeling of being. And that's where inverting goggles come in. Degenaar had an insight into the hard problem of consciousness around day 30 of his experiment. Until then, he had found that coordinating his movements with what he saw required effort, and he had begun to worry that he would only ever be able to compensate for his impairment, never really adapt to it. That changed when he suddenly noticed that objects appeared to be where they actually were. In his write-up of the experiment, he describes what happened next: "A few moments later, when I had not moved my head for a while, I fell back in the other way of experiencing the visual field again, so that the objects once again appeared to be in places where they were not actually located. But when I continued looking around again, by slowly moving my head, I could now see objects where they were." In other words, he now had access to two perceptual worlds, whereas most of us spend our whole lives trapped inside one.

Previous goggle-wearers have described a stage where they saw two versions of the same object, one more ghostly than the other, though with time the ghostly one became more substantial until it replaced the first. Degenaar's experience was slightly different: his two "percepts" were rivals. He compares this bi-stable state to what people experience when they look at an ambiguous image such as the Necker cube or duck/rabbit illusion. "It can't be described as the flipping of an image," he says. "It's more like a gestalt switch." He was seeing the same objects, and nothing had moved, but the raw feel of seeing had changed. The reason, he thinks, is that his sensorimotor engagement with the world – the bodily act of seeing – had also been transformed.

Enactivism hasn't won everybody over. One sceptic is Colin Klein, a philosopher at the Australian National University in Canberra. While impressed by Degenaar's descriptive powers, he says they still leave room for different interpretations. The perceptual breakthrough he recorded on day 30, for instance, could have been the result of his brain learning to extract information from an inverted internal image – in the same way that a trained sonographer can decode an ultrasound image that appears to a patient as meaningless black and white splodges. "In one sense they are seeing the same image, but one is seeing it with expert knowledge and one isn't," says Klein.

Jesse Prinz, a philosopher at the City University of New York, expresses similar reservations. When you look in a mirror, he says, "You know the image is reversed, but you develop the skills to cope with a world that has been turned backwards."

Both Klein and Prinz cite a study from 1999 that circumvents the problem of describing what it feels like to experience visual inversion. David Linden, then at the Max Planck Institute for Brain Research in Frankfurt, Germany, and colleagues, tested four wearers of up-down inverting goggles on a simple visual trick that involved showing them discs drawn on a flat, grey background and shaded vertically from white to black. Normally, observers assume that the discs are lit from above, and see those that are white at the top as convex and those that are black at the top as concave (see Illustration). Linden's volunteers made this assumption too, but when they put the glasses on they reported that the discs they had originally seen as convex were now concave and vice versa. What's more, this interpretation persisted throughout the 10-day experiment, despite the fact that they adapted behaviourally to the glasses (Perception, vol 28, p 469).
For Klein and Prinz, this is clear evidence for a picture model of vision. The internal image is inverted by the goggles and does not adapt or flip; rather, behaviour adapts to the inverted image. For Degenaar and O'Regan, it merely demonstrates that vision can be fragmented until a person relearns all the ways in which they can interact visually with their environment.

The two interpretations have different implications. A robot built on the picture model would passively register a photo-like image of the world, while a sensorimotor robot would learn to see the world by actively exploring it. More fundamentally, if the sensorimotor approach is correct, a newborn baby might have to learn consciousness. And the quality of another species' consciousness might differ radically from our own, given that it explores the world with different senses, such as eyes on the side of its head or the ability to echolocate or sense magnetic fields.

O'Regan's ultimate goal is to understand how the brain mediates these sensorimotor interactions to create phenomenological experience: how, for example, it generates a feeling of redness across all the different conditions in which a red object can be observed. At root, he thinks, raw feel springs from something elementary and predictable – the laws of physics – but understanding how it does so could give us an insight into what it means to be human. Imagery, symbolism, metaphor and language – the things that set our species apart – are, after all, grounded in sensory experience.

The debate over the hard problem continues, and we surely haven't seen the last of inversion goggles. Degenaar would like to repeat his experiment with a group of volunteers, having them describe their visual experiences while observers simultaneously record changes in their behaviour. So look out for people trying to spoon soup into their foreheads, or throwing themselves to the ground in an attempt to stand up straight. They will be doing it in a good cause.

This article appeared in print under the headline "Goggle eyed"

Laura Spinney is based in Lausanne, Switzerland