How the brain accomplishes perception

Sensory information that comes into the brain is necessarily ambiguous and noisy. In order to make sense of these signals, the brain has to combine its prior expectations about the causes of these signals.

Anil Seth

Professor of Cognitive and Computational Neuroscience

View Author ->

Anil Seth

Professor of Cognitive and Computational Neuroscience

20 Apr 2025

Key Points

The brain extracts information from sensory inputs. In order to make sense of these electrical signals, it has to combine them with its prior expectations about the causes of these signals and form its best guess about what's out there in the world.
We have many more than five ways in which the brain senses the world. But, critically, none of these sensory channels relay the state of the world as it is. Perception in the brain is always a process of interpretation of these sensory signals.
Time is illusory in the same way that colour is illusory – the way in which we experience time is not a direct readout of what's actually happening in the world. Our experience of time is also a construction.
We all inhabit our own distinctive perceptual universes – we all have a unique, conscious experience of ourselves and of the world around us.This diversity is illustrated beautifully by people with synesthesia. Synesthesia means mixing of the senses.
I like to think of perception as controlled hallucination. It's a construction, but it is controlled by what's out there in the world. What we call hallucination, when people see things that other people don't, is a form of uncontrolled perception.

How the brain accomplishes perception

Photo by Svitlana Pavluik

There's a natural way to think about how the brain accomplishes perception, which is that the brain takes in sensory signals from the world and it reads out these sensory signals in the brain. In this view, perception is a kind of bottom-up or outside-in process in which the brain is extracting information from sensory input.

But this isn't how things are. Sensory information that comes into the brain is necessarily ambiguous and noisy. Imagine that you are a brain: you’re locked inside a bony skull, you have no direct access to what's out there in the world. All you get are electrical signals, and these electrical signals don't come with labels, they just arrive at the brain.

In order to make sense of these signals, the brain has to combine its prior expectations about the causes of these signals. These are prior beliefs about what's out there in the world, and it has to combine them with sensory signals to form its best guess about what's out there in the world. This is what we consciously see – it’s a best guess about what caused the sensory signals that the brain receives.

Many ways of sensing the world

It's tempting to think there are just five ways in which the brain senses the world. These are the classic senses of sight, sound, taste, touch and smell. This list of five, which goes back to Aristotle, is highly incomplete; we have many more ways of sensing the world and sensing the body. For instance, we have thermoception to detect temperature. We have kinesthesia, which senses how the body is moving, and proprioception, which senses the body's position in space, the configuration of the limbs.

There are many different ways in which the brain has access to what's going on in the world. But, critically, none of these sensory channels relay the state of the world as it is in some ideal, objective sense. It's all just noisy, sensory signals that are indirectly related to things that are actually out there, which is why perception in the brain is always a process of interpretation of these sensory signals. This process of interpretation, that's what we really mean by perception. It's all about how the brain makes sense of the barrage of sensory input that it’s continually being exposed to.

Studying perception in the lab

As a neuroscientist, one of the challenges of working on consciousness is how to study it experimentally. You open your eyes in the morning, you get down to your kitchen and make some breakfast. There's a whole world of experience going on, a rich panorama of perceptual content. This all has to be boiled down into very restricted situations that we can study under laboratory conditions.

One example of how to study perception in the lab is using a phenomenon called binocular rivalry. What happens in binocular rivalry is that we present two different images, one to each eye. These images are typically quite different; there might be an image of a house shown to one eye, and an image of a face shown to another eye. The brain doesn't combine these images into a single percept: rather, perceptual content alternates between them. At some points we see a house, and then we consciously experience a face. Then, it goes back to a house, and so on.

What's interesting here is that the overall sensory input that we're providing to the brain is constant, but conscious experience is flipping back and forth between house and face. This situation allows us to look inside the brain to figure out what changes when the conscious experience changes. We know that it’s not driven by sensory input because sensory input stays exactly the same. Situations like this, where we can begin to tease apart what people perceive from what sensory information the brain is receiving, provide experimental windows onto the brain basis of conscious perception.

How we perceive time

Photo by Photo_Traveller

One of the most puzzling aspects of conscious perception is how we perceive and experience time. We experience it as flowing from one moment to the next, but every moment seems to have an extension a little bit into the future, with a bit of residue from the past as well. We experience events as having particular durations, but there is no ‘clock in the head’ by which all this happens.

Like other kinds of perception, our perception of time is also a “best guess”, and it’s biased in some ways, too: we systematically misperceive objective time. Sometimes we experience events as being shorter than they really are, and sometimes we experience them as lasting longer than they really are. Studying time perception in the brain is not about looking for a little stopwatch somewhere inside the skull that measures out events in terms of clock time, it's about understanding how the brain makes inferences about how long things last and about what flows.

Time is an illusion

Physicists sometimes say that time is an illusion, that it's not in the fundamental equations of physics. In neuroscience and psychology, time is also illusory, but in a different way. For me, time is illusory in the same way that colour is illusory – the way in which we experience time is not a direct readout of what's actually happening in the world. When we say that a flower is red, it doesn't mean that that redness that we experience is actually out there in the world. The redness is a property of how the brain interacts with the world. The artist Paul Klee said, ‘Colour is the place where the brain and the universe meet’. I think the same thing applies to time: our experience of time is also a construction. This doesn't mean that time doesn't exist physically, in the same way that there is something about a red flower that makes us see it as a red flower, it just means that what we experience this time is not identical to what the physical phenomenon of time actually is. As to what that physical phenomenon is, that's where you have to go and ask a physicist.

The synesthetic experience

One assumption that's easy to make, but which is very definitely wrong, is that we all see the world in the same way. On the contrary, there's a great diversity among our conscious experiences. This diversity is illustrated beautifully by people with synesthesia. Synesthesia means mixing of the senses, which is a good description of what perception is like for a person with synesthesia.

There are many different varieties of synesthesia. One of the more common is what's called “grapheme–colour” synesthesia. This is a condition in which people will have an additional experience of colour when they lettering – even if the letters are just black and white. For example, when they see the letter A in black ink, it’s not that they necessarily will see the A as being red or blue instead, but, instead, that the black letter will induce an additional experience of a very specific colour. If they see the same letter again, they will have exactly the same colour experience – synaesthetic experiences are highly consistent.

We are all synesthetic to some degree

There are many other forms of synesthesia. There are people for whom sounds have particular shapes; there are those for whom numbers have a particular arrangement in space. To me, what's important is that we're all synesthetic to some degree – a metaphor, if you think about it, is a kind of synesthesia. For each of us, certain tastes seem ‘sharp’, while others don't. What happens with synesthesia is that this metaphorical association becomes perceptually literal, so people have additional experiences.

Synesthesia is a unique window into consciousness. Firstly, we can use it as a way of understanding what underpins any kind of conscious experience, because for the same situation – the same sensory input – a synaesthete will be having a different conscious experience compared to a non-synaesthete, so we can look at what's different. But there is also the larger lesson here that we all inhabit our own distinctive perceptual universes; we all have a unique, conscious experience of ourselves, and of the world around us.

Hallucinations are false perceptions

Photo by ysuel

A hallucination is typically thought of as a false perception. It's when somebody experiences something that's not there, or that other people don't experience. People with certain kinds of psychosis – schizophrenia, for example – may have auditory or visual hallucinations, and you can also experience hallucinations when you take various kinds of psychedelic drugs.

It's a mistake to think of hallucination as completely distinct from the normal business of perception – in an important sense, we're all hallucinating all the time. Every perception we have is an act of construction, an act of interpretation in which the brain's expectations about what's out there in the world are reined in, or controlled by, sensory signals coming from the world.

I like to think of perception as controlled hallucination. It's a construction, but it is controlled by what's out there in the world. What we typically call hallucination, when people see things that other people don't, we can think of as a form of uncontrolled perception in which the brain's predictions are not being reined in by what's out there in the world.