Seeing in 3D with just one eye: Q & A

One of our papers, recently published in Psychological Science, got some media attention. We received many questions from the general public about the study and its implications. The following Q&A post is geared at answering some of these questions in a non-specialist way.

What is this research about?
Most people are familiar with the special visual effect one gets when watching 3D movies: the feeling of objects looming out of the screen, the feeling of real empty space between things, a crispness and clarity to depth, the feeling you can reach out and touch things. This visual phenomenon is referred to colloquially as “seeing in 3D”. Our research is about understanding the source and functional significance of this visual phenomenon. Conventional belief for a very long time has been that this effect is an all-or-nothing phenomenon. Either you have normal binocular (2-eye) vision and can get the 3D effect, or you can’t. We show that this assumption is not true.

What did you find?
We found that this phenomenon of “seeing in 3D” (technically referred to as “stereopsis”) can be induced simply by looking at a single picture of an appropriate 3-dimensional scene with one eye through a small aperture (a pea-sized hole). While 2-eye vision is important in many ways for depth perception, we’ve shown experimentally that it is not necessary for experiencing “seeing in 3D”. Anyone can try it at home using the method described in our paper.

Can’t most people see some form of 3-dimensionality and depth with one eye anyway?
Yes. There are many sensory cues that are available to a single eye that the brain can use to understand the 3-dimensional layout of the scene in front of them. In fact, even when one looks at a conventional 2D photograph of a 3D scene, one is able to perceive 3-dimensionality and depth. However, in these cases, even though 3-dimensionality and depth are perceived, one doesn’t get the same special perceptual “experience” that is usually associated with 3D movies. The term “seeing in 3D” is a bit misleading in this regard. A more appropriate term might be “seeing tangible depth”.

Is the effect you describe due to motion parallax?
No. There was no motion of the picture or observer. Moreover, even if there was, the motion would not be consistent with the depth seen in the picture. When you move a picture (or your head) when looking at the picture, motion parallax tells your brain that everything in front of you is flat. While motion parallax has previously been reported to also induce an impression of “seeing in 3D”, we show that parallax is not necessary for the effect.

Is this a pinhole effect?
No. The aperture used is not a pinhole. The aperture (hole) can be 1cm or more in diameter. Pinhole optics causes an entirely different effect which many people with myopia (short sightedness) are familiar with. If a person with myopia looks at a distant object normally with their bare eye, things appear blurred. If they look through a pinhole, the object appears to be in focus. The pinhole can therefore bring a blurry image into focus or increase the depth of field of an already focused image. These pinhole effects are purely optical (they have to do with light rays and how they can be focused). The pinhole effect has nothing to do with psychology or how the brain computes 3D space.

Is this a new discovery?
Not exactly. Since the discovery of perspective during the Renaissance, the idea that the perception of depth improves when a picture is viewed with one eye (compared to two-eye viewing) has been noted by several scientists and artists; but it has been generally ignored or dismissed by vision scientists after the discovery of the stereoscope 185 years ago (the device that is the basis for 3D movie technology). Moreover, it has been unclear if the improvement with one eye was really the same type of improvement seen in binocular stereoscopes. We have provided the first empirical evidence that the effect is the same in both cases (but not necessarily of the same strength). We also provide evidence that previous explanations of why depth perception in pictures improves with one-viewing are incorrect. We describe how a new theory of stereopsis can potentially explain the effect, and a range of other observations.

Some scientists are uncomfortable with studies that look at the qualitative nature of perception; what’s your response?
In our study, only one experiment was purely qualitative, the other two were quantitative.The work is aimed at understanding the nature, functional role, and neural underpinnings of a qualitative perceptual experience. Without doing qualitative experiments to understand the phenomenon, this will not be possible.
The objection to examining qualitative phenomena by some scientists is not limited to this study. There is a general reluctance in vision science to tackle the important qualitative aspects of perception because they are harder to study. However, a qualitative approach to research in visual perception was responsible for solving one of the major puzzles of vision—how colour is represented in the brain. In the 1870’s, Ewald Hering proposed the opponent theory of colour perception which was based primarily on qualitative observations and phenomenological analysis. For many decades, those insisting on a quantitative approach dismissed this theory. However, now it has been proven based on recordings from neurons in the retina and all the way through to the visual cortex that Hering was correct. The neurophysiology and the psychology of color perception are both now firmly understood in terms of opponent mechanisms.

Is it necessary to show that the strength of the monocular 3-D effect is the same as the strength of the binocular 3-D effect?
One cognitive scientist claims that it is necessary to show that the monocular (one-eye) 3D effect has the same strength as the binocular (stereoscopic) 3D effect in order to conclude that they are both the same psychological phenomenon. This argument confuses the strength of a psychological effect with identity of a psychological effect. Two occurrences of the same phenomenon at different intensities, does not imply that the two occurrences are different types of phenomena.

To illustrate this point, consider the following:
You put a new energizer battery into your flashlight, turn it on, and it produces a bright light. Now a chemist friend of yours says (s)he has found a new way to make a really cheap battery that can produce electricity to light up a flashlight. You put her battery into your flashlight, turn it on, and it also produces light but the light is not as bright as in the first case. You wouldn’t argue with your friend that her/his battery didn’t produce ‘electricity’ and ‘light’. You might argue that you prefer the energizer battery.

What is your explanation of the 3D effect you find with one eye?
It is related to an alternative hypothesis about what stereopsis or “seeing in 3D” really is. The specific explanation for the one-eye effect is given in detail in the paper.

What is your hypothesis of what it is to “see in 3D”?
The hypothesis is that it is your conscious phenomenal experience of how well your brain has calculated the scale of the scene (the actual size of things and actual distance between things). The better it is able to make the calculation, the stronger is the 3D effect.

The brain has various sources of sensory information that it can use to calculate scale. However, it is best at doing these calculations using the difference in the images between two eyes (binocular disparity); so you get a strong 3D effect when the brain has 2-eye information, for example, in 3D movies.

However, the hypothesis (described in detail in a forthcoming paper) suggests that 2-eye information (binocular disparity) is not the direct cause of the effect; it’s only indirectly responsible. In fact, 2-eye is information not even sufficient for a strong 3D effect. When we view a real scene, the 3D effect (stereopsis) diminishes rapidly with the distance of the objects you are looking at. It’s only very strong in the space right in front of you.

This diminishment is conventionally attributed to the fact that the differences between the two eyes images (binocular disparities) become smaller with viewing distance. The alternative theory claims that this is not the explanation. Instead, it proposes that it is linked to availability and quality of two types of visual information: Information on how things you are looking at are laid out in 3-dimensions (relative depth information), and information about the distance of the things you are looking at (egocentric distance information). These two types of information are carried by different sensory signals to the brain.

Based on these and other observations, the alternative hypothesis argues that “seeing in 3D” (stereopsis, tangible depth, or whatever you want to call it) is a special qualitative visual effect not to be confused with simply being able to determine depth relations. You can see 3 dimensions and depth even in the absence of this special 3D effect—for example, you can see depth in normal pictures.

Moreover, the new hypothesis proposes that the effect is not all or nothing. It exists in a continuum from conditions where you don’t get that effect at all (e.g. watching a conventional movie) to the point you get it the strongest (in 3D movies). In all these cases, regardless of whether or not the effect is present, you can still see 3-dimensionality and depth relations; e.g., the 3-d shape of objects, what’s in front of, or behind, what. Consistent with empirical observations, the new theory proposed predicts that the strength of the monocular 3D effect has to be weaker than binocular stereopsis (in some viewing situations, but not all) because of the nature of the sensory signals involved in the two cases.

Are you testing this theory further?
Yes, we are doing both standard quantitative behavioral experiments (psychophysics) as well as brain imaging (fMRI)

What is the implication for those with only one eye or those with trouble with 2-eye vision (strabismus)?
We don’t expect this finding to directly lead to therapeutic intervention to fix strabismus or one-eye vision. Rather, we think our method might be able to help certain individuals who have never experienced the effect to have a direct experience of what it means to “see in 3D”. We are testing a group of strabismics to confirm that this is indeed the case.

There is a lot of confusion out there about what kind of depth perception individuals with misaligned eyes or a single eye have compared to those with 2-eye vision, and research is needed to get a better understanding of this, so as to lead to better therapies. The first step is to find out what sort of visual experiences and capacities each of these groups of individuals have. This will involve both qualitative and quantitative experiments.

Does this imply that strabismics, or individuals with one eye, have never experienced “seeing in 3D”?
This will likely depend on the individual and their specific history and condition.   For example it is likely that those with only one eye have a different experince of depth than those with strabismus.  Even though vision in the latter case is usually assumed to be monocular even when viewing with 2 eyes, there are intercoular suppressive effects that might make depth perception worse that if viewing with one eye closed. So it is possible that some individuals have never experienced depth in the way that is described in the paper, but that others have. We need to test a large number of individuals with different conditions to answer this question, and we have started to do that.