What is the relationship between neural activity in the brain and the perceptual world we experience? We explore this connection by using physiological and psychophysical techniques in animals and humans. This allows us to answer questions about how visual information is represented in the activity of neurons, where in the system critical perceptual computations are performed, and how neural activity is related to our visual experience.
Natural vision, Visual context, Eye movements
Most laboratory studies of vision use simple isolated stimuli such as spots of light or sinewave luminance gratings. Human or animal subjects examine these stimuli by holding their gaze fixed and avoiding eye movements. A great deal has been learned about brain processing with this reduced approach, but it clearly neglects much of the complexity of vision in the real world. We are exploring and quantifying brain processing in natural vision which involves complex visual input and exploratory eye movements. We find that visual context and eye movements have potent effects on visual processing and perception. We also find that there are temporal interactions in which stimuli seen on one fixation of the eyes alter perception on a subsequent fixation.
Vision Prosthetic for the Blind
We are developing a prosthetic device that will allow the blind and people with low vision to recognize and locate objects in their environment. The system takes input from wearable cameras and uses computer vision techniques to interpret the visual scene. The user receives visual information through a combination of auditory signals and direct stimulation of visual cortex.
Eye movement signals, Temporal parsing for object recognition, Stability of the visual world
Human visual perception takes place primarily during eye fixations separated by rapid saccadic eye movements. An important function of saccades is to bring the fovea to objects of interest. However, there are multiple lines of evidence suggesting that saccades are directly involved in visual processing rather than simply providing a series of snapshots to the brain. The brain may use information about eye movements to understand vision across saccades and fixations. It appears that vision may require two transsaccadic processes that paradoxically are concurrent but opposite -- temporal integration and temporal parsing. It was noted long ago that a push to the eyeball makes the world appear to jump, but a similar eye movement generated internally leaves the world stable. As suggested by Helmholtz long ago, it appears that parts of the brain responsible for visual perception are informed about the occurrence of saccades. This information may be used to create the sense that our visual environment is stable even when the eyes jump from one viewpoint to another. A process of integration responsible for visual stability may benefit from the suppression of perception for objects seen during saccades. At the same time that transsaccadic visual integration occurs, it is essential that the continuum of neural activity in visual areas of the brain can be interpreted as distinct objects on distinct fixations of the eyes. This may involve a process of temporal parsing or visual reset at the start of each fixation. In our studies, we have found that there is precise information about the direction and amplitude of saccades and the timing of fixations in the activity of visual neurons. We are exploring how eye-movement signals, saccadic suppression, visual stability, and visual parsing are based on neural activity in visual areas of the brain.