Visual Perception

headed by Dr. C. Tursun

Visual perception is the study of how our eyes and brain process visual stimuli to create meaningful interpretations of the world around us. Humans are the main observers of the content produced by image processing, computer vision, and computer graphics techniques. Hence, visual perception principles are instrumental in optimizing the visual content according to the properties and limitations of the Human Visual System (HVS) for the best user experience with our limited computational and display hardware resources.

Visual perception has many useful applications that have proven useful in the past couple of years. For instance, foveated rendering leverages non-uniform distribution of photoreceptor cells on our retinas, where the fovea has the highest resolution and peripheral vision is blurrier; hence, high-resolution rendering targets only where the gaze is focused, saving computational resources. Actually, foveated rendering is one example of a wide range of gaze-contingent display systems, which adjust content based on where a user is looking, improving user experience by providing real-time visual content adaptation. While designing quality metrics, understanding visual perception allows the design of algorithms that can better assess image quality based on human perceptual criteria. Especially Augmented and Virtual Reality (AR/VR) displays benefit from visual perception insights by optimizing color, depth, and motion in ways that align with how humans naturally perceive these elements. Finally, in generative deep learning, models can be trained to produce images that are perceptually convincing to human viewers, thanks to the knowledge from visual perception.

Some examples of these applications might be: a VR game utilizing foveated rendering to only render high-detail graphics at the player's gaze point, making the game run smoother on less powerful hardware. Similarly, a gaze-contingent e-reader might allow users to delve deeper into what specifically captures their interest while skipping over items they find less appealing from their gaze direction and duration. A perceptually inspired quality metric might rate an image's clarity not just on technical details but on how a human would perceive its quality. An AR system, informed by visual perception, might overlay virtual objects in the real world with optimized depth cues, making them appear more realistic. Finally, a generative deep learning model, like those used in creating deepfakes, would benefit from visual perception knowledge to generate facial features that appear more lifelike to human observers (although the ethical aspect of deepfakes is still being actively discussed by the society).