The #1 XR Blog in the World
- English
- Sign In
Foveated rendering is a process used in virtual reality to improve performance and graphical experience. It works by only rendering the portions of a scene the user is currently looking at while surrounding areas are blurred. Since Sony announced that PlayStation VR2 will feature this innovative rendering technology and premium mixed reality headsets like Meta Quest Pro alike, the topic has been ablaze within the industry.
In this article, we will cover:
What is foveated rendering?
How does foveated rendering work?
Types of foveated rendering?
What does foveated rendering mean for virtual reality?
Conclusion
Let’s start with the name. Foveated rendering comes from the tiny area of your eye where resolution is at its peak. This area is called the fovea and is only 3˚ to 5˚ wide cone. So when we construct our visual surroundings, we do so like a laser pointer, consciously directing attention to specific areas. Outside this laser pointer zone is called the ‘gaze area’, an area about 18˚ cone that is slightly blurred. This means that as you move further toward your peripheral vision, your vision becomes less clear and outright black and white in the far edges. This is, for example, why you move your eyes when you read.
In other words, dynamic foveated rendering means that only one-tenth of the total resolution is rendered. Offloading a substantial amount of processing and allows for much better virtual reality visual experiences. Some estimate it to result in ten times better graphics than we see today! We will circle back to this point later in the article with the effects we see in virtual reality headsets currently.
For foveated rendering to be reliable, you need sophisticated eye tracking technology that not only covers pupils’ movements and size differences but also people’s different kinds of pupils. Going back to the fovea and gaze area mentioned above. Our eyes can scan an environment within the 21-23˚ cone by 900˚ per second. It, therefore, takes the average human eye 0,024 seconds to move one degree.
As a result, to ensure that the real-time rendering works, eye tracking must be able to predict where your eyes are likely to be. In fact, if the eye tracking fails with foveated rendering, the visual experience is ruined and immersion is removed. These predictions are usually computed by a Neural Processing Unit seen in Qualcomm’s Snapdragon SoC chips.
Foveated rendering can be categorized into two types: fixed and dynamic foveated rendering.
In virtual reality headsets that don’t feature eye tracking, fixed foveated rendering is an option. Essentially, the headset pre-determines a fixed point that needs to be in full resolution and the circle of blurring expands outward from there. For example, Oculus/Meta Quest 2 has fixed foveated rendering.
In VR headsets where eye tracking is featured, most have the dynamic foveated rendering and is what we covered above under “What is foveated rendering?” Headsets with dynamic foveated rendering can be seen in Varjo’s premium HMDs VR-3 and XR-3, as well as the upcoming PlayStation VR2.
Foveated rendering in virtual reality enables game developers to focus and allocate resources to what is important and lessen what is not. This means that visual and immersive fidelity will be able to improve dramatically in the coming years. For example, today, the Meta/Oculus Quest 2 VR headset with fixed foveated rendering saves on average between 34% to 43% graphical performance. Whereas Meta Quest Pro with dynamic foveated rendering currently saves between 36% to 52%. However, this will increase as visual fidelity improves in the coming years and the technology is more optimized. For example, the unreleased PlayStation VR2 will be 3.6x faster with dynamic foveated rendering.
Courtesy Varjo.
This is why foveated rendering has been referred to as the technology to bring Retinal Resolution – or human eye resolution – to standalone VR headsets. A resolution density of between 60 to 70 pixels per degree is currently only seen in prototypes or the center area of Varjo VR-3 and XR-3.
In any case, Sony’s commitment to foveated rendering and eye tracking for commercial VR headsets will push the technology within the industry to new levels. Soon we will see other headsets also offer foveated rendering and eye tracking, and more VR content to take advantage of the technologies.
Foveated rendering is a technique that has the potential to improve virtual reality graphics tenfold, by reducing the number of pixels rendered. Thus, making it possible for low-end hardware to power high-quality VR experiences. With Sony’s commitment to this rendering technique, we can expect other players to also feature this innovative technology, providing even better VR visuals soon.
Jakob Pii is the Head Writer for VR Expert and currently lives in the UK. He started his career in VR gaming in 2015 and has stayed in XR since, from exposure therapy in VR to 360-degree video documentaries. He is fascinated by how emerging technologies change how we live, play and work.
