Nvidia has shed some light on the inner workings of its latest technology, revealing that DLSS 5 doesn’t have access to the 3D renderer. The tool leverages the final 2D frame plus motion vectors to generate its ‘enhanced’ image, which results in some inconsistencies compared to the original render.
Following the official reveal of DLSS 5, many started screening the official comparison pictures and videos to find out how the technology works. One of these is Daniel Owen on YouTube, who contacted Nvidia to ask about some discrepancies he noticed. For example, DLSS 5 seems to have added hair to a Starfield character that wasn’t present in the original game-rendered frame. Another was regarding the creator’s intent in Resident Evil Requiem, where the main character seems to have makeup in the DLSS 5 version, which isn’t the case for the original.
Among the questions asked by Daniel was the processes used by DLSS 5 to generate its so-called enhanced image, because the official wording on Nvidia’s marketing language around geometry, materials, and scene understanding led to the belief that DLSS 5 had access to the underlying 3D game engine. Turns out, no, DLSS 5 only takes the final 2D image plus motion vectors to generate the enhanced image.
In other words, it’s like taking a screenshot of the game and passing it through an AI filter. This may be an oversimplification, but what is certain is that DLSS 5 doesn’t leverage information such as 3D geometry, depth, or material properties; it mainly infers what each texture should look like based on its training.
Why that wording, then, you may ask? Well, by appreciating Nvidia means that the model understands scene semantics such as hair, fabric, skin, and lighting conditions by analysing the 2D frame, it explains the aforementioned hair change on Starfield, as the model could have mistaken the missing hair for a low fade haircut.
Daniel also asked about the creators’ intent and how such tampering could steer the final image from the artist’s vision. Simple aspects like character makeup could break the atmosphere and emotional impact by adding bright or incorrect hues to a scene, like in Resident Evil Requiem, where Grace is supposed to be going through some old trauma, and thus has no time to get pretty and apply makeup.
To this, Nvidia said that developers will have detailed controls such as intensity and colour grading, which would allow them to adjust blending, contrast, saturation, and gamma, to maintain the games’ unique aesthetic, adding that they can also mask specific objects or areas to be excluded from these enhancements.
Nvidia sees DLSS 5 as a tool that allows developers to achieve their artistic vision without being bound by traditional real-time rendering limitations. While I understand the appeal, these early examples hasn’t put DLSS 5 in the best light. They surely were impressive, but not appealing to many. To say that the DLSS 5 teaser was badly received by the gaming community would be an understatement. Many pointed out its aggressive filter effect that strays dramatically from the creators’ intent, likening it to an Instagram filter.
Nvidia needs to carefully monitor community and developer feedback to improve the quality and detail persistence between the rendered and enhanced image. I wouldn’t be surprised to see a drastic shift in user reception after a couple of generations of this technology. Just look how DLSS 1 was received; now, nearly everyone enables DLSS 4.5.
