Speaking from experience, best-case just the capturing of such videos adds 50% to the amount of time it takes to conduct benchmarking. Analyzing the videos and composing useful content from them is something more usable on an individual game basis, rather than for graphics reviews.
I wish that weren’t the case. I wish it was possible to get all the benchmarks from all the potential configurations with clear image quality comparisons using every possible setting. It’s not. And it’s foolhardy to think otherwise. It’s also why, for the time being, our GPU reviews will primarily focus on non-upscaling, non-framegen performance as the baseline, where image quality shouldn’t differ too much. We can do some testing of upscaling and framegen as well, but that will be a secondary consideration.
And we feel that’s pretty fair. Because no matter how much marketing gets thrown at the problem, frame generation differs from rendering and upscaling. It adds latency, and while it makes the visuals on your display smoother, 100 FPS with framegen doesn’t feel the same as 100 FPS without framegen — and certainly not the same as 100 FPS with multi-frame generation! Without framegen, user input would get sampled every ~10ms. With framegen, that drops to every ~20ms. With MFG, it could fall as far as sampling every ~40ms.
Upscaling is a different matter. What we’ve seen in the past clearly shows that DLSS upscaling delivers superior image quality to FSR2/3 and XeSS upscaling. And now, Nvidia is about to overhaul DLSS to further improve image fidelity thanks to a transformers based AI model. It will run slower than the older CNN model but it will look better. How much better? Yeah, things just became that much more complex.
There’s more to the benchmarking discussion, including AI and professional workloads. We test all these areas on our graphics card reviews, and what Nvidia shows already agrees with what we’ve been doing for the most part. If you have any thoughts or feedback on the matter, let us know in the comments. The full deck from the session is included below for reference.