In Defence Of: Ray Tracing...

 

I wrote an article, now over 10 years ago, about my disdain for the logical argument equivalent of ingnoring the problem/issue. At that point in time, it was "get over it" and, whatever "it" was, was obviously something that the person in question, uttering those three words, had mentally, internally and socially accepted and moved on. However, there was no leeway or credence that someone else could be still within those early stages of acceptance or denial or that someone might be struggling to, intellectually, come to terms with that new paradigm. Or that they might not even ever accept the concept.

I'm writing this post because of the conversations I've had on youtube and twitter within the last three months. I'm writing these thoughts because of the conversation surrounding, and indeed preceeding, Nvidia's latest imperial SNAFU/FUBAR situation about a focus on a feature they have tied themselves to: ray tracing. And while Nvidia are doing their best to present the worst face of their behaviour, there is an underlying problem with how we review graphics cards. I think this may be quite an unpopular opinion, especially in the blind rallying to reviewers' collective defence - but this is a separate, though related issue that needs to be addressed...

I've always been a low-spec gamer. However, as I've stated many times, I am a scientist. This means that I am able to look at and interpret data that is provided to me: I can extrapolate, integrate and differentiate in order to extract interesting titbits of comparison data. This is something I consider a strength and something that I have been trained to do. Similarly, I have been trained to accept, actually to expect, that I can be, will be and am wrong and, with an additional bit of data, can have my conclusions changed. 

That's science and that's how it works.

So I am at a loss as to the whole "ray tracing doesn't really matter" philosophy of hardware reviews. Are we really in the infancy of ray tracing? (I'll get to that in an addendum below)

The problem is that Hardware Unboxed are not wrong in what they're saying. However, that opinion that reviews should not be focussed on ray tracing performance has informed their coverage of the hardware. Worse still, it seems many reviewers are in parallel agreement. In my opinion, that's not a great situation to be in as I hope to explain over the course of the following paragraphs.

The issue I have with HWUB (for some reason they're usually abbreviated to HUB but that makes no sense - either it's HU or HWUB; the letter begining of each word or the leading syllables of each part of the word... but, I digress) and the way they have been approaching ray tracing is that they view it as a sort of tragedy of the commons*: the average user does not care about ray tracing performance therefore we will not attempt to care about it either. OR the user does care about the RT performance but they are doing it in a way that doesn't make sense - *I* do not perceive visual benefit from this technology, therefore, it is of limited benefit and, as a consequence of its implementation, it affects an aspect of performance that I value a great deal (frames per second) which, obviously the actual user who prefers RT doesn't understand they are losing that performance.

*I'm aware this is a terrible analogy - if you have a better one, please post it in the comments!

You may think I'm projecting there but I am not so sure that's true. The number of people I've recently had discussions with who think that raster performance, specifically "fps", is the only applicable attribute to high end gaming that matters has been considerable. For me that's considerably disconcerting. Steve, himself, has said many times that he prefers higher frame rate gaming and that he can't go back to 60 fps. That's perfectly fine but it's a very narrow-minded view.

My own opinion stems from this premise - niche products should be reviewed from the perspective of the niche(s) that apply to them. Mainstream products can be reviewed from both a mainstream perspective and a specialist perspective.

A high-end sports car is not primarily reviewed on its general road handling or cruising at 40 kph. It's not reviewed for its road-trip-worthiness or its ability to transport the weekly grocery shopping. You can review it from that perspective but it defies the main purpose of the technology - to push the boundaries of engineering in certain and specific ways. Jeremy Clarkson, James May and Richard Hammond made a fantastic turn away from traditional car reviews by mixing the specific with the general for the revamp of the aging and decrepit Top Gear show format. However, in so doing, they moved the show from purely factual to op-ed entertainment.

I think that reviewers that ignore the application focus of high-end computer hardware in order to adhere to traditional metrics and viewpoints are missing the point of both the hardware and the review. In the process, they diminish their review's value to the consumer and do the hardware a disservice.

To give a specific example, it's like taking a Threadripper CPU and only reviewing the gaming performance of the chip. That's not its intended application and it's not the most common use-case for the product... it's also not the most logical use-case. In the same way, many reviewers have not been recommending the RX 6900 XT and RTX 3090 because they are not really gaming-focussed cards but they have been benchmarking their performance for the sake of completeness - and that's the correct way to go.

Now, I understand that gaming is WAY more complicated than those simple scenarios. Gaming is complicated because it's diverse. That means high-end gaming encompasses various different realities: A simulation focussed setup is not the same as an e-sports gaming setup; is not the same as a photorealistic setup; is not the same as an ultrawide setup; is not the same as a VR setup.

Each and every one of those applications is high-end and each of them has different requirements in both software and hardware support. Each of those gamer types comes into each hardware review expecting different information and a different focus that can feed their needs. This is all good stuff. The problem lies in when reviews or reviewers do not see their own bias as a preference, instead of a necessity.

Sites such as PC World specifically cater to certain subsets of audiences because they are aware that there is demand out there for that information. They don't not test for ultrawide setups because they don't, themselves, game at those resolutions. In the same way, other sites cater for e-sports titles/gaming or simulation gaming. What amazes me in the conversation is that high resolution has gotten a free pass, despite contributing to only 2.25% of the steam hardware settings... However, the narrative surrounding ray tracing performance has virtually always been from a negative angle. Most commentators accept that this may be the future of game rendering but a great many appear to refuse to accept that consumers wish to know how these nascent cards are performing in the available games, predicating the argument around terrible performance. Well, 4K resolution performance was terrible until relatively recently but everyone happily tested the hell out of that!*

*It's very frustrating to me to see all the commentary that RT isn't worth it, that performance is terrible and yet those 1080 / 1080 Ti performance at 4K had the exact same issue but no one was suggesting that 4K resolution wasn't a main part of a review or that it shouldn't be worried about for another 3-4 years or 1-2 generations. Nobody said that!

It's entirely acceptable to see ray tracing as a niche product/feature... but all high-end graphics cards are niche products. We're talking about 7% of the steam hardware survey market being above an RTX 2070 and, IMO, actual decent ray tracing performance starts at RTX 2080 Ti levels of performance so that's an incredibly minor segment of the marketshare of all GPUs.

It boggles my mind that reviewers, of all people, that are spending inordinate amounts of time benchmarking incredibly niche products refuse to accept that specific hardware optimisations or specialisations (which can thus be conisdered niche in their implementation) should be a major part of those reviews of that hardware.

YES! Ignore ray tracing performance in the low-end. Ignore VR-performance and ultrawide performance in the low-end as well. You already do that for 4K performance! But don't do it in the high-end because this is exactly what these cards are designed to do.

As a reviewer, you cannot benchmark for every scenario or every piece of hardware - the time limit  to release date or limitations for your putrid meat sack bodies means you cannot do this. However, omitting data because it cannot be conceivably attained is very different from omitting data because you don't see the value in the data. It's also highly different than omitting data because you specialise in another area: I don't expect those sim reviewers to focus on RT performance or even necessarily high framerate or e-sports performance. That's what specialist reviews are for!

Hardware Unboxed did not do that, though. They deprioritised ray tracing performance without a "specialisation" justification. It was just a preference. A preference without any real backing in reality or for the products they were reviewing. It's just as valid to say, benchmarking hardware for high framerate gaming above 60 fps is worthless when the vast majority of the market do not have monitors that can display more than a fixed 60 Hz image. In the same way, you can say that the vast majority of gamers don't play on ultrawide displays or at 4K resolution. I think it's perfectly valid to target a specific audience and to cater to that market but you should be honest about the reason for it - not give a reason to ignore or minimise that coverage that can equally be applied to your current focus as well. Maybe you personally believe that reason but it's not a logical reason.

Now, I don't want you to get the impression that I am hating on Hardware Unboxed. I'm not. I actually think the data they do provide is of very high quality - they even test ray tracing performance (begrudgingly) and it's very telling that it's Tim who is testing and presenting this information, not their primary graphics reviewer (i.e. Steve). However, I think their view on this aspect of gaming performance is not ideal and not in line with their stances on other high-end aspects/features of gaming tech.

And this applies to a swathe of the tech press and gaming community. This is an ongoing discussion throughout Twitter, message boards, tech review websites, and youtube videos and comments. It continues to amaze me how in denial many people are about this. I've singled out HWUB because they've been in the limelight over this and because I follow their reviews and have commented about this issue in their youtube comments.

At the end of the day, the vast majority of gamers play at 30-60 fps, at 1080p (or below) resolution on hardware that has less than 4 physical CPU cores and less than 16 GB RAM (actually, due to consistently cheap RAM prices, we've come a long way in this regard and when taken individually 16 GB of RAM is the mode!). I mean, the GTX 1060 is still the most popular GPU on Steam. If you are addressing segments of the market other than those data points then you are a specialised outlet producing specialised content. Don't pretend to normalise one type of speciality over another... as if the feature is not ready for review. And don't discount it based on reasons that you don't apply to other specialist features that exhibit the same performance issues when being targetted...

__________________________________________________________________________

Is ray tracing in its infancy?

This is a question that has been bubbling across several conversations I've had over the last day or two. There is a perception that "real time ray tracing" in games is in its infancy and is thus to be taken with a grain of salt or kid gloves or whatever way you wish to express this feeling. However, I believe there is a confusion between the software and hardware support sides of things.

Due to the long-standing implementations of ray tracing in film and research, ray tracing algorithms are well-researched and the topic is quite widely open to engine designers/coders. In terms of real-time applications, from what I can tell from a brief search, actual implementations and discussions of those implementations (or analogous implementations - see: voxel cone tracing, as seen in The Tomorrow Children) began publicly around 2010-2011.

We are now in a situation for which we have mature real-time software implementations and APIs across multiple platforms, game engines and rendering hardware. DX12 Ultimate and Vulcan ray tracing are publicly released, RTX has been out since 2018 and the basis for these implementations are still, at their core, from those beginnings in 2010-2011.

Developers have had a minimum of two years (and probably a bit longer) to become familiar with this upcoming technological shift and, believe me, I'm sure many of them were - whether that's behind the scenes based on discussions with the graphics hardware manufacturers, the console manufacturers or at graphics/game  dev conferences...

Seriously, the new consoles were in development for 4-5 years. RDNA 1/2 and RTX were in development around the same time, if not before. Real-time RT has been the holy grail for CAD-style and production environments for a LOT longer than that... So, I am fairly confident when I say that the software side of things is quite mature.

Where things fall apart a bit is the hardware.

RTX was a proprietary solution that was sort of rolled out to certain developers in a preferential (and seemingly highly controlled) manner. RDNA 2 RT was rumoured to have issues and had to be put back in the oven until it was done - hence the release of RDNA 1 minus ray tracing*.

*There were a lot of news stories about this back in the day but they are drowned out by RX 6000 news right now and I can't find one to link...

Putting this into perspective, developers have only had 2.5-3 years to address Nvidia hardware and maybe 1 year to address AMD hardware. Yes, it takes time to develop a game but already we're expecting innate support for DX12U feature sets in major game engines by mid-2021, meaning that I'm expecting a large majority of games released for console to support ray tracing features, going forward.

Yes, we do have the issue that a PS5/XSX are RX 6700/RX 6700 XT levels of performance but there's nothing we can do about that...

So, yes. I believe that specific levels of attainable RT on specific hardware is still in their developmental infancy but in terms of software implementation? I think RT and equivalent technologies are already well-developed from a software point of view...