2 November 2023

Alan Wake 2 Performance Analysis...


Alan Wake 2 is the new hotness in the games industry. Love it or hate it, you cannot deny the impact that it has had on the general conversation with regards to hardware, software, and game development. While I'm not as on-board with the near universal, unfettered praise for the title as most reviewers appear to be, I do find the discussions surrounding the hardware requirements and performance of that hardware near and dear to my heart.

So, without further ado, let's take a look at how current mid-range hardware performs in this game and whether that actually lines up with the hardware requirements that the developers put out just before the game's release...

The test setup...


The test system is what many would likely call "high end" but it lies squarely in the middle of mid-to-high-mid-range for currently available hardware:
  • Intel i5-12400
  • Gigabyte B760 Gaming X AX
  • Corsair Vengeance DDR5 2x16 GB 6400
  • Sapphite Pulse RX 7800 XT
  • MSI RTX 4070 Ventus 2X
  • XFX Speedster SWFT 319 RX 6800
  • Zotac Twin Edge OC RTX 3070
The AMD drivers used were 23.10.2 (since no new WHQL were released for this release so far).
For Nvidia, I used the 545.92 drivers.

Regarding the settings, all settings used were at 1080p 'native' unless otherwise specified


Asleep at the Wheel...


Everyone and their dog has been claiming that AW2 is one of the best looking games ever released thus far. From a art direction perspective, I'd say it's very high on the list of games of all time but I just cannot agree from a technical art perspective.

During the time I've played I've felt significantly let-down based partially on the hype but also in comparison with other games on the market. Aside from copious amounts of pop-in on small objects (at max setting), along with world and object mesh geometry LOD being WAY too close to the camera many times (especially noticeable in the forest during Saga's missions there), the big no-nos from my point of view were the grainyness and lack of clarity in the shadows, reflections and the softness of the anti-aliasing solutions.

Addressing the last one first, I found the image to be quite 'soft' with regards to the general presentation. I guess that's a purposeful choice from the developer but it's dissappointing when you increase the resolution and do not really get a clearer image. In a way, the cynic in me would say that it's like this because it masks potential softness introduced when using an upscaler - so there is not a large difference between 1440p Native and with Balanced or Performance upscaling at that output resolution...

As a result, Alan Wake 2 looks the "same" at whatever resolution you run it at. I could not tell the difference between native 1080p and 2160p FSR/DLSS performance (1080p render with upscale) or even balanced/quality upscaling... there was not really any improvement in clarity. Which, well, you can argue that if everyone's experience is the same then that's an artistic choice and a 'great leveller' but if you have more powerful hardware, you'd want to see a benefit.

There's no ability to affect the sharpness of the image in the graphics settings menu...


The lack of clarity in the shadows and the way that transparencies are handled was really a big disappointment for me. Shadows are grainy - even at "native" resolution. It's harder to observe in the forest but the problem still persists. It does go away when utilising Nvidia's ray reconstruction but that's just a terrible cop-out. I've seen people say that the game is using signed distance fields in order to handle some of these things but I've played other games using that technology and the performance and visual presentation was better.

Screenshots taken at high settings, no RT or PT... Predator glasses, stippled shadows, completely unrealistic shading and transparency on that water bottle (which is a blue plastic when seen head-on without a background light).


I think a lot of the big excitement around this game from a visual perspective is from people taking screenshots at max settings, max resolution, with max path tracing, probably with ray reconstruction on and with DLSS upscaling along with frame generation. That's not a realistic playing scenario for the vast majority of players in the here and now. Yes, I agree - it does looks amazing and impressive, but that's not the game most people will experience.


More issues with reflections and shadows...


FSR and DLSS are a bit of a disaster in this title. They accentuate the grainy, undefined look of the image presented to the screen when utilising ray tracing modes, pressenting the user with a pixellated mess. However, the newly implemented "ray reconstruction" technology from Nvidia clears these issues up very nicely. I presume that this is solely due to the low quantity of rays traced within a scene to draw it out and light/reflect it? Or possibly, the default denoising algorithms just aren't very good/clean...

So, I find myself questioning this state of affairs on two fronts - when the game was being developed, the RTX 4090 didn't exist, nor did ray reconstruction (at least it was not announced and I refuse to believe that Remedy were made aware of the tech in any reasonable time in advance of the release date - i.e. more than a year out). Does this mean that the game honestly looked pretty terrible from a technical standpoint and that was how they intended to release it?

The other side of the coin is did they just not bother trying to optimise raytracing for AMD cards? What role is the release playing on the console side of the equation? It seems it's adding no benefit to RT*.
*Of course, I don't expect AMD cards to perform as well as Geforce cards in ray tracing due to the lack of physical silicon dedicated to the feature but other studios and engines have shown that it is possible to have better performance!! The RT mode performs worse than other, equivalent titles on AMD hardware...

It's hard to capture but, in motion, Framegen causes intense banding on the blacks in the image, along with very strong ghosting. Additionally, even with medium raytracing settings, you're getting terrible temporal instability - just look at the edges of the character on the still image of Saga...


Aside from the PC requirements (above) being very high - there were commentators saying that the "low" settings on one game are not equivalent to another. Well, that is true. However, it conveniently side steps the fact that typically low settings look pretty poor. If 99 out of 100 games work like this, it is a ridiculous expectation that developers of a single game that decides not to do this would think that gamers would understand the settings nomenclature in a different manner. That's just deluded crazy talk.

In my opinion, Remedy had terrible communication regarding the system requirements; how good they are; the expected image quality; AND the requirement for mesh shaders - which took the industry by surprise and which should have been front and centre on the system requirements image. Hopefully, they will learn from this experience and improve that point going forward...


Ray reconstruction (right) looks significantly better than without (left). Of course, AMD doesn't have this tech and, how did the game look before this feature was even shared with the developers...?


Anyway, with that pretty negative look at the visual quality of the game, let's move onto a more positive (?) look at the frame presentation performance...


Banging on the door...


One of the benefits of targeting mid-range hardware in my reviewing, is that a lot of these modern titles are not holding back in their visions or, conversely, not able to optimise enough so the types of graphics cards and CPUs they are targetting lie directly within my sphere. 

Let's take a look back up there at the system requirements:
  • Recommended: Medium settings, performance upscaling, 1080p60fps, R7 3700X + RTX 3070/RX 6700 XT
  • Ultra: High settings, performance upscaling, 2160p60fps, R7 3700X + RTX 4070/RX 7800XT
  • Low RT: Medium settings, RT low, Quality upscaling, 1080p30fps, R7 3700X + RTX 3070
  • Medium RT: Medium settings, RT medium, Quality upscaling, 1080p60fps, R7 3700X + RTX 4070

Two of the specified settings appear to be good but for some reason the RTX 4070 never met the fps target set out in the requirements table. I am not sure why!


I honestly don't know why outlets don't test the recommended requirements to corroborate what was communicated by developers. Just looking at the results above, we can see the testing for the RTX 3070 shows good results but weirdly, the settings for the RTX 4070 don't match what was indicated. Bear in mind that this is when using a system that is, in theory, better than what was listed in that requirements matrix.

I don't know if this is a fault of the Nvidia driver or if there is an issue in the game for the RTX 4070 that hasn't been widely reported as yet...


Absolutisms...


With that out of the way, let's take a look at performance scaling for these four cards:

This was the toughest segment of the game I have encountered thus far but both cards can have settings which would provide 60 fps...

Interestingly, the AMD cards perform better than their counterparts in rasterisation-only testing...


Performance scaling, with respect to settings adjustment appears to be very limited in this title and this is made worse by the observation I made above regarding the visual muddiness of the game meaning that most of the settings really don't have that big of an impact on the quality of the presented image.

We're looking at a maximum of around 17 fps difference between low and high settings for the RTX 4070 and RX 7800 XT - not a massive win, that's for sure! What is interesting here is that it does appear that there is a generational difference for both Nvidia and AMD cards: the current generation are performing better (relatively) than you would expect in this game compared to the average difference across a range of titles.

Normally, you'd be looking at a ~10 - 15 % difference between the 6800 and the 7800 XT but here we're seeing a 25 - 30 % uplift in rasterised settings. For the 3070 to 4070, we'd expect a ~ 20 - 25 % uplift, whereas we observe a 30 - 40 % uplift. This might be due to more heavy use of compute resources on the cards as postulated by Digital Foundry's Alex Battaglia - which could be a fair assumption given the architectural changes in the RX 7000 series from AMD. It's possible that Nvidia's increase in the L2 cache is helping on their side of the fence given that AMD already increased their L3 cache last generation...


There are some strange frametime spikes on the AMD side of things, as was noted by Hardware Times, though I am experiencing them to a much lesser extent...


Normally, I'd include frametime data in the bar charts, along with power usage but, for one, my RX 7800 XT doesn't (or won't) report power metrics to Frameview and, if you'll take a look up at the frametime graph examples above, the consistency of the sequential frame presentation is very good in this title - so I didn't particularly feel the need.

I do want to point out those small spikes in the AMD graphs. I didn't find them noticeable at all and they didn't detract from the experience. The spikes are much smaller in magnitude and frequency thatn those reported by Hardware Times and this could possibly be explained by two issues cropping up at the same time in their testing. I observed similar spikes to the ones they reported due to an issue where the mouse input would frequently try and override the controller input (I mostly play with controller, these days due to RSI caused by heavy Quake 3 Arena playing in the early 2000s). 

If you're getting constant stuttering, try unplugging or turning your mouse over...


I didn't have time to also test the RX 6800 and RX 7800 XT for FSR scaling but I presume it will be similar...

Finally, let's take a look at the effect of upscaling on performance.

Despite the numbers being smaller, there is actually better scaling on the 30 series card over the current generation card - with an uplift of 54 % for the 3070 compared to 41 % for the 4070.

When combining all the possible tools at the user's disposal, this means that 60 fps can be achieved on the RTX 3070 at high settings with 1080p DLSS Quality or balanced, though the image will look a little softer.

There is one last point I would like to raise - the cost of using the available technologies to improve performance. 

Ray reconstruction typically had a frametime cost of around 4 fps, or around 5 - 10 % compared to simple denoising at the frame rates I was achieving with the RTX 4070. There was similarly a cost for  utilising upscaling, too, with a ~20 % lower framerate going from 1080p native to 1260p Quality upscaling, and a ~30 % lower framerate going up to 2160p Performance upscaling. These are, in theory, all the same render resolution but the frametime cost for doing so is quite extreme... As a result, I think it is better to use native settings rather than rely on upscaling to a higher resolution, if you are able.

The cost in frametime for DLSS becomes more pronounced as the output resolution increases relative to the rendered resolution...




Conclusion...


As an experience, so far, Alan Wake 2 is up there with Control for atmosphere, world-building and visual style. Aesthetically, it's a beautiful game which is, in my opinion, marred by some choices in technical visual presentation. Perhaps, I have some setting wildly wrong on my monitor or I am doing some other random thing incorrectly but I find the presented image very soft.

Additionally, the game does not perform well on, what many would consider, pretty strong hardware. At low settings at native 1080p, the best I was able to achieve was 90 - 95 fps which, while is acceptable from a performance standpoint, falls far short of where I'd expect these cards to land in other, similarly good looking titles (yes, other games' high settings are as good or better than this game's low settings).

Unfortunately, I did see this coming but it does disappoint me to say so - I picked up an RTX 3070 for 1080p high gaming. The same with the RX 6800. I never intended these GPUs to be targetting 1440p or 2160p gaming at all because, to be honest, I grew up playing games on low and medium graphics settings because I couldn't afford the higher end hardware and because I couldn't upgrade as often as I would have liked. It worked out okay but now that I have more disposable income I actually dislike that games and hardware are completely out of sync when it comes to the expectations of the consumer. 

On the hardware side, I expect a €500 - 700 graphics card to be able to manage 1440p60 high settings, without RT and upscaling. On the software side, I expect there to be fall-back systems in the engine to handle the cases where the best-of-the-best rendering is not able to be performed - that's the way it always was. 

In this instance, Alan Wake 2 sets up the future poorly and doesn't provide enough to the past. We are not expecting big gains in future GPU hardware generations because we have not received them over the last 1-2 (depending on SKU). So, sure, the developers can say they're targetting feature sets that can be enjoyed in the future but I am not so sure that will be the case. The visual quality of the game they've supplied in the here and now on mid-range hardware is not that much greater than Control (which I still think looks good!) and in some cases looks much worse, with much more powerful hardware. 

The performance scaling of the game with respect to the options is pretty poor with a miserable 13 - 17 average fps between the low and high settings (which can be further tweaked to add around 20 - 30 fps with upscaling) on all four of the cards tested here... and those averages scale from 55 - 77 average fps, which is not extravagant in terms of user experience.

In my opinion, the upscaling looks okay - it looks similar to the native image but that's because the developers chose to present the native image in a soft format, which is a negative in my opinion. There are already .ini tweaks to improve the native visual output of the game and it wouldn't surprise me if those expand over the coming months.

The quality and performance of ray tracing in this game is pretty poor considering the visual upside is incredibly minor. Path tracing does look good but absolutely tanks the performance to a level where Nvidia's frame generation is required, which also introduces graphical artefacts which, in my opinion, are not worth the trade.

No comments: