Analyse This: The Technical Performance of Last of Us Part 1...

Where we're going, we don't need roads...

Although I'm fairly sure we do... The Last of Us Part 1 (from here on in: TLOU) has had a rocky start on its PC release. Problems have been reported far and wide, and players have complained, and complained, about performance issues on this PS3 port of a game.

But is this really a fair comparison? Does TLOU really have the performance problems that people ascribe it? Let's find out!

There will be no spoilers here, today, because I have barely played ten minutes of this release. However, I think I know enough to have a very short post on the demands of this game.

So, without further ado:

I never had a Post Thoughts on TLOU when it released on the PS3. In fact, I don't even remember if that "structure" existed on this blog at that point in time. At any rate, I hold the original game with a certain fondness in my heart, if not my mind*. This was one of those few games that you could sit, absorb and continue to enjoy past the moment.

*The ending made no scientific sense, and as such, I found it very easy to just blast away at the fraudulent people who were trying to chop up Ellie for no gain...

The game has since been updated for PS4 and then re-mastered for PS5... leading to this very moment where exclusive PC gamers get to experience the story and world that was crafted by Naughty Dog. Personally, I think this port is long overdue. It could and should of happened after the PS4 port but it didn't. What we have been granted with, now, appears to be - at the surface level - a difficult port based on the PS5 remaster. 

What this is not, essentially, is an old game. This is a new game, using new technologies. However, this game is based on a game that has been transistioned between various hardware, perhaps without forethought on its interoperability with the PC gaming ecosystem. As such, in its current state, it has strange demands on PC hardware that are difficult to explain beyond being simple technological patches on the original engine.

Texture settings have a noticeable effect on the visuals of the game but do scale well with PC resources...

Getting over it...

TLOU is not a difficult game to run, graphically. The options included in the game are quite versatile and enable decent levels of scaling for the available GPU power. However, this isn't the whole story. 

Aside from lacking a benchmarking feature to enable players to understand how their choices really affect the technical performance, the options in this game scale in a way that most other games do not: For instance, VRAM is easily eaten up by the higher texture settings in the game (see above). But scale very strongly with the available settings, meaning that they will work with whatever hardware is available - unfortunately, with an associated loss in visual fidelity...

I would say that "medium" settings are fine... though not pretty compared to various other titles.

As a result, it is not quite clear why or how the game is using so many resources to generate visuals which are on par with other titles of the current generation - without ray tracing. However, users choosing not to realistically set their settings to lie within the VRAM requirements of the game (which are realistically set, aside from one menu bug) is most likely one of the main causes of the current conflagration surrounding TLOU on PC.

The VRAM utilisation bar in the settings menu is very accurate, aside from this one bug I noticed when playing with settings - the High and Medium settings must be accidentally swapped...

Worse still, this title exhibits a strange reliance on CPU processing power, which may be leading many users to incorrectly finger the wrong piece of their hardware as being the culprit.

I have two systems available to me and I have quickly tested the game on both - at the very start. It may not be representative of scenes later into the game but this testing does highlight some differences in the setups and identify potential solutions to those looking for them.

System Limitations...

The systems I use are covered in prior blogposts, so I won't go over them again. However, the long and short of it is that I have a weaker processor paired with a stronger GPU and a stronger processor (and RAM) paired with a weaker GPU. This is not some Machiavellian stroke of genius to whittle-out issues on PC releases... it's just dumb luck*.

*If it is even that...

The point I'm trying to make, here, is that we are potentially able to observe where system-level bottlenecks or application (a.k.a. game) bottlenecks, or demands, are focussed.

The aspirations I will look at for this analysis are 60 fps at 1080p. These limitations are a reflection of the hardware available to me but also information which the any user will find useful in terms of their own experience. Many other reviewers will be able to spit out bar charts covering the average and minimum or percentile lows (wrong as they may be) but they do not explore the more common situation of a v-sync'ed experience.

This analysis will focus on frametimes, instead of 'fps', with a target of 16.666 ms for our "perfect" user experience. The benchmark is an approximate two minute period during the escape of Joel and his daughter during the initial outbreak, covering in-engine cutscene and movement gameplay, both in an ubran environment with a large number of AI NPCs and a small, lightly forrested area with minimal AI NPCs.

During the analysis I will often refer to each system as the AMD/Intel system, for ease of writing - this is only referring to the platform in each system... and does not infer any poor performance on either manufacturer.

First up, let's look at the effect of the two systems on the game:

The game scales well, graphically, but it seems that the texture detail is where the majority of the problems are emanating from...

The AMD system with an R5 5600X paired with an RX 6800 does well in terms of presentation to the user - with minimal frame drops or missing frames at medium or high quality settings. Ultra quality, on the other hand, provides a sudden challenge to the system, with a frametime spike in the hundreds of milliseconds.

Dialling back the four texture setting options (seen above in the image) to medium, we see that these spikes are severely reduced at ultra quality settings. What is odd in this benchmark run is that we should not be anywhere near to breaching the RX 6800's 16 GB of VRAM. Additionally, we do not consistently observe frametime spikes at the same points during the benchmark run (taking into account run to run variance).

Moving onto the Intel system:

The RTX 3070 gives a much worse presentation to the player... but behind the scenes we see a different story playing out...

The i5-12400 paired with the RTX 3070 is struggling.

Presentation-wise, we're looking at lots of stutters and dropped frames throughout the benchmark. Yes, at ultra we're seeing terrible frametime spikes - though much better than with the AMD system. Primarily, the reason for these is the 3070's 8 GB VRAM buffer - at high and ultra quality, the game flat-out tells the user that this is above the poor GPU's VRAM limit. So, fair enough. 

However, dropping the four texture settings to medium at ultra quality, as we did with the RX 6800, we do not observe a huge improvement in presentation compared to the high quality preset.

This indicates that the issue is not only related to VRAM quantity.

Playing the long game...

The thing is - are we really saying that the RTX 3070 is that much worse than the RX 6800 at 1080p with a non-raytraced game? Personally, I would not expect it to be... So what IS going on?

Removing graphical considerations, as much as we can (in this particular comparison) I used the optimised settings I have previously validated for the Intel system at DDR4 3200 at the medium quality preset. For the main portion of the testing, the RAM was operating at the DDR4 3800 optimal settings.

With the faster RAM speed, (aka higher bandwidth) the game plays smoother...

What we are able to observe is an appreciable effect of memory bandwidth on the presentation of the game! The faster RAM reduces sequential frametime variance and also reduces delays in frame presentation for the user on the screen.

This isn't the whole story though.

Look back up at the graphs for each system and compare the each quality setting between them. At both medium and high quality settings, the 12400 outperforms the 5600X. At the higher quality settings, the GPUs become more of a limiting factor. This means that the game is also heavily affected by the throughput of the CPU: frametime spikes are smaller on the 12400 with faster RAM.

This is specifically happening in the sections of the benchmark where there are physics interactions and large numbers of AI NPCs running around. Looking at the 5600X/RX 6800 system, we can see that in the later sections when we move away from those situations, the sequential frametime reaches its true potential, inbetween some undue frametime spikes.

There is some (post?) processing happing that is heavily impacting the AMD CPU... on medium quality settings.

Conclusion...

The Last of Us is, apparently, a very heavily CPU and RAM-limited game. From my brief time with it, the game seems to scale well with available graphical processing resources but the CPU limitation is a difficult aspect to explain without an understanding of how the game engine is working. 

It is important to note that the game will heavily load up the system memory of any system. I have observed that the game will easily eat up 16 GB as a buffer between storage and the VRAM - this is actually optimal behaviour and can explain why the bandwidth of memory is quite an important performance attribute for this game.

Speaking of VRAM, the game does not look its best at medium settings - with smeary surfaces and horrid reflection quality. High or Ultra is where you would like to be for the best experience. Which means you would ideally have greater than 8 GB VRAM on your GPU...

Medium quality setting reflections look absolutely terrible...

However, neither the RX 5600X or i5-12400 are weak CPUs compared to what is available in the current generation of consoles but, here we are... a game wreaking havoc on any CPU that is not at the peak of current performance levels. In this, I am at a loss as to what exactly, the game is doing to warrant such high CPU usage. In the portion I've been testing, I can see that the worst portions are those with explosions and heavy physics interactions. However, this isn't the only game with large numbers of NPCs (Hogwart's Legacy/Spider-man) or physics interactions...

Could it be that the engine on PS5 uses the CPU in some unusual manner which doesn't translate well to the hardware setup on PC?

At any rate, players of this game, at this point in time, ideally need a strong, modern CPU with fast, low latency system memory and a GPU with a larger than 8 GB VRAM framebuffer in order to have the best experience...

It's possible that some of this can be addressed in an eventual patch but if the CPU demands are hard-coded into the engine, it seems like that is not a reasonable item to fix, going forward... This means that, overall, TLOU is a disappointing technical release on PC, years late from the original experience.