A Timely Discovery: Examining Our AMD 2nd Gen Ryzen Results
by Ian Cutress & Ryan Smith on April 25, 2018 11:15 AM ESTLast week, we published our AMD 2nd Gen Ryzen Deep Dive, covering our testing and analysis of the latest generation of processors to come out from AMD. Highlights of the new products included better cache latencies, faster memory support, an increase in IPC, an overall performance gain over the first generation products, new power management methods for turbo frequencies, and very competitive pricing.
In our review, we had a change in some of the testing. The big differences in our testing for this review was two-fold: the jump from Windows 10 Pro RS2 to Windows 10 Pro RS3, and the inclusion of the Spectre and Meltdown patches to mitigate the potential security issues. These patches are still being rolled out by motherboard manufacturers, with the latest platforms being first in that queue. For our review, we tested the new processors with the latest OS updates and microcode updates, as well as re-testing the Intel Coffee Lake processors as well. Due to time restrictions, the older Ryzen 1000-series results were used.
Due to the tight deadline of our testing and results, we pushed both our CPU and gaming tests live without as much formal analysis as we typically like to do. All the parts were competitive, however it quickly became clear that some of our results were not aligned with those from other media. Initially we were under the impression that this was as a result of the Spectre and Meltdown (or Smeltdown) updates, as we were one of the few media outlets to go back and perform retesting under the new standard.
Nonetheless, we decided to take an extensive internal audit of our testing to ensure that our results were accurate and completely reproducible. Or, failing that, understanding why our results differed. No stone was left un-turned: hardware, software, firmware, tweaks, and code. As a result of that process we believe we have found the reason for our testing being so different from the results of others, and interestingly it opened a sizable can of worms we were not expecting.
An extract from our Power testing script
What our testing identified is that the source of the issue is actually down to timers. Windows uses timers for many things, such as synchronization or ensuring linearity, and there are sets of software relating to monitoring and overclocking that require the timer with the most granularity - specifically they often require the High Precision Event Timer (HPET). HPET is very important, especially when it comes to determining if 'one second' of PC time is the equivalent to 'one second' of real-world time - the way that Windows 8 and Windows 10 implements their timing strategy, compared to Windows 7, means that in rare circumstances the system time can be liable to clock shift over time. This is often highly dependent on how the motherboard manufacturer implements certain settings. HPET is a motherboard-level timer that, as the name implies, offers a very high level of timer precision beyond what other PC timers can provide, and can mitigate this issue. This timer has been shipping in PCs for over a decade, and under normal circumstances it should not be anything but a boon to Windows.
However, it sadly appears that reality diverges from theory – sometimes extensively so – and that our CPU benchmarks for the Ryzen 2000-series review were caught in the middle. Instead of being a benefit to testing, what our investigation found is that when HPET is forced as the sole system timer, it can sometimes a hindrance to system performance, particularly gaming performance. Worse, because HPET is implemented differently on different platforms, the actual impact of enabling it isn't even consistent across vendors. Meaning that the effects of using HPET can vary from system to system, as well as the implementation.
And that brings us to the state HPET, our Ryzen 2000-series review, and CPU benchmarking in general. As we'll cover in the next few pages, HPET plays a very necessary and often very beneficial role in system timer accuracy; a role important enough that it's not desirable to completely disable HPET – and indeed in many systems this isn't even possible – all the while certain classes of software such as overclocking & monitoring software may even require it. However for a few different reasons it can also be a drain on system performance, and as a result HPET shouldn't always be used. So let's dive into the subject of hardware timers, precision, Smeltdown, and how it all came together to make a perfect storm of volatility for our Ryzen 2000-series review.
242 Comments
View All Comments
eek2121 - Sunday, April 29, 2018 - link
Steam benchmarks are meaningless unless you can filter based on several factors. OEM vs Custom builds, country, etc. That is why I consider anybody that brings up steam survey users to not know what they are talking about. The US has around 300 million people in the world, we spend the highest amount on PC hardware in the world, yet China has billions of people, and they spend the least. Steam groups everyone together. Second, OEM systems force HPET to on. I just checked my laptop running an i7 mobile 6700HK or whatever, and HPET was on in both the BIOS as well as in Windows. So no, you can't make assumptions. Custom builders typically have HPET off, and OEM builders has HPET on. If I were AT, I'd force HPET on. Not to screw one company over vs another, but to force them to improve their HPET implmentations.Sancus - Wednesday, April 25, 2018 - link
Yeah benchmarking CPUs exclusively with via GPU bottlenecked tests is a great idea.IndianaKrom - Wednesday, April 25, 2018 - link
One does have to question the usefulness of a 4k benchmark in a CPU review, other than "yep, its still GPU limited". Whole bunch of graphs showing +/- 3%, content to pad the ads with I guess...GreenReaper - Friday, April 27, 2018 - link
I imagine the intent is to let people who want to be 4K gamers right know that it doesn't matter what CPU they get. Or just to find interesting anomolies. You don't know what you don't test.RafaelHerschel - Wednesday, April 25, 2018 - link
This makes no sense. I use a GTX 1070 and game on 1080p, let's forget about the reasons why for now, I'm not alone in this. Most gamers, even most of those with slightly above average cards use 1080p monitors. HPET is not an issue, very few people force HPET on. There is a reason only AnandTech got these numbers.SkyBill40 - Friday, April 27, 2018 - link
I'm in a similar situation but with a 1080. I'm playing at 1920*1200 @ 60Hz only because I've got other financial priorities keeping me from buying a new monitor.eek2121 - Sunday, April 29, 2018 - link
You are also alone. You cannot rely on steam survey numbers to claim supremacy, as I mentioned in an earlier comment, a steam survey can be nothing more than a dell machine with a GTX 1070 running 1080p, or it can be a Threadripper machine running dual 1080tis @ 1080p. In my situation, I game at 1440p on a 1950x and a 1080ti running 1440p. HPET off. does that make me a minority? No it does not. There is no way to measure per capita spending on PC hardware due to the second hand market, and different demands in different countries. My thought is to force HPET on for all, and may the best company win....just like with Anti Aliasing in gaming.Peter2k - Wednesday, April 25, 2018 - link
As for the last questionMaybe it should be made clear if its on/forced on and so forth
This is anecdotal
But my testing has shown in a handful of games that HPET is detrimental to gaming (7600K at 5.2Ghz)
Fps where the same, but HPET introduced a stutter the whole time
Now I also could've sworn HPET was default off in the UEFI on my Z170
Maybe that's something to look into as well
How is HPET set at default in the UEFI
If its default on with Z370 then it should be made clear it's on
Its default off for older/newer chips it should be made clear me thinks
GreenReaper - Friday, April 27, 2018 - link
The default would be for it to be enabled, since it is a standard feature of the platform nowadays. However, forcing it to be used instead of the CPU's TSC (when it is also available) is not standard in most modern operating systems where the TSC is known to be reliable, or can be made so.eek2121 - Sunday, April 29, 2018 - link
A stutter is a defect. I favor neither Intel nor AMD in this case, however IMHO there can be only 2 outcomes: a) Intel fixes it's HPET implementation or b) Microsoft removes HPET altogether. Only then will we receive the true numbers.