The AMD Threadripper 2990WX 32-Core and 2950X 16-Core Review
by Dr. Ian Cutress on August 13, 2018 9:00 AM ESTOur New Testing Suite for 2018 and 2019
Spectre and Meltdown Hardened
In order to keep up to date with our testing, we have to update our software every so often to stay relevant. In our updates we typically implement the latest operating system, the latest patches, the latest software revisions, the newest graphics drivers, as well as add new tests or remove old ones. As regular readers will know, our CPU testing revolves an automated test suite, and depending on how the newest software works, the suite either needs to change, be updated, have tests removed, or be rewritten completely. Last time we did a full re-write, it took the best part of a month, including regression testing (testing older processors).
One of the key elements of our testing update for 2018 (and 2019) is the fact that our scripts and systems are designed to be hardened for Spectre and Meltdown. This means making sure that all of our BIOSes are updated with the latest microcode, and all the steps are in place with our operating system with updates. In this case we are using Windows 10 x64 Enterprise 1709 with April security updates which enforces Smeltdown (our combined name) mitigations. Uses might ask why we are not running Windows 10 x64 RS4, the latest major update – this is due to some new features which are giving uneven results. Rather than spend a few weeks learning to disable them, we’re going ahead with RS3 which has been widely used.
Our previous benchmark suite was split into several segments depending on how the test is usually perceived. Our new test suite follows similar lines, and we run the tests based on:
- Power
- Memory
- Office
- System
- Render
- Encoding
- Web
- Legacy
- Linux
- Integrated Gaming
- CPU Gaming
Depending on the focus of the review, the order of these benchmarks might change, or some left out of the main review. All of our data will reside in our benchmark database, Bench, for which there is a new ‘CPU 2019’ section for all of our new tests.
Within each section, we will have the following tests:
Power
Our power tests consist of running a substantial workload for every thread in the system, and then probing the power registers on the chip to find out details such as core power, package power, DRAM power, IO power, and per-core power. This all depends on how much information is given by the manufacturer of the chip: sometimes a lot, sometimes not at all.
We are currently running Prime95 as our main test, however we are recently playing with POV-Ray as well.
Memory
These tests involve disabling all turbo modes in the system, forcing it to run at base frequency, and them implementing both a memory latency checker (Intel’s Memory Latency Checker works equally well for both platforms) and AIDA64 to probe cache bandwidth.
Office
- Chromium Compile: Windows VC++ Compile of Chrome 56 (same as 2017)
- PCMark10: Primary data will be the overview results – subtest results will be in Bench
- 3DMark Physics: We test every physics sub-test for Bench, and report the major ones (new)
- GeekBench4: By request (new)
- SYSmark 2018: Recently released by BAPCo, currently automating it into our suite (new)
System
- Application Load: Time to load GIMP 2.10.4 (new)
- FCAT: Time to process a 90 second ROTR 1440p recording (same as 2017)
- 3D Particle Movement: Particle distribution test (same as 2017) – we also have AVX2 and AVX512 versions of this, which may be added later
- Dolphin 5.0: Console emulation test (same as 2017)
- DigiCortex: Sea Slug Brain simulation (same as 2017)
- y-Cruncher v0.7.6: Pi calculation with optimized instruction sets for new CPUs (new)
- Agisoft Photoscan 1.3.3: 2D image to 3D modelling tool (updated)
Render
- Corona 1.3: Performance renderer for 3dsMax, Cinema4D (same as 2017)
- Blender 2.79b: Render of bmw27 on CPU (updated to 2.79b)
- LuxMark v3.1 C++ and OpenCL: Test of different rendering code paths (same as 2017)
- POV-Ray 3.7.1: Built-in benchmark (updated)
- CineBench R15: Older Cinema4D test, will likely remain in Bench (same as 2017)
Encoding
- 7-zip 1805: Built-in benchmark (updated to v1805)
- WinRAR 5.60b3: Compression test of directory with video and web files (updated to 5.60b3)
- AES Encryption: In-memory AES performance. Slightly older test. (same as 2017)
- Handbrake 1.1.0: Logitech C920 1080p60 input file, transcoded into three formats for streaming/storage:
- 720p60, x264, 6000 kbps CBR, Fast, High Profile
- 1080p60, x264, 3500 kbps CBR, Faster, Main Profile
- 1080p60, HEVC, 3500 kbps VBR, Fast, 2-Pass Main Profile
Web
- WebXPRT3: The latest WebXPRT test (updated)
- WebXPRT15: Similar to 3, but slightly older. (same as 2017)
- Speedometer2: Javascript Framework test (new)
- Google Octane 2.0: Depreciated but popular web test (same as 2017)
- Mozilla Kraken 1.1: Depreciated but popular web test (same as 2017)
Legacy (same as 2017)
- 3DPM v1: Older version of 3DPM, very naïve code
- x264 HD 3.0: Older transcode benchmark
- Cinebench R11.5 and R10: Representative of different coding methodologies
Linux
When in full swing, we wish to return to running LinuxBench 1.0. This was in our 2016 test, but was ditched in 2017 as it added an extra complication layer to our automation. By popular request, we are going to run it again.
Integrated and CPU Gaming
We are in the process of automating around a dozen games at four different performance levels. A good number of games will have frame time data, however due to automation complications, some will not. The idea is that we get a good overview of a number of different genres and engines for testing. So far we have the following games automated:
- World of Tanks, encore (standalone benchmark)
- Final Fantasy XV (standalone benchmark, standard detail to avoid overdraw)
- Far Cry 5
- Shadow of War
- GTA5
- F1 2017
- Civilization 6
- Car Mechanic Simulator 2018
We are also in the process of testing the following for automation, with varying success:
- Ashes of the Singularity: Classic (is having issues with command line)
- Total War: Thrones of Britannia (will not accept mouse input when loaded)
- Deus Ex: Mankind Divided (current test not portable, might be Denuvo limited)
- Steep For Honor
- Ghost Recon
For our CPU Gaming tests, we will be running on an NVIDIA GTX 1080. For the CPU benchmarks, we use an RX460 as we now have several units for concurrent testing.
In previous years we tested multiple GPUs on a small number of games – this time around, due to a Twitter poll I did which turned out exactly 50:50, we are doing it the other way around: more games, fewer GPUs.
Scale Up vs Scale Out: Benefits of Automation
One comment we get every now and again is that automation isn’t the best way of testing – there’s a higher barrier to entry, and it limits the tests that can be done. From our perspective, despite taking a little while to program properly (and get it right), automation means we can do several things:
- Guarantee consistent breaks between tests for cooldown to occur, rather than variable cooldown times based on ‘if I’m looking at the screen’
- It allows us to simultaneously test several systems at once. I currently run five systems in my office (limited by the number of 4K monitors, and space) which means we can process more hardware at the same time
- We can leave tests to run overnight, very useful for a deadline
- With a good enough script, tests can be added very easily
Our benchmark suite collates all the results and spits out data as the tests are running to a central storage platform, which I can probe mid-run to update data as it comes through. This also acts as a mental check in case any of the data might be abnormal.
We do have one major limitation, and that rests on the side of our gaming tests. We are running multiple tests through one Steam account, some of which (like GTA) are online only. As Steam only lets one system play on an account at once, our gaming script probes Steam’s own APIs to determine if we are ‘online’ or not, and to run offline tests until the account is free to be logged in on that system. Depending on the number of games we test that absolutely require online mode, it can be a bit of a bottleneck.
Benchmark Suite Rollout
This will be the first review with our new benchmark suite, at least the CPU portion of it. We are still working on the new gaming suite. So far for this review we tested 8-9 processors, and I am expecting to iron out any inconsistencies further into September, after several key industry events over the next few weeks.
As always, we do take requests. It helps us understand the workloads that everyone is running and plan accordingly.
A side note on software packages: we have had requests for tests on software such as ANSYS, or other professional grade software. The downside of testing this software is licensing and scale. Most of these companies do not particularly care about us running tests, and state it’s not part of their goals. Others, like Agisoft, are more than willing to help. If you are involved in these software packages, the best way to see us benchmark them is to reach out. We have special versions of software for some of our tests, and if we can get something that works, and relevant to the audience, then we shouldn’t have too much difficulty adding it to the suite.
Facebook
Twitter
RSS
171 Comments
View All Comments
ibnmadhi - Monday, August 13, 2018 - link
It's over, Intel is finished.milkod2001 - Monday, August 13, 2018 - link
Unfortunately not even close. Intel was dominating for last decade or so. Now when AMD is back in game, many will consider AMD but most will still get Intel instead. Damage was done.It took forever to AMD to recover from being useless and will take at least 5 years till it will get some serious market share. Better late than never though...tipoo - Monday, August 13, 2018 - link
It's not imminent, but Intel sure seems set for a gradual decline. It's hard to eke out IPC wins these days so it'll be hard to shake AMD off per-core, they no longer have a massive process lead to lead on core count with their margins either, and ARM is also chipping away at the bottom.Intel will probably be a vampire that lives another hundred years, but it'll go from the 900lb gorilla to one on a decent diet.
ACE76 - Monday, August 13, 2018 - link
AMD retail sales are equal to Intel now...and they are starting to make a noticeable dent in the server market as well...it won't take 5 years for them to be on top...if Ryzen 2 delivers a 25% increase in performance, they will topple Intel in 2019/2020HStewart - Monday, August 13, 2018 - link
"AMD retail sales are equal to Intel now"Desktop maybe - but that is minimal market.
monglerbongler - Monday, August 13, 2018 - link
Pretty much this.No one really cares about workstation/prosumer/gaming PC market. Its almost certainly the smallest measurable segment of the industry.
As far as these companies' business models are concerned:
Data center/server/cluster > OEM consumer (dell, hp, microsoft, apple, asus, toshiba, etc.) > random categories like industrial or compact PCs used in hospitals and places like that > Workstation/prosumer/gaming
AMD's entire strategy is to desperately push as hard as they can into the bulwark of Intel's cloud/server/data center dominance.
Though, to be completely honest, for that segment they really only offer pure core count and PCIe as benefits. Sure they have lots of memory channels, but server/data center and cluster are already moving toward the future of storage/memory fusion (eg Optane), so that entire traditional design may start to change radically soon.
All important: Performance per unit of area inside of a box, and performance per watt? Not the greatest.
That is exceptionally important for small companies that buy cooling from the power grid (air conditioning). If you are a big company in Washington and buy your cooling via river water, you might have to invest in upgrades to your cooling system.
Beyond all that the Epyc chips are so freaking massive that they can literally restrict the ability to design 2 slot server configuration motherboards that also have to house additional compute hardware (eg GPGPU or FPGA boards). I laugh at the prospect of a 4 slot epyc motherboard. The thing will be the size of a goddamn desk. Literally a "desktop" sized motherboard.
If you cant figure it out, its obvious:
Everything except for the last category involves massive years-spanning contracts for massive orders of hundreds of thousands or millions of individual components.
You can't bet hundreds of millions or billions in R&D, plus the years-spanning billion dollar contracts with Global Foundries (AMD) or the tooling required to upgrade and maintain equipment (Intel) on the vagaries of consumers, small businesses that make workstations to order, that small fraction of people who buy workstations from OEMs, etc.
Even if you go to a place like Pixar studios or a game developer, most of the actual physical computers inside are regular, bone standard, consumer-level hardware PCs, not workstation level equipment. There certainly ARE workstations, but they are a minority of the capital equipment inside such places.
Ultimately that is why, despite all the press, despite sending out expensive test samples to Anandtech, despite flashy powerpoint presentations given by arbitrary VPs of engineering or CEOs, all of the workstation/Prosumer/gaming stuff is just low-binned server equipment.
because those are really the only 2 categories of products they make;
pure consumer, pure workstation. Everything else is just partially enabled/disabled variations on those 2 flavors.
Icehawk - Monday, August 13, 2018 - link
I was looking at some new boxes for work and our main vendors offer little if anything AMD either for server roles or desktop. Even if they did it's an uphill battle to push a "2nd tier" vendor (AMD is not but are perceived that way by some) to management.PixyMisa - Tuesday, August 14, 2018 - link
There aren't any 4-socket EPYC servers because the interconnect only allows for two sockets. The fact that it might be difficult to build such servers is irrelevant because it's impossible.leexgx - Thursday, August 16, 2018 - link
is more then 2 sockets needed when you have so many cores to play withRelic74 - Wednesday, August 29, 2018 - link
Actually there are, kind of, supermicro for example has created a 4 node server for the Epyc. Basically it's 4 computers in one server case but the performance is equal to that if not better than that of a hardware 4 socket server. Cool stuff, you should check it out. In fact, I think this is the way of the future and multi socket systems are on their way out as this solution provides more control over what CPU. As well as what the individual cores are doing and provides better power management as you can shut down individual nodes or put them in stand by where as server with 4 sockets/CPU's is basically always on.There is a really great white paper on the subject that came out of AMD, where the stated that they looked into creating a 4 socket CPU and motherboard capable of handling all of the PCI lanes needed, however it didn't make any sense for them to do so as there weren't any performance gains over the node solution.
In fact I believe we will see a resurrection of blade systems using AMD CPU's, especially now with all of the improvements that have been made with multi node cluster computing over the last few years.