Evaluating Futuremark's Servermark VDI on the Supermicro SYS-5028D-TN4T
by Ganesh T S on September 1, 2016 8:00 AM EST- Posted in
- Benchmarks
- Futuremark
- Supermicro
- Servers
- Xeon-D
Standardized benchmarks with industry-wide acceptance are a must for evaluating computing systems. These benchmarks may evaluate the system as a whole, or target specific aspects. Vendors such as Futuremark and BAPCo have various offerings for both consumer and business-use PCs. However, the market for such programs in the server space is largely untapped. Futuremark has been working on Servermark for more than two years now, and they finally felt it was stable enough to let us test drive one of their recent beta versions. Servermark will have a number of sub-categories, but, our focus was on VDI (virtual desktop infrastructure) - a way to determine how many virtual machines a server can support while providing an acceptable level of performance for all users.
Introduction and System Setup
Modern servers are very powerful and come with virtualization capabilities. This capability allows the work done by multiple physical machines to be be consolidated into a single physical server using the concept of 'virtual machines' (VMs). Virtualization brings about a host of benefits to system administrators. VMs can be used in a headless manner (for example, as a mail server or IoT controller), or as a virtual desktop with a thin client front-end. The latter can enable businesses to provide basic low-cost / low-power computers at the employee's desk. Business applications which require plenty of processing power can be processed on a virtual desktop running on the VM host / server. This aspect is the focus of Servermark VDI and our investigation today.
Prior to going into the details of Servermark VDI and its various features, let us take a look at our test machine for the evaluation. The Supermicro SYS-5028D-TN4T is a very popular server in the mini-tower form factor amongst home-lab enthusiasts. We already covered it in our Xeon-D review. Our unit (from last year) came with a Xeon-D 1540 (8C / 16T @ 2.00 GHz) SoC and four hot-swappable 3.5" drive bays. The following table summarizes the specifications of the Supermicro SYS-5028D-TN4T used in this review.
| Supermicro SYS-5028D-TN4T Specifications | |
| Form Factor | 4-bay mini-tower / mITX motherboard |
| Platform | Intel Xeon D-1540 |
| CPU Configuration | 8C/16T Broadwell-DE x86 Cores 12 MB L2, 45W TDP 2.0 GHz (Turbo: 2.6 GHz) |
| SoC SATA Ports | 6x SATA III (for four hot-swap bays and two fixed drives) |
| Additional SATA Ports | None |
| I/O Ports | 2x USB 3.0 2x USB 2.0 1x D-Sub 2x RJ-45 GbE LAN 2x RJ-45 10GBASE-T LAN 1x RJ-45 IPMI LAN 1x COM1 Serial Port |
| Expansion Slots | 1x PCIe 3.0 x16 (Unused) 1x M.2 2242 / 2280 PCIe 3.0 x4 |
| Memory | 4x 16GB DDR4-2133 ECC RDIMM Micron 36ASF2G72PZ-2G1A2 15-15-15-36 |
| OS Drive | Samsung SSD 840 EVO (120 GB; 2.5in SATA 6Gb/s; 19nm; TLC) |
| Data Drives | 4x Crucial MX200 (500 GB; 2.5in SATA 6Gb/s; 16nm; MLC) |
| Chassis Dimensions | 240mm x 210mm x 279mm |
| Power Supply | 250W Internal PSU |
We used four Crucial MX200 500GB SSDs in the four hot-swap bays and configured them in RAID-0 with Intel's BIOS-assisted software RAID.
Futuremark's Servermark VDI came by default with scripts for a VMWare-based virtualization setup. However, being personally used to Hyper-V, I decided to perform the evaluation using Windows Server 2016 TP5.
The RAID-0 volume with the Crucial MX200 SSDs was used as the VM data store. A switch was setup for the VMs and they were uplinked to the main network using one of the 10GBASE-T LAN ports. For home lab setups, this is acceptable, though we must make a note here that the Xeon-D 1540 doesn't support SR-IOV and ends up increasing CPU load for this type of internal switch setup. The newer version of the SYS-5028D-TN4T comes with the Xeon-D 1541. It has better I/O virtualization support, and doesn't have this problem.
In addition to the Hyper-V role, Windows Server 2016 TP5 was also configured with a DHCP Server role on an internal network accessible to both the host OS and the guest OS. Windows 10 VMs were used as guests. Each VM was configured with two network adapters - the first one connected to one of the 10GBASE-T LAN ports of the host and the second one connected to the internal network. Both interfaces were configured as DHCP clients.
In the next section, we will talk about Futuremark Servermark VDI in detail, followed by details of how we benchmarked the Supermicro SYS-5028D-TN4T. At this juncture, we must stress that the Xeon-D machine is not really geared for large-scale virtualization workloads. Our purpose here is to take Futuremark's Servermark VDI for a test drive and create a comprehensive evaluation script that can be utilized for evaluating 'production' machines.
Facebook
Twitter
RSS
18 Comments
View All Comments
ddriver - Thursday, September 1, 2016 - link
Absent a base for comparison, those marks are a meaningless metric.ganeshts - Thursday, September 1, 2016 - link
Intent of the article is to show what Servermark VDI is, and how it can be used. Benchmark numbers are secondary.Also, benchmark numbers can be compared within the same 'set' to show how increasing the load on the server causes a drop in the effectiveness of the server.
Lastly, the benchmark numbers are just PCMark 8 Work preset scores. Plenty of PCs have been evaluated with that benchmark. Here is a graph from our latest mini-PC review with PCMark 8 work scores : http://images.anandtech.com/graphs/graph10595/pcm8... - so that you can get an idea of the performance of the Xeon-D 1540 -equipped server against some of the more powerful mini-PCs.
lioncat55 - Thursday, September 1, 2016 - link
While I did not read the full article and I agree this is more about what the software can do. I believe it would help to see the benchmark numbers from the PCMark 8 Work preset to show how much of a difference there is from a single session to 2+ VMs.It does seem like this server is a bit under powered for VM work. Any word on when there might be a follow up? It would be a lot more interesting to see some numbers that show how much of a change there is as you add more VMs.
extide - Wednesday, September 7, 2016 - link
You probably should have read the article then, because it DOES show the scores as they change vs the number of VM's...powerarmour - Thursday, September 1, 2016 - link
What is the point of this article exactly?, Advertising?ganeshts - Thursday, September 1, 2016 - link
Oh, right.. that is why we mention competitors like SPEC and LoginVSI? Never knew advertising meant talking about competitors in good light!!Futuremark is well respected in benchmarking circles, and we were given the opportunity to beta-test their upcoming product. Anyone interested in performance evaluation of computing systems would jump in at this opportunity. We did, and we thought it would be interesting to share our impressions with readers - they can give their own feedback to Futuremark.
No solicitation is made in this article to get readers to go out and 'purchase' Servermark - In fact, it is not even available for purchase yet.
Don't impugn the editorial integrity of the writers here who spend countless hours in attempting to present a fair and balanced view of hardware and software of interest to the computing community. It is insulting to read comments like this.
powerarmour - Thursday, September 1, 2016 - link
FM is thoroughly decent company which I have no issue with. But I do have issues with articles about software which have no base for comparisons. You might mean well and intend to present in a fair and balanced method, but it sure doesn't come across like it.Then again, if you don't care for the opinions of your readers, then I can clearly see why these kind of articles are becoming more common.
ganeshts - Thursday, September 1, 2016 - link
What is your rationale for saying : "it sure doesn't come across like it" ?You have made an insinuation without properly backing up your comment.
We value opinions of all readers, and we will hear them out if they are presented in the right manner with proper reasoning - definitely can't respond with 'care' if they are off-the-cuff remarks that are baseless and demeaning to the efforts put by the editor.
powerarmour - Thursday, September 1, 2016 - link
My rationale is that this type of software ideally needs to be OS agnostic, and not limited to a Windows platform. There is no critique in the article to mention that, and it comes off as a one dimensional viewpoint, which is quite far away from typical server usage, especially in VM workloads.ganeshts - Thursday, September 1, 2016 - link
That would not be a valid criticism of Futuremark's efforts. The aim of Servermark *VDI* is to test how many virtual desktops a server can support. Almost all office environments that use *VDI* do so with Windows as the guest environment. I do agree that when it comes to *VMs*, a lot of them are Linux-based and the like - but it is very rare to see Linux desktops as part of VDI in office environments.Futuremark had a particular scenario in mind while framing Servermark VDI, and I have to say that they are doing the right thing for that scenario.
What would be valid is that they did not supply benchmarking environment scripts for non-VMWare environments, and we had to develop our own. Futuremark has taken this feedback and promised to provide support for Hyper-V also in the final release. This, I have pointed out as a drawback that Futuremark will address in the future.
We can only point out valid issues in the review, and we have done that on the basis of our hands-on evaluation of the benchmark.