Performance Consistency

We've been looking at performance consistency since the Intel SSD DC S3700 review in late 2012 and it has become one of the cornerstones of our SSD reviews. Back in the days many SSD vendors were only focusing on high peak performance, which unfortunately came at the cost of sustained performance. In other words, the drives would push high IOPS in certain synthetic scenarios to provide nice marketing numbers, but as soon as you pushed the drive for more than a few minutes you could easily run into hiccups caused by poor performance consistency. 

Once we started exploring IO consistency, nearly all SSD manufacturers made a move to improve consistency and for the 2015 suite, I haven't made any significant changes to the methodology we use to test IO consistency. The biggest change is the move from VDBench to Iometer 1.1.0 as the benchmarking software and I've also extended the test from 2000 seconds to a full hour to ensure that all drives hit steady-state during the test.

For better readability, I now provide bar graphs with the first one being an average IOPS of the last 400 seconds and the second graph displaying the IOPS divided by standard deviation during the same period. Average IOPS provides a quick look into overall performance, but it can easily hide bad consistency, so looking at standard deviation is necessary for a complete look into consistency.

I'm still providing the same scatter graphs too, of course. However, I decided to dump the logarithmic graphs and go linear-only since logarithmic graphs aren't as accurate and can be hard to interpret for those who aren't familiar with them. I provide two graphs: one that includes the whole duration of the test and another that focuses on the last 400 seconds of the test to get a better scope into steady-state performance.

Steady-State 4KB Random Write Performance

The SMI 2256 manages to pull off decent average IOPS under a sustained random IO workload. It can't challenge the 850 EVO that uses faster 3D V-NAND, but compared to the BX100 with SM2246EN and 16nm MLC the drop in performance isn't massive -- better yet the SMI 2256 is quite a bit faster than SanDisk's TLC drive Ultra II. 

Steady-State 4KB Random Write Consistency

Unfortunately, the performance isn't very consistent, though, but then again the SM2246EN isn't either as the BX100 is only marginally better.

SMI2256 500GB
Default
25% Over-Provisioning

The steady-state behavior of the SM2256 appears to be similar to its predecessor SM2246EN. The baseline performance is fairly low at roughly 2,000 IOPS, but bursts occur frequently and go all the way to up to 25K IOPS, although this only lasts for about a second. 

SMI2256 500GB
Default
25% Over-Provisioning
Introduction, The Drive & The Test AnandTech Storage Bench - The Destroyer
POST A COMMENT

35 Comments

View All Comments

  • dcaxax - Thursday, June 18, 2015 - link

    "admittedly Samsung has had some issues with TLC"

    That's the understatement of the Millenium. My 840 and 840 Evo have such severe performance degradation (60-70%) that their speed is now visibly impaired and actually compares with some of the fastest HDDs.
    Considering the price per GB of an SSD that is completely unacceptable and constitutes a ripoff.

    Vendors like Samsung are fobbing off substandard parts supported by immature and unreliable technology to unsuspecting customers.

    This is the equivalent of a car with flashy design and big engine which is guaranteed to develop problems as soon as you drive 100 miles. They have a name for it and it's called 'a lemon'.
    Reply
  • Oxford Guy - Saturday, June 20, 2015 - link

    Apple reportedly abandoned TLC because of the defects. Reply
  • serndipity - Saturday, June 20, 2015 - link

    Have noticed that, as the NAND manufacturers shrunk the die process, issues with both MLC and TLC based SSDs have begun surfacing.

    Glad to see that Samsung, with its 3D NAND technology, used in the 850 PRO and 850 EVO, has been able to return to a much more stable process size (e.g. almost 3X that of current MLC).
    Reply
  • viktorp - Thursday, July 2, 2015 - link

    TLC NAND isn’t welcome at any price.
    Ask yourself simple question – what parts of your personal data will you trust to storage designed specifically to have 10 times less endurance than technology it is supposed to replace (MLC).
    I am upset just thinking that someone thought of it as being a good idea.
    Reply
  • cbjwthwm - Thursday, September 3, 2015 - link

    The big problem here is a newer gen product at 2x the capacity of the equivalent architecture Sandisk Ultra II which for the most part gets outperformed by it. It would be interesting to see what the shipping firmware of this type of product eventually performed like compared to the reference design, but otherwise I see no reason to consider this vs the Marvell-based Ultra II. Reply

Log in

Don't have an account? Sign up now