Performance Consistency

We've been looking at performance consistency since the Intel SSD DC S3700 review in late 2012 and it has become one of the cornerstones of our SSD reviews. Back in the days many SSD vendors were only focusing on high peak performance, which unfortunately came at the cost of sustained performance. In other words, the drives would push high IOPS in certain synthetic scenarios to provide nice marketing numbers, but as soon as you pushed the drive for more than a few minutes you could easily run into hiccups caused by poor performance consistency. 

Once we started exploring IO consistency, nearly all SSD manufacturers made a move to improve consistency and for the 2015 suite, I haven't made any significant changes to the methodology we use to test IO consistency. The biggest change is the move from VDBench to Iometer 1.1.0 as the benchmarking software and I've also extended the test from 2000 seconds to a full hour to ensure that all drives hit steady-state during the test.

For better readability, I now provide bar graphs with the first one being an average IOPS of the last 400 seconds and the second graph displaying the IOPS divided by standard deviation during the same period. Average IOPS provides a quick look into overall performance, but it can easily hide bad consistency, so looking at standard deviation is necessary for a complete look into consistency.

I'm still providing the same scatter graphs too, of course. However, I decided to dump the logarithmic graphs and go linear-only since logarithmic graphs aren't as accurate and can be hard to interpret for those who aren't familiar with them. I provide two graphs: one that includes the whole duration of the test and another that focuses on the last 400 seconds of the test to get a better scope into steady-state performance.

Steady-State 4KB Random Write Performance

The SMI 2256 manages to pull off decent average IOPS under a sustained random IO workload. It can't challenge the 850 EVO that uses faster 3D V-NAND, but compared to the BX100 with SM2246EN and 16nm MLC the drop in performance isn't massive -- better yet the SMI 2256 is quite a bit faster than SanDisk's TLC drive Ultra II. 

Steady-State 4KB Random Write Consistency

Unfortunately, the performance isn't very consistent, though, but then again the SM2246EN isn't either as the BX100 is only marginally better.

SMI2256 500GB
Default
25% Over-Provisioning

The steady-state behavior of the SM2256 appears to be similar to its predecessor SM2246EN. The baseline performance is fairly low at roughly 2,000 IOPS, but bursts occur frequently and go all the way to up to 25K IOPS, although this only lasts for about a second. 

SMI2256 500GB
Default
25% Over-Provisioning
Introduction, The Drive & The Test AnandTech Storage Bench - The Destroyer
POST A COMMENT

35 Comments

View All Comments

  • watzupken - Wednesday, June 17, 2015 - link

    Looking at the state of the Samsung 840, I am still not convinced with TLC based SSDs. With value in mind, SSD is also walking down the path of cheap but unreliable storage solutions from my opinion. E.g. mechanical drives used to last a very long time, but not now even though they are cheap. Reply
  • leexgx - Saturday, June 20, 2015 - link

    the problem with 840 and 840 evo is just that its not refreshing the cells when it should be (the data is still retained even if its slow doing it) SSDs problem with retaining data is an issue towards end of life but that happens on all SSDs (more a concern for commercial use then consumer drives) Reply
  • der - Wednesday, June 17, 2015 - link

    Wow great! Reply
  • RU482 - Wednesday, June 17, 2015 - link

    IS THAT....A HAIR? Reply
  • i7 - Wednesday, June 17, 2015 - link

    Looks like it to me. Reply
  • KAlmquist - Wednesday, June 17, 2015 - link

    "OEMs can't price their TLC drives similarly to the MLC ones and expect it to be a good sale."

    Agreed. If you look at the prices of Samsung's 850 series, it's around $0.90 per GB of nand cells plus a fixed cost of $30. So you can get an MLC model for better performance at $0.45/GB of capacity, or a TLC model with lower performance at $0.30/GB of capacity. If that type of pricing is adopted by other SSD manufacturers, then TLC becomes very tempting; otherwise not so much.

    The other thing about the 850 line is that the relatively large cell size associated with 3D Nand appears to have eliminated the problem with data deterioration that we saw on the 840 EVO. So TLC will become more attractive next year when 3D Nand becomes available from other manufacturers.
    Reply
  • nwarawa - Wednesday, June 17, 2015 - link

    10% my tush. Try 20%+. Considering I can get a good MLC 256GB-class drive like a BX100 right now for around $100, if they can't get a similar TLC drive under $80, I won't even give it a second glace. Reply
  • revanchrist - Wednesday, June 17, 2015 - link

    TLC is going to be real cheap. Tigo, a Chinese 3rd-party ssd manufacturer has announced its tlc ssd based on Silicon Motion controller and SK Hynix nand chips last week, available in quantity up to 2TB. They've only disclosed the price of the 240gb model, which is RMB 399, roughly 65 USD. FFS that could translate to 260 USD for a 1TB model OMG. Reply
  • revanchrist - Wednesday, June 17, 2015 - link

    The controller is exactly SM2256 and the nand is 16nm 128Gb TLC from SK Hynix. Reply
  • npz - Thursday, June 18, 2015 - link

    It worries me that "real cheap" will mean actually worse value, or disproportionately worse performance and reliability in relation to the lower price. With 2D, tiny process NAND, TLC relies more heavily on ECC which means less performance and more potential reliability issues/bugs, and more spare capacity which means prevents going "real cheap", unless you want to sacrifice long term reliability. Reply

Log in

Don't have an account? Sign up now