Understanding Storage Options for IBM i

In the past when it came to data center infrastructure and specifically servers, most of the hyped up innovation was focused around the CPU, peripherals, and higher memory ceilings.  For many years disk wasn’t really that sexy.  For most IBM i shops all that mattered was having enough total storage, enough arms to support I/O needs, and a solid raid configuration for resiliency.  Most customers had spinning 15k hard drives up until SSD drives (Solid State Disk) became more readily available and financially palatable.

SSD and more specifically the underlying flash technology they are built on, paved the way for big innovation and massive performance gains at the storage level.  These innovations were so impactful for overall compute performance due to storage becoming a major bottleneck as a result of CPU and interconnect speeds increasing over the years, while storage tech stayed fairly stagnant.

Today Flash technology is driving big I/O and drastically increasing the density of storage in the data center.  In this article we are going to look at these drive options, what makes them unique, how they perform, and where they are available.

Below is a diagram we will reference throughout the rest of the article.

SAS HDD’s

These drives have much higher latency, longer read/write times, and as a result, are much slower than the other drive options we will talk about here.  These drives use the SCSI protocol to communicate, which we will touch on later. The biggest reason for the speed is that these drives have a motor-driven spindle that holds flat circular disks (called platters).  They are coated with a thin layer of magnetic material. Read-and-write heads are positioned on top of each plater that moves back and forth as they read and write to disk.  While these platters spin extremely fast at 15,000 RPM (Revolutions per minute) and the heads moving rapidly across them, it’s still subject to physical movement and has limitations.

Suggestion: If you get a chance google “slow motion video of hard drive seeking”. It’s incredible to watch them in action.

SAS SSD’s

This is our entry point into Flash storage.  As you can see represented in the red outlined gray box in the above image, all the remaining drive options are built with Flash storage. SAS based SSD’s have no moving parts and are essentially a memory chip or interconnected integrated circuit.  As a result of not having any physical movement required for operation they have much lower latency, faster read/writes speeds, and support more IOPs (input-output operations per second) compared to HDD.  As you can see in the yellow shaded section of the above chart, SAS-based SSD’s communicate across the same SCSI protocol originally built to support spinning disk. This is a key concept to understand as we discuss the difference between the remaining drive options in this article.

Let’s do some quick math here.

  • 16x 15k RPM HDD’s that give you roughly 200 IOPS per drive. You are looking at 3,200 IOPS from your storage.  SCSI is more than capable of supporting this level of IO.
  • 16x SAS based SSD’s that give you roughly 6,000 IOPS per drive. Over the SCSI interface, you are looking at 96,000 IOPS from your storage.

I know this is an oversimplification here and there are many factors that play into actual IO performance but bear with me.  The thing to know here is that SSD’s and lack of moving parts offer up much higher IO performance than spinning disk.

SAS SSD’s have been supported for internal IBM i storage on IBM Power Systems for many years now and are still supported today.  They are also supported in IBM FlashSystem SAN storage and can be combined with higher performance drives and EasyTier in a tiered storage configuration.

NVMe SSD’s

Without further ado, I would like to introduce you to NVMe SSD’s.  These drives are making a massive splash in our modern data centers.  Probably shouldn’t talk about water and data center in the same sentence but thought I would keep you on your toes.  NVMe drives have essentially the same NAND Flash technology under the covers as SAS SSD’s but operate across the PCIe bus instead of SCSI.  This also greatly simplifies the logic for getting things on and off disk as well as widens the path.  Think of NVMe SSD’s as our introduction to how Flash storage technology can perform when you take the SCSI governor off them.  Data is all over the place as far as performance increases go and range from 10x to 100x the throughput as SAS SSD’s.

Think about the impact this has within the data center from a rack density standpoint.  Ten years ago a company would have had drawers and drawers of 15k spinning disks stacked up in their racks to support the IO needs of their business applications.  Now they can get more throughput with a handful of NVMe drives in a 1-2U SAN.  Think about what that storage density means for the modern data center.  What it delivers from an energy consumption standpoint.

When it comes to IBM i storage specifically, these drives are great because they are supported in IBM Power9 servers.  A couple of caveats, you need to be at IBM i 7.4 and they are installed as PCIe cards in the back of the server.  This can cause a bit of a real estate issue when it comes to available card slots in the smaller Power9 machines.  You can always add on a PCIe expansion drawer if need be.

IBM Flashcore Modules

These drives are NVMe with a supercharger and safety features you won’t be able to find anywhere else.  I know, sounds like a car commercial right.  IBM Flashcore modules deliver extreme performance, ultra-low latency, greater density, and mission-critical reliability with purpose-engineered NVMe flash storage modules.  IBM took similar NAND flash storage and packed it on two boards instead of a single board you would find in SAS SSD’s and NVMe units.  FlashCore Modules have a decent amount of extra Flash (overprovisioned) to support the replacement of failing flash cells. They then packed in the following to increase performance and reliability.

  • Large Controller – Serves as the brain of the FCM and has approximately 10x the performance and capabilities of a commodity flash controller. It facilitates much of the advanced function and logic which provide FCMs with their uniqueness.
  • DRAM – FCMs contain a fairly large amount of DRAM which is used to handle extensive high-performance Write operations as well as provide a place for compression tables to be handled.
  • Capacitors – designed to maintain power long enough during a power loss to facilitate the completion of outstanding transactions involving DRAM, MRAM, NAND flash, and the FPGA Controller.

This results in extremely high storage density with module sizes all the way up to 38.4TB.  With powerful 2:1 inline hardware-accelerated compression and encryption on each FlashCore Module that results in zero performance impact. Compressing a 38.4TB drive at the 2:1 guarantee means you can pack 76.8TB on a single FCM drive.  FCM drives also include advanced Smart data placement logic that positions data for optimal performance and health.

IBM Flashcore modules are only available on IBM FlashSystem SAN storage which compliments IBM i very well.  Unfortunately, this technology is not supported as internal IBM i storage at the Power System level.  With that being said, external storage offers up some great benefits for IBM i shops with Encryption at rest, Flashcopy for backups, SAN to SAN remote mirroring, easy tier, and immutable safeguarded copy services.

Conclusion

Storage has come a long way in the last few years with the advancement and increased approachability of Flash.  Utilization of the PCIe bus for connectivity stands out to me as the anchor for all this amazing tech and releasing Flash storage from the confines of the SCSI protocol.  All these drive options have their place in the data center but with applications demanding more and more from their storage infrastructure, I believe the days of seeing racks of spinning disk are numbered.

More from this month:

Leave a Comment

Your email address will not be published.