Options for IBM Storage and POWER9

Patrick Nip

Almost three decades ago, back in 1997, IBM announced the flagship 9337 Disk Array Subsystem, a rack-mount disk unit that offers 1,084 MB to 33.55 GB of DASD for the AS/400 9406 Models B, D, E, and F as well as AS/400 Advanced System Model.  Back in the early ’80s, I engaged in a benchmark of the 9337 Disk Subsystem on an enterprise AS/400 against the EMC Symmetric system in Rochester, Minnesota.  I also presented RAID technology at a local user group meeting, explaining how to protect the data on the 9337 storage for the AS/400 with RAID 5 technology.

We have come a long way in storage for the Power System. In addition to internal hard disk drives (HDDs), there are Solid State Drives (SSDs), Non-Volatile Memory express (NVMe) drives, and the external IBM FlashSystems with all NVMe storage option. Gone are the days when AS/400 internal storage was usually the first choice in achieving the best performance. This blog discusses various IBM storage options for POWER9 and highlights the cost difference of deploying different storage options on a FlashSystem fiber attached to a POWER9.

Internal Storage for POWER9

Internal storage is generally the storage of choice for small entry-level systems like the 4-core S914.  This server supports a maximum of 10 HDD drives or 10 SSDs or a combination of 10 HDDs and SSDs in the system unit.  Alternatively, in the system unit, a maximum of two NVMe PCIe devices (no SAS drives) are allowed, but SAS drives located in I/O drawers such as the EXP24SX (#ESLS) are not supported.  However, attachment to SANs is supported on the 4-core S914. Many storage options are available for systems with six cores and up. The choice depends on what backplane is used and whether external storage will be a better option.

The following is a summary of storage backplane features for POWER9, S914, and S924, for IBM i.

Table 1.  POWER9 Backplane Storage Features

Backplane Description Supported Write Cache Protection Comments
EJ1C(1)

 

Base storage backplane with 12 SFF-3 bays with an optional split card (EJ1E) to support split backplane capability. 5xx or 4-KB HDD or SSD are supported No write cache.

 

RAID 0, 5, 6 or 10.

 

Use EJ1D or EJ1M for write cache protection.
EJ1E(1)

 

Adds a second SAS controller, the EJ1C can be split into two sets of 6 SFF-3 bays, each with one SAS controller in the system unit. 5xx or 4-KB HDD or SSD are supported

 

No write cache.

 

 

RAID 0, 5, 6 or 10.

 

 

 Use EJ1D or EJ1M for write cache protection.
 

EJ1D

 

 

 

18 SFF-3 bays/Dual IOA with Write Cache.

 

 

 

5xx or 4-KB HDD or SSD are supported

 

 

1.8GB write cache, provides up to 7.2 GB

with compression.

RAID 0, 5, 6 or 10, RAID 5T2(2), RAID 6T2(2), or RAID 10T2.(2) Easy tier support, moving hot data to SSD and cold data to HDD.
EJ1M

 

 

 

 

12 SFF-3 Bays/RDX Bays with write cache and optional external SAS port.

 

 

 

5xx or 4-KB HDD or SSD are supported

 

 

 

Dual integrated SAS controllers with write cache. RAID 0, 5, 6 or 10, RAID 5T2(2), RAID 6T2(2), or RAID 10T2.(2)

 

Easy tier support – moving hot data to SSD and cold data to HDD.

 

EC59

 

 

Optional Non-Volatile Memory express (NVMe) carrier card with two M.2 module slots PCIe3 x8 1.6TB, 3.2TB or 6.4TB NVMe adapter

 

Maximum of 3 EC6V (1.6TB), EC6X (3.2TB) or EC6Z (6.4TB).

Note (1) IBM i Clients with write-sensitive HDD workloads should upgrade from the base storage backplane (EJ1C/EJ1E) to the expanded function storage backplanes (EJ1M/EJ1D) to gain the performance advantage of write cache.  Except for the 4-core S914, EXP24SX SAS Storage Enclosure (ESLS) and EXP12SX SAS Storage Enclosure ((ESLL) can be used to increase storage capacity for S914 and S924.

Note (2) RAID 5T2, RAID 6T2 and RAID 10T2 are RAID protection for two different tiers of disks.   RAID 5 stripes data and writes parity data on all drives across Tier 0 SSD disks and Tier 1 HDD disks, with the capacity equivalent of one parity drive per disk array.  RAID 6T2 uses the capacity equivalent of two parity drives on Tier 0 SSD and two parity drives on Tier 1 HDD disks arrays.  RAID 10T2 stripes data across all disks on each array on both Tier 0 SSD and Tier 1 HDD.  All disks in the array in both tiers are mirrored.

A brief summary on disk storage protection for POWER9 is detailed in Table 2 below.

Table 2. Storage protection using RAID protection for POWER9

RAID Protection for POWER9 – Symbols used: F = Fair, G = Good, VG = Very Good, EXC = Excellent
RAID Type Protection Level Configuration Per Array Parity Drive Capacity Per Array Usable Capacity Performance
RAID 0 None Minimum: 1 drive 0 100% Read: VG; Write: EXC
Maximum: 32 drives
RAID 5 VG Minimum: 3 drives 1 67% to Read: VG;  Write: G
Maximum: 32 drives 94%
RAID 6 EXC Minimum: 4 drives 2 50% to Read: VG; Write: G
Maximum: 32 drives 89%
RAID 10 EXC Minimum: 2 drives Mirrored 50% Read: EXC; Write VG
Maximum: 32 drives

The most common and cost-effective storage protection for POWER9 client is RAID 5 plus one hot-spare for each array.  Should one of the disk drive fail, the hot-spare will take over while the failed drive in the array is replaced, greatly reducing the degraded mode penalty without hot-spare. When two drives in the same RAID 5 array fails, the array will be down. A better protection scheme than RAID 5 is RAID 6 with two hot-spares. RAID 6 arrays stripe data across all drives but uses the capacity equivalent of two drives for parity data per array. Even when two disk drives in the same array fail, the array will continue to function with the hot-spares in the array while the failed disks are being replaced. Should a third disk fail in the same array, the array will be in a failed state.  There is a slight write penalty for RAID 6 with parity data written to two drives in the array.  Such a write penalty is greatly diminished with the use of SSD.

External Storage for POWER9

Many external storage options are now available for POWER9.  In this discussion, we focus on the IBM All-Flash and Hybrid FlashSystems, which include the FlashSystem 5000, 7200, 9200, and 9200R.  The IBM FlashSystem 5000, 7200, and 9200 have a latency of 70µS, offering two times (2X) faster throughput than the high-end EMC PowerMax at 100 µS.  Data reduction pools enable selectable data to be reduced to optimize the investment and have a guaranteed reduction level of up to 5:1.  Utilizing hardware-accelerated compression, the IBM FlashSystem achieves the highest performance with the IBM FlashCore Module.

Earlier this year, IBM simplifies the FlashSystem family as shown in Table 3 below.

Table 3. New FlashSystem Summary

Previous Systems New FlashSystems in 2020
All Flash Hybrid Flash
Storwize V5010E FlashSystem 5010 FlashSystem 5010H
Storwize V5030E FlashSystem 5030 FlashSystem 5030H
Storwize V5100, V5100F FlashSystem 5100 FlashSystem 5100H
Storwize V7000 Gen 3 FlashSystem 7200 FlashSystem 7200H
FlashSystem 9100 Family FlashSystem 9200 Family
FlashSystem 9100
FlashSystem 9150 FlashSystem 9200
FlashSystem 9200R

With IBM Spectrum Virtualize, the IBM FlashSystems offer POWER9 clients a best of breed external storage that is low cost, less complex and has the best innovation in the industry.  The FlashSystem 5100 provides unsurpassed value and performance to most entry to mid-tier IBM i clients.

IBM FlashSystem 5100, formerly Storwize V5100, is a virtualized, software-defined storage system that is most popular with IBM i POWER9.  It is designed to consolidate workloads into a single storage system for ease of management, offering highly scalable capacity, high performance, and high availability.

[ View IBM FlashSystem 5100 Data Sheet ]

The IBM FlashSystem 5100 model 4H4, an all-flash storage system, is used in a cost comparison analysis detailed below.  The small form factor (SFF) control enclosure of the FS5100 model 4H4 features two node canisters, each with an 8-core processor and integrated hardware-assisted compression acceleration.  64 GB cache (32 GB per canister) is standard with options from 192 GB to 576 GB (per system), with eight 10 Gb Ethernet ports available for iSCSI connectivity.  FS5100 also provides 16 or 32 Gb Fiber Channel (FC) connectivity options with FC-NVMe support and accommodates up to 24 2.5-inch NVMe flash drives in a 2U, 19-inch rack-mount enclosure.

The following cost comparison examines the cost difference of attaching a FlashSystem 5100 model 4H4 with different types of storage drives to a POWER9 S914 or the S924.  The 5100 4H4 FlashSystem was configured with the following common features:

  • (1) 16 GB fiber channel 4-port adapter
  • 128 GB cache upgrade
  • 120 GB M.2 boot-drive
  • Raw 27 TB storage

Three different types of storage drives were used for each of the FS5100 FlashSystems:

  • System 1: (30) 900 GB 10K HDD with (2) 24-G Enclosure. This system is used as a base for comparison
  • System 2: (7) 3.8 TB 2.5” Flash Drives
  • System 3: (7) 3.8 TB 2.5” NVMe Drives

Summary of the cost deltas are highlighted in Table 4 below. 

Table 4. Comparing the cost of drive storage options for FS5100 Model 4H4

Features of IBM FlashSystem FS5100 Model 4H4
System 1, HDD System 2, Flash System 3, NVMe
Base Common HW Features (1) 16 G Fiber Channel 4-port Adapter Pair
128 GB cache upgrade
120 GB M.2 Boot-drive
Common SW Functions Included Encryption enablement
Flash Copy
Spectrum Virtualize
24X7 SWMA Y Y Y
Storage drive features (30) 900GB 10K HDDs (7) 3.8 TB 2.5″ Flash Drives (7) 3.8 TB NVMe 2.5″ NVMe Flash Drives
Additional HW features required (2) 2077-24G Expansion Enclosure (1) 2077-24G SFF Expansion Enclosure All in the FS5100 NVMe Enclosure
Additional SW and SWMA required Yes, X2 Yes, X1 No
Cost System 1, HDD System 2, Flash System 3, NVMe
Hardware $89,249 $119,477 $106,027
Software(1) $32,753 $24,262 $15,771
Total(2) $122,002 $143,739  

$121,798

Cost delta compared to Base System 1 Base $21,717 -$204
Cost delta compared with Flash Drives -$21,941
Cost of storage drives alone(3) $33,090 $66,234 $60,200

 

Observations in this FS5100 Model 4H4 cost comparison:

  • Software for the FS5100 using HDDs is the highest because two (2) 2770-24G SFF Enclosure units are required for the 30 units of 900 GB 10K drives. Each enclosure requires its own SW and SWMA for IBM Spectrum Virtualize.
  • It is no surprise that FlashSystem FS5100 with NVMe is the best choice:
    1. System 3 is almost $22K cheaper with AET3 NVMe Flash Drives than System 2 using the ACM1 Flash Drives.
    2. System 3 with all NVMe Flash Drive is even slightly cheaper than System 1 with 10K HDD and the performance of NVMe is far superior to the traditional HDDs.
    3. System 2 with the ACM1 Flash Drives is the most expensive, due to the cost of the drives and the additional cost of the expansion enclosure for the ACM1. In this particular example, the FS5100 is an all-Flash model with a default NVMe control enclosure that houses NVMe drives.  A 2770-24G Enclosure is required for the ACM1 drives.
  • ATE3 NVMe Flash Drives cost less than the ACM1 Flash Drives. Surprised?

Is Internal or External Storage better for POWER9?

The discussion above highlights the internal and external storage options for POWER9, with a focus for the entry and mid-level POWER9 system clients. One often asks, “What is better, internal or external storage”?  The answer is, it depends. It depends on the total amount of storage you need, the performance requirements throughout the system, IT budget, and a lot of other things. But one thing is clear, IBM Power Systems running IBM i remains a robust and viable option for clients who run their critical applications on this proven technology.

Please refer to another iTech blog that illustrates an example of internal vs external storage here.

Feel free to contact iTech Solutions if you have any questions on Power system storage, both internal and external, or if you like to explore storage options for your Power system. We can help. You can contact us here. 

View More Articles from this Month:

Leave a Reply

Your email address will not be published. Required fields are marked *

*