IBM i Storage by the Numbers: Internal vs External SSD
I wrote an article on LinkedIn called “Shocking… IBM i Cost of Storage beyond the CEC: EXP-24S or V3700 SAN?” back in 2016 that sparked many conversations on what place external storage has in the IBM i hardware stack. As a result of the popularity of my previous article, big innovation in the storage market, changes in storage costs, and a much more open IBM i infrastructure stack, it seemed like a really good time to go through the exercise again.
I don’t think it’s my place to sell you on whether internal or external storage is better. Honestly, I believe there is a strong case to be made for both, depending on the customer situation. With that being said I think it’s important I share my experience and knowledge, so you have what you need to make an informed decision for the businesses you serve.
For IBM i clients, typically there are a few reasons why they start to ponder a move to external storage:
- You have multiple locations with separate LPAR’s and want to consolidate segregated systems onto a badass S922 which requires external storage.
- You have a new compliance requirement to encrypt your storage.
- You need to shrink your point in time backup window drastically and heard rumors about external storage tech that can deliver.
- You are moving to HW based replication and need to offload the replication workload to a SAN.
- Maybe it’s as simple as just feeling like you should at least see what’s involved in moving to external storage before you push forward with internal storage because it’s what you’ve always done.
Whatever that reason may be, this article should serve your interests very well. If you still have questions, let’s connect and talk through your needs.
One of the things that have always been quite funny to me is how many people avoid talking about the price at all costs, especially in the B2B space. The reality is value matters and price is a major component of the value equation. So this article is going to serve that up on a silver platter for you.
Let’s get introduced to the contenders of this welterweight battle of IBM i Storage.
On the left side of the screen, we have the rock-solid, reliable, and long-standing workhorse EXP-24SX expansion draw (internal storage) and on the right side of the screen we have the quick, nimble, and scalable V5030E SAN storage (external storage).
|Internal Storage||External SAN Storage|
|EXP24SX Disk Expansion Unit(s)
w/ EJ14 PCIe3 12GB Cache RAID PLUS SAS Adapter Quad-port 6Gb x8
|V5030E SFF CONTROL Enclosure
w/ Cache Upgrade, (2x) 16GB FC ADAPTERS, Software (FLASHCOPY, Remote Mirror, Encryption), Redundant Fiber Switches, & Dual PCIe3 16Gb 2-port Fibre Channel cards for the P9’s.
V5030 SAN Expansion Enclosure(s)
|Disk Unit: 775GB Enterprise SAS 4k SSD for IBM i||Disk Unit: 800GB 2.5 FLASH DRIVE|
The results you will find as you progress through this article include all the components you need, for a solid and resilient storage environment. It does NOT include costs for the Power9 server.
Both the EXP24SX and the V5030E pricing that follows includes redundant cards to connect additional storage outside the CEC. The V5030E SAN has redundant 16GB Fiber cards feeding into redundant rack-mounted 1U Fiber switches and the EXP-24SX has redundant 12GB Quad-port RAID cards to support drawer two and beyond.
In my original article from 2016, SSD was still not widely adopted by IBM i shops, so the configurations built were based on 15k RPM spinning disk. Much has changed in the last 4 years and SSD’s have taken the main stage. It’s fairly rare for us to sell spinning disk anymore and I don’t see a big ramp-up of spinning disk usage in the near future. So, the current configs are using 775GB and 800GB Enterprise-class SSD’s. I thought about getting crazy and factoring in the IBM i disk unit overhead loss resulting from IBM i’s 520-byte sector size into the equation, but will save that for another day.
As you can see from the image above, the V5030E stack is slightly more complex compared to the very straight forward internal expansion units.
As you can imagine the cost of industry-standard disk units is less than the cost of disk units specific to IBM i. This means there would be a point where the added upfront costs of the SAN would be offset by the net savings per drive. Finding that inflection point was my real motivation back in 2016 and I decided it was worth sharing. Back then the transition point for the then V3700 SAN was at drive 58.
So where does the V5030E SAN with all SSD storage take over as the more cost-effective option for IBM i storage? Let’s take a look.
The transition point where V5030E SAN storage actually becomes cheaper then internal IBM storage is at disk number 24. You almost wonder if IBM had some pricing logic that says we want the cost of a fully populated V5030E SAN to cost roughly the same amount as a fully populated drawer of internal disk. Crazy right.
You also see a major spike on the internal storage when you hit drawer number 2. The biggest reason for that is the addition of a pair of RAID Adapters.
Now let us take a look at the cost per GB comparison over the same number of drives. As you can see we have the same transition point and the V5030E has almost half the Cost/GB at our 96th drive. I know for most IBM i shops that would be overkill and you would probably be looking at a much heavier SAN to support those workloads.
I think it’s also important to point out that we are comparing just the total cost of storage outside the CEC. In reality, you could pack 18 additional 775GB SFF3 drives in the IBM i system unit, before you had to even add your first drive. This would give the internal storage a slight head start but remember the actual drawer costs for each option is a small percentage of the overall cost of the solution. The cost is in the disk units themselves. Which makes sense because they are actually doing the work.
I created a similar graph as in chart one but included 18 additional 775GB drives in the IBM Power9 system unit before diving into the first expansion unit. The results really aren’t that much different with the transition point being at drive 28 instead of drive 24.
Richie’s reflection on the results.
It has long been an argument by some folks that external storage added an additional layer of unnecessary complexity into a solid IBM i environment that was prized for its simplicity. I couldn’t argue against that logic back then, and I still won’t today. It is another layer and it requires different tools and knowledge to manage it. With that being said, the tools IBM serves up for Storwize SAN storage are very intuitive and honestly quite pretty when you’re used to pounding a green screen. If SAN storage management is what’s holding you back from at least looking at external storage as an option, send us a note and we can give you a quick tour of the GUI. After you carve up the virtual disk units on the SAN and serve them up to IBM i, you can go right back over to the greener pastures of IBM i.
SAN storage offers some additional features and functions that you can’t get with internal IBM i storage. These include things like Flashcopy (which seems like Houdini magic when you’re trying to explain it to people). Basically, Flashcopy creates a complete copy of your environment in a matter of seconds, which you can use for testing, development, and even backups. Imagine being able to complete a full save with the only impact to your production machine being a few second pause. Or remote mirroring, which allows two SAN storage devices to push changed blocks of data from a source machine to the target. This allows you to remove the replication burden from your Power Server when using a hardware-based HA solution like POWERHA. I would be doing a big disservice if I didn’t mention encryption at the SAN level. With the V5030E, you can encrypt all data at the SAN level. This is huge when you look at the requirements for meeting audits in most business sectors these days.
The last thing I want to touch on (I promise), is the excitement I have for what IBM i and it’s growing options for storage will look like in the coming years. Heck, I didn’t even mention NVMe in this article, which I truly had to hold myself back from. NVMe or Non-Volatile Memory Express is a storage technology that has been around for a little while but has recently gained some big momentum and traction with the IBM install base, including IBM i clients. That tech might do some crazy things to my charts in the coming years.
Let us know if you have any storage-related questions, or if you simply want some help deciding if a move outside IBM i internal storage is right for you.