How many mirrors?
Over the past few months we’ve deployed several Storage Spaces Direct solutions, and one question that seems to pop up is the expected performance difference for a 2-way mirror vs a 3-way mirror. The obvious answer is a 2-way will out-perform a 3-way, but by how much?
I won’t go into the resiliency or design decisions or when either option is preferred, this is purely from a performance based perspective..
The chosen system for this test was a fairly cost effective low-end system.
4-node Hyper-Converged. Each node being:
2x Xeon E5-2640 v4 (10C/20HT)
128 GB Memory
2x 400GB SSD in RAID1 for OS
4x 1.92TB SSD SATA 6Gbps
8x 6TB 7.2K SATA 6Gbps
1x Mellanox ConnectX-3 Pro
The tool of choice responsible for our S2D stress testing was none other than VMFleet.
We could do a myriad of tests here, but to keep things simple I used the below settings for our testing. The theory here is to simulate the average workload
Byte Size: 4K
Threads: 4
Queue Depth: 20
Read/Write: 70%/30%
First, the 2-way mirror – approx. 660K IOPS
Now, the 3-way mirror – approx. 520K IOPS
*these tests were ran over a period of 30minutes and the above indicate the average results.
So in this particular deployment, we see approx. 22% performance overhead by using a 3-way mirror. Not that much of a hit considering the resiliency benefits. But again, this is dependent on the deployment..
Don’t use this data as a constant for all environments, it was posted simply to show that a 3-way mirror doesn’t impact performance on a grand scale, just a little bit 
For anyone speculating the performance characteristics of a 2-way vs 3-way mirror, I hope this helps in some way
Enjoy!
Dan