EVE Information Portal

Can you run Minecraft on it?

Out of curiosity... where will come from all the additional players needed to use/justify such powerful hardware?

We did an intensive comparison with couple of vendors but came to this conclusion out of couple of reasons

This is a vague answer and does not tell you much apart from that we did our du diligence

Plus our relationship with the Icelandic vendor is excellent after decade of cooperation, I literally can call the lead IBM SAN expert anytime 24/7 and they are quick to support us in a critical scenario with good escalation path into IBM

Hope this answer helps and please keep on asking about TQ Tech III

What's the warranty on the new kit? Default 3 year or extended?

Being someone that builds complex, large datacenters for both private and public use on a rather period basis, I'm not that impressed on the path of physical architecture that you are looking at. For whatever reason, why are you looking at a traditional, silo-based solution with storage and compute in different silos and traversing a "slow" FC link to make it work when not hitting the cache in RAM. Any particular reason why you are not looking on a more modern, flexible and scalable platform then the one described in the blogpost?

Seeing that everything not in the RAM-cache have to traverse the FC switch we can quickly give a few numbers on the actual latency and round-trip on several different ways of accessing data on different locations

L1 cache reference 0.5 ns
Branch Mispredict 5 ns
L2 cache reference 7 ns 14x L1 cache
Mutex lock/unlock 25 ns
Main memory reference 100 ns 20x L2 cache, 200x L1 cache
Compress 1KB with Zippy 3,000 ns
Sent 1KB over 1Gbps network 10,000 ns 0.01 ms
Read 4K randomly from SSD 150,000 ns 0.15 ms
Read 1MB sequentially from memory 250,000 ns 0.25 ms
Round trip within datacenter 500,000 ns 0.5 ms
Read 1MB sequentially from SSD 1,000,000 ns 1 ms, 4x memory
Disk seek 10,000,000 ns 10 ms, 20x datacenter round trip
Read 1MB sequentially from disk 20,000,000 ns 20 ms, 80x memory, 20x SSD
Send packet CA -> Netherlands -> CA 150,000,000 ns 150 ms

Looking at the figures, as soon as we start to traverse several layers we add up latency on the whole request, if we need to traverse the FC, to hit the storage nodes, then hit the disk and back, latency can rather quickly add up. Keeping the data as local as possible is the key, mainly in Memory, or as close to the node as possible without traversing the network (Sure, FC is stable, proven and gives a rather low latency, but from other standpoint you could argue that it is dead in the upcoming years as we are moving towards utilizing RDMA over Converged Fabrics or the like).

On another note, if we look on a mainstream enterprise SSD we can find a few figures:
500MB/s Read and 460MB/s Write
If we put these into the following calculation to see when we saturate a traditional storage network:
numSSD = ROUNDUP((numConnections * connBW (in GB/s))/ ssdBW (R or W))

We get the following table:
Network BW SSDs required to saturate network BW
Controller Connectivity Available Network BW Read I/O Write I/O
Dual 4Gb FC 8Gb == 1GB 2 3
Dual 8Gb FC 16Gb == 2GB 4 5
Dual 16Gb FC 32Gb == 4GB 8 9
Dual 1Gb ETH 2Gb == 0.25GB 1 1
Dual 10Gb ETH 20Gb == 2.5GB 5 6

This is without taking into account the roundtrip to access the data, and is counting with unlimited CPU power as this can also become saturated. We can see that we don't need that many SSD to saturate a network. Key point here, try and keep the data as local as possible, once again not traversing the network with the added latency and network limitation.

We can also do calculations on difference on hitting per say a local storage cache in the memory, or hitting a remote storage cache (Per say, SAN controller with caching). I know from the top of my head which are the fastest, key point, once again, keep the data as local as possible.

There are several technologies pin-pointing these issues that are seen in traditional silo datacenters, have you looked at any, and if so what is the reason that these do not fit your needs?

Nerdgasm,

Be interesting to see if you get SQL to perform on top of HyperV, seen it done wrong so many times to the detriment of performance.

Hey dears

Is there anyone around that can hop on and chat regarding storage for a while. Are you using Child pools on Storwize V5k/SVC, is SSD used and in what configuration, is it Easy Tier enabled or just standalone allocation, Flash Copy usage? SVC Generation? Is it 32 or 64GB cache model and are you consider using Compression? What primary protocol is used (for SVC - V5k is definately FC but is it 8 or 16 FC Frontend protocol or 10Gbps iSCSI for Flex connection?

Ouch this is strange consideration.

If you wanna see close numbers lets assume they going to have 8 SSD drives each 800GB capacity in RAID 5 (so one global spare = 9)
And 80 drives 1.2TB each 10k rpm SAS all in RAID 5 = 8 drives for each RAID ( 7+Parity) and 3 Global spare - there we go for basic Storwize V5000 configuration in Dev Blog.

Lets assume all of them are in one pool so have overall capacity available for mapping - 84000 GB (SAS) + 5600 GB (SSD)
Lets say we have one host connected through 4 x 8Gbps FC from server side and 8 x 8Gbps FC from Storage side with allocated usage of capacity at 70000 GB (not gonna push the limits.

According to what Storwize is capable of. Lets not be conservative and load it a little bith with 16KiB block size for transfers and start fairly from 5000 IOPS

Assuming Cache statistic
Read Percentage - 70%
Read Sequential - 20%
Read hit - 60%
Random Read hit - 40%
Sequential Read hit - 20%
Write Percentage - 30%
20% of all writes will be sequential
Seek percentage - 33%
Random Write Efficiency - 35%


At 5000 IOPS we will be here:
Total Service Time: 1.0 ms
Read Service Time: 1.3 ms
Write Service Time: 0.2 ms
Channel Queue Time: 0.0 ms
Processor Utilization for I/O: 1.5 %
Channel Utilization: 1.2 %
Host Adapter Utilization: 1.1 %
SAS Interface Utilization: 3.3 %
Flash Drive Utilization: 2.2 %
SAS 10K Drive Utilization: 9.0 %

While increasing load before drive get 60-70%
it will be close to this metrics:


Service Time with IO rate growth

And

Highest SAS interface utilization with IO rate growth

So you might be right considering 10k as for clear SAS performance (tho it still depend from a lot of factors) But you definitely not right saying 10k IOPS while there is SSD and Tiering involved.

As of Utilization

Utilization overview with IO rate


Edited.
And this is not considering how powerful SVC is that stands on top of this :)

And I'm sure there is more than 70% read :)

I hear you when it comes to a rewrite on the concept when it comes to processing the DB in memory, but in this case this was not my intention, I'm mainly talking about a storage solution that for most of the part scales better without any limitation in the SAN nodes, SAN network and the like, while also giving increased performance with storage cache in the actual compute node, access to data without traversing a SAN network (Latency is latency, and no matter how high or low it adds up to every transaction).

There are several HCI solutions on the market, giving same or better performance as a high-end SAN, smaller footprint in Us, lower energy-consumption, lower cooling needs, scales linear and you scale only when you need to.

HCI ain't a one-size fits all, henche the questions.

As a DBA who's just recently started to get invovled on the SAN storage side, everything you say its well beyond me :) But it's interesting!

Can you give some real examples of what you are talking about and not just buzz words? :)

Edit: ok, so looking here: http://purestorageguy.com/2015/03/12/hyper-converged-infrastructures-are-not-storage-arrays/

This seems to be how hadoop and these other similar systems work? It's a bunch of servers with local disks that sit behind some shared filesystem to distribute the data cross all the server nodes?