These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

Out of Pod Experience

 
  • Topic is locked indefinitely.
 

Advice: Fixing the Bottleneck

Author
Xenuria
#1 - 2012-11-23 22:44:25 UTC  |  Edited by: Xenuria
~Consider the following~

You have a computer with a SSD that has a 6GB/s Transfer Rate. The OS is Windows 8 64bit Pro.

What hardware is required to prevent the SSD from being bottle-necked?


Rules: You must list the full name of the part(s) required as well as where/how it can be purchased/built.

~Clarifications~


Price is no object
This is not for gaming
The computer needs to be able to actually utilize 6Gb/s transfer rates.
OBVIOUSLY the motherboard needs to be SATA 3 capable.
Bane Necran
Appono Astos
#2 - 2012-11-24 00:25:52 UTC  |  Edited by: Bane Necran
You'd need a motherboard with a 6gb SATA.

There are too many to name one specifically.

"In the void is virtue, and no evil. Wisdom has existence, principle has existence, the Way has existence, spirit is nothingness." ~Miyamoto Musashi

Angelo Doelman
Bacon Diplomacy Project
#3 - 2012-11-24 08:37:24 UTC
Xenuria wrote:
~Consider the following~

You have a computer with a SSD that has a 6GB/s Transfer Rate. The OS is Windows 8 64bit Pro.

What hardware is required to prevent the SSD from being bottle-necked?


Rules: You must list the full name of the part(s) required as well as where/how it can be purchased/built.

Everything is the bottleneck. Or at a minimum, something is.

If you have a fast data retrieval and/or storage device (these rates are asymmetrical on all primary storage (SSD/HDD) and thus affect each other), you can still be bottlenecked by the CPU. If the CPU can't process the bits fast(er) than the SSD can supply them, then a ceiling is hit.

If your memory is designed in such a way that it doesn't maximize throughput to the CPU, there is another ceiling. If the chipset sitting between the memory and CPU can't handle the throughput, there is another ceiling.

If your hard drive controller is sub-optimal, there is yet another ceiling.

If you are performing any type of fault-tolerance on the drive (volume), other than JOBD or RAID 0 (both of which aren't tolerant of fault, just to let you know), there is another ceiling. RAID1 doesn't have a calculation bottleneck, but crappy controllers and/or drivers might not be able to perform simultaneous read/write actions.

If a user can't supply input and process output in such a way that a hard drive is not fully consumed, there is another ceiling.

If there is power management, a ceiling has been met.

If there are poor drivers written for any of the aforementioned subsystems, including subsystems not mentioned.... there is a ceiling.

As you can see here, there are plenty of reasons why a uber-performance drive could perform similar to a good-performing drive.

After all that scarasm... here is my only real advice: Do not shoot for the most expensive thing(s). There is a performance / price curve for everything (at least computer-related). That curve has an 'elbow' in it. Buy at the elbow. This will grant you the most performance for the least cost per unit of performance.

As you can see from above there are plenty of legitimate reasons why a "6GB" SATA drive won't actually provide that throughput.... set aside marketing, forever.

Johnny Bloomington
The Scope
Gallente Federation
#4 - 2012-11-24 08:41:44 UTC
CCP is the bottleneck... Your PC well be old by the time you get WiS. Lol

CCP wish list: show damage on ships and open that door!

Xenuria
#5 - 2012-11-24 15:56:25 UTC
Johnny Bloomington wrote:
CCP is the bottleneck... Your PC well be old by the time you get WiS. Lol


Funny but in all seriousness I wanted some solid answers.


What kind of RAM or CPU would I need to be able to not be bottle-necked?

I don't know how to convert GHz to Gbps. Something tells me though that the RAM would be the first thing to cause problems. I cannot find any ram on the market that is as fast as a CPU.
Bane Necran
Appono Astos
#6 - 2012-11-24 16:18:21 UTC  |  Edited by: Bane Necran
Angelo Doelman wrote:
As you can see from above there are plenty of legitimate reasons why a "6GB" SATA drive won't actually provide that throughput.


The main selling point for 6gb SATA on a motherboard is so you can use a SSD properly.

The reason you don't hear that said about any other component is because it's the most important bottleneck, by far. Fixing it would improve performance much more than everything you mentioned combined. You probably even know that. If the person has the funds, then by all means, they should just go out and buy a whole new system, but it didn't sound like that's what they wanted to do.

And you're talking about CPUs that would also require a motherboard with a 6gb SATA in order to perform at the speed desired, and acting like the motherboard isn't important. If he runs out and follows your advice to buy a CPU, it'll still be seriously bottlenecked by the older 3gb SATA motherboard, and again, you probably know this.

"In the void is virtue, and no evil. Wisdom has existence, principle has existence, the Way has existence, spirit is nothingness." ~Miyamoto Musashi

Angelo Doelman
Bacon Diplomacy Project
#7 - 2012-11-24 17:40:26 UTC
Xenuria wrote:
Johnny Bloomington wrote:
CCP is the bottleneck... Your PC well be old by the time you get WiS. Lol


Funny but in all seriousness I wanted some solid answers.


What kind of RAM or CPU would I need to be able to not be bottle-necked?

I don't know how to convert GHz to Gbps. Something tells me though that the RAM would be the first thing to cause problems. I cannot find any ram on the market that is as fast as a CPU.


I will take the liberty and assume that this computer is for personal use. If not, then please let me know. What I can tell you from my experiences and analysis, a personal computer's storage subsystem is rarely the actual reason why your computer is 'slow'. The reason for this is because the drive for a PERSONAL, GAMING computer is only really taxed when loading a game. After that action is performed, the drive is only occasionally touched in order to grab additional information or to page out bits between real memory and virtual memory.

This can be confirmed by opening Performance Monitor and fire up the following monitors:
Logical Disk \ % Idle Time
Logical Disk \ Avg. Disk Read Queue Length
Logical Disk \ Avg. Queue Length
Logical Disk \ Avg. Disk Write Queue Length

When the queue lengths are twice the number of drives that you have in the volume, then you are possibly experiencing a drive bottleneck.

But here is the rub: How long does the queue length stay greater than twice the number of drives you have? If it is only for seconds, then is that really your bottleneck? Is it worth spending hundreds of dollars to possibly eliminate a few seconds of suspect bottleneck?

Also take a look at that % Idle Time. This is the amount of time that your drive is actually doing nothing. That needs to be factored into your analysis.

I just did a 1 minute, 40 second analysis on my laptop, loading Eve... this laptop is a business-grade, four-year old laptop (means it wasn't built for awesome performance, just reliability). My disk idle time average was 73.5%. My Average Disk Read Queue was .339. My average disk write queue was 0.081. My average disk queue was 0.420.

Now, the max for my write queue was 1.152. The max for the read queue was 1.889. But these were instantenous readings, not sustained readings.

Now, from just these numbers, you'd think that eve would play awesomely on this laptop. Far from it. In fact I rarely undock if using this thing because the performance just sucks.

Which goes to my original point... the bottleneck isn't always where you suspect it lies and only proper analysis may help tease out information that could point you towards it.
Xenuria
#8 - 2012-11-24 19:31:21 UTC  |  Edited by: Xenuria
~Clarifications~

Price is no object
This is not for gaming
The computer needs to be able to actually utilize 6Gb/s transfer rates.
OBVIOUSLY the motherboard needs to be SATA 3 capable.



The main problem I am running into myself is finding components like RAM, CPU, etc.
I will reiterate that I do not have very good math skills and I have no idea how to convert mhz to Gb/s. However the question remains, if it is possible to build a system that has a 6Gb/s Push all the way through then How?
Angelo Doelman
Bacon Diplomacy Project
#9 - 2012-11-24 23:21:22 UTC
Xenuria wrote:
~Clarifications~

Price is no object
This is not for gaming
The computer needs to be able to actually utilize 6Gb/s transfer rates.
OBVIOUSLY the motherboard needs to be SATA 3 capable.



The main problem I am running into myself is finding components like RAM, CPU, etc.
I will reiterate that I do not have very good math skills and I have no idea how to convert mhz to Gb/s. However the question remains, if it is possible to build a system that has a 6Gb/s Push all the way through then How?

There is no conversion between mhz and Gb/sec.

Obviously.

Why do you suspect that the hard drive is your limiting factor (as I have asked previously) in this complex equation? (Don't answer, I don't care anymore.)

I'm tired of your obstinate behavior, but I will leave you with this last bit of advice: hard drives scale out really well. This means that whatever your solution will be, it will likely contain a great deal of drives. Probably in the neighborhood of 6-12 of them. I have seen LSI (Hard Drive Controller Manufacturer) benchmarks using eight drives on a high-end controller pushing 6GB (capital B) / second. This should excite you. This also shouldn't really excite you, since benchmarks can't tell you how your system is going to perform under production loads. It can only say that under the specific condition(s) that the benchmark tested, performed a certain way.

Since price is no object, I honestly suggest that you hire someone to develop your hard drive subsystem. You obviously have no clue what you are talking about or looking for. Let someone who does know what they're talking about submit a proposal to you for this. This way you won't be disappointed with the results, because you'll have someone to go back to if it doesn't meet your requirements.

Good luck!
Bane Necran
Appono Astos
#10 - 2012-11-24 23:41:07 UTC
Xenuria wrote:
The main problem I am running into myself is finding components like RAM, CPU, etc.


8GB of RAM is pretty much standard on gaming rigs these days, even though nothing really uses that much yet.

After you find a motherboard you like, look for CPU's with the same chipset that are recommended for it.

"In the void is virtue, and no evil. Wisdom has existence, principle has existence, the Way has existence, spirit is nothingness." ~Miyamoto Musashi

Xenuria
#11 - 2012-11-24 23:47:55 UTC
Bane Necran wrote:
Xenuria wrote:
The main problem I am running into myself is finding components like RAM, CPU, etc.


8GB of RAM is pretty much standard on gaming rigs these days, even though nothing really uses that much yet.

After you find a motherboard you like, look for CPU's with the same chipset that are recommended for it.


Ok...

But how do I know if the RAM or CPU is fast enough to process 6 GB/s?
Mars Theran
Foreign Interloper
#12 - 2012-11-25 00:26:11 UTC  |  Edited by: Mars Theran
What you are looking for is a Motherboard with a PCIe 3.0 interface on it. If you have that, then the motherboard can handle the data throughput. If you have a CPU which is capable of enabling PCIe 3.0, then the CPU can handle it. CPUs have a bandwidth capability, just like anything else to, and you can find Intels on ark.

As linked, you are looking for a 3rd generation processor. Second generation doesn't have the capability. Intels 'Northbridge' is on the CPU now, so the only chipset you have to worry about is the Southbridge, or storage controller and that will be essentially as advertised capability on the motherboard.

I'm not sure about AMD anymore, but generally assume you will run into multiple potential bandwidth issues if you go that route.

Basically, if you use Intel, all you need to avoid the bottleneck is the appropriate hardware for the job. RAM is maybe a small exception, but DDR3 used on the hardware in question is of minimum specification and aside from overclocking, doesn't exceed 1333MHz. That means that without an OC, your system hardware including RAM, will pretty much perform as expected consistently.

Obviously, if you check the ark, you'll find bandwidth specifications for the CPU along with other detailed specifications. This will give you an idea of what the CPU can handle, but it is also fairly standard in the LGA 1155 socket CPUs: 25.6 GB/s.

On your board, you are looking for more capable PCIe lanes. Assuming you don't use them all for GPUs, you shouldn't choke your bandwidth on Storage. It's basically a balancing act, but a good motherboard should be more than capable of handling a GPU and 2-4 SDDs without any real issues.

Anything you plug into a PCIe slot uses bandwidth, and your SDDs or HDDs are using some of those lanes too. This is why more capable set ups rely on a board with more capable PCIe lanes. More is better, provided you don't try to stuff everything onto the same lanes anyway.

It can get a bit complicated if you're really trying to squeeze every last bit out of it, but you probably won't need to even bother unless you are benchmarking and loading everything for it. So really, it depends on your intended use for it and your expectations.

Hard to answer this question in any short detail, or summarize how you could go about getting exact capabilities out of a system with specific hardware for one purpose or another.

Much easier to just suggest 3rd Gen. i5, a mid-to-high end motherboard supporting PCIe 3.0, a PCIe 3.0 GPU, 4-16 GB of 1600MHz*(*OC) RAM with Heatspreaders, (Kingston KHX, Patriot, Corsair, G.Skill,, Mushkin, or whatever.), your SDDs in either a pair mirrored or 4 mirrored and Striped in RAID 1+0, a decent PCIe Soundcard, and whatever you like to go with it.

edit: Also,your SSD Read and Write speeds are somewhat more important than the SATA speed itself. 540 MB/s Read for example, is what you will actually be looking at as your maximum throughput for read speed on average with newer SSDs.. About the same for Write speeds.
zubzubzubzubzubzubzubzub
Bane Necran
Appono Astos
#13 - 2012-11-25 00:40:28 UTC  |  Edited by: Bane Necran
Xenuria wrote:
But how do I know if the RAM or CPU is fast enough to process 6 GB/s?


Any CPU compatible with a 6gb SATA motherboard will handle it fine, because it's designed to, and 8gb of RAM is more than enough.

These other guys certainly know their tech stuff better than i do, but i might be better at simplifying. Blink

"In the void is virtue, and no evil. Wisdom has existence, principle has existence, the Way has existence, spirit is nothingness." ~Miyamoto Musashi

Xenuria
#14 - 2012-11-25 00:49:35 UTC
Mars Theran wrote:
What you are looking for is a Motherboard with a PCIe 3.0 interface on it. If you have that, then the motherboard can handle the data throughput. If you have a CPU which is capable of enabling PCIe 3.0, then the CPU can handle it. CPUs have a bandwidth capability, just like anything else to, and you can find Intels on ark.

As linked, you are looking for a 3rd generation processor. Second generation doesn't have the capability. Intels 'Northbridge' is on the CPU now, so the only chipset you have to worry about is the Southbridge, or storage controller and that will be essentially as advertised capability on the motherboard.

I'm not sure about AMD anymore, but generally assume you will run into multiple potential bandwidth issues if you go that route.

Basically, if you use Intel, all you need to avoid the bottleneck is the appropriate hardware for the job. RAM is maybe a small exception, but DDR3 used on the hardware in question is of minimum specification and aside from overclocking, doesn't exceed 1333MHz. That means that without an OC, your system hardware including RAM, will pretty much perform as expected consistently.

Obviously, if you check the ark, you'll find bandwidth specifications for the CPU along with other detailed specifications. This will give you an idea of what the CPU can handle, but it is also fairly standard in the LGA 1155 socket CPUs: 25.6 GB/s.

On your board, you are looking for more capable PCIe lanes. Assuming you don't use them all for GPUs, you shouldn't choke your bandwidth on Storage. It's basically a balancing act, but a good motherboard should be more than capable of handling a GPU and 2-4 SDDs without any real issues.

Anything you plug into a PCIe slot uses bandwidth, and your SDDs or HDDs are using some of those lanes too. This is why more capable set ups rely on a board with more capable PCIe lanes. More is better, provided you don't try to stuff everything onto the same lanes anyway.

It can get a bit complicated if you're really trying to squeeze every last bit out of it, but you probably won't need to even bother unless you are benchmarking and loading everything for it. So really, it depends on your intended use for it and your expectations.

Hard to answer this question in any short detail, or summarize how you could go about getting exact capabilities out of a system with specific hardware for one purpose or another.

Much easier to just suggest 3rd Gen. i5, a mid-to-high end motherboard supporting PCIe 3.0, a PCIe 3.0 GPU, 4-16 GB of 1600MHz*(*OC) RAM with Heatspreaders, (Kingston KHX, Patriot, Corsair, G.Skill,, Mushkin, or whatever.), your SDDs in either a pair mirrored or 4 mirrored and Striped in RAID 1+0, a decent PCIe Soundcard, and whatever you like to go with it.

edit: Also,your SSD Read and Write speeds are somewhat more important than the SATA speed itself. 540 MB/s Read for example, is what you will actually be looking at as your maximum throughput for read speed on average with newer SSDs.. About the same for Write speeds.


Thank you!
You have been very helpful.

FYI PCI-E SSDs can go to several GBp/s Transfer speeds. One of the SSDs I am looking at is 2-3 grand and fits in a PCI-E slot. It has a transfer rate of 1GB/s. Obtaining a SSD that by itself does 6~Gb/s is not the hard part. You won't find them on newegg but that dose not mean they can't be bought by civilians.
Mars Theran
Foreign Interloper
#15 - 2012-11-25 01:04:53 UTC
Xenuria wrote:


Thank you!
You have been very helpful.

FYI PCI-E SSDs can go to several GBp/s Transfer speeds. One of the SSDs I am looking at is 2-3 grand and fits in a PCI-E slot. It has a transfer rate of 1GB/s. Obtaining a SSD that by itself does 6~Gb/s is not the hard part. You won't find them on newegg but that dose not mean they can't be bought by civilians.


np Smile I know of the Revodrive and the newer offerings from OCZ; pricey but amazing. In this case, your primary concern is your PCIe capability, and of course, amount of RAM. I'd look at 16-32 GB of RAM for one of these, and a motherboard with multiple PCIe lanes. Probably even start looking at Xeons and Enthusiast Server/Workstation Boards. Almost a must given the expense.
zubzubzubzubzubzubzubzub