These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE General Discussion

 
  • Topic is locked indefinitely.
 

The Infrastructure of EVE Online

First post
Author
Previous Topic Next Topic
Sala Kyss
The Scope
Gallente Federation
#41 - 2013-01-28 01:32:15 UTC
Ai Shun wrote:

You have been given a fair bit of information to read and you've basically glossed over it because nobody provided you with a simple, three word summary.


Xenuria is a moron

Dang that's 4 words...
Xenuria
#42 - 2013-01-28 01:32:50 UTC
mechtech wrote:
The Database is on a RAMSAN, which is a storage device made up entirely of RAM. It's thousands of times faster than enterprise hard drives, and much faster than SSDs.

The spaceship simulation is single threaded, due to the fact that the core of the game was coded back before dual cores existed. Some functions like market processing are offloaded to separate clusters, but the actual processing of pew-pew is done on a single core. That's just the nature of the architecture, and there is a very slim chance of this ever changing. MMOs just don't scale well to multiple cores because every entity on screen is accessing the same dataset and needs to be synced.

They have in house thin clients that can simulate 1000 player fleet battles at will, and they have reporting tools that break down the latency of each stage of each server tick with fine granularity.

The server hardware, as far as I know, is getting a bit old. CPUs used are 3.3GhZ wolfdales. It's about time for a CPU upgrade, but unfortunately new CPUs are not increasing single threaded performance much at all, so it's not as urgent as it may seem. It's likely just not the right time for CCP to move to new hardware yet, both because of financial reasons (DUST will kill CCP financials if it tanks, but will give them breathing room if it takes off) and because the hardware landscape might not quite be where they want it to be yet. At the moment, each new generation of Intel CPU gives about a 10-15% boost to single threaded performance, clock for clock. That's just not a lot. AMD has dropped completely off the radar when it comes to single threaded performance.

CCP does experiment with new hardware. TQ-SOL-099 or "Frankenserver" is used to host the biggest fleet battles and Jita. It's an Intel Xeon X5698 (hex core with 4 cores disabled, running at 4.4GhZ and meant for high frequency trading) with 2100mhz DDR3 and 2x offloaded NICs. This is a test system that is not suitable for a rollout into a datacenter. When this level of performance is available in mainline server chips, it's likely CCP will feel the urge to do a full scale rollout.


Thank You!
You have given me a much better understanding of the situation.


CSM 12 Candidate
Nariya Kentaya
Ministry of War
Amarr Empire
#43 - 2013-01-28 02:02:13 UTC  |  Edited by: Nariya Kentaya
Xenuria wrote:
I don't think this thread is going to get very far with all the trolls and flames. It really is sad and depressing that when somebody asks a question or requests help in understanding something, you people dogpile on me and play up anything and everything I have done wrong in your eyes.

I mean really, come on now. Don't you have something better to do than :BadPost: and attack me? I think I should apologize too. I am so VERY sorry for causing you what I can only imagine is extreme discomfort and frustration over the mere concept of me trying to better my understanding of eve.

I asked some questions and so far NONE of the linked articles and Dev Posts answered any of them. If it were as simple as searching through dev posts or using google I would not have made a thread about it. You can throw your mindless hate at me all you want but the fact remains I am here trying to learn something and you are being nasty because of it.

The ISSUE Xenuria, si as fr as you had posted, you have not attempted to look up any of this information yourself.

if you ahd just posted in OP "guys ive looked but couldnt find the information, any of you know where to look, or perchance have that info on hand?", you just post "tell me infos so i can make judgements".

*frak my kindle and its horriebl touchscreen)
RubyPorto
RubysRhymes
#44 - 2013-01-28 02:44:37 UTC
INB4 Xenuria demonstrates a complete lack of knowledge about the topic he brings up and a complete unwillingness to put any effort into learning about it.


... oh... too late.

"It's easy to speak for the silent majority. They rarely object to what you put into their mouths." -Abrazzar "the risk of having your day ruined by other people is the cornerstone with which EVE was built" -CCP Solomon
Ai Shun
#45 - 2013-01-28 03:05:16 UTC  |  Edited by: Ai Shun
mechtech wrote:
The Database is on a RAMSAN, which is a storage device made up entirely of RAM. It's thousands of times faster than enterprise hard drives, and much faster than SSDs.

The spaceship simulation is single threaded, due to the fact that the core of the game was coded back before dual cores existed. Some functions like market processing are offloaded to separate clusters, but the actual processing of pew-pew is done on a single core. That's just the nature of the architecture, and there is a very slim chance of this ever changing. MMOs just don't scale well to multiple cores because every entity on screen is accessing the same dataset and needs to be synced.

They have in house thin clients that can simulate 1000 player fleet battles at will, and they have reporting tools that break down the latency of each stage of each server tick with fine granularity.

The server hardware, as far as I know, is getting a bit old. CPUs used are 3.3GhZ wolfdales. It's about time for a CPU upgrade, but unfortunately new CPUs are not increasing single threaded performance much at all, so it's not as urgent as it may seem. It's likely just not the right time for CCP to move to new hardware yet, both because of financial reasons (DUST will kill CCP financials if it tanks, but will give them breathing room if it takes off) and because the hardware landscape might not quite be where they want it to be yet. At the moment, each new generation of Intel CPU gives about a 10-15% boost to single threaded performance, clock for clock. That's just not a lot. AMD has dropped completely off the radar when it comes to single threaded performance.

CCP does experiment with new hardware. TQ-SOL-099 or "Frankenserver" is used to host the biggest fleet battles and Jita. It's an Intel Xeon X5698 (hex core with 4 cores disabled, running at 4.4GhZ and meant for high frequency trading) with 2100mhz DDR3 and 2x offloaded NICs. This is a test system that is not suitable for a rollout into a datacenter. When this level of performance is available in mainline server chips, it's likely CCP will feel the urge to do a full scale rollout.


Or as presented earlier, but being told that doesn't answer the question :)

CCP Yokai wrote:

Servers
64 x IBM HS21
2x Dual Core 3.33GHz CPU's
32GB of RAM Each
1x72GB HDD Each

2 x IBM X3850 M2's
2x Six Core 2.66GHz
128GB of RAM
4 x 146GB HDD

Cores
- 280 total Cores
- ~1 THz

RAM
- 2.3TB of Total RAM

Storage
- 4.8TB of Local Storage
- 2TB of SSD SAN
- 256GB of RAM SAN

Network
- Gigabit Ethernet
- 4Gb/s Fiber Channel


and the very detailed and informational software linkfest from CCP Fallout

CCP Fallout wrote:
Over the last week we've published a number of detailed blogs on the myriad steps we have been taking to tackle the lag issue many players have been reporting. If you've not yet done so, we encourage you to check out the following blogs:

Need for MOAR Speed - CCP Zulu announces the Fixing Lag series
Fostering Meaningful Human Interaction, Through Testing and Communication - CCP Tanis shares recent mass test results
The Long Lag - CCP Warlock discusses the problems inherent with distributed real-time systems
And I, For One, Welcome Our New Automaton Overlords - CCP Atropos unveils the new "thin client" developer tool
Carbon and the Core Technology Group - CCP Unifex shares CCP's Carbon framework and discusses the Core Technology Group
Character Nodes - CCP Atlas shares the work being done on character nodes
Module Lag: Why Not All Bugfixes Are a Good Idea - CCP Veritas reveals how module lag can be problematic


which covers a wide array of the programming challenges and problems, their distributed architecture and more. They are really open as a company about their architecture, challenges and what testing they do. I'm surprised it was not discoverable.

Sala Kyss wrote:
Xenuria is a moron

Dang that's 4 words...


That is a bit rude. He/she is just not up to assimilating the information that CCP has put there. Hence my immediate push-back at somebody who cannot gather that information from what CCP has already provided pushing themselves as a member of the CSM (And by picking up this issue almost de-facto making it part of their manifesto)

Somebody like mechtech - now there is a person that read it, listened to CCP and has a good grounding Big smile
Kaylar Prime
Doomheim
#46 - 2013-01-28 08:43:40 UTC
tldr: Why is Eve dying? Discuss
Whitehound
#47 - 2013-01-28 09:17:14 UTC  |  Edited by: Whitehound
I came to read something on freighters. Turns out Xenuria wants tell CCP how to do their jobs.

This borders pretty strong on ranting. Do not tell CCP how to do their jobs. Just tell them what your problem is and they will see what they can do for you. Do not patronize them or tell them what to do or we will tell you what you can do...

Loss is meaningful. Therefore is the loss of meaning likewise meaningful. It is the source of all trolling.
James Amril-Kesh
Viziam
Amarr Empire
#48 - 2013-01-28 09:36:45 UTC
Kaylar Prime wrote:
tldr: Why is Eve dying? Discuss

EVE is dying because not everyone in this thread sent me 100 million ISK each.
With your generous donation I can ensure the future of this wonderful gaming experience we all share.

Enjoying the rain today? ;)
Rebecha Pucontis
Doomheim
#49 - 2013-01-28 09:55:43 UTC
Miss DSA wrote:
Rebecha Pucontis wrote:
I think CCP should focus on rebuilding the whole game from the ground up. Scrap all the existing code, just reuse the art assets, and then completely recode the game. If this game is to last another ten years then its the only way.
Worst idea ever.


Thats an interesting article, and I wouldn't want to disagree with all those bad experiences. But in the case of Eve in which it is optimised for only one core at the moment, it is putting a serious bottleneck on the game it seems. And according to information given this seems impossible to change without a fundamental restructuring of the code.
kakmonstret
Domain Mining and Trading Corp
#50 - 2013-01-28 14:16:47 UTC
Rebecha Pucontis wrote:
Miss DSA wrote:
Rebecha Pucontis wrote:
I think CCP should focus on rebuilding the whole game from the ground up. Scrap all the existing code, just reuse the art assets, and then completely recode the game. If this game is to last another ten years then its the only way.
Worst idea ever.


Thats an interesting article, and I wouldn't want to disagree with all those bad experiences. But in the case of Eve in which it is optimised for only one core at the moment, it is putting a serious bottleneck on the game it seems. And according to information given this seems impossible to change without a fundamental restructuring of the code.


Well a "restructuring of the code" is a hell of a long way from "scrap all the existing code". The article is very relevant because rebuilding big stuff from the start is very seldom a god idea. Especially as EVE works fairly well over all.
Benny Ohu
Royal Amarr Institute
Amarr Empire
#51 - 2013-01-28 14:32:20 UTC
makin up busywork to look like an csm eh

righto
Jonah Gravenstein
Machiavellian Space Bastards
#52 - 2013-01-28 14:39:11 UTC  |  Edited by: Jonah Gravenstein
CCP are doing their best to cope with large fights, unfortunately not every large fight gives 24 hours notice so that a reinforced node is available. I think that the fact that circa 2800 players (10% of logged on players at the time I believe) can duke it out in a single system without a reinforced node is a testament to just how good the hardware and software actually is.

Besides if we didn't break the infrastructure occasionally, CCP would have nothing to aim for.

In the beginning there was nothing, which exploded.

New Player FAQ

Feyd's Survival Pack
Skippermonkey
Deep Core Mining Inc.
Caldari State
#53 - 2013-01-28 14:44:33 UTC
Posting in a thinly disguised 'VOTE FOR ME' thread

Xenuria, you are never going to get on the CSM, even I got more votes than you last year

COME AT ME BRO

I'LL JUST BE DOCKED IN THIS STATION
Docter Daniel Jackson
Fleetworks Training
#54 - 2013-01-28 15:10:05 UTC
Just because you know the specs to a server, dose not mean u know a thing about how servers/computers work, this person is going to do a lot more then reading before they can tell ccp what to do when it comes to servers and hardware... Roll
Zagdul
Federal Navy Academy
Gallente Federation
#55 - 2013-01-28 15:14:58 UTC
mechtech for CSM

Dual Pane idea: Click!

CCP Please Implement
Blake Armitage
The Scope
Gallente Federation
#56 - 2013-01-28 15:18:09 UTC  |  Edited by: Blake Armitage
I also remember that at one of the Dev talks during the last Fanfest presentation, they stating that a lot of the server time is spent on not WHAT happened, but rather TELLING everyone what happened.

Consider if you had to call someone over the phone and tell them that you are running late. Now scale that up to 100 or 1,000 phone calls and the time involved increases dramatically.
Elistea
BLUE Regiment.
#57 - 2013-01-28 16:09:35 UTC
CSM 8 candidate... made me laugh.
Since u are unable to do anything yourself, what do u think will be your contribution to CSM?
Are u planning to "outsource" your responsibilities there as well?
Rebecha Pucontis
Doomheim
#58 - 2013-01-28 18:22:02 UTC
kakmonstret wrote:
Rebecha Pucontis wrote:
Miss DSA wrote:
Rebecha Pucontis wrote:
I think CCP should focus on rebuilding the whole game from the ground up. Scrap all the existing code, just reuse the art assets, and then completely recode the game. If this game is to last another ten years then its the only way.
Worst idea ever.


Thats an interesting article, and I wouldn't want to disagree with all those bad experiences. But in the case of Eve in which it is optimised for only one core at the moment, it is putting a serious bottleneck on the game it seems. And according to information given this seems impossible to change without a fundamental restructuring of the code.


Well a "restructuring of the code" is a hell of a long way from "scrap all the existing code". The article is very relevant because rebuilding big stuff from the start is very seldom a god idea. Especially as EVE works fairly well over all.


What I am getting at is perhaps keeping parts of the code, but the fundamental structure of the code needs to be completely rewritten. Although what I was originally thinking is that to alter the fundamental structure of the code to allow for multiple cores would probably require almost all existing code to be re written anyway.

The point is though I think CCP should set aside a groups of Devs who can work for the next two or so years alongside normal eve development rewriting and optimising the entire code. I'd gladly have the Dust team working on something like that instead of Dust, or any other part of the CCP dev team for that matter.
Ranzabar
Doomheim
#59 - 2013-01-28 18:26:51 UTC
My understanding is that the scientists at CCP have proven the Quantum Field Theory and achieved a Singularity. There are no servers anymore, just three dudes with implants and high as a kite running the virtual reality.

Abide
Feyd Rautha Harkonnen
Doomheim
#60 - 2013-01-28 18:28:44 UTC
IMHO the intent of this thread has merit, as everyone who plays the game is a stakeholder and just wants to ensure the infrastructure team, dev's and wallets are best pursuing long term viability beyond stopgaps; so asking for info on this is not a sin.

What I haven't heard (beyond all the reasons and blog links saying what the problems are at a fine-grained level and why you 'cant' fix things) is a strategic vision from CCP on how they plan to position the architecture for the long term, and move beyond time dilation 'stopgap' measures.

i.e. Are you planning on having surplus computing power that is available and *dynamically* spun up & allocated to big fleet battles without intervention, 24 hour notice, and no lag to said fleet (or other players)? How do you plan on getting there?

As the concurrent user counts continue to grow, I foresee a brick wall being hit on this issue one day. You can defer dealing with it by tuning, pre-allocating extra nodes to specific trade hubs or systems, but the real answer to a long term viable 'single shard' universe is dynamic allocation of processing, that is infinately scalable by adding additional servers underneath.




Would you like to know more?