These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE General Discussion

 
  • Topic is locked indefinitely.
 

TiDi

First post
Author
Lucas Kell
Solitude Trading
S.N.O.T.
#21 - 2015-12-29 14:02:30 UTC  |  Edited by: Lucas Kell
Kagura Nikon wrote:
Lucas Kell wrote:
Tidi was a stopgap measure. It allowed them to give the server more time to get through the event queue so that they could still run everything on a single heavily overclocked core. At some point the promoted to "the fix" and now it's a burden. Things like BIAB help, but in the long run the solution has to be modernising their server code.
Sorry, I might be mistaken, but you sound as someone that has no clue what multithreading is. What do you think BIAB is? It IS ALREADY paralelization (external) of the most consuming task of the node.
BIAB is taking a service and moving it outside of the solar system service, much like how chat or the market is unaffected by the tidi in a system. Effectively if you undocked or jumped into a system in tidi, the math to work out your stats used to get queued with other action in that system, but now it does not. While that helps, it doesn't fix the issue (and isn't actually multithreading, it's relocating a process) because every action taken in that system by players is still in a single queue relying on a single CPU to process them. A modern game server would have spread the bulk of those actions across multiple CPU cores so that multiple complex calculations could occur simultaneously.

This is why the new sov system takes you into multiple systems, because it spreads players across multiple nodes and thus across multiple CPUs. If they want to be able to run large battles without tidi however then the only real solution is to allow system actions to be spread across CPU cores, i.e. multithreading.

Ed: Oh, and yes, you are mistaken.

The Indecisive Noob - EVE fan blog.

Wholesale Trading - The new bulk trading mailing list.

Kagura Nikon
Native Freshfood
Minmatar Republic
#22 - 2015-12-29 14:20:04 UTC
Lucas Kell wrote:
Kagura Nikon wrote:
Introducing paralelism on a task whose bottleneck is I/O with the database and sincronization is not always the best thing to do. If you cannot change the model to remove data dependency you are not going to solve anything. Usually it is better to remodel the system. The things that are needed for an eve fleet battle are NOT huge and do nto justify even a single modern CPU being bottlenecked.
But we already know that's not the bottleneck. The bottleneck is CPU. This is why when you fill in that form they use a reinforced server, which is one where most of the cores are shut off on the CPU and used as a heat spreader while the core in use is overclocked like crazy. A single modern CPU has multiple cores, all (or most) of which will be used by most CPU heavy game servers. EVE doesn't. A single core has to process all of the information coming in from the thousands of players in the system. Even on a modern CPU, that is too much.



Do not dare to think I do not know how a CPU work or a high parallelism system, #!#!@ I have a master degree on it (not an imaginary one, a real one from a good university). And All that do not means that a system is bottlonecked on a task with high parallelism potential.


Let me give you a simple example, sorting a list. You may do it in hunreds of ways, and in 99.99% of time the bottleneck will be CPU. If you are using quick sort (a.k.a. partition search) you have a very high paralelism potential and using several CPU will boost you. If you are doing a bubble sort ( I know very horrible example , but it is an extreme example) and to use even a second CPU you would need to add so much overhead of syncronization that your code would likely get slower.


So if their model, and I do not mean simply the data model but the information flow and data dependency model has bad paralelism qualities , then taking a huge effort to make that code tun in several cores might have a meaningless gain, and on some cases even reduce the performance.

Whenever a problem is defined by a context sensitive grammar, then trying to execute the problem in a paralel way will scale very badly. When that happens the solution is to change your problem in a context free one.

"If brute force does not solve your problem....  then you are  surely not using enough!"

Kagura Nikon
Native Freshfood
Minmatar Republic
#23 - 2015-12-29 14:25:15 UTC
Lucas Kell wrote:
Kagura Nikon wrote:
Lucas Kell wrote:
Tidi was a stopgap measure. It allowed them to give the server more time to get through the event queue so that they could still run everything on a single heavily overclocked core. At some point the promoted to "the fix" and now it's a burden. Things like BIAB help, but in the long run the solution has to be modernising their server code.
Sorry, I might be mistaken, but you sound as someone that has no clue what multithreading is. What do you think BIAB is? It IS ALREADY paralelization (external) of the most consuming task of the node.
BIAB is taking a service and moving it outside of the solar system service, much like how chat or the market is unaffected by the tidi in a system. Effectively if you undocked or jumped into a system in tidi, the math to work out your stats used to get queued with other action in that system, but now it does not. While that helps, it doesn't fix the issue (and isn't actually multithreading, it's relocating a process) because every action taken in that system by players is still in a single queue relying on a single CPU to process them. A modern game server would have spread the bulk of those actions across multiple CPU cores so that multiple complex calculations could occur simultaneously.

This is why the new sov system takes you into multiple systems, because it spreads players across multiple nodes and thus across multiple CPUs. If they want to be able to run large battles without tidi however then the only real solution is to allow system actions to be spread across CPU cores, i.e. multithreading.

Ed: Oh, and yes, you are mistaken.



No I am not, since in a few posts you have proven to be very limited in knowledge of parallel systems. Taking a process to outside of another process is the DEFINITION OF PARALELISM. Since the BIAB is running on a different machine and therefore in a different CPU, by DEFINITION it is running in a different thread since not a single architecture of computers that is used outside academics have a single thread run at same time on different CPUs.

The simple basics of operating systems for you.. a process can have one or more threads, 2 process will have at least 2 threads.. therefore.. multi thread. Now, since it is clearly established that you have no clue what multi threaded really means, maybe you should stop trying to say to CCP how to solve anything?

"If brute force does not solve your problem....  then you are  surely not using enough!"

Lykouleon
Noble Sentiments
Second Empire.
#24 - 2015-12-29 14:51:42 UTC
Can't we just run the server in the cloud? Why don't we use the cloud more? Big Data. MapReduce. Buzzwords.

My work here is done. Someone go tell HR I'm taking the week off.

Lykouleon > CYNO ME CLOSER so I can hit them with my sword

Lucas Kell
Solitude Trading
S.N.O.T.
#25 - 2015-12-29 14:56:53 UTC
Kagura Nikon wrote:
Do not dare to think I do not know how a CPU work or a high parallelism system, #!#!@ I have a master degree on it (not an imaginary one, a real one from a good university). And All that do not means that a system is bottlonecked on a task with high parallelism potential.
Good for you? Though more important than a degree is working experience. Most of us that work as developers do, even new starters with very limited knowledge.

The thing is, we already know what the problem is here, as CCP have shared this in the past, and the fact that it would add overhead is irrelevant. Even if they were just offloading some of the math to other cores and still processing the actual events on a single CPU, that would be better than what they currently do which is wait more than 10 times as long to process the events.

Kagura Nikon wrote:
So if their model, and I do not mean simply the data model but the information flow and data dependency model has bad paralelism qualities , then taking a huge effort to make that code tun in several cores might have a meaningless gain, and on some cases even reduce the performance.
But we know what happens in MMORPG servers. There is wide ranging research into it, and even information given out by CCP. If they threaded the servers and reduced performance that would simply mean they wrote it wrong. I don't even think CCP would suggest that they wouldn't benefit, they simply aren't willing to dive that heavily into their legacy code.

Kagura Nikon wrote:
No I am not, since in a few posts you have proven to be very limited in knowledge of parallel systems. Taking a process to outside of another process is the DEFINITION OF PARALELISM. Since the BIAB is running on a different machine and therefore in a different CPU, by DEFINITION it is running in a different thread since not a single architecture of computers that is used outside academics have a single thread run at same time on different CPUs.
No, you really are. And I didn't say parallelism, you did. I said multithreading, which can be a form of parallelism but isn't the same thing. Kinda like how an apple is a fruit but fruit isn't just apples.

See, what you are talking about is how they took the old attribute code that existed within the solar system service and they added it to a new BIAB service which operates as a separate process on the server and thus can run on it's own core. That reduces the amount of work inside the solar system service. There's limited scope to do that however as a lot of what bottlenecks the solar system service can't be handed over to a separate service (at least not easily) as it requires granular control over how that service would operate and a lot of shared data.

Taking the actual solar system service however and allowing it to use threads to distribute its workload across multiple cores would allow it to do some of the calculations simultaneously with the main service thread only needing to worry about the events as a whole. Say for example 10 people fire lasers at a target, each of those damage calculations could be done together on different cores and the service thread would take the results of those and apply them to the target.

Kagura Nikon wrote:
The simple basics of operating systems for you.. a process can have one or more threads, 2 process will have at least 2 threads.. therefore.. multi thread. Now, since it is clearly established that you have no clue what multi threaded really means, maybe you should stop trying to say to CCP how to solve anything?
So if I open a copy of notepad, then I open another copy of notepad, have I multithreaded my notepad? The answer is no, those are two separate processes. Did you actually pay attention when you got that degree?

The Indecisive Noob - EVE fan blog.

Wholesale Trading - The new bulk trading mailing list.

Solecist Project
#26 - 2015-12-29 14:59:52 UTC
lol

That ringing in your ears you're experiencing right now is the last gasping breathe of a dying inner ear as it got thoroughly PULVERISED by the point roaring over your head at supersonic speeds. - Tippia

Harrison Tato
Yamato Holdings
#27 - 2015-12-29 15:16:20 UTC
*nerd rage intensifies.
Kagura Nikon
Native Freshfood
Minmatar Republic
#28 - 2015-12-29 15:28:12 UTC  |  Edited by: Kagura Nikon
Lucas Kell wrote:
lots of ignorance...



A multi process system is by definition multi threaded. If you run a task in a different machine, you are running it in a different process, therefore a different thread. More than one thread is multi thread. You can have one or more threads within a proccess but you cannot have a process that is not a thread since a process by definition shares its data with itself. Since the addressing model used in x86 and MAD 64 architectures do not allow for sharing of address space, when you run something in another computer you are running in another process. If you have another process you have another thread.

So bad that your work experience has thought you nothing . At least 1+1 > 1(one thread + other thread means multi) should be the very least to expect.

Just to be clear, I do not even care with what you think, ignorance and stupidity are things not worth to fight against, but other people read these forums and they should not learn wrong things by people like you. To the people learning computer science out there, the difference in in sharing or separation of data, a process is in a separate address space from other process, while a thread might be on same address space of other thread. A process can have one or more threads, when you have 2 process executing a task you therefore have at least 2 threads executing the task, but you can have more, but at least 2 of those do not share the same address space. When you are trying to solve a problem where the concurrency is over completely independent set of data, there is no need to keep the threads on the same address space. Those cases are usually the best candidates to be offload to a completely different machine.

That is why kids you should all go to school, to not end up like this guy.

"If brute force does not solve your problem....  then you are  surely not using enough!"

Lucas Kell
Solitude Trading
S.N.O.T.
#29 - 2015-12-29 15:48:45 UTC
Kagura Nikon wrote:
A multi process system is by definition multi threaded. If you run a task in a different machine, you are running it in a different process, therefore a different thread. More than one thread is multi thread.
Are you aware that one process can have multiple threads, and that parallelism by multi-threading a single process and parallelism by creating multiple processes are not the same thing? Running a single-threaded process twice or running two single-threaded processes that communicate with each other does not make the processes multi-threaded. The bulk of the work of the solar system process cannot be split into multiple processes, and so the only way to increase throughput without increasing the speed of a single CPU core is by multi-threading the single process and distributing the load across multiple cores.

Kagura Nikon wrote:
You can have one or more threads within a proccess but you cannot have a process that is not a thread since a process by definition shares its data with itself. Since the addressing model used in x86 and MAD 64 architectures do not allow for sharing of address space, when you run something in another computer you are running in another process. If you have another process you have another thread.
See, here you seem to be confused. At no point did I claim you could have a 0 thread process, and you're going on about shared address space and multiple computers as if that's part of the conversation. It's not. I get the feeling that you think the solar system service uses all of the cores of a single CPU and that we're talking about distributing the workload onto another machine. We are not, we are talking about how the solar system services uses one of many cores on a single CPU, and how by utilising more cores the single process could get more work done in less time even with the overhead.

What I find amazing here is that you have the audacity to insult me as if I'm the one that is confusing the terminology. Perhaps you should go back and re-read the posts. You seem to have no understanding of why "parallelism" and "multi-threading" are not directly interchangeable. I call bullshit on you having a degree if you can't even work that out. Screenshot or it never happened.

The Indecisive Noob - EVE fan blog.

Wholesale Trading - The new bulk trading mailing list.

Lord Molly
The Tuskers
The Tuskers Co.
#30 - 2015-12-29 16:07:48 UTC
Kagura Nikon wrote:
Lord Molly wrote:
This video documents some real bad tidi issues

If you look closely, you might also notice some glitching, such as smart bombs doing damage in warp.......

All of this is caused by server node overload when to many people are in one system activating too many mod.


To combat this very frustrating trait of gameplay, CCP made a jolly form that you fill in and effectively tell them there is gonig to be a big fight and etc etc, so that they may reinforce a node and divert resources to said systems node and hopefully make it all ok.

Only issue is, that's not really the case is it. Have you ever been to a public event or a large pre-arranged scrap where tidi is down to 10%? where you fire a volley, then go and make dinner and come back to find you are still on the same cycle?

Id be interested to hear from CCP about how they intend to improve the cancer that is TiDi.



TIDi cancer? spoiled kid. before Tidi when we jumped into a large fight what happened is that you would wait between 30 minutes to 2 hours with a black screen. then 30% of people would disconnect, other 33% would load grid and be unable to use any controls of the ship. Other 13% would be able to load and press fire a single time per hour. The rest would wait another hour and try their luck again...



I remember thems dark and horror-some days but i mean jita has no lag, that entire area has no lag and that has possibly the most high traffic gates in use in the EvE universe.

Why can they not automatically re-allocate server resources as pilots move around. Use some sort of code whereby each pilot in space is given a value and when the combined pilot values peak over a specific figure the server automatically compensates by taking away resources from other areas.

I know that in theory this does happen / CCP claim it happens but surely there is a less complicated way of doing it where it actually works out in the pilots favor.
Solecist Project
#31 - 2015-12-29 16:14:48 UTC
Lord Molly wrote:
Kagura Nikon wrote:
Lord Molly wrote:
This video documents some real bad tidi issues

If you look closely, you might also notice some glitching, such as smart bombs doing damage in warp.......

All of this is caused by server node overload when to many people are in one system activating too many mod.


To combat this very frustrating trait of gameplay, CCP made a jolly form that you fill in and effectively tell them there is gonig to be a big fight and etc etc, so that they may reinforce a node and divert resources to said systems node and hopefully make it all ok.

Only issue is, that's not really the case is it. Have you ever been to a public event or a large pre-arranged scrap where tidi is down to 10%? where you fire a volley, then go and make dinner and come back to find you are still on the same cycle?

Id be interested to hear from CCP about how they intend to improve the cancer that is TiDi.



TIDi cancer? spoiled kid. before Tidi when we jumped into a large fight what happened is that you would wait between 30 minutes to 2 hours with a black screen. then 30% of people would disconnect, other 33% would load grid and be unable to use any controls of the ship. Other 13% would be able to load and press fire a single time per hour. The rest would wait another hour and try their luck again...



I remember thems dark and horror-some days but i mean jita has no lag, that entire area has no lag and that has possibly the most high traffic gates in use in the EvE universe.

Why can they not automatically re-allocate server resources as pilots move around. Use some sort of code whereby each pilot in space is given a value and when the combined pilot values peak over a specific figure the server automatically compensates by taking away resources from other areas.

I know that in theory this does happen / CCP claim it happens but surely there is a less complicated way of doing it where it actually works out in the pilots favor.

The latest TQ devblog hints at them being able to direct resources at will ... soon ...
... thanks to virtual environments that allow to sidestep pythons problematic use of multithreading.

Don't forget ... it's still python with C, or C++.
Doesn't make things easier.


That ringing in your ears you're experiencing right now is the last gasping breathe of a dying inner ear as it got thoroughly PULVERISED by the point roaring over your head at supersonic speeds. - Tippia

Lucas Kell
Solitude Trading
S.N.O.T.
#32 - 2015-12-29 16:15:50 UTC
Kagura Nikon wrote:
When you are trying to solve a problem where the concurrency is over completely independent set of data, there is no need to keep the threads on the same address space. Those cases are usually the best candidates to be offload to a completely different machine.
Now that your edits are showing up, this explains some of where your confusion is coming from. It's true enough that like BIAB, some parts can be moved to a new process, but the problem is that most of the work done on the solar system service can't be offloaded to a different machine and needs to share address space.

Also Rem, while your brother may have read the isolated buzz words and unconnected fragments of fact in Kagura's post, for the most part he's missed the entire problem that CCP have that means that tidi is needed. You're doing him a disservice by posting his support without full understanding of the context.

The Indecisive Noob - EVE fan blog.

Wholesale Trading - The new bulk trading mailing list.

Lucas Kell
Solitude Trading
S.N.O.T.
#33 - 2015-12-29 16:27:20 UTC  |  Edited by: Lucas Kell
Lord Molly wrote:
I remember thems dark and horror-some days but i mean jita has no lag, that entire area has no lag and that has possibly the most high traffic gates in use in the EvE universe.

Why can they not automatically re-allocate server resources as pilots move around. Use some sort of code whereby each pilot in space is given a value and when the combined pilot values peak over a specific figure the server automatically compensates by taking away resources from other areas.

I know that in theory this does happen / CCP claim it happens but surely there is a less complicated way of doing it where it actually works out in the pilots favor.
Depending on what you are doing you'll already be distributed over multiple services. Undocking does a lot on BIAB now, market are on a service, so is chat, etc. The problem service that causes the lag is SOL, the solar system service. This only has a finite amount of resource that can be used (up to one maxed out CPU core) at which point it's at it's limit. If they know in advance, they can make sure the target solar system is on one of the machine with really beastly CPUs, but they can't move the node with a fight already occurring as everyone will get booted. Then the best they can do is move other solar systems off of the node and crank up the processor a bit. That can go wrong though, like in the battle of Z9PP. In theory though, the new virtual environments will allow them to move or expand a running node without booting people, but we'll have to see how that works out in practice once it's all rolled out.

The Indecisive Noob - EVE fan blog.

Wholesale Trading - The new bulk trading mailing list.

Max Fubarticus
Raging Main
Bullets Bombs and Blondes
#34 - 2015-12-29 16:31:07 UTC
Fancy that! I have a Masters Degree as well...
Sadly it has nothing to do with computers so I guess I will keep moving along.
Epeen fencing should be an Olympic sport. Just saying...

Max

Civil discourse is uniquely human. After all, when is the last time a pride of lions and a herd of water buffalo negotiated SOV over a watering hole? Never. Someone either gets their ass kicked or eaten. At the end of the day someone holds SOV.

Lucas Kell
Solitude Trading
S.N.O.T.
#35 - 2015-12-29 16:34:42 UTC
Max Fubarticus wrote:
Fancy that! I have a Masters Degree as well...
Sadly it has nothing to do with computers so I guess I will keep moving along.
Epeen fencing should be an Olympic sport. Just saying...

Max
Now I'm curious... You should start a threadnaught on your chosen subject.

The Indecisive Noob - EVE fan blog.

Wholesale Trading - The new bulk trading mailing list.

ISD Supogo
ISD BH
ISD Alliance
#36 - 2015-12-29 21:37:07 UTC
Removed some posts.

Quote:

Forum rules

4. Personal attacks are prohibited.

Commonly known as flaming, personal attacks are posts that are designed to personally berate or insult another forum user. Posts of this nature are not conductive to the community spirit that CCP promotes. As such, this kind of behavior will not be tolerated.

ISD BH Supogo

Bughunter

Equipment Certification and Anomaly Investigations Division (ECAID)

Interstellar Services Department

Pix Severus
Empty You
#37 - 2015-12-30 00:37:17 UTC
Always Shi wrote:
The new TQ hardware coming in a couple of months time (plus the recent and ongoing software improvements like BitB) are all helping to make TiDi occur less often.


I am so looking forward to this.

MTU Hunter: Latest Entry - June 12 2017 - Vocal Local 5

MTU Hunting 101: Comprehensive Guide

Nat Silverguard
Aideron Robotics
Aideron Robotics.
#38 - 2015-12-30 03:10:48 UTC
Lykouleon wrote:
Can't we just run the server in the cloud? Why don't we use the cloud more? Big Data. MapReduce. Buzzwords.

My work here is done. Someone go tell HR I'm taking the week off.


i understood this reference! yehey! o7

Just Add Water

ISD Buldath
#39 - 2015-12-30 05:53:19 UTC
Greetings Pilot!

I would like to invite you to apply to www.ccpgames.com/jobs if you believe you are capable of creating a multi threaded experience For Tranquility that CCP could use to remove Tidi, since you believe it to be so easy. CCP Could really use your Expertise and assistance on such a grand task!

~ISD Buldath

Instructor King of the Forums! Knight of the General Discussion

Support, Training and Resources Division

Interstellar Services Department

I do not respond to EVE-Mails regarding forum moderation.

MidnightWyvern
Fukamichi Corporation
SAYR Galactic
#40 - 2015-12-30 06:00:13 UTC
ISD Buldath wrote:
Greetings Pilot!

I would like to invite you to apply to www.ccpgames.com/jobs if you believe you are capable of creating a multi threaded experience For Tranquility that CCP could use to remove Tidi, since you believe it to be so easy. CCP Could really use your Expertise and assistance on such a grand task!

Critical Sass Levels

Rattati Senpai noticed us! See you in the next FPS!

Alts: Saray Wyvern, Mobius Wyvern (Dust 514)