These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

 
  • Topic is locked indefinitely.
 

Eve Client and its Threads
Author
Previous Topic Next Topic
carnivore2k4
The Herd
#1 - 2015-03-17 09:13:35 UTC
A few days ago, i was for some reason looking at the windows 8.1 process monitor and noticed that my eve clients were listed having 50+ threads each. It is said, that the client is written in Stacklass Python only, which is not able of parallelisation und runs all those threads on a single core. However, the CPU load were evenly distributed over all 8 cores (i7 with 4 real and 4 hyperthreading cores, 3 clients running at the time).

Why?

Is windows distributing load even though the threads are not parallelized?
Is eve not as single-threaded as I thought?
Each client is causing ~10% CPU load, which should be enough to be noticed.
Is the process monitor garbage? Am I stupid?

Any input is appreciated, especially if anyone knows a tool that lists which thread of which process is actually running on which CPU core.
Aineko Macx
#2 - 2015-03-17 12:51:52 UTC  |  Edited by: Aineko Macx
Just because a process has threads doesn't mean it's doing useful work in parallel. You could have many threads but still have the performance critical path be executed by only a single one of them while the others wait. Also, a single process can and is moved between cores frequently by the scheduler (unless you explicitly set CPU affinity), so what you see in the process monitor with its 1 second interval is an average over that period.

Usually when people talk about eve's lack of multithreading they are generally referring to the server side of things (the gerbils). CCP used to do technical devblogs, if you look for them you'll eventually find things like python and the GIL, and the problems associated with it. Stackless python offers "tasklets" which are described as microthreads but which do not actually provide parallel processing (execution is still serial), but a convenient way of handling processing tasks in an asynchronous fashion.

iveeCore 3.0: The PHP engine for industrial activities and CREST library
carnivore2k4
The Herd
#3 - 2015-03-17 14:09:21 UTC
Aineko Macx wrote:
Just because a process has threads doesn't mean it's doing useful work in parallel. You could have many threads but still have the performance critical path be executed by only a single one of them while the others wait.


True.

Aineko Macx wrote:
Also, a single process can and is moved between cores frequently by the scheduler (unless you explicitly set CPU affinity), so what you see in the process monitor with its 1 second interval is an average over that period./quote]

This is a good point, although I am not yet convinced this is the only answer to my question. If I find some time, I will experiment with the CPU affinity, maybe this will allow some further conclusions.

[quote=Aineko Macx]
Usually when people talk about eve's lack of multithreading they are generally referring to the server side of things (the gerbils). CCP used to do technical devblogs, if you look for them you'll eventually find things like python and the GIL, and the problems associated with it. Stackless python offers "tasklets" which are described as microthreads but which do not actually provide parallel processing (execution is still serial), but a convenient way of handling processing tasks in an asynchronous fashion.


Stackless Python's single-threadness und single-coreness refers only to a single process. The Eve server is, however, not just a single process, but a lot of them distributed over many nodes that are communicating. The performance limiting process seems to be the one that is handling all the stuff that is happening in space in a specific system. And, as far as I understand the current implementation plans, CCP is currently trying to decouple a few things from this process to free up computing time.

Now back to the client side:
Are things already decoupled there?
I don't really have any idea how Direct X calls are handled? Not every call needs to be answered (e.g. play_some_music) and could theoretically be executed independently. I just learned that Direct 3D supports multi threaded rendering from version 11 on, but somehow i doubt that this is used.
And there are lot of things that could be done programmatically.

Thanks for your answers so far.
Dragonaire
Here there be Dragons
#4 - 2015-03-17 15:30:11 UTC
Quote:
Stackless python offers "tasklets"
IE trying to recreate things like the original Amiga task manager and co-op task scheduling all over again which works just as well as preemptive until something doesn't return when it's suppose to and locks up your whole system Blink I'm not surprised that they might be moving away from it as it has some issues that aren't easy to solve. Preemptive does too but the problems there are better understood and there are more people working on them since every major OS uses it plus some embedded stuff etc. Stackless has only CCP working on it and that a lot of extra work that could be better spent IMHO some where else but of course there the cost of switching over as well Blink I would say though if you are really seeing 10% across all processor that don't sound normal for Eve. As Aineko Macx pointed out it could be just being rescheduled a lot and that's what you are seeing but if that the case I'd be looking at why my system is doing that so much because normally that only starts happening when something is not right either to much load or some wildly varying one that is cause problems for the OS task scheduling.

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.