EVE Technology Lab

After EVEMon release we had to disable Eve Addicts as an endpoint, to ensure the smooth operation of the site, as data flow was surging and bringing the server down.
Now after several hours, things have smoothed out. Currently we are at 33 calls/sec and expecting the rate to drop even further.
So if Snarf wants to enable Eve Addicts again as an endpoint, please get in touch.

I believe it wouldn't hurt to enable it from time to time and observe the load.
What we can also do is add Eve Addicts to the endpoint list but disabled and rely on the user to enable it (which only a few users do).

Is this a CPU/Memory/OS problem? Which hoster/VPS package are u using atm, i used various VPS over quite some time and could check if there is a similar priced, but more powerful VPS offer.

We go over this in the EMDR documentation, but there is substantial overhead in accepting a ton of HTTP connections. TCP sockets have to be opened from all over, managed, sending/receiving happens, then they are closed (and hang around for a while in a FINISHED_WAIT state if the server isn't properly configured), and this is done for EVERY individual upload.

Contrast this to EMDR, which uses ZeroMQ. ZeroMQ PUB/SUB arrangements are a persistent, always-on, one-way feed. The consumers just listen in and don't bother replying. You only use one socket connection, and because it stays open, you don't have the overhead associated with a ton of rapidly opening/closing TCP sockets.

I'm not trying to plug EMDR here, but I am not at all surprised people are having issues handling the traffic. EMDR's gateways are designed in a very specific way to be able to handle the volume, but I had to build for performance from the very beginning. It wasn't nearly as much of a concern when some of the existing market sites started development, due to the much lower volume of market data. It now, all of the sudden, is, and we're seeing that.

Though, this is no fault of EVEMon. It's only doing exactly what it's supposed to.

As you can see from the stats, i'm still getting data from EVEMon users. The volume is low enough that it doesn't cause any problems, so no worries there.

Like Ilyk said, it was the opening and closing of tcp sockets that was one of the issues. Trying to process all of the incoming data at once also did not help.... When I looked for the first time, there were 160k history and about 50k orders waiting to be processed.

I cant get this to work with IE9, while it works in FF13? The problem is that Excels Web Data import only uses IE, so is there a way around this? I would like to import orders directly from the web-page tables into Excel.

bye


PS: There is a excel/IE error, "Line 439, Char 2, 'console' is undefined code 0", no clue if this has anything to-do with this problem.

I know, but i prefer Excel over GoogleDocs/OpenOffice, but unfortunately they forget to add a option to choose your web import renderer, i wonder why :)

Thx for trying

bye

I now see the order tables in excels IE renderer, but unfortunately it wont let me select the sell/buy tables and i only get the header table imported :(

I will try a workaround and see if i can import the tables via googledocs and than import this to excel.


[UPDATE]
I cant seem to get the sell/buy tables via googledocs "importXML/HTML" queries, maybe someone got this working for eve addicts?
The "importHTML" with the "table" option always results in a "#na" error and the "importXML" always with a bad xref error.

Seems i'm having problems constructing the correct xpath to the tables, since the "importHTML" function don't work.
I tried using "XPath Checker" for FF and use "importXML("url....";"id('buyPriceTable')", which gives me the table in XPath, but not in googledocs.
Also "id('buyPriceTable')/x:tbody/x:tr

/x:td[1]" gives me the first column in Xpath, but not google-docs?

Is there some syntax difference i'm missing?


any hints?

thx Andy

Hi,

i just noticed that the 5% sell/buy are "just" the mathematical 5% percentile of the actual buy/sell price entries?
If this is correct, would it be possible to change this to a actual sell/buy simulation of 5% of the total stock?

Thx Andy

As you all might have noticed, i'm not that active in eve/forums lately...
I'll try and look at the requests, but i can't make any promises.

This should not be too hard, only problem is that the table will get kind of crowded.



I don't think i understand what you mean or want. Can you give me an example?




Again, i'll see what i can do :)
The 5 minute delay is caused by the fact that i cache the queries, so that is intentional.