These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

 
  • Topic is locked indefinitely.
 

EVE-Central - The classic is back with updates
Author
Previous Topic Next Topic
Desmont McCallock
#61 - 2012-06-03 12:42:22 UTC  |  Edited by: Desmont McCallock
Andy DelGardo wrote:
Maybe its the new evemon uploader?
No way, as we have tested the uploaded messages with Kaladr before allowing them to get included in EC data. And I haven't changed anything since, in the message format.
Andy DelGardo
#62 - 2012-06-03 12:57:30 UTC  |  Edited by: Andy DelGardo
I tried "fixing" the broken reports by re-uploading, but seems u can't override those broken reports with fresh data?
I'm not sure whats going on, but i get more and more missing data via API-Calls. I also tried removing the "sethour" option completely, but i still wont get the "broken" data listed via api, while this works in the web version at least.
Andy DelGardo
#63 - 2012-06-03 13:07:11 UTC  |  Edited by: Andy DelGardo
Oki something is broken here, i just re-uploaded using evemon and now the website basically shows all my send entries in this broken "06-03 1" time format. I'm not sure what causes this behavior, but can u double check evemons date format, since it basically breaks all API calls.
I will test the old eve-marketeer uploader next and see if this is maybe a eve-central problem, for all uploads.

Here is a broken example item: http://eve-central.com/home/quicklook.html?typeid=30122

The "Reported Time" is all "06-03 1" and all those entries are filtered out and wont show up, if u use the API and not the website.
Andy DelGardo
#64 - 2012-06-03 13:20:04 UTC  |  Edited by: Andy DelGardo
Oki this actually "seems" like a eve-mon problem, i just re-uploaded a single entry using marketeers uploader which did fix all the Jita entries to the correct "06-03 13:16:40" format.

Could also be someone just fixed eve-central, will try break the entry again with eve-mon to double check.

UPDATE: Oki i also could again break the entry using eve-mon, so u might want to disable uploading until u found the problem.

Could also be different handling of incoming eve-marketeer uploader data vs the new universal data?


PS: Seems eve-addicts is handling this problem a bit better, there a empty/broken "Updated" value is shown, but after 1 minute, the broken entry's are "fixed" and changed to a 1min entry.
Desmont McCallock
#65 - 2012-06-03 14:31:47 UTC
OK, I just checked again and it looks like an EC parsing issue. EVE-Marketeer is not using the same format as EVEMon, so you can't compare.
Andy DelGardo
#66 - 2012-06-03 14:44:06 UTC
Ah oki, hopefully this can be fixed soon, since it basically breaks all spreadsheets that import XML data via API.
Andy DelGardo
#67 - 2012-06-05 12:17:58 UTC
Any update on this parser error?
Andy DelGardo
#68 - 2012-06-08 23:22:15 UTC
mhh i still see lots of "06-08 1" so there is no eta on this problem?
Felixjet CA
Roghnaithe
#69 - 2012-06-11 03:20:39 UTC
is api down? i cant get it to work since yesterday
Kaladr
Viziam
Amarr Empire
#70 - 2012-06-11 06:29:42 UTC  |  Edited by: Kaladr
Hey folks, just looked into the parser error - its not a parsing error (input data is correct), its a display issue on the older Python code.

The API should be unaffected, only the web display was affected (and is now kludge fixed).

If at all possible, please use the API for automated tasks :)

Creator of EVE-Central.com, the longest running EVE Market Aggregator
Andy DelGardo
#71 - 2012-06-11 08:28:35 UTC  |  Edited by: Andy DelGardo
Kaladr wrote:
Hey folks, just looked into the parser error - its not a parsing error (input data is correct), its a display issue on the older Python code.

The API should be unaffected, only the web display was affected (and is now kludge fixed).

If at all possible, please use the API for automated tasks :)


Are u sure thats the case?
I use the API and for me any order that had this invalid date/time was filtered out, mainly because the api seems to have problems calculating the "report time" correctly. For me my whole spreadsheet basically had "holes" in it if items had those bad dates? So can u double check this, simply use PLEX or whatever popular item and try to get orders via api with "sethours" set and without and compare the result to the web output. So basically try to get those "bad" entries listed via api.
I could consistently upload "bad" data using evemon and basically could break my whole spreadsheet, since i would not "see" any orders at all in my uploaded regions/items.
Desmont McCallock
#72 - 2012-06-11 08:55:03 UTC
I would really much appreciate if you don't link EVEMon to this issue as EVEMon is not sending 'bad' data as you call them. You are creating misconceptions. Please try to find a different expression.
Andy DelGardo
#73 - 2012-06-11 09:15:17 UTC  |  Edited by: Andy DelGardo
Desmont McCallock wrote:
I would really much appreciate if you don't link EVEMon to this issue as EVEMon is not sending 'bad' data as you call them. You are creating misconceptions. Please try to find a different expression.


Um, sorry might have chosen my words unwisely :p I really would like to use eve-mon as updater, but because of this problem i had to revert back to the marketeer updater until now.

The problem seems "fixed" now, so i will test this later today again using eve-mon + XML API, so i'm sorry for "stirring" up troubles for u guys and i absolutely appreciate your hard work!

bye Andy
Muscaat
EVE Markets
#74 - 2012-06-11 23:25:49 UTC
Is "1" a success response when uploading to the UU endpoint? Based on earlier posts in this thread it is, but just want to check ;)
Kaladr
Viziam
Amarr Empire
#75 - 2012-06-12 01:46:23 UTC
Andy DelGardo wrote:
Desmont McCallock wrote:
I would really much appreciate if you don't link EVEMon to this issue as EVEMon is not sending 'bad' data as you call them. You are creating misconceptions. Please try to find a different expression.


Um, sorry might have chosen my words unwisely :p I really would like to use eve-mon as updater, but because of this problem i had to revert back to the marketeer updater until now.

The problem seems "fixed" now, so i will test this later today again using eve-mon + XML API, so i'm sorry for "stirring" up troubles for u guys and i absolutely appreciate your hard work!

bye Andy


Let me know if you see any major discontinuities in API vs Web data - there may be a several minute caching period, but it shouldn't be major.

Creator of EVE-Central.com, the longest running EVE Market Aggregator
Kaladr
Viziam
Amarr Empire
#76 - 2012-06-12 01:46:33 UTC
Muscaat wrote:
Is "1" a success response when uploading to the UU endpoint? Based on earlier posts in this thread it is, but just want to check ;)


Yes :)

Creator of EVE-Central.com, the longest running EVE Market Aggregator
Kaladr
Viziam
Amarr Empire
#77 - 2012-06-12 01:48:44 UTC
As a note: Peak response times sometimes took a nosedive due to the classic "collect the world" parallel garbage collector syndrome. As I've updated to Java 7u4, I've turned on the G1 collector which should significantly improve the "collect 10GB of heap" pauses. (On a side note, 'top' in Linux appears to cap out at 999% CPU).

The amount of new data thanks to EVEMon has been immense.

Creator of EVE-Central.com, the longest running EVE Market Aggregator
Ilyk Halibut
Deep Core Mining Inc.
Caldari State
#78 - 2012-06-14 02:21:00 UTC
Is that CPU load just in the HTTP gateway, or is that more so the storage/statistical/analytical stuff?

EVE Market Data Relay - A real-time feed of EVE Market data http://www.eve-emdr.com
Kaladr
Viziam
Amarr Empire
#79 - 2012-06-15 05:55:21 UTC
Ilyk Halibut wrote:
Is that CPU load just in the HTTP gateway, or is that more so the storage/statistical/analytical stuff?


That is the combined REST server (sitting behind nginx), API server, analytics, and upload parsing process.

Note that the high CPU use only came across when all 12 threads were dedicated to a full compacting collection. Normally, it averages to 20-30%.

With the new G1 collector in use, and a "small" memory leak fixed in the SMTP service, I'm happy to report peak response time is significantly better.

Creator of EVE-Central.com, the longest running EVE Market Aggregator
Kaladr
Viziam
Amarr Empire
#80 - 2012-07-30 23:38:25 UTC
A notice to everyone using the SMTP Push service:

The SMTP Push Service will be closed (i.e., no longer deliver new mail, accept signups, acknowledge its existence) on August 15th, 2012.

I advise everyone to use the EVE Market Data Relay as a (much better) replacement.

For more information, I suggest reading the blog article: http://yannramin.com/2012/07/30/sunsetting-the-mail-push-service-in-favor-of-emdr/

Creator of EVE-Central.com, the longest running EVE Market Aggregator