These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

 
  • Topic is locked indefinitely.
 

RETURNED - EVE Marketeer - Market Data Aggregation & Information
Author
Previous Topic Next Topic
Callean Drevus
Perkone
Caldari State
#421 - 2012-02-07 22:22:42 UTC  |  Edited by: Callean Drevus
Nope, not blacklisted. Seems the problem had something to do with InnoDB. Stupid innodb size... I hope it has been fixed for now.

Developer/Creator of EVE Marketeer
Jacob Lyon Chieve
Sanguis Mortem Industrial
#422 - 2012-02-08 04:23:12 UTC
Having issues once more. On mouseover, the cell containing the XML call says "error: the XPath query did not return any data"

FWIW, I can get onto the website just fine.

Need to know how much ore to mine in order to yield a certain order of minerals? https://forums.eveonline.com/default.aspx?g=posts&t=67477 It will even calculate how long you can expect for it to take!
Dragonaire
Here there be Dragons
#423 - 2012-02-08 05:01:34 UTC
If you're running out of database connections then you have something not releasing them correctly some where. Make sure when you're done with the connection on a page you set you variable for your connection to NULL should help make sure it gets released. Something like $mysql->close();$mysql=NULL; works wonders at times Blink

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.
Aineko Macx
#424 - 2012-02-08 06:53:22 UTC  |  Edited by: Aineko Macx
Some random comments/questions:
- in the API description for the historical averages, the description for 'averages', 'maximum' and 'minimum' are wrong. Those refer to the item volume, not transaction count.
- When querying using station_info, in the returned document station_id is null
- How are you estimating the sell/buy percentages?
- I'd very much like to have the historical data expanded to have a row per day, instead of just one averaged for the whole period. Would you consider doing that?

Keep up the good work!

iveeCore 3.0: The PHP engine for industrial activities and CREST library
Callean Drevus
Perkone
Caldari State
#425 - 2012-02-08 09:02:32 UTC  |  Edited by: Callean Drevus
Jacob Lyon Chieve wrote:
Having issues once more. On mouseover, the cell containing the XML call says "error: the XPath query did not return any data"

FWIW, I can get onto the website just fine.


This might be because Google caches the request your xml call makes for a while, so it remembers whatever happened when the site was down. If the call you are doing by XML is going wrong if you enter the URL in a browser, please give me that URL, so I can investigate.

Quote:
If you're running out of database connections then you have something not releasing them correctly some where. Make sure when you're done with the connection on a page you set you variable for your connection to NULL should help make sure it gets released. Something like $mysql->close();$mysql=NULL; works wonders at times

Ah, turns out it wasn't connections, but InnoDB log file size, the odd thing is that it continued to work until it really couldn't do anything anymore. I do not exactly understand the reason, but the size has been increased and I do not expect to see this problem again.

- in the API description for the historical averages, the description for 'averages', 'maximum' and 'minimum' are wrong. Those refer to the item volume, not transaction count.

I think you are quite correct. Number of transactions is a meaningless number anyway. I'll fix this to state what it means correctly.

- When querying using station_info, in the returned document station_id is null

Ah, yes, this is an unfortunate artifact of not saving the station id when orders are added together. This will probably change some day, but as you already know what station it is when querying, it isn't exactly high on my list of priorities.

- How are you estimating the sell/buy percentages?

Using magic! No, the calculation is simply to see whether the average lies more in the direction of the highest transaction that day, or the lowest transaction that day. High has a good chance of being sell, low has a good chance of being buy.

- I'd very much like to have the historical data expanded to have a row per day, instead of just one averaged for the whole period. Would you consider doing that?

I thought this was already possible! Where the hell has my history API gone... Anyway, will be added ASAP. UPDATE: Yes! I already had it, but it's not on the API page: http://www.evemarketeer.com/api/history/24698/10000002/xml

Developer/Creator of EVE Marketeer
Aineko Macx
#426 - 2012-02-08 19:49:19 UTC
Callean Drevus wrote:
- I'd very much like to have the historical data expanded to have a row per day, instead of just one averaged for the whole period. Would you consider doing that?

I thought this was already possible! Where the hell has my history API gone... Anyway, will be added ASAP. UPDATE: Yes! I already had it, but it's not on the API page: http://www.evemarketeer.com/api/history/24698/10000002/xml

Awesome!
I see you are returning the whole history set there. You might want to introduce a time period parameter (today minus n days) so we can query for just the needed history without making your servers angry Blink

iveeCore 3.0: The PHP engine for industrial activities and CREST library
Callean Drevus
Perkone
Caldari State
#427 - 2012-02-08 21:37:48 UTC
Filtering the days will probably not make it much faster to load, but it will make it easier on you to use ;) I'll add that to the list of stuff to do :P

In addition, I've added support for character groups today, which will allow me to do some very cool things that some other market programs already do. For more information, see the newsitem on the website. And please report any issues you find.

Developer/Creator of EVE Marketeer
Domi Etymology
Violent Alternatives
#428 - 2012-02-09 01:35:15 UTC
Getting a lot of
Quote:
Warning: mysql_select_db() expects parameter 2 to be resource, boolean given in /var/www/www.evemarketeer.com/core/lib/cordb/cordb.php on line 38 Warning: mysql_ping() expects parameter 1 to be resource, boolean given in /var/www/www.evemarketeer.com/core/lib/cordb/cordb.php on line 29 Database connection #0 failed


Got it both trying to access the index page, as well as the regions page. You messing around with something at the moment?
Dragonaire
Here there be Dragons
#429 - 2012-02-09 06:00:16 UTC
Just to go with the above:
EVE Marketeer: Upload unknown problem:
Warning: mysql_select_db() expects parameter 2 to be resource, boolean given in /var/www/www.evemarketeer.com/core/lib/cordb/cordb.php on line 38

Warning: mysql_ping() expects parameter 1 to be resource, boolean given in /var/www/www.evemarketeer.com/core/lib/cordb/cordb.php on line 29
Database connection #0 failed

So it's also effecting uploads now P

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.
Scrapyard Bob
EVE University
Ivy League
#430 - 2012-02-09 06:42:30 UTC
Dragonaire wrote:

So it's also effecting uploads now P


Days like these, I just point my in-game browser at either eve-marketdata market scanner page or eve-central's upload suggestions page.
Callean Drevus
Perkone
Caldari State
#431 - 2012-02-09 07:28:39 UTC
Damn, the reproduction of the problem is bothering me. I cannot. I'll try to enter a nice new check in the page to see exactly what error is occuring. Seems it had little to do with the mass of errors I fixed in the mysql log.

I suspect my changes to yapeal are to blame.

Developer/Creator of EVE Marketeer
Dragonaire
Here there be Dragons
#432 - 2012-02-09 08:24:40 UTC
If it's something with Yapeal contact me and I'll go over it with you to figure it out. Also I've been cleaning some stuff up that might have been causing some slow downs in Yapeal in the last couple days so you might want to look at updating as well. I pushed it to the hg already just haven't updated the archives yet that gets to wait until tomorrow I need to head to bed Blink

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.
6ie
The Kiwis
#433 - 2012-02-09 12:31:19 UTC
Did you fix the search? Its acting weird - put in Projectile Weapon Rigging
You get a page: Could not find 'Projectile Weapon Rigging'
Perhaps you meant one of these items?

And it only lists one item... Projectile Weapon Rigging
Scrapyard Bob
EVE University
Ivy League
#434 - 2012-02-10 16:20:22 UTC
Database errors are still happening on a regular basis, both in the uploads queue as well as various pages on the website.

At a rough guess, about 1/4 of all uploads are failing, and 10-20% of the web page hits.
Zifrian
Federal Defense Union
Gallente Federation
#435 - 2012-02-10 17:52:42 UTC
I've noticed that some people are having this same error:

Warning: mysql_select_db() expects parameter 2 to be resource, boolean given in /var/www/www.evemarketeer.com/core/lib/cordb/cordb.php on line 38 Warning: mysql_ping() expects parameter 1 to be resource, boolean given in /var/www/www.evemarketeer.com/core/lib/cordb/cordb.php on line 29 Database connection #0 failed

I'm getting this with the API, sometimes it works fine and other times it doesn't.

I added this to EVE IPH a few weeks ago to query the API for average price data. I'm not sure if there is an increase in queries? Let me know if I can help.

Maximze your Industry Potential! - Download EVE Isk per Hour!

Import CCP's SDE - EVE SDE Database Builder
Scrapyard Bob
EVE University
Ivy League
#436 - 2012-02-10 19:23:45 UTC
From the uploader over the past few days:

EVE Marketeer - 51.6k
EVE MarketData - 68.4k

So EVEMarketeer, which usually runs neck and neck with EMD, is losing about 25% of the upload attempts due to the database error.
Callean Drevus
Perkone
Caldari State
#437 - 2012-02-10 19:49:45 UTC  |  Edited by: Callean Drevus
Fixing it now

UPDATE: Should be fixed now. Will continue to monitor over the weekend. The cause was... yapeal running rampant on the server and eating all connections. Yapeal runs quite some time, so eventually all mysql connection slots were filled.

This is NOT the fault of Dragonaire, whom very correctly built in a measure to stop yapeal from going mad, which I, in my stupidity, disabled.

Besides that, the server was limited to 100 concurrent connections, which made the problem appear early and often. If it had been higher there would have been 1000 yapeal instances hammering the EVE API servers, which would probably get me banned very fast and thus stop all the yapeals from going mad :P

But seriously, I do hope this fixes the problems. Yapeal has been restored, and the connection limit is now 1000.

Developer/Creator of EVE Marketeer
Dragonaire
Here there be Dragons
#438 - 2012-02-11 05:33:25 UTC
Now I've got to know what you were trying to change in Yapeal that caused the problem Question but I'll let you do it in a Eve-mail if you want instead of here Smile

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.
Scrapyard Bob
EVE University
Ivy League
#439 - 2012-02-11 06:57:40 UTC
Another wish item for the LP "best loyalty points".

- Sort the two drop-lists in alphabetical order before presenting it to the client.

And another argument in favor of only showing the list of corps that sell a particular LP item rather then every single station.

http://www.evemarketeer.com/item/info/9956

3393 rows returned on the LP Stores tab for that item, which takes a bit to load. :)
Callean Drevus
Perkone
Caldari State
#440 - 2012-02-11 08:24:35 UTC
Oh, yes, sorting should be something that is easliy implemented, but very handy to have.

Regarding the LP store items: I agree with you that it should be done on a different corporation page (listing all the stations), the loading is not the issue for me though :P just that 3300 items are not very reasy to browse :)

Developer/Creator of EVE Marketeer