These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

 
  • Topic is locked indefinitely.
 

Reading Whole Market Prices with API, eve-marketdata or eve-central?
Author
Previous Topic Next Topic
Tanin Majere
Orange Lazarus Petroleum Inc.
#1 - 2015-12-08 17:13:18 UTC
Hello,

I am trying to build an Excel sheet to help me with manufacturing/trading.

I would like to read all the prices from the market for a specific station or system. I have been using the documentation by fuzzysteve to get me started so far and it was a ton of help.

My question is, does eve-marketdata update its prices from the Crest? Is it still being supported by the developer? When I read the prices using it, I see a significant difference in the update time of prices for various items, which made me question its update status.

Is there a way to do what I want using the API of eve-central? I read that the only think I can do is write down the typeIDs I need for the web query for each item I want. But is there a way to access all available items?

Also for the Excel gurus out there, whenever I try to use "import data from web" function in excel it takes me to a stupid MSN webpage that throws ton of errors. Its very annoying as I am just testing things and reuse that function a lot. Is there a way to make the default or homepage on that function blank for instance? Google failed me on solving that one.

Thanks,
Tanin
Jaantrag
#2 - 2015-12-09 10:06:08 UTC
eve-marketdata seems to be still useing the uploader method ..
last update according to the site nesw/blog section was in summer but some random items i loked from november are in there .. might be best prolly to actualy directly mail the dev there

eve central is prolly better option .. u can collect all the id-s to one query and go from there, as pointed out in the link u provided .. u can deddicate one page in the excel to data pulling list all the ids u plan on pulling and then other pages use that one .. havent used excel much for doing things like this, as google docs is little easyer imo to handle and collabarate with others ..



EVElopedia < add this to your sig to show u WANT it back
Tanin Majere
Orange Lazarus Petroleum Inc.
#3 - 2015-12-10 00:11:40 UTC  |  Edited by: Tanin Majere
Thank you for your input! I will avoid eve-marketdata for now since what you described was my impression as well.

I was looking for a more flexible way to do it for excel, with station and typeIDs as cell parameters to the query, but I guess I will settle with the method described by fuzzysteve. I did found about the FILTERXML(WEBSERVICE()) function in excel, which is exactly like importXML in googledocs and exactly what I wanted. But even with a mere 20 reads it takes too long each time I make a different read.

I decided to take the excel over google docs route because I read people having trouble with large amount of data in docs. But I am not so sure about it anymore since excel has been very slow for me as well, even though only thing I read from the internet is the prices.
Louis Vitton
Viziam
Amarr Empire
#4 - 2015-12-14 20:47:28 UTC
Google docs is very good I personally find, however I write additional scripts to allow for more integration and use of my own functions.

Steve also has written a nice script that will allow you to pull from eve-central.
The script is far better than using the import xml function as the import function kinda randomly wont work.

Link to the scripts: https://github.com/fuzzysteve/eve-googledocs-script
Documentation from Google Docs on adding your own functions: https://developers.google.com/apps-script/guides/sheets
Tanin Majere
Orange Lazarus Petroleum Inc.
#5 - 2015-12-14 23:25:35 UTC  |  Edited by: Tanin Majere
Thanks for the input Louis! That script was the reason I was trying to find more optimal xml import commands for Excel. Performing an xml read for each item simply takes too long.

I managed to fix my problem using the code sample posted here.

Right now I am able to read all information for about 80 items with a single xml load and parsing within VBA in mere seconds. I will expand my database and see how it performs for larger data reads but even if I end up using a couple reads I believe the execution time will be low enough for my needs.

Thanks,
Tanin

PS: I eve-mailed the developer of eve market-data, asking if he has any plans to update the site. I have not received any reply but as of today the site seems to be down. So that answers the main question I asked in the beginning I guess.
Lemkor Gengod
Stacmon Production and Trade Institue
#6 - 2015-12-16 01:57:55 UTC  |  Edited by: Lemkor Gengod
Eve market-data doesn't update from CREST, it still uses the uploader.

You can pull almost a whole region by simply omitting any typeID from the XML. (I say almost because it is limited to 10,000 lines and since the ship skins came in, we have gone a little above the 10,000 now).

Eve market data goes down every few months, you just learn to deal with it and have a contingency plan. The developer i believe is a one man team doing it for a hobby and no longer has the time to dedicate to it as he once had. He also rarely replies to mails, but he usually gets it back online within a few days to a week.

Don't use Excel. Yuk - Googledocs all of the way. Easily 250 items in one call. I haven't had any trouble with eve central and googledocs apis since introducing a cachebuster. If you are using excel before 2013, you will have visions of doing bad things to Bill Gates in your sleep. Mainly because it is very complicated to link a list of cell contents into your URL. Post 2013 works a little better, but as usual, Microsoft entered the game so late, Google had already been declared the winner and is sat around a bar celebrating before Microsoft arrived for the kickoff.

To get better performance with the evemarketdata api in googledocs, use the IMPORTDATA command with the .txt file

Eve central is more accurate, but you can't do a full region/system dump like you can with EMD, eve central also doesn't have sales data.

Finally, if you are wanting to use huge amount of data, and you have experience with coding, build your own system. It will allow you to insert time delays so as to not overload the servers by making too many calls at once. I've had success with Python, JavaScript, Java and Php with the Eve central API. Eve Market Data is a little harder with the web based languages because it isn't set up to send data to another web server.
AzTrackGuy
Advanced Excavators Inc.
#7 - 2015-12-19 20:08:47 UTC
its too bad EMD is done, all of my market info came from there & now all of my google docs sheets are broken since the site is down., which is too bad, I really liked how easy it was to use the xml from there

"If you put forth the effort to succeed... you will"