EVE Technology Lab

Hello forums o/

I was wondering if anyone out there has the know-how on importing the EVE market cache data you get every time you fetch item data to an Excel 2013 file. I know that some programs like EVE-Mentat can do it, but the program doesn't have the layout I desire. I've seen a lot of tutorials on how to do this with Google Sheets, but the data isn't as accurate as I'd like it to be. I'll say again please do not link a tutorial on how to do this in Google Sheets, I've already made 2 and they don't have the performance that I'm seeking.

Basically what I'm looking to do is make a spreadsheet that will hold buy/sell/quantity/etc. data for multiple stations. For instance, if I had a station trader in Jita and another character in any other system, I'd like to be able to fetch certain item data in-game on both characters and then import the data to the Excel file.

Thanks in advance for any help,
Price

I haven't done it yet, as that's a task which requires either some from-scratch programming (either creating a native Win32 or COM DLL consumable by Windows applications, which either scraps the cache itself or might act as an EMDR endpoint) or cobble together a tool chain by using i.e. Entity's reference Phyton stuff to scrap the cache, produce some kind of "readable" files from it and import those into Excel.

Either way - it's most likely something you won't put together within a couple of lazy sunday afternoon hours.

Forgot to mention the new CREST APIs. Not sure though if someone has put something easy to use together yet.

It's also worth mentioning that for the time being, the nowadays practiced cache scraping - although technically a vaiolation of the EULA - is tolerated (but discouraged) by CCP. But CCP FoxFour (I think it was him) has mentioned that with the release of the CREST market API, the EULA on cache scraping will be enforced rather sooner than later.

tbh, most people will continue to use eve-central. The feed is useful, but it's substantially more work than asking eve-central for the data. And when you only need approximate pricing, it's more work than you need.

I'd say trying to do it in Excel is going to be painful.

Visual basic script has really poor json handling.

However, what might be possible, is an application which exists purely to pull market data and store a file locally, which Excel could then interact with. Probably putting it into a database would be the best option, with an odbc connection for Excel.

It's not the fastest thing to update, when I was doing it in php, it was taking around half a second per request.


Does that sound like the kind of thing you'd want? What kind of information would you be looking for?

I would recommend using the EVE Central API. It is a lot easier to work with, especially for spreadsheets, and they are pulling their data from CREST so should pretty much be real time. Their API if I remember should also have all the information you are after.

Ah, does their API cover that? I had no idea :P I've noticed though that their order prices will sometimes differ drastically from the actual orders, leading me to believe that the data was pretty old. Is this just the current limitations of the 'real-time' APIs?

Great news I'm glad to hear that that fact that the data is inaccurate is because they're not actually pulling from the API yet.

Yeah, I thought that the players were using some kind of uploader for the data, I saw that they do that on 'eve-marketdata.com' as well. I would never use the uploader myself haha, while it would give me up-to-date data, it would also send the data to every other person that has a spreadsheet, effectively turning it into a double-edged sword. If I find a deal, I'd like as few people to know about it as manageable, that way I can squeeze every lost drop of ISK I can out of it.

Would you be willing to share with me how you're pulling the data? I can't for the life of me find anything helpful.

The endpoints are https://crest-tq.eveonline.com/market/{regionId}/orders/{buy | sell}/?type=https://api.eveonline.com/types/{typeId}/ and https://crest-tq.eveonline.com/market/{regionId}/types/{typeId}/history/, but as pointed out this is hard stuff if you're not up to commercial web development skills. There are APIs available to abstract some of this away, but consuming an API or rolling your own is a tiny fraction of the effort of wrapping your head around OAuth.

A short list of your hurdles are redirecting to EVE and receiving the response on your web service; constructing web requests for both auth and data with the necessary headers; managing access and refresh tokens; etc. CCP certainly didn't make this easy for the casual consumer, who will be far better off waiting for a 3rd party to do the legwork.