These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

 
  • Topic is locked indefinitely.
123Next page
 

Citadel Market Data

First post
Author
Haile Korhal
Professional Amateurs
#1 - 2016-09-11 05:04:11 UTC
My search-foo is pretty terrible and I haven't found any real resources for helping me with this problem, so I thought I'd post here and see if anyone has encountered similar issues or can point me in the right direction. This is going to go everywhere, so please bare with me.

I'm trying to learn how to import market data from Eve Market, using Fuzzworks (thank you!) as my guide and sole repository of knowledge (if there are any others I'd love to see them!). I've got a spreadsheet built in Excel that pulls data out of Jita just to test the principle to see if I'm even on the right track, and it works; I've got lots of data. So I am now trying to pull the same data out of a citadel.

I found the station_ids of my target test citadel and plunked that into the import, and I'm getting nothing out of it. Or rather, I'm getting everything with a static value of price=0 out of it. So I started digging into it to see if I could figure out why. I've discovered something even more confusing. So let me build you picture.

I've got a tab with the citadel's information imported into it.
I've got a tab with all the typeids in Eve imported into it.
I've got a tab with Jita's information imported into it.

I look up various items in the in-game market, search them up in the typeids tab, and then search for that typeid in either market tab. I'm always coming up with "No Results." At first I thought "maybe it was due to it being a citadel and not having information." So I checked the Jita tab, no results either. I've searched for "Dragonfly I", "Helium Isotopes", "Capital Hyperspatial Velocity Optimizer II", "Wyvern", "Revenant", etc. But, there are similarities. Plagioclase exists in both sheets just fine, but there aren't any on sale in the market in the citadel, so it wasn't an adequate target for this test.

I'd like to be able to pull all the same information you can out of Jita, out of this citadel. I thought it would be as simple as replacing all of the identifiers for Jita with the identifiers for this citadel, but that has proven to be moot so far. Is this not something that can be done yet with Citadels?

So I guess what I am getting at is that I really have no idea what I am looking at because it just doesn't make sense. Why do some items show up, but others aren't? I'm using the url on the Fuzzworks site with my name, a station id (for the citadel) / a region id (for Jita), and buysell=s. Am I missing something important?

I can't really find any other resources for learning how to build a proper url to pull from the Eve Market except for on Fuzzworks (and I do have to admit, it is still extremely hard to decipher even there). I know I need to put in some more effort to 'learn this for myself' but I cannot honestly find any more documentation or resources that I can read at my level. It is either "too advanced" or "not holistic" enough to actually mean anything. I wouldn't have posted here without doing as much homework as I can, and I can say that I am finally stumped enough to post here.

If you have any guidance, links, or anything, please eve-mail me or reply here. I'd love a discussion where I can learn something new. Even just a simple link to "go read this" would be sufficient if it helps me towards my goal. Thank you in advance,

Egregious Spreadsheet Services - For Spreadsheets as a Service to businesses, corporations, and higher, look no further!

Blacksmoke16
Resilience.
#2 - 2016-09-11 20:48:30 UTC
https://market.fuzzwork.co.uk/api/

Check out the power query section towards middle of the page. Might help you out a bit.
Haile Korhal
Professional Amateurs
#3 - 2016-09-11 20:57:20 UTC
Those urls look a bit weird, are they eve-central?

I'll take a look at the Power Query when I get home tonight. I believe I am utilizing that function, I've just called it a different name. The steps listed there do not look familiar, as in I don't believe I had them as an option. But, I haven't slept in a while so I'm probably just misremembering things. Thanks for the suggestion, I'll post the results when I get them.

I'm mostly concerned about why my pulled market information doesn't even contain the whole list from Jita; when I clearly pulled the whole typeids from the game.

Would the theory at least be the same for a citadel? As in, replace the Jita identification information, for example, with a target citadel? After digging to find the citadel stuff of course. I would hope similar information can be used from citadels as is possible with npc stations, especially if we're supposed to start using them in place of!

Thanks again. More testing will follow this evening/tomorrow morning.

Egregious Spreadsheet Services - For Spreadsheets as a Service to businesses, corporations, and higher, look no further!

Blacksmoke16
Resilience.
#4 - 2016-09-11 20:59:33 UTC
If you are using eve-central i would switch to steve's aggregate data. Should be much more accurate.
Haile Korhal
Professional Amateurs
#5 - 2016-09-11 21:10:14 UTC
You know, I've been using fuzzworks for my entire sole repository of knowledge (because I cant seem to find reputable help anywhere else) and I hadn't yet stumbled across this particular part of his site. I hadn't heard of his own aggregate stuff, although if it is just for the regions or hubs, then it won't be super useful for me. I'll still look into it, however. It looks like yet more knowledge to plumb.

I have a very basic understanding of how html works, and I'm trying to learn xml and all the other fiddly bits with this import stuff. My newest project is taxing my limits of comprehension and pushing me towards new horizons. Until now I've stayed away from all of this stuff because "it's so hard" and I'm not unimpressed with its apparent complexity. I wonder, after having lurked through the Eve Technology Lab for over a year quietly, if there is some underlying subject matter that I need to know first before trying to get involved with this craziness. It's almost as though everyone's speaking a foreign language, and the worst part is, I understand what every word means, but not the sentence in itself!

The json dataset example above the Power Query section looks to be the information that I want. I just need to figure out how to use json or whatever its called. I suppose vlookups will work wonderfully, but only if it is formatted properly I wager. Is there a way to get it to push from that code-look-a-like to an excel table? Or does that happen automatically? (Curses not being able to work on this here!)

I'm learning so much!

Egregious Spreadsheet Services - For Spreadsheets as a Service to businesses, corporations, and higher, look no further!

Messenger Of Truth
Butlerian Crusade
#6 - 2016-09-12 12:17:42 UTC
I think you might be running into the problem where CCP don't expose sell orders in citadels via CREST. Ranged buy orders show up but not sell orders.

Trade Hub Price Checker: stop.hammerti.me.uk/pricecheck

Visit "Haulers Channel" in game for all matters courier-related.

Structure name/system API: stop.hammerti.me.uk/api

Haile Korhal
Professional Amateurs
#7 - 2016-09-12 18:50:08 UTC
Oh damn, I had read about that somewhere but I was hoping it had changed. Well this complicates things. I don't suppose it is possible to get other information out of it then? If sell orders aren't exposed yet (which I hope they are soon!), is it possible to get anything else?

I doubt anyone knows, but any idea when CCP might be expanding this functionality for us? Would it be worth filing a suggestion support ticket?

Egregious Spreadsheet Services - For Spreadsheets as a Service to businesses, corporations, and higher, look no further!

Messenger Of Truth
Butlerian Crusade
#8 - 2016-09-12 20:29:05 UTC
According to a forum post from CCP last week, CCP have plans to do more API support for citadels, but the team that would/will be working on it is working on support for a mobile app at the moment. So there will be stuff...just not yet...

Trade Hub Price Checker: stop.hammerti.me.uk/pricecheck

Visit "Haulers Channel" in game for all matters courier-related.

Structure name/system API: stop.hammerti.me.uk/api

Haile Korhal
Professional Amateurs
#9 - 2016-10-09 22:23:34 UTC
Has there been any news regarding this topic yet?
Can we pull from the API any citadel market data yet?
Where can I check to follow up on this, so I don't have to spam the Technology Lab every time I'm looking for an update?

I'm sure it has to be soon! Citadels have been out for a while, and those markets are getting pretty jam packed. Plus I really need to finish up my citadel market spreadsheet! Not only am I anxious for it, so is my customer. :o

Egregious Spreadsheet Services - For Spreadsheets as a Service to businesses, corporations, and higher, look no further!

Draconas109
Dova Industry
#10 - 2016-10-17 00:53:18 UTC
+1 I want citadel market info in crest asap, having to do it all by hand is getting ridiculous
Haile Korhal
Professional Amateurs
#11 - 2016-11-03 02:50:49 UTC
I am still learning about all of this craziness, but I believe I read somewhere that CREST is something like a 3d data architecture, which Excel uses a 2d architecture, so in order to get it to play together you have to flatten it. I believe I also read something about Fuzzworks saying 'it's possible, but hard' or something like that. Considering I don't want to get into programming an entire 3rd party tool, and I'd rather stay within the realm of a spreadsheet, I don't believe I can use CREST; unless I am completely not understanding what I am reading, and I will admit everything I've read about CREST seems to be written in some form of older than old english. lol-.-lol

So, while we still need CREST data for citadel stuffs, can we have that for XML so we can use that in a spreadsheet? Right now I have to manually fly to 24 different citadels spread out over vast distances, and manually copy their entire content (from the sell market) into a spreadsheet so I can crunch that raw data it into information. It's a lot of work. :/

I know how to find the Citadel station_id, why cant we just import like we do for stations? ccpls! :o

Egregious Spreadsheet Services - For Spreadsheets as a Service to businesses, corporations, and higher, look no further!

Blacksmoke16
Resilience.
#12 - 2016-11-03 02:56:44 UTC  |  Edited by: Blacksmoke16
https://market.fuzzwork.co.uk/api/

Look towards the top of the page for the 'Power Query' section. May be of some use for you.
Haile Korhal
Professional Amateurs
#13 - 2016-11-11 18:54:35 UTC
Thank you for the reminder, I looked at that before when you first mentioned it and I looked at it again, can that pull the data out of a Citadel? That's what I'm trying to get at. I can't seem to get anything to pull anything out of a Citadel.

Anyway, I am thinking I need to redirect all of my attentions at this point. CCP mentioned ESI and I have never heard of that stuff before, could it do what I am looking to do?

1. Import list of items for sale in a citadel

That's it. I can use vlookup and sums and such to get the data to look how I want, but I'm having trouble getting the data into a spreadsheet to start with. Right now I fly to each citadel, which are several dozen jumps apart, and copy the market information one line at a time, by hand, into a spreadsheet, and then I can get it to act how I want.

If I need to learn ESI or CREST or XML for this, I want to, but I am finding it nearly impossible to teach myself. I really feel like I am missing something really basic that would be the key to unlocking all of this knowledge.

I know other people have said CREST doesn't expose such data, so I guess that's out until they do. But, I'm not a programmer, and I know very little scripting, but I'm willing to learn what I need to know in order to get this spreadsheet to work; I just need to know what I need to learn so I can go find it.

Egregious Spreadsheet Services - For Spreadsheets as a Service to businesses, corporations, and higher, look no further!

Blacksmoke16
Resilience.
#14 - 2016-11-11 19:15:56 UTC  |  Edited by: Blacksmoke16
IIRC market data is not in CREST atm except buy orders. Will have to wait for ESI market endpoints to get that.

So yea, are pretty much limited to only stations for the moment.

Once an endpoint is released you could prob make a script to get that information and put it into your sheet. However I don't know what you are doing with your sheet so depending on how many items/regions you are fetching it could take a while. At that point it may be better to use an aggregate like steve's.
Haile Korhal
Professional Amateurs
#15 - 2016-11-11 20:07:39 UTC  |  Edited by: Haile Korhal
Damn, that is what I was really afraid of. I don't suppose you know of a way to copy large amounts of data out of the market window by hand? The game only lets you select one row at a time, grr. Thank you for the quick reply, I'll continue to sit and stew. CCPlease!

I'm just trying to import a list of items, how many are for sale, and then the sum of their value (probably just an estimate off Jita value multiplied by their quantity).

Egregious Spreadsheet Services - For Spreadsheets as a Service to businesses, corporations, and higher, look no further!

Blacksmoke16
Resilience.
#16 - 2016-11-11 20:26:48 UTC
Could bring up that item in the market viewer, click 'export to file' at the bottom and then parse the data in that file by copy/pasting it into your sheet.

However not sure how you would parse which are buy and which are sell.
Kendepp
State War Academy
Caldari State
#17 - 2016-11-18 19:25:18 UTC
Dear ccp, we REALLY need market data for our apps. Please give any info whether will you do it or not and any estimated date. I dont even know if i need to collect personal api for marketdata or to wait till you refresh your api code.
thx!
GoldLeader
Zansha Expansion
#18 - 2016-11-22 15:04:07 UTC
One more bump for this one.

CCP: This is the ONLY thing keeping my alliance from moving into a citadel. We can't reliably stock a citadel market if we can't reliably access the data.
Haile Korhal
Professional Amateurs
#19 - 2016-12-02 12:02:43 UTC
Another bump for the original problem.

On another issue, I couldn't get the eve central API to work at all, it would never return a value no matter what I did with it, so I tried to use Eve Marketdata and I feel like I've gotten further with it, but I'm still getting an error where it says "imported content is empty." I built the query with the query builder for Google Docs API Features > Location Prices
API to use: price_type_station_sell_min
Your character name: Haile Korhal
type_id is in column: P
Data is in column: ??? I don't know what Data it is talking about. So I put in R because I dunno what to put in it.
Starting on row: 45
Ending on row: 69

Also, how do I parse it the Jita station id?

It gave me this: =ImportXML("http://eve-marketdata.com/api/price_type_station_sell_min.xml?char_name=Haile Korhal&type_id="&JOIN(",",$P45:$P69)&"&v="&JOIN(",",$R45:$R69), "/emd/val")

Whic returns that error. Here's a picture of the very few items I'm trying to get, and their TypeIDs which I pulled out of the Fuzzworks document.

Any recommendations?

Egregious Spreadsheet Services - For Spreadsheets as a Service to businesses, corporations, and higher, look no further!

Awkward Pi Duolus
Pator Tech School
Minmatar Republic
#20 - 2016-12-02 19:39:24 UTC
GoldLeader wrote:
One more bump for this one.

CCP: This is the ONLY thing keeping my alliance from moving into a citadel. We can't reliably stock a citadel market if we can't reliably access the data.


This. It confused this life out of me when some of my orders were not showing up in downstream analysis, until I realized that CREST does not pull data from citadels.

CCP... what gives?
123Next page