These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

 
  • Topic is locked indefinitely.
 

Market API sell orders and Citadels PROBLEM

First post
Author
Vrot Kompot
AirGuard
LowSechnaya Sholupen
#1 - 2017-05-03 19:10:39 UTC
Hey! Do you fix API results for market sell orders from citadels? Sell orders from the citadels is not showing!
Pete Butcher
The Scope
Gallente Federation
#2 - 2017-05-03 19:34:08 UTC
Vrot Kompot wrote:
Hey! Do you fix API results for market sell orders from citadels? Sell orders from the citadels is not showing!


To fetch orders from citadels you need to use authed ESI: https://esi.tech.ccp.is/latest/#!/Market/get_markets_structures_structure_id

http://evernus.com - the ultimate multiplatform EVE trade tool + nullsec Alliance Market tool

Vrot Kompot
AirGuard
LowSechnaya Sholupen
#3 - 2017-05-03 20:12:23 UTC
I need all sell orders in region! How I get all structures? get_markets_structures_structure_id - needless ****...
But ESI and CREST market buy orders is work well and showing buy orders in all structures!
Steve Ronuken
Fuzzwork Enterprises
Vote Steve Ronuken for CSM
#4 - 2017-05-03 21:19:43 UTC
Vrot Kompot wrote:
I need all sell orders in region! How I get all structures? get_markets_structures_structure_id - needless ****...
But ESI and CREST market buy orders is work well and showing buy orders in all structures!



Nope, they're only showing ranged buy orders. Ones you can sell to, regardless of docking rights.

If you want the other structures, you have to ask for each in turn.

Woo! CSM XI!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter

Vrot Kompot
AirGuard
LowSechnaya Sholupen
#5 - 2017-05-04 11:11:23 UTC
This is stupid mechanics... If seen in game, should be seen in api
Steve Ronuken
Fuzzwork Enterprises
Vote Steve Ronuken for CSM
#6 - 2017-05-04 11:47:49 UTC
Vrot Kompot wrote:
This is stupid mechanics... If seen in game, should be seen in api



It's because of privileges and caching.

If it was in the main feed, that would need to be authenticated, and couldn't be cached. Which isn't a viable option.

Woo! CSM XI!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter

Lil Afro
Viziam
Amarr Empire
#7 - 2017-05-06 10:52:04 UTC
Does this mean that you need to iterate through all public citadels to find the ones you are interested in and then dumping out all of their orders to pick out the ones for the items that you are interested in?

If that is the case then that seems highly inefficient and complicated
Pete Butcher
The Scope
Gallente Federation
#8 - 2017-05-06 12:06:26 UTC
Lil Afro wrote:
Does this mean that you need to iterate through all public citadels to find the ones you are interested in and then dumping out all of their orders to pick out the ones for the items that you are interested in?

If that is the case then that seems highly inefficient and complicated


It would be great to have some kind of bulk endpoint. At this moment you need to make thousands of requests to get the market.

http://evernus.com - the ultimate multiplatform EVE trade tool + nullsec Alliance Market tool

Althalus Stenory
Flying Blacksmiths
#9 - 2017-05-07 01:53:13 UTC
they won't, as it would mean a cache per existing ACL on that endpoint, not speaking on how they'll get all structures you have access to etc etc..

EsiPy - Python 2.7 / 3.3+ Swagger Client based on pyswagger for ESI

Steve Ronuken
Fuzzwork Enterprises
Vote Steve Ronuken for CSM
#10 - 2017-05-08 01:41:28 UTC
Pete Butcher wrote:
Lil Afro wrote:
Does this mean that you need to iterate through all public citadels to find the ones you are interested in and then dumping out all of their orders to pick out the ones for the items that you are interested in?

If that is the case then that seems highly inefficient and complicated


It would be great to have some kind of bulk endpoint. At this moment you need to make thousands of requests to get the market.



You've got on the order of 120-200 odd requests to get all the station based markets. with the majority of that being the forge

for citadels, last I looked there were around 350 structures with public docking rights, with markets in them. Most with less than 10k orders per, so you're looking at around 500 requests total.

Woo! CSM XI!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter

Pete Butcher
The Scope
Gallente Federation
#11 - 2017-05-08 06:15:04 UTC
Steve Ronuken wrote:
You've got on the order of 120-200 odd requests to get all the station based markets. with the majority of that being the forge

for citadels, last I looked there were around 350 structures with public docking rights, with markets in them. Most with less than 10k orders per, so you're looking at around 500 requests total.


Just checked and The Forge alone has around 900 known citadels - that's 900 paginated requests. In other words, we're looking at lower bound of 900 for this single region. If you add multiple regions, we're going up to thousands of requests.

http://evernus.com - the ultimate multiplatform EVE trade tool + nullsec Alliance Market tool

Steve Ronuken
Fuzzwork Enterprises
Vote Steve Ronuken for CSM
#12 - 2017-05-08 16:48:46 UTC  |  Edited by: Steve Ronuken
Pete Butcher wrote:
Steve Ronuken wrote:
You've got on the order of 120-200 odd requests to get all the station based markets. with the majority of that being the forge

for citadels, last I looked there were around 350 structures with public docking rights, with markets in them. Most with less than 10k orders per, so you're looking at around 500 requests total.


Just checked and The Forge alone has around 900 known citadels - that's 900 paginated requests. In other words, we're looking at lower bound of 900 for this single region. If you add multiple regions, we're going up to thousands of requests.



However, most of them don't have markets.

Sure, you have to, on an irregular basis, run a check on what the structures are, to see if they might have a market (all astrahus can be ignored, for example) but once you've run through that, (around 3000 public structures), you just need to look. And while yes, it _might_ be paginated, if there's less than 10k orders, you don't need to ask for another page. my ballpark of 250 with markets wasn't a number just pulled out the air. I check this stuff on a semi regular basis.

And you can't judge every region based on the Forge. It, and Delve, are quite different from the 'average'.

Woo! CSM XI!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter

Pete Butcher
The Scope
Gallente Federation
#13 - 2017-05-08 16:58:36 UTC  |  Edited by: Pete Butcher
Steve Ronuken wrote:
However, most of them don't have markets.

Sure, you have to, on an irregular basis, run a check on what the structures are, to see if they might have a market (all astrahus can be ignored, for example) but once you've run through that, (around 3000 public structures), you just need to look. And while yes, it _might_ be paginated, if there's less than 10k orders, you don't need to ask for another page. my ballpark of 250 with markets wasn't a number just pulled out the air. I check this stuff on a semi regular basis.

And you can't judge every region based on the Forge. It, and Delve, are quite different from the 'average'.


Unfortunately, I can't make any guesses until I actually check a given structure (except for Astrahus). My users want real-time data without any assumptions, which might turn out to be false an hour from the last check, like access rights. I checked and the total of known citadels is about 11k. Even with one page - that's 11k requests. Even if we make some assumptions and declare half of them don't have any orders, that's still 5.5k requests.

EDIT: Exactly 5139 citadels at this moment.

http://evernus.com - the ultimate multiplatform EVE trade tool + nullsec Alliance Market tool

Messenger Of Truth
Butlerian Crusade
#14 - 2017-05-10 17:47:56 UTC  |  Edited by: Messenger Of Truth
I've also thought to myself how ridiculous it is that you have to fetch all the station orders, then individually poll hundreds of structures to get any structure orders (despite there being no API that tells you whether there is a market module or not).

I've got an API that aggregates all the orders from a region (npc station, outpost and public upwell structures) with no paging. From my point of view,, this is how CCP should have designed their API.

However Domain results in a 38MB json page, and The Forge generates a 67MB json page. Sending this kind of data to people every 5m or so isn't viable so I haven't made this public.

I might be able to put this behind a CDN and make it publicly available, butI had a look at cloudflare and found that its interested in objects with a lifetime of several hours, not several minutes...

Bittorrent or a similar p2p protocol would be ideal for this use case (I have visions of a service that provides a new torrent file per region every 5 minutes or so, then people fetch this torrent file and start fetching data from me and sharing it with others), but it would take a lot of heavy lifting and setup for people to get involved (as bittorrent is not very standard) and that would kind of defeat the purpose of making a convenient API intended to address the shortcomings of CCP's less convenient API!

Trade Hub Price Checker: stop.hammerti.me.uk/pricecheck

Visit "Haulers Channel" in game for all matters courier-related.

Structure name/system API: stop.hammerti.me.uk/api