These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

 
  • Topic is locked indefinitely.
 

Citadel market orders showing up in crest bulk data.
Author
Previous Topic Next Topic
Dulock Sental
Center for Advanced Studies
Gallente Federation
#1 - 2016-07-05 12:51:50 UTC  |  Edited by: Dulock Sental
/market//orders/all/ is returning some weird stationIDs (13 digits instead of the usual 8) in Black rise some of the orders have stationID: 1021332707232 and different regions have different IDs (1021332707232 is the only 13 digit ID in black rise but there are two different IDs in Derelik).

It could be that citadel market orders have been partially added to crest or it could be just be a bug somewhere in crest.
Dropbox (too big for Pastebin) of black rise orders: https://www.dropbox.com/s/zpty6wct5f4iepn/log.txt?dl=0
and since that file is really hard to read here is one of the orders in question:
{"buy": true, "issued": "2016-06-22T05:25:24", "price": 858461000.0, "volume": 1, "duration": 90, "id": 4574663681, "minVolume": 1, "volumeEntered": 1, "range": "20", "stationID": 1021332707232, "type": 23911}

this one appears to be a buy order for a Thanatos placed on the 22nd of June (I've been pulling bulk market data for a while so it would have shown up before now if it had been included in crest data previously)

I'm gonna travel to Black rise and see where this order was placed

Edit: The order above was placed in a citadel ("Black Rise Central Hub - Public") in Black rise. so it seems citadel market orders are now showing up in crest data.

Edit2: It's likely that this was caused by todays patch
Quote:
Ranged buy orders from non-public Citadels are now shown in the market order list, as they can fulfilled from stations in range.
Ydnari
Estrale Frontiers
#2 - 2016-07-05 16:08:18 UTC
Hmm, could explain why my site is now chucking errors for many but not all requests, don't think I have stationid as a bigint on that route.

--
Dulock Sental
Center for Advanced Studies
Gallente Federation
#3 - 2016-07-05 16:41:19 UTC
I don't think it's just Black rise and Derelik those are just the first regions my code pulls data for. (haven't checked other regions)
Ydnari
Estrale Frontiers
#4 - 2016-07-05 18:14:43 UTC
Yup, it's everywhere (not just the bulk endpoints, the individual ones too), and my code is falling over because it doesn't know what solarsystem this unknown station ID is in, so ranges make no sense.

--
Ydnari
Estrale Frontiers
#5 - 2016-07-05 18:57:00 UTC
Having public citadel orders in the data is nice and all, but it's useless without a corresponding API to get at least the name and location of the citadel represented by the ID. I still don't know of one, was one secretly released along with this change?

--
Dulock Sental
Center for Advanced Studies
Gallente Federation
#6 - 2016-07-05 19:07:20 UTC  |  Edited by: Dulock Sental
I'm not sure this change was even intended, it may just be a result of the change to buy order visibility:
Quote:
Ranged buy orders from non-public Citadels are now shown in the market order list, as they can fulfilled from stations in range.


As for the API endpoints I don't think the API gives any data about citadels (except the market orders) though since there are still a few undocumented endpoints (or at least I can't find any documentation for /universe/locations/) I may be wrong.

Edit: I think it must be an unintended change since using https://crest-tq.eveonline.com/market/10000069/orders/buy/?type=https://crest-tq.eveonline.com/inventory/types/23911/ shows the the citadel order from before and gives the location data as:
"location": {"id_str": "1021332707232", "href": "https://crest-tq.eveonline.com/universe/locations/1021332707232/", "id": 1021332707232, "name": "Black Rise Central Hub - Public"}

which does give the name of the citadel but the address given returns a 500 server error
KenFlorian
Jednota Inc
#7 - 2016-07-06 00:55:49 UTC
Citadel data now present in my import...broke my code due to data size.
salacious necrosis
Garoun Investment Bank
Gallente Federation
#8 - 2016-07-07 02:26:28 UTC
Is there now an endpoint to map citadel location IDs to names?

Use EveKit ! - Tools for EVE Online 3rd party development
Dulock Sental
Center for Advanced Studies
Gallente Federation
#9 - 2016-07-07 10:48:10 UTC
as far as I can tell there isn't a direct way but https://crest-tq.eveonline.com/market/Region/orders/buy/?type=https://crest-tq.eveonline.com/inventory/types/Type/
Where Type is a type that is being bought in a citadel in the given Region returns a list of market orders which include some location data (the ID of the citadel and the name only along with a non-functioning link to a /universe/locations/ resource), I don't think there's any way to get the system the citadel is in though (it's worth noting that most market citadels include the system name
Dragonaire
Here there be Dragons
#10 - 2016-07-08 00:22:57 UTC
Just got to say all eve IDs have been using 64bit numbers for several years now since they stopped wanting to reset WalletJournal/Transactions IDs every couple months way back when. Every since then they have been drawing all the IDs from a single generator so not really sure how you all missed that change and never noticed before. That change was made even before CREST existed they just expanded where they are using it since CREST has been around.

salacious necrosis - There's an API in XML API that should map any ID into a name plus one to go the other way as well. Sorry not where I can look up names right now but think it's something like eve/charactorID.xml.aspx. I know name seems to suggest only work for chars but has worked for everything for a long time.

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.
salacious necrosis
Garoun Investment Bank
Gallente Federation
#11 - 2016-07-08 01:52:34 UTC
Dragonaire wrote:

salacious necrosis - There's an API in XML API that should map any ID into a name plus one to go the other way as well. Sorry not where I can look up names right now but think it's something like eve/charactorID.xml.aspx. I know name seems to suggest only work for chars but has worked for everything for a long time.


I did NOT know that! Thanks, I'll give it a try!

Use EveKit ! - Tools for EVE Online 3rd party development
Ydnari
Estrale Frontiers
#12 - 2016-07-08 22:00:03 UTC
Still doesn't get us very far without a way of getting at least a solarsystem for the citadel IDs.

--
Dulock Sental
Center for Advanced Studies
Gallente Federation
#13 - 2016-07-08 22:23:02 UTC
Well since most market citadels include the name of the system they're in in their names (at least the citadels I've seen do) you could get the list of systems in a region (their names and IDs), and the list of citadels with active orders (get bulk orders from region and use one of the methods posted above to get the names) and check for any cases where the name of a system is part of the name of a citadel.

This probably wouldn't get the complete list (any citadels with no market orders or citadels that don't include the system name in their own name would be excluded) and there might be some false positives but it's a start.
Ethan02
Succurit Research and Development
#14 - 2016-07-09 06:04:16 UTC
Done that and turned out quite a list of Citadels. Most don't have the system in their name. And as twitter pointed out, the data is only for ranged buy orders. Thus sell orders and buy order with range "Station" aren't supplied. Cry

Currently only in DEV, thus site may throw errors from time to time: http://dev.adam4eve.eu/stats_citadels.php

@CCP: Obviously it is easy to include the Citadel order information. As a first step please expose all buy and sell orders. Giving us an endpoint with name, location, owner, tax rates etc. would then be a later bonus... Bear

BTW Dragonaire, checked the XML API out you referenced, however it is limited to 32bit as the CCP error message notes.
Messenger Of Truth
Butlerian Crusade
#15 - 2016-08-15 00:16:52 UTC  |  Edited by: Messenger Of Truth
OK these citadel IDs have been annoying me. I've scraped all citadels that get exposed in CREST via market orders and made an API for anybody that wants this data.

some info here: https://stop.hammerti.me.uk/api
and here: https://stop.hammerti.me.uk/citadelhunt

Basically, once you have a mysterious citadel ID go to e.g. https://stop.hammerti.me.uk/api/citadel/1021696460797 and if there have been some buy orders listed at the citadel (or someone taking part in the citadel hunt has bookmarked it), then you will receive a JSON response stating the name of the citadel, e.g.:

{"1021696460797": {"typeId": 35833, "systemId": 30003522, "name": "Sarum Prime - Star Emporium", "typeName": "Fortizar", "lastSeen": "2016-09-09T04:21:02Z", "systemName": "Sarum Prime", "regionId": 10000043, "firstSeen": "2016-08-14T23:33:52Z", "regionName": "Domain"}}


You can also go to https://stop.hammerti.me.uk/api/citadel/all to get a list of all citadels that have been exposed. Since I launched the citadel hunt crowd-sourcing initiative to fill in citadels not exposed by market orders, the list has become quite large so I recommend checking that "Accept-Encoding: gzip" is enabled if you use the API in any of your tools.

Trade Hub Price Checker: stop.hammerti.me.uk/pricecheck

Visit "Haulers Channel" in game for all matters courier-related.

Structure name/system API: stop.hammerti.me.uk/api