These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

 
  • Topic is locked indefinitely.
 

Testing a major revision to CREST

First post First post
Author
Previous Topic Next Topic
Bruce Warhead
Incursion Citadel Network
#61 - 2016-05-11 17:20:40 UTC
I noticed that if you send an invite to a character and the character is in a fleet already, you just get the back the 201 created.
Could that be changed to actually give an other error code back with a message just like invite an offline character does?
Would be awesome!
Rectar en Meunk
Center for Advanced Studies
Gallente Federation
#62 - 2016-05-16 10:18:05 UTC
Is there a way to get the Leadership skills to automatically assign squad leaders?
DoToo Foo
Garoun Investment Bank
Gallente Federation
#63 - 2016-05-17 15:15:42 UTC
CCP FoxFour wrote:
Current test setup looks like this:

{
"totalCount_str": "20000",
"items": [
{
"buy": false,
"issued": "2016-04-29T14:16:02",
"price": 1200000000.01,
"volumeEntered": 3,
"stationID": 60000970,
"volume": 3,
"range": "region",
"minVolume": 1,
"duration": 90,
"type": 1944,
"id": 4051697671
},

20,000 entries per page. The largest region is The Forge and as of last night it had just shy of 287,000 orders. So even on that region it would only be 15 requests to get all of it. Smaller regions would of course be less. Hopefully can get this out next release.

I also stripped out all the links and such to make this response as small as possible.


Any update on this? I would love it.

http://foo-eve.blogspot.com.au/
CCP FoxFour
C C P
C C P Alliance
#64 - 2016-05-17 15:57:11 UTC
DoToo Foo wrote:
CCP FoxFour wrote:
Current test setup looks like this:

{
"totalCount_str": "20000",
"items": [
{
"buy": false,
"issued": "2016-04-29T14:16:02",
"price": 1200000000.01,
"volumeEntered": 3,
"stationID": 60000970,
"volume": 3,
"range": "region",
"minVolume": 1,
"duration": 90,
"type": 1944,
"id": 4051697671
},

20,000 entries per page. The largest region is The Forge and as of last night it had just shy of 287,000 orders. So even on that region it would only be 15 requests to get all of it. Smaller regions would of course be less. Hopefully can get this out next release.

I also stripped out all the links and such to make this response as small as possible.


Any update on this? I would love it.


The next big release which I think is May 31st.

@CCP_FoxFour // Technical Designer // Team Tech Co

Third-party developer? Check out the official developers site for dev blogs, resources, and more.
Pete Butcher
The Scope
Gallente Federation
#65 - 2016-05-19 07:39:04 UTC
CCP FoxFour wrote:
DoToo Foo wrote:
CCP FoxFour wrote:
Current test setup looks like this:

{
"totalCount_str": "20000",
"items": [
{
"buy": false,
"issued": "2016-04-29T14:16:02",
"price": 1200000000.01,
"volumeEntered": 3,
"stationID": 60000970,
"volume": 3,
"range": "region",
"minVolume": 1,
"duration": 90,
"type": 1944,
"id": 4051697671
},

20,000 entries per page. The largest region is The Forge and as of last night it had just shy of 287,000 orders. So even on that region it would only be 15 requests to get all of it. Smaller regions would of course be less. Hopefully can get this out next release.

I also stripped out all the links and such to make this response as small as possible.


Any update on this? I would love it.


The next big release which I think is May 31st.


Small question: are you sure pagination is necessary? I can't think of any use case when someone would fetch only single pages instead of everything; after all, it's not like market journal where you would like to browse a list page by page. Having a single request fetch everything would make things simpler and probably speed everything up.

http://evernus.com - the ultimate multiplatform EVE trade tool + nullsec Alliance Market tool
CCP FoxFour
C C P
C C P Alliance
#66 - 2016-05-19 09:00:19 UTC
Pete Butcher wrote:
CCP FoxFour wrote:
DoToo Foo wrote:
CCP FoxFour wrote:
Current test setup looks like this:

{
"totalCount_str": "20000",
"items": [
{
"buy": false,
"issued": "2016-04-29T14:16:02",
"price": 1200000000.01,
"volumeEntered": 3,
"stationID": 60000970,
"volume": 3,
"range": "region",
"minVolume": 1,
"duration": 90,
"type": 1944,
"id": 4051697671
},

20,000 entries per page. The largest region is The Forge and as of last night it had just shy of 287,000 orders. So even on that region it would only be 15 requests to get all of it. Smaller regions would of course be less. Hopefully can get this out next release.

I also stripped out all the links and such to make this response as small as possible.


Any update on this? I would love it.


The next big release which I think is May 31st.


Small question: are you sure pagination is necessary? I can't think of any use case when someone would fetch only single pages instead of everything; after all, it's not like market journal where you would like to browse a list page by page. Having a single request fetch everything would make things simpler and probably speed everything up.


Purely the size of the response and the time needed to generate the response. Actually changing it to 10,000 orders per page. At this time that is 29 pages for The Forge. When it was at 30,000 orders per page it would take upwards of 8 seconds on my local to generate the response (CREST is slow... I know). In that time we could end up having to process a large number of requests as many people request it at the same time and all get cache misses. Also it was 8 megabytes in size. Whelp.

@CCP_FoxFour // Technical Designer // Team Tech Co

Third-party developer? Check out the official developers site for dev blogs, resources, and more.
CCP FoxFour
C C P
C C P Alliance
#67 - 2016-05-19 09:01:07 UTC
Putting this new market orders resource out to Sisi today to test. If it goes well I am going to squeeze it into TQ on Tuesday the 24th.

@CCP_FoxFour // Technical Designer // Team Tech Co

Third-party developer? Check out the official developers site for dev blogs, resources, and more.
Pete Butcher
The Scope
Gallente Federation
#68 - 2016-05-19 09:15:48 UTC
CCP FoxFour wrote:
Purely the size of the response and the time needed to generate the response. Actually changing it to 10,000 orders per page. At this time that is 29 pages for The Forge. When it was at 30,000 orders per page it would take upwards of 8 seconds on my local to generate the response (CREST is slow... I know). In that time we could end up having to process a large number of requests as many people request it at the same time and all get cache misses. Also it was 8 megabytes in size. Whelp.


Ah, too bad. Although you could queue clients waiting for data on cache misses, so that only one will do the processing and storing in cache, and then everyone in the queue will be sent that one response (which will at this point be cached for all further requests). That way each client will wait at most the time needed to generate the request, and that would be the worst case scenario (of course I'm assuming using some kind of Proactor pattern for IO, so no request will stall). Splitting the response into pages actually doesn't solve the problem of cache misses, but merely splits the big response into smaller ones. That in turn can lead to situations where multiple clients will wait the same time for all the responses, in case of cache miss for every page. Which in turn makes the total time waiting for all clients to be at least the time to generate the data without cache.

http://evernus.com - the ultimate multiplatform EVE trade tool + nullsec Alliance Market tool
CCP FoxFour
C C P
C C P Alliance
#69 - 2016-05-19 10:08:55 UTC
Pete Butcher wrote:
CCP FoxFour wrote:
Purely the size of the response and the time needed to generate the response. Actually changing it to 10,000 orders per page. At this time that is 29 pages for The Forge. When it was at 30,000 orders per page it would take upwards of 8 seconds on my local to generate the response (CREST is slow... I know). In that time we could end up having to process a large number of requests as many people request it at the same time and all get cache misses. Also it was 8 megabytes in size. Whelp.


Ah, too bad. Although you could queue clients waiting for data on cache misses, so that only one will do the processing and storing in cache, and then everyone in the queue will be sent that one response (which will at this point be cached for all further requests). That way each client will wait at most the time needed to generate the request, and that would be the worst case scenario (of course I'm assuming using some kind of Proactor pattern for IO, so no request will stall). Splitting the response into pages actually doesn't solve the problem of cache misses, but merely splits the big response into smaller ones. That in turn can lead to situations where multiple clients will wait the same time for all the responses, in case of cache miss for every page. Which in turn makes the total time waiting for all clients to be at least the time to generate the data without cache.


I would love to queue people however we don't have that ability. :(

@CCP_FoxFour // Technical Designer // Team Tech Co

Third-party developer? Check out the official developers site for dev blogs, resources, and more.
DoToo Foo
Garoun Investment Bank
Gallente Federation
#70 - 2016-05-19 11:51:51 UTC
CCP FoxFour wrote:
Putting this new market orders resource out to Sisi today to test. If it goes well I am going to squeeze it into TQ on Tuesday the 24th.


For singularity, do you have an endpoint I can hit?

http://foo-eve.blogspot.com.au/
CCP FoxFour
C C P
C C P Alliance
#71 - 2016-05-19 21:42:00 UTC
DoToo Foo wrote:
CCP FoxFour wrote:
Putting this new market orders resource out to Sisi today to test. If it goes well I am going to squeeze it into TQ on Tuesday the 24th.


For singularity, do you have an endpoint I can hit?


https://api-sisi.testeveonline.com/market/10000001/orders/all/

@CCP_FoxFour // Technical Designer // Team Tech Co

Third-party developer? Check out the official developers site for dev blogs, resources, and more.
Pete Butcher
The Scope
Gallente Federation
#72 - 2016-05-20 07:07:11 UTC
CCP FoxFour wrote:
DoToo Foo wrote:
CCP FoxFour wrote:
Putting this new market orders resource out to Sisi today to test. If it goes well I am going to squeeze it into TQ on Tuesday the 24th.


For singularity, do you have an endpoint I can hit?


https://api-sisi.testeveonline.com/market/10000001/orders/all/


What will be the endpoint name in region resource?

http://evernus.com - the ultimate multiplatform EVE trade tool + nullsec Alliance Market tool
Pete Butcher
The Scope
Gallente Federation
#73 - 2016-05-20 07:26:05 UTC
Also, the last page has the "next" href point to the first. it would be nice to stop there, so we can just jump be those hrefs.

http://evernus.com - the ultimate multiplatform EVE trade tool + nullsec Alliance Market tool
CCP FoxFour
C C P
C C P Alliance
#74 - 2016-05-20 08:53:49 UTC
Pete Butcher wrote:
Also, the last page has the "next" href point to the first. it would be nice to stop there, so we can just jump be those hrefs.


Yea that was a bug. Have fixed. It actually just always linked to page 2, no matter if there was another page or what page you requested. Derp.

@CCP_FoxFour // Technical Designer // Team Tech Co

Third-party developer? Check out the official developers site for dev blogs, resources, and more.
Pete Butcher
The Scope
Gallente Federation
#75 - 2016-05-20 09:17:41 UTC
Could you add solar system id to order object?

http://evernus.com - the ultimate multiplatform EVE trade tool + nullsec Alliance Market tool
CCP FoxFour
C C P
C C P Alliance
#76 - 2016-05-20 16:19:52 UTC
Pete Butcher wrote:
Could you add solar system id to order object?


Unfortunately I don't have that information for the order unless I make another request as orders are just stored on the per station level. You can request the station and that will have the solar system information and then cache that.

@CCP_FoxFour // Technical Designer // Team Tech Co

Third-party developer? Check out the official developers site for dev blogs, resources, and more.
Ideki
Wolf Brothers INC
United Neopian Federation
#77 - 2016-05-20 18:03:11 UTC
I did not see an endpoint related to PI.
Is that excluded from it ?

Creator of The EVE Planetary Planner
Ncc 1709
Fusion Enterprises Ltd
Pandemic Horde
#78 - 2016-05-21 09:48:21 UTC
is there an issue with TQ market pulls at present?

my pull is detecting data there, but error'ing before any of it makes it to the sheet.
Crashys
State War Academy
Caldari State
#79 - 2016-07-13 10:45:39 UTC
I'm trying to understand the fleet endpoints...

1. How can i 'list' from the CREST the active fleets of the CREST char?
(tried https://crest-tq.eveonline.com/fleets/characterID/ with a denied response) :p

Is there a truly list of known endpoints/calls/etc about this crest stuff? All i see is an outdated 3rd party docs, broken links on the 'developer' website of CCP, etc... It seems that people who know the stuff keep it for them self... :/
Steve Ronuken
Fuzzwork Enterprises
Vote Steve Ronuken for CSM
#80 - 2016-07-13 11:43:01 UTC
Crashys wrote:
I'm trying to understand the fleet endpoints...

1. How can i 'list' from the CREST the active fleets of the CREST char?
(tried https://crest-tq.eveonline.com/fleets/characterID/ with a denied response) :p

Is there a truly list of known endpoints/calls/etc about this crest stuff? All i see is an outdated 3rd party docs, broken links on the 'developer' website of CCP, etc... It seems that people who know the stuff keep it for them self... :/



Due to how the fleet stuff works, you can't list the fleet you're a member of. It's not actually stored on your character. You have to hand in a link manually, from the client.

https://github.com/fuzzysteve/fleetTracker

is a _very_ basic fleet tracker.


As for the 3rd party docs, the readthedocs site isn't outdated. it's just not complete. (i.e, what is on it is accurate, it just doesn't have everything.)

If you want to chat with people who've been working with this stuff for a while, I'd suggest the #devfleet channel, on the tweetfleet slack.

Woo! CSM XI!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter