These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

 
  • Topic is locked indefinitely.
Previous page123Next page
 

CREST changes coming to Sisi

First post
Author
Previous Topic Next Topic
Squizz Caphinator
The Wormhole Police
#21 - 2016-01-04 21:38:39 UTC
Max Kolonko wrote:
Btw, what kind of rate limit Are You guys planning for it? Can I for example have 100 persons loggged in into my app and check their locations every 10 seconds? Or 300? Or 1000?


Just fyi, zKillboard hits the CREST api on a consistent 50-100 per second basis and has no issues with rate limits. I could probably increase how much I hit CREST and still be ok.

Various projects I enjoy putting my free time into:

https://zkillboard.com | https://evewho.com
narthollis
Perkone
Caldari State
#22 - 2016-01-04 22:50:11 UTC  |  Edited by: narthollis
Risingson wrote:
CCP FoxFour wrote:
It is more likely to go up but I am hoping after some optimizations made today we can actually go lower. Just need to see what this does to load on TQ. TQ behaves so differently than anything else. :(


when i thought about those upcoming changes i thought of why not have the eve client tell stuff via a localhost?
that would mean instant info and no TQ load at all?
i am a dummy at such things so i don't know about the technical sides of that brainstorm though.



Talking to localhost from a website is a pita, particularly if your website uses SSL/TLS (and there is no reason these days why it shouldn't)

Also, FoxFour: Is there any chance we could get the current ship the user is in as part of the current location endpoint?
Max Kolonko
Caldari Provisions
Caldari State
#23 - 2016-01-04 23:53:30 UTC
Squizz Caphinator wrote:
Max Kolonko wrote:
Btw, what kind of rate limit Are You guys planning for it? Can I for example have 100 persons loggged in into my app and check their locations every 10 seconds? Or 300? Or 1000?


Just fyi, zKillboard hits the CREST api on a consistent 50-100 per second basis and has no issues with rate limits. I could probably increase how much I hit CREST and still be ok.


I'm well aware. But its public. For private You have to make separate connection for each request (diffrent player / diffrent token), unless I'm missing something obvious

Read and support: Don't mess with OUR WH's What is Your stance on WH stuff?
narthollis
Perkone
Caldari State
#24 - 2016-01-05 00:05:11 UTC
Max Kolonko wrote:
Squizz Caphinator wrote:
Max Kolonko wrote:
Btw, what kind of rate limit Are You guys planning for it? Can I for example have 100 persons loggged in into my app and check their locations every 10 seconds? Or 300? Or 1000?


Just fyi, zKillboard hits the CREST api on a consistent 50-100 per second basis and has no issues with rate limits. I could probably increase how much I hit CREST and still be ok.


I'm well aware. But its public. For private You have to make separate connection for each request (diffrent player / diffrent token), unless I'm missing something obvious


You should be able to utilise the same connection(s), different character/token is just a different header, which you should be able to set per-request.
Kelath Erebus
Federal Navy Academy
Gallente Federation
#25 - 2016-01-05 03:59:58 UTC
narthollis wrote:
Also, FoxFour: Is there any chance we could get the current ship the user is in as part of the current location endpoint?


What narthollis said. Getting the current ship with the location endpoint would be really useful.
Pete Butcher
The Scope
Gallente Federation
#26 - 2016-01-05 05:22:30 UTC
Any chance for bulk market order import? It would be better for both the client and the server to make just one request to fetch the market for given region/types instead of thousands. And we know how fast that currently works Sad

http://evernus.com - the ultimate multiplatform EVE trade tool + nullsec Alliance Market tool
CCP FoxFour
C C P
C C P Alliance
#27 - 2016-01-05 08:09:05 UTC
Pete Butcher wrote:
Any chance for bulk market order import? It would be better for both the client and the server to make just one request to fetch the market for given region/types instead of thousands. And we know how fast that currently works Sad


I will bump that higher on my priority list to look into. Not for this release as this release comes out next week and I am already getting stared at for putting these changes in so late. :)

@CCP_FoxFour // Technical Designer // Team Tech Co

Third-party developer? Check out the official developers site for dev blogs, resources, and more.
Jarno Midumulf
Riders of Sleipnir
Backdoor Crashers
#28 - 2016-01-05 08:29:54 UTC
what is this madness?! is it Christmas again?

CCP FoxFour wrote:

It will return an empty dictionary of the character is offline.


is it possible te return the last known location when a player is offline?
this is already in the game so why not add it to crest? (corp -> members -> location)
CCP FoxFour
C C P
C C P Alliance
#29 - 2016-01-05 08:39:29 UTC  |  Edited by: CCP FoxFour
Jarno Midumulf wrote:
what is this madness?! is it Christmas again?

CCP FoxFour wrote:

It will return an empty dictionary of the character is offline.


is it possible te return the last known location when a player is offline?
this is already in the game so why not add it to crest? (corp -> members -> location)


Possible yes, but at this time I am trying to keep this resource as light as possible. Right now the request goes something like this:

Internet (and whatever load balancers and such network ops has) -> NGINX -> CREST proxy (at this point we verify the character in the authorization header matches the requested character, the token should have been looked up and cached, we then use the character ID to determine which sol node to send the request to, at this point we are doing all in memory lookups) -> sol node running the character manager for the requested character (at this point we look to see if a session belonging to that character exists on this node, if it doesn't we return None, if it does we pull the solar system from the session) -> CREST proxy (format the response) -> back to the NGINX (cache the response here) -> internet

So at this point in time we are getting away with completely in memory information, a few simple dictionary lookups, and that is about it. If this is the first request for a given authorization token we need to make a network hop to the SSO to make sure it is valid and then we cache that response for a few minutes.

Getting the characters location at time of logoff would require a DB dip. :( Considering how much we expect this to get hit I am hoping to avoid that at this time. You can get that information from the XML API on a slower cache time.

When we expose more resources in CREST we can look at getting that information moved over in a more sane manner. Just not this resource at this time. :)

@CCP_FoxFour // Technical Designer // Team Tech Co

Third-party developer? Check out the official developers site for dev blogs, resources, and more.
Jarno Midumulf
Riders of Sleipnir
Backdoor Crashers
#30 - 2016-01-05 09:22:01 UTC
CCP FoxFour wrote:
Jarno Midumulf wrote:
what is this madness?! is it Christmas again?

CCP FoxFour wrote:

It will return an empty dictionary of the character is offline.


is it possible te return the last known location when a player is offline?
this is already in the game so why not add it to crest? (corp -> members -> location)


Possible yes, but at this time I am trying to keep this resource as light as possible. Right now the request goes something like this:

Internet (and whatever load balancers and such network ops has) -> NGINX -> CREST proxy (at this point we verify the character in the authorization header matches the requested character, the token should have been looked up and cached, we then use the character ID to determine which sol node to send the request to, at this point we are doing all in memory lookups) -> sol node running the character manager for the requested character (at this point we look to see if a session belonging to that character exists on this node, if it doesn't we return None, if it does we pull the solar system from the session) -> CREST proxy (format the response) -> back to the NGINX (cache the response here) -> internet

So at this point in time we are getting away with completely in memory information, a few simple dictionary lookups, and that is about it. If this is the first request for a given authorization token we need to make a network hop to the SSO to make sure it is valid and then we cache that response for a few minutes.

Getting the characters location at time of logoff would require a DB dip. :( Considering how much we expect this to get hit I am hoping to avoid that at this time. You can get that information from the XML API on a slower cache time.

When we expose more resources in CREST we can look at getting that information moved over in a more sane manner. Just not this resource at this time. :)


well.. if its in the xml api then you don't need to do a db pull right?
or you could do a db pull and then set a longer cache time? (and clear that cache when a character enters the game)

but this basically means we have to do a xml pull if we get an empty result back, thx for the answer!
Max Kolonko
Caldari Provisions
Caldari State
#31 - 2016-01-05 09:31:20 UTC
Jarno Midumulf wrote:
what is this madness?! is it Christmas again?

CCP FoxFour wrote:

It will return an empty dictionary of the character is offline.


is it possible te return the last known location when a player is offline?
this is already in the game so why not add it to crest? (corp -> members -> location)


As long as it would retain the "logged off" information i dont mind.

Read and support: Don't mess with OUR WH's What is Your stance on WH stuff?
Kyra Lee
Doomheim
#32 - 2016-01-09 07:53:25 UTC
I have a probably dumb question regarding the Character location thing that you all are talking about. I am a complete noob when it comes to crest, api, and all of that other stuff :(

My question is this; can someone other than I, that I have not given access to, look up my location? I am worried about this being used like a locator agent but out of game and with more speed and accuracy.

I assume this is meant to be used on programs like EveMon and other things that the user authorizes themselves, but I wanted to make sure. Thanks!
Louis Vitton
Viziam
Amarr Empire
#33 - 2016-01-09 11:06:42 UTC
If you use any programs that you use crest it will ask for permission from you for that application to view the scope. In order for it to ask for this permission / scope also you need to login via the sso.
You can also manage all the applications that have access to your data in crest.
Jack Hayson
The Scope
Gallente Federation
#34 - 2016-01-09 21:04:04 UTC
Just found a (probably caching related) bug in the location ressource:

Jumping back into the system you just came from causes CREST to not update your location for several minutes.

Reproduction:
-login character in System A (Vlillirier) pull location, CREST returns System A within 10 seconds.
-jump into system B (Alsavoinon), pull location, CREST gives me System B within 10 seconds.
-now I jump BACK into System A but CREST continues to give me SYSTEM B for >5 minutes.

If I then jump to System C (Oicx) CREST updates to System A while jumping and then to System C.



(not sure under which category I should bug report this on Sisi)
CCP FoxFour
C C P
C C P Alliance
#35 - 2016-01-10 00:25:21 UTC
Jack Hayson wrote:
Just found a (probably caching related) bug in the location ressource:

Jumping back into the system you just came from causes CREST to not update your location for several minutes.

Reproduction:
-login character in System A (Vlillirier) pull location, CREST returns System A within 10 seconds.
-jump into system B (Alsavoinon), pull location, CREST gives me System B within 10 seconds.
-now I jump BACK into System A but CREST continues to give me SYSTEM B for >5 minutes.

If I then jump to System C (Oicx) CREST updates to System A while jumping and then to System C.



(not sure under which category I should bug report this on Sisi)


Any chance you can check the headers? We should return the cache status of the requests. I wont have time to fix this before it ships but hopefully can track it down and fix it as soon as possible.

Huge thank you for testing this and responding here with this information. This is a huge help!

@CCP_FoxFour // Technical Designer // Team Tech Co

Third-party developer? Check out the official developers site for dev blogs, resources, and more.
Jack Hayson
The Scope
Gallente Federation
#36 - 2016-01-10 10:39:20 UTC  |  Edited by: Jack Hayson
CCP FoxFour wrote:
Any chance you can check the headers? We should return the cache status of the requests.


After jumping from 6-CZ49 to Reblier and back to 6CZ49:
HTTP/1.1 200 OK Server: nginx/1.2.0 Content-Type: application/vnd.ccp.eve.CharacterLocation-v1+json; charset=utf-8 Connection: keep-alive Vary: Accept-Encoding Date: Sun, 10 Jan 2016 10:28:58 GMT Cache-Control: private, max-age=10 Access-Control: allow <*> Access-Control-Max-Age: 1728000 Access-Control-Expose-Header: CONTENT-TYPE Access-Control-Allow-Headers: AUTHORIZATION, X-REQUESTED-WITH, CONTENT-TYPE, ACCEPT Access-Control-Allow-Origin: * Content-Length: 143 Access-Control-Allow-Methods: GET, PUT, POST, DELETE, HEAD, OPTIONS Content-Language: en-US Vary: Accept-Language X-Cache-Status: EXPIRED 
{"solarSystem": {"id_str": "30005328", "href": "https://api-sisi.testeveonline.com/solarsystems/30005328/", "id": 30005328, "name": "Reblier"}}


The X-Cache-Status continues to change from HIT to BYPASS to EXPIRED and then again to HIT but the returned system doesn't change to 6-CZ49 as it should.


Some of my characters on Sisi are stuck... maybe it has something to do with that?

EDIT: now it seems to be stuck at BYPASS
Dusty Meg
Echelon Research
Goonswarm Federation
#37 - 2016-01-11 17:40:24 UTC
Would it be possible in a later update to get something like the fleet name if the character is in a fleet?
We currently use the igb to track shares during my corps mining ops and atm we cannot see a easy way to track this when the igb is removed.
The location will definitely help with us moving tocrest and plans have been done for a version using crest but we would need to have more data exposed through crest to reliable be able to track through it.

Creater of the EVE animated influence map http://www.youtube.com/user/DustMityEVE
CCP FoxFour
C C P
C C P Alliance
#38 - 2016-01-11 19:10:31 UTC
Dusty Meg wrote:
Would it be possible in a later update to get something like the fleet name if the character is in a fleet?
We currently use the igb to track shares during my corps mining ops and atm we cannot see a easy way to track this when the igb is removed.
The location will definitely help with us moving tocrest and plans have been done for a version using crest but we would need to have more data exposed through crest to reliable be able to track through it.


Is fleet information included in the IGB headers?

@CCP_FoxFour // Technical Designer // Team Tech Co

Third-party developer? Check out the official developers site for dev blogs, resources, and more.
CCP FoxFour
C C P
C C P Alliance
#39 - 2016-01-11 19:11:40 UTC
I am hoping to get a CREST resource out later that lets fleet bosses basically see the fleet composition window via CREST.

@CCP_FoxFour // Technical Designer // Team Tech Co

Third-party developer? Check out the official developers site for dev blogs, resources, and more.
Max Kolonko
Caldari Provisions
Caldari State
#40 - 2016-01-11 21:15:18 UTC
As for location endpoint, are You planning to fill it with remaining information current headers provide? Like shipname / ship type / ship id for example?

HTTP_EVE_SHIPID - This header will contain the users ship id
HTTP_EVE_SHIPNAME - This header will contain the users ship name
HTTP_EVE_SHIPTYPEID - This header will contain the users ship type id
HTTP_EVE_SHIPTYPENAME - This header will contain the users ship type nam
HTTP_EVE_STATIONID - This header will contain the unique identifier number assigned to the station at which the user’s character is presently located. If the character is in space, or otherwise not in a station, this header will not be present. *Note* This header does not appear to be used.
HTTP_EVE_STATIONNAME - This header will contain the name of the station in which the end user’s character is presently located. If the character is in space, or otherwise not at a station, this header will be set to "None".

the rest of headers is not important or obtainable differently with info already provided (like corp, system name, constelation name, alliance, etc)

Read and support: Don't mess with OUR WH's What is Your stance on WH stuff?
Previous page123Next page