These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

 
  • Topic is locked indefinitely.
 

Devsite blog: Nom nom tasty market data!

First post First post
Author
Previous Topic Next Topic
Steve Ronuken
Fuzzwork Enterprises
Vote Steve Ronuken for CSM
#101 - 2014-11-27 23:31:39 UTC
Max Kolonko wrote:
Steve Ronuken wrote:
Callback?


The url shouldn't need to be changed at all. you just need to fill in the app id, secret and a refresh token.

If you use postman to do the initial authentication, you should have a refresh token available to you (no idea how it'll make it available to you, I'm afraid)

This code doesn't do the initial auth for you, it's for use after that, once you've authed, and have the token.


ahhh, got it... postman shows refresh token as blank :( so i'll wait for someone with more expierience figure it out :) I can only assume that You plan on expanding it to full auth support including login process?



For the Crest handler? Nope.

I have auth code ( https://github.com/fuzzysteve/eve-sso-auth ) which is easily expandable to work with a scope (you just need to fill it in, rather than leaving it blank in the place it sends you to the login server)

The callback for that is also easy to then store the refresh token in your session. 

$auth_token=$response->access_token;

becomes

$auth_token=$response->access_token;
$refresh_token=$response->refresh_token;


and you add it to the bit that deals with $_SESSION.

I used that to just output to screen, and then copied and pasted it.

Leaving the crest handler to /just/ doing crest, and being given the tokens (it'll take an access token, and an expiry time as a unix style timestamp as optional parameters) means it's more flexible, for use in backend products. It's not well suited for interactive ones, due to the lack of caching of the item and region data. That bit will change in future, but that's not in this release. It's just a bit slow for that.

Woo! CSM XI!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter
Kaladr
Viziam
Amarr Empire
#102 - 2014-11-28 04:16:40 UTC
I've started a library and scanner for navigating the CREST APIs. There are a bunch of missing features (walking pages, pooled parallel fetching, types beyond Region), but it's capable of acquiring and caching tokens and has a few nice wrappers.

APIs will change to clean them up as the right abstraction is found.

https://github.com/theatrus/crestmarket

Creator of EVE-Central.com, the longest running EVE Market Aggregator
CCP FoxFour
C C P
C C P Alliance
#103 - 2014-11-28 08:20:24 UTC  |  Edited by: CCP FoxFour
Kaladr wrote:
I've started a library and scanner for navigating the CREST APIs. There are a bunch of missing features (walking pages, pooled parallel fetching, types beyond Region), but it's capable of acquiring and caching tokens and has a few nice wrappers.

APIs will change to clean them up as the right abstraction is found.

https://github.com/theatrus/crestmarket


Your application is going to break on Monday when the route for types changes from /inventory/types/ to just /types/ (don't get me started on why I have to revert that change). I highly recommend NOT hard cording URI's. We do, and will continue, to change routes. Follow links.

Also, if I am not mistaken, it appears you are always sending the same accept header "application/vnd.ccp.eve.MarketTypeCollection-v1+json" which for anything other than the market type collection will result in just the newest version.

@CCP_FoxFour // Technical Designer // Team Tech Co

Third-party developer? Check out the official developers site for dev blogs, resources, and more.
Kaladr
Viziam
Amarr Empire
#104 - 2014-11-28 08:54:26 UTC
CCP FoxFour wrote:

Your application is going to break on Monday when the route for types changes from /inventory/types/ to just /types/ (don't get me started on why I have to revert that change). I highly recommend NOT hard cording URI's. We do, and will continue, to change routes. Follow links.


Already fixed :-)

Quote:

Also, if I am not mistaken, it appears you are always sending the same accept header "application/vnd.ccp.eve.MarketTypeCollection-v1+json" which for anything other than the market type collection will result in just the newest version.


I was going to bring up this as a question - how are the variants of the API versioned based on the Accept header?

Creator of EVE-Central.com, the longest running EVE Market Aggregator
CCP FoxFour
C C P
C C P Alliance
#105 - 2014-11-28 09:06:00 UTC
Kaladr wrote:
CCP FoxFour wrote:

Your application is going to break on Monday when the route for types changes from /inventory/types/ to just /types/ (don't get me started on why I have to revert that change). I highly recommend NOT hard cording URI's. We do, and will continue, to change routes. Follow links.


Already fixed :-)


:) Yay!

Kaladr wrote:
CCP FoxFour wrote:

Also, if I am not mistaken, it appears you are always sending the same accept header "application/vnd.ccp.eve.MarketTypeCollection-v1+json" which for anything other than the market type collection will result in just the newest version.


I was going to bring up this as a question - how are the variants of the API versioned based on the Accept header?


Not sure I understand the question. How do we handle it on our end or how do you handle it?

@CCP_FoxFour // Technical Designer // Team Tech Co

Third-party developer? Check out the official developers site for dev blogs, resources, and more.
Yongtau Naskingar
Yongtau Naskingar Corporation
#106 - 2014-11-28 10:51:25 UTC
CCP FoxFour wrote:
when the route for types changes from /inventory/types/ to just /types/ (don't get me started on why I have to revert that change)

Now I kinda want to know Shocked
CCP FoxFour
C C P
C C P Alliance
#107 - 2014-11-28 13:24:30 UTC
Kali Izia wrote:
POSthing Market


Done

@CCP_FoxFour // Technical Designer // Team Tech Co

Third-party developer? Check out the official developers site for dev blogs, resources, and more.
Grimdahl
Aliastra
Gallente Federation
#108 - 2014-11-28 14:26:09 UTC  |  Edited by: Grimdahl
please add access for:

Evemeep
403cd668eb674ebe9991621ce74dbabd
Kaladr
Viziam
Amarr Empire
#109 - 2014-11-28 18:51:26 UTC
CCP FoxFour wrote:

Not sure I understand the question. How do we handle it on our end or how do you handle it?


More specifically, what are/are there accept headers to use for stable result versioning for the other resources yet (I.e. Regions, inventory types)?

If you're returning them in a header already I probably missed it.

Creator of EVE-Central.com, the longest running EVE Market Aggregator
Steve Ronuken
Fuzzwork Enterprises
Vote Steve Ronuken for CSM
#110 - 2014-11-28 20:00:08 UTC
Kaladr wrote:
CCP FoxFour wrote:

Not sure I understand the question. How do we handle it on our end or how do you handle it?


More specifically, what are/are there accept headers to use for stable result versioning for the other resources yet (I.e. Regions, inventory types)?

If you're returning them in a header already I probably missed it.



They're in the content-type header of the returned resource. Specify it in an Accept header to continue to get the same one, until it's deprecated (check for the deprecation header for that step. x-deprecated, I believe) then removed.

Woo! CSM XI!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter
CCP FoxFour
C C P
C C P Alliance
#111 - 2014-11-28 20:21:07 UTC
Steve Ronuken wrote:
Kaladr wrote:
CCP FoxFour wrote:

Not sure I understand the question. How do we handle it on our end or how do you handle it?


More specifically, what are/are there accept headers to use for stable result versioning for the other resources yet (I.e. Regions, inventory types)?

If you're returning them in a header already I probably missed it.



They're in the content-type header of the returned resource. Specify it in an Accept header to continue to get the same one, until it's deprecated (check for the deprecation header for that step. x-deprecated, I believe) then removed.


Yup. An example of X-Deprecated is application/vnd.ccp.eve.Api-v2+json for the root API.

You can also do an OPTIONS call on a resource to get a description of all versions and their mediatype.

@CCP_FoxFour // Technical Designer // Team Tech Co

Third-party developer? Check out the official developers site for dev blogs, resources, and more.
Kaladr
Viziam
Amarr Empire
#112 - 2014-11-28 22:33:36 UTC  |  Edited by: Kaladr
Perfect!

(Its obvious I don't do web APIs much Shocked)

Creator of EVE-Central.com, the longest running EVE Market Aggregator
Nuke Cherenkov
The Scope
Gallente Federation
#113 - 2014-11-29 23:24:44 UTC
Very basic question, I've gone to https://developers.testeveonline.com/applications/create to register an account so I can start trying to figure things out. I'm targeting a standalone Python app which uses Excel as a GUI (using the ExcelPython library). I don't know what to put in for the Callback URL. I've read the whole topic and it's been mentioned that a standalone app should be possible but I'm stuck at the beginning...

Anybody else done a standalone app with Python? I'm planning on starting with the Requests and Requests-OAuthlib libraries. I'm working ok with public CREST, getting market price and industry info.

Thanks
Steve Ronuken
Fuzzwork Enterprises
Vote Steve Ronuken for CSM
#114 - 2014-11-29 23:54:07 UTC  |  Edited by: Steve Ronuken
Nuke Cherenkov wrote:
Very basic question, I've gone to https://developers.testeveonline.com/applications/create to register an account so I can start trying to figure things out. I'm targeting a standalone Python app which uses Excel as a GUI (using the ExcelPython library). I don't know what to put in for the Callback URL. I've read the whole topic and it's been mentioned that a standalone app should be possible but I'm stuck at the beginning...

Anybody else done a standalone app with Python? I'm planning on starting with the Requests and Requests-OAuthlib libraries. I'm working ok with public CREST, getting market price and industry info.

Thanks



What you /should/ be doing is:

Send the user to their regular browser to authenticate

Have a callback url like eve://localhost/callback.py, where you've registered your application to handle the eve:// protocol (something other than eve is a good idea. Something unique) so when it gets called, it sends the request to your application from the browser.

http://msdn.microsoft.com/en-us/library/aa767914.aspx is of interest.


(an alternate option is to embed a webserver in the application, and use that as the location for your callback. so http://localhost:43252/callback/ or something)

Ideally, your application should never see the username and password.


So:
someone starts your application, without ever having authed. It sends them to log in and closes.
They log in, and get sent to a custom url.
This url scheme is configured to launch your application, and pass in the details provided to it.
you parse these out, and complete the authentication (ideally storing a refresh token, so they don't need to do it again)



Alternatively, have a helper program registered to the url scheme, which hands the access and refresh tokens to your main program Smile

Woo! CSM XI!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter
Kaladr
Viziam
Amarr Empire
#115 - 2014-11-30 00:22:06 UTC  |  Edited by: Kaladr
Along with the above two there is the third option of an embedded web view with the in-application URL handler to pick up the redirect.

(Disclaimer: this is significantly less secure for the user since your application would have login credentials effectively typed into it. It is common practice on mobile applications however)

Creator of EVE-Central.com, the longest running EVE Market Aggregator
Steve Ronuken
Fuzzwork Enterprises
Vote Steve Ronuken for CSM
#116 - 2014-11-30 00:30:15 UTC
Can I get 7e5d5a8e8e5f45d78e43f302cceeb43d (Fuzzwork Native) authorized for the public data scope, thanks.

just want to see about working up an example of how to do it at least semi properly on windows Smile

Woo! CSM XI!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter
Nuke Cherenkov
The Scope
Gallente Federation
#117 - 2014-11-30 01:06:48 UTC
Kaladr wrote:
Along with the above two there is the third option of an embedded web view with the in-application URL handler to pick up the redirect.

(Disclaimer: this is significantly less secure for the user since your application would have login credentials effectively typed into it. It is common practice on mobile applications however)


To keep the learning curve reasonable (not done any significant web dev), I'm focused more on a single user app (= probably me and some close friends) so this sounds like the best approach. I'm trying to write the Python in a modular enough fashion that it would be useful in a web based app at which point I will follow Steve's advice.

This also seems to be helpful to understand the process: https://dev.twitter.com/web/sign-in/implementing ?

Thanks to you both!
Nuke Cherenkov
The Scope
Gallente Federation
#118 - 2014-11-30 01:19:59 UTC
Please add access for:

Python standalone app
e337559af9f145f08d83be165b3263b7
Kaladr
Viziam
Amarr Empire
#119 - 2014-11-30 02:08:59 UTC
Nuke Cherenkov wrote:
Kaladr wrote:
Along with the above two there is the third option of an embedded web view with the in-application URL handler to pick up the redirect.

(Disclaimer: this is significantly less secure for the user since your application would have login credentials effectively typed into it. It is common practice on mobile applications however)


To keep the learning curve reasonable (not done any significant web dev), I'm focused more on a single user app (= probably me and some close friends) so this sounds like the best approach. I'm trying to write the Python in a modular enough fashion that it would be useful in a web based app at which point I will follow Steve's advice.

This also seems to be helpful to understand the process: https://dev.twitter.com/web/sign-in/implementing ?

Thanks to you both!


The process is similar, except that Twitter uses OAuth1, whereas CREST is OAuth2. The concepts of redirecting are similar, just the token steps are different (i.e., OAuth requires signing the requests, whereas OAuth2 simply assumes transport security is perfect and forwards your "secret").

Google uses OAuth2 for its services, and this guide may be more relevant:

https://developers.google.com/api-client-library/python/guide/aaa_oauth

Creator of EVE-Central.com, the longest running EVE Market Aggregator
Steve Ronuken
Fuzzwork Enterprises
Vote Steve Ronuken for CSM
#120 - 2014-11-30 04:09:38 UTC  |  Edited by: Steve Ronuken
One option which remains open for people is:

Have a client which sends you to a particular website (forming a session with it)

Have that website redirect people to the auth.
Have the site do all the talking with CCP's server, to get the access token (retaining the refresh token)
Send the access token back to the client.

That way, the client doesn't every see the refresh token, or the other detailed needed to create one. It can just ask the website for a new access token.

The client is also capable of doing authentication by (different) usernames and password, for an added level of security, just storing the refresh tokens against those.

Woo! CSM XI!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter