EVE Technology Lab

Very nice! I've been trying to find a way to hook python up to Authed CREST; this will be useful.

One question (warning: from a clueless newbie): if I'm writing a python app rather than a web app, what should I put in for the redirect_uri? As I understand the Oauth flow, the authorization server will be returning the access token back through the redirect UI; does that mean I need to be running a web server to use the interface? I'm wanting to just write a simple python program to read and evaluate market data.

Cool; thank you. I'll play around and see if I can get that to work.

I'm getting an error in requests trying to use PyCrest:

requests.exceptions.SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:581)

My code:

import pycrest

eve = pycrest.EVE()
eve()

This doesn't have anything to do with PyCrest but hoping there is some request/networking expertise out there.. This code has the same problem as the above (created in a fresh virtualenv that only has requests installed in it):

import requests

params = None
res = requests.get('https://public-crest.eveonline.com/', headers={'Accept': 'application/json', 'User-Agent': 'PyCrest - v 0.0.1 testing'}, params=params if params else {})

I'm using Python 2.7.9 (32 bit) on Win 7 64. The code is running in a virtualenv with requests 2.5.1 (latest).

I'm also using the Wing Pro IDE and I set breakpoints down into ssl.py where the error is raised but I don't see anything that indicates a problem to me (but then I'm more of a hardware guy). I found a post on how to force requests to use a specific SSL version, https://lukasa.co.uk/2013/01/Choosing_SSL_Version_In_Requests/ , selected PROTOCOL_TLSv1, but get the same error. I checked with https://www.digicert.com/help/ and it shows that https://public-crest.eveonline.com/ uses TLS 1.0, SSL 3.0 (and notes that 3.0 has a bad vulnerability (POODLE Bug attack)). Note that I also ran the code outside the IDE and got the same error.

Looking on Stackoverlfow I found:



This page: https://sslanalyzer.comodoca.com/?url=https%3A%2F%2Fpublic-crest.eveonline.com%2F says the crest server uses TLS_RSA_WITH_RC4_128_MD5. One of the main reasons for Python 2.7.9 was to update SSL:

Is this a server and latest version of Python issue or?

Edit 1/26
Found another PC with Python 2.7.8, installed requests 2.5.1 (1.2.3 worked too) and the second code example runs fine. Looks like CREST and the latest versions of Python won't play nicely with each other?

Yes, this is a problem because the public CREST serveronly supports TLS_RSA_WITH_RC4_128_MD5, and ciphers using MD5 were explicitly disabled in the latest Python releases. To get around this, a custom transport adapter should be used with the requests library, so this is something that should either be handled by Dreae or by CCP changing their server configuration to allow other ciphers (CCPlease?). I've submitted a pull request with that and a few other fixes/changes, so there might be a fix soon-ish.

I've merged Paxswill's pull request, and I've updated the package on pip so it should work now.




I was not, I'll update the library to respect the Cache-Control header when I get the chance

Having trouble connecting to auth CREST from a desktop app. I'm pretty sure I can negotiate the authorization process, auth_conn.whoami() looks like it returns good data but region().marketSellOrders(type=item) fails with a 404 (pycrest version 0.0.3).

I've put the main code below, the server code is at https://gist.github.com/EricE/fab125c74082ea337faa (my first code at GitHub ), I think it is working ok.

Tracing down into eve.py, the problem is here (line 49):

res = self._session.get(resource, params=prms)

resource is u'https://public-crest.eveonline.com/market/10000067/orders/sell/'
prms is {'type': u'https://public-crest.eveonline.com/types/34/'}

It's trying to use public CREST for an auth CREST resource? I can't figure out how to tell pycrest to use https://crest-tq.eveonline.com/ instead. I tried setting a breakpoint and manually changing resource to crest-tq but then I get a 401, I assume it's because I wasn't sending the auth info (I'm in over my head at this point, very limited web development background). I'd appreciate some pointers, feel free to point out if I could be doing this a better way...

import multiprocessing
import urlparse
import uuid
import webbrowser

import pycrest
import server

#----------------------------------------------------------------------
def main():
    # Launch the server in a separate process, pass back the code and state in a queue
    queue = multiprocessing.Queue()

    eve_auth = pycrest.EVE(client_id="client id goes here",
                           api_key="api key goes here",
                           redirect_uri="http://localhost:12345/callback/")

    state_str = uuid.uuid4().get_hex()
    webbrowser.open(eve_auth.auth_uri(state=state_str))

    p = multiprocessing.Process(target=server.start_server, args=(queue,))
    p.start()

    code_raw = queue.get()

    queue.close()
    queue.join_thread()

    p.join()

    query_str = dict(urlparse.parse_qsl(code_raw))

    eve_auth()

    auth_conn = eve_auth.authorize(query_str['code'])

    region = getByAttrVal(eve_auth.regions().items, 'name', 'Catch')
    item = getByAttrVal(getAllItems(eve_auth.itemTypes), 'name', 'Tritanium').href
    
    print getAllItems(region().marketSellOrders(type=item))
        
# Below lifted from http://pycrest.readthedocs.org/en/latest/
def getByAttrVal(objlist, attr, val):
    ''' Searches list of dicts for a dict with dict[attr] == val '''
    matches = [getattr(obj, attr) == val for obj in objlist]
    index = matches.index(True)  # find first match, raise ValueError if not found
    return objlist[index]

def getAllItems(page):
    ''' Fetch data from all pages '''
    ret = page().items
    while hasattr(page(), 'next'):
        page = page().next()
        ret.extend(page().items)
    return ret

if __name__ == '__main__':
    main()

Thanks for the quick reply! Made the change:

region = getByAttrVal(auth_conn().regions().items, 'name', 'Catch')
item = getByAttrVal(getAllItems(eve_auth.itemTypes), 'name', 'Tritanium').href
print getAllItems(region().marketSellOrders(type=item))

However, getting a 401 from this line:

print getAllItems(region().marketSellOrders(type=item))

I set a conditional breakpoint in APIConnection :

def get(self, resource, params=None):
    logger.debug('Getting resource %s', resource)
    if params is None:  # <-- breakpoint here, condition is 'orders/sell/' in resource
        params = {}

resource == u'https://crest-tq.eveonline.com/market/10000014/orders/sell/'
params == {'type': u'https://public-crest.eveonline.com/types/34/'}

Going into the cache check have:

prms == {'type': u'https://public-crest.eveonline.com/types/34/'}

I noticed the necessary info seems to be present in key after the cache check:

# check cache
key = (resource, frozenset(self._session.headers.items()), frozenset(prms.items()))
cached = self.cache.get(key)  # cached == None

key is set to:

(u'https://crest-tq.eveonline.com/market/10000014/orders/sell/', frozenset([('Accept', 'application/json'), ('Connection', 'keep-alive'), ('Accept-Encoding', 'gzip, deflate'), ('Authorization', u'Bearer 9t long string A2'), ('User-Agent', 'PyCrest/0.0.1')]), frozenset([('type', u'https://public-crest.eveonline.com/types/34/')]))

The actual call to requests doesn't have any auth info (or user agent)?:

res = self._session.get(resource, params=prms)
if res.status_code != 200:  # <-- status_code == 401
    raise APIException("Got unexpected status code from server: %i" % res.status_code)

Success! Thanks for hanging with me, apologize about missing adding scopes, I'd even copied and pasted the line from your docs in a comment above my line... Also needed to update the application for public data.

No but thats my fault and something I hope to fix soon.

Depends on your definition of "convenient". If you already have a list of valid regionIDs and typeIDs, you can just "cheat" by building your own URLs.

So, for example, to get yesterday's Tritanium from The Forge, you could just use:

>>> import pycrest
>>> eve = pycrest.EVE()
>>> history = eve.get('https://public-crest.eveonline.com/market/10000002/types/34/history/')
>>> print history['items'][-1]
{u'volume': 16016039000, u'orderCount': 2760, u'lowPrice': 5.92, u'highPrice': 6.05,
u'avgPrice': 5.99, u'volume_str': u'16016039000', u'orderCount_str': u'2760',
u'date': u'2015-03-01T00:00:00'}

I know, it's not the recommended way of using CREST, but it works. (unless FoxFour changes URLs without notice ;-) )