These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

 
  • Topic is locked indefinitely.
 

API unresponsive after ~1500 requests.

First post
Author
Previous Topic Next Topic
Driveways
The Scope
Gallente Federation
#1 - 2014-12-31 06:12:59 UTC
I'm pulling market data over authed CREST in a python and after making about 1500 requests the API seems to become unresponsive. The issue appears to be unrelated to any individual resource, and resources which become unresponsive can be successfully pulled later by themselves.

The errors I receive are similar to the one below. This example is from the requests library in Python27, but I've seen the same result using urlfetch in the AppEngine dev environment.
An unexpected error was encountered: HTTPSConnectionPool(host='crest-tq.eveonline.com', port=443): Max retries exceeded with url: /market/10000002/orders/buy/?type=https://crest-tq.eveonline.com/types/15210/ (Caused by class 'socket.gaierror': [Errno -2] Name or service not known)


Not ruling out any logic or networking issues on my end yet, but I'm curious if I've missed something obvious.
Steve Ronuken
Fuzzwork Enterprises
Vote Steve Ronuken for CSM
#2 - 2014-12-31 10:10:22 UTC
Are you using the same access key for all your requests? (or at least a very limited number of them?)

Woo! CSM XI!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter
Pete Butcher
The Scope
Gallente Federation
#3 - 2014-12-31 11:04:44 UTC
I can confirm you can make > 20k requests for orders and they do come back fine.

http://evernus.com - the ultimate multiplatform EVE trade tool + nullsec Alliance Market tool
Driveways
The Scope
Gallente Federation
#4 - 2014-12-31 16:04:46 UTC
I request a fresh access token off of a stored refresh token at the start of every batched run, but all of those requests are using the same access token.
Messir Astaroth
Chemosh Ascendancy
#5 - 2014-12-31 16:08:33 UTC  |  Edited by: Messir Astaroth
Confirmed

i have the same s***t with all my 3rd party programs, that rely on api.

EVE Mon, pyfa and EFT are not working. They just dont connect to the API database, which pisses me off.

Can the CCP dudes do anything about that?

ADD: Evernus is working well. But EFT, PYFA, and EVE Mon still unable to connect to eve.api for some reason.What? which is frustrating
Driveways
The Scope
Gallente Federation
#6 - 2014-12-31 17:59:01 UTC
Thanks for the help, I think I've figured it out.

I didn't configure the HTTP library I was using to correctly keep-alive and reuse any TCP connections, so I was quickly building up idle connections which caused the program to lock up on my end. Got that fixed and everything's working beautifully.
Messir Astaroth
Chemosh Ascendancy
#7 - 2014-12-31 18:05:22 UTC
Driveways wrote:
Thanks for the help, I think I've figured it out.

I didn't configure the HTTP library I was using to correctly keep-alive and reuse any TCP connections, so I was quickly building up idle connections which caused the program to lock up on my end. Got that fixed and everything's working beautifully.


Can you please tell me in detail step by step what you have done.

Im afraid im still having that issue :(
Driveways
The Scope
Gallente Federation
#8 - 2014-12-31 18:17:30 UTC
It sounds like you're using programs created by someone else, you might have to contact the creator for help.

What I fixed was a bug with logic for a new program, so it's unlikely to apply to any other applications.

I'd guess this is a networking problem on your end though, so I'd start by checking any firewalls installed on your computer or router and seeing if they might be blocking inbound/outbound connections from any of the applications you're using.
Messir Astaroth
Chemosh Ascendancy
#9 - 2014-12-31 20:02:43 UTC
Driveways wrote:

1.It sounds like you're using programs created by someone else, you might have to contact the creator for help.

2.What I fixed was a bug with logic for a new program, so it's unlikely to apply to any other applications.

3.I'd guess this is a networking problem on your end though, so I'd start by checking any firewalls installed on your computer or router and seeing if they might be blocking inbound/outbound connections from any of the applications you're using.


1. Nope. My software is original.
2. ... erm... ok, I`ll pretend that i understood that...
3. ... not even gonna pretend :(
Driveways
The Scope
Gallente Federation
#10 - 2014-12-31 20:11:04 UTC
I thought you were saying you had problems with EVEMon and other 3rd party applications.

What project are you working on?