EVE Technology Lab

A little warning about this might have been nice. I got some pings on Jabber about API pulls not happening, checked the worker process and it's upset about 403s. Well, it's actually upset about getting a bunch of 500s in between 403s but that's about par for the course. The errors I see with any frequency are:

* Herp derp the API server has decided this perfectly valid request is invalid, have a 24 hour cache timer. This is always honoured, even though I can run the exact same call a few minutes later and it's valid again.
* 500 Internal server error or the server exploding, 5/10/20/40s delays are added in between requests.
* Trying to be nice about using /char/Locations I bulk all updates into a single request. This often fails even if it's called literally seconds after retrieving /char/Assets because who the hell knows.
* Occasional (like a few per day at most) permanent errors when keys expire/are deleted/etc. The key is disabled immediately and no new jobs are triggered for it.

All cachedUntil timers are always honoured, I'm not really sure what else I'm meant to do to be a 'good API citizen'. I am not scraping the API and I try to handle the ridiculous error situation gracefully :(


Edit: oh btw, /corp/OutpostList flat out doesn't work for any key ever.

exactly how much warning time do you need?
this was introduced more than 1 1/2 years ago:

http://community.eveonline.com/devblog.asp?a=blog&bid=924

and if someone is not even reacting on the code that sais "dude your doing it wrong, and your temp blocked for it, change or it will go perma" (yes i paraphrased it), then actually he deserves the permanent ban =)

Ooops

The whole "helping the enablers" doesn't seem to be the highest priority, seeing as high profile sites such as EVEWho, EVE-KILL and most likely Battleclinic got banned, ditto for alliances such as Goonswarm and it seems TEST too.

The main problem isn't that the API is getting hammered, it's that it's coded in a way, that doesn't return data that has been cached.

Best example of this is the killlog.
If EVE-KILL accesses a killlog belonging to a key, noone (not even eve-kill) can access that killlog for that character, before the cachedUntil value is over. And if anyone does it, they get an error 119 thrown at them. Which incidentally also counts as an error.

Now if you make several thousand of those a day (Which EVE-KILL does) you're apparently screwed now.
And the best part? you have no idea if you'll make a 119 or not.

On another note, if CCP were to allow us to pay say 100$ a month for privileged access to the API server, or let us buy a server and place at them, we'd do it.

:v

Then can you please start being more consistent with your errors? I basically have to treat 211 errors as advisory at this point and just back off a little, because this error seems to be thrown at random, regardless of the actual state of the account. I've had entire swaths of vCodes return 210 errors, and I stop and invalidate the key and stop using it, but if you use the exact same vCode later, it works just fine, resulting in logic like "less than X 210 errors per run and it's probably safe to invalidate them, greater than X 210 errors per run and it's :ccp: and just abort the run and wait an hour."

I don't want to end up getting throttled because of exceptions being tossed that I'm not actually responsible for generating. I try and do everything else according to the accepted rules. I scrub vCodes (both when I collect them and during polling), I honour CachedUntil, I delay between requests, making sure I'm not crushing the API server, etc. Maybe CCP should consider setting up feedback loops with developers of well-used community tools (ala EVEWho and EVE-Kill) or those who use the API to manage large blocs (IE, TEST, Goonswarm, PL, etc) so that IPs registered to these entities won't just be blindly nuked, but somebody can send out an email asking WTF and sorting out what may be the underlying issue (CCP backend, utility backend, etc) rather than suddenly tools suddenly ceasing to function.

Anyways, just my $0.02. I'm now going to go off and implement a 904 kill switch so that if at any time, any API polling system within the GSF encounters a 904 it'll raise a flag and immediately terminate all current and future polling until the cachedUntil timer expires.


Edit: WTF? My entire post vanished and replaced by a quote I mostly deleted? Excellent. Forums are hard to develop.

Don't blame the forums for your own stupidity.
Pages have a timeout, SSL pages an even shorter timeout.
If you take to long to type your post and post it, your browser needs to reacquire the SSL connection losing your POST data in the process.

Type it in notepad (or whatever) next time, and copy/pasta.



I'm interested in the FW part aswell. My forummod uses that to verify if an user is part of FW, always throwing a certain error.
Would be weird if this could lead to a ban, as it's a clear design flaw on CCPs own side.

No worries, it did the same to mine, three times. Thank god chrome allows you to go back and get the text you just tried to post.

And i wholly agree with you, the fact we get random errors thrown at us, for something that shouldn't even produce an error, coupled with the caching issues, is just ridiculous.

Doesn't take 5 minutes to hack together some code that stores a cached result from the backend, and serves it till a timer runs out.

They said over a year and half ago they will do this.

Can you post an example of this? I don't think I've ever run into this and given the number of Russian corps in our alliance, that's rather surprising.