These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

 
  • Topic is locked indefinitely.
Previous page123Next page
 

IMPORTANT NOTICE: We have started banning IPs again!

First post First post
Author
NUXI7
Sebiestor Tribe
Minmatar Republic
#21 - 2013-02-02 17:23:28 UTC  |  Edited by: NUXI7
Solo Drakban wrote:
Wollari wrote:
Russian characters that make XML invalid


Can you post an example of this? I don't think I've ever run into this and given the number of Russian corps in our alliance, that's rather surprising.



I'm not sure its actually a Russian character (it doesn't render as one for me) but at one point in the description for XCAPITALSX there was a character which did not get XML escaped by CCP's API server before being transmitted.

I can't remember anything more, I'm still searching my IRC logs. I know I brought it up in #eve-dev and the guy who made the API Jackknife had also seen it.

Edit: found it and confirmed its still there.
The xCPAITALSx (corp ID 320586188) still contains the character. The character is not Russian, it is actually just a plain non-printable ASCII character that is not allowed in XML. The character is 0x0C aka \f aka form feed. The API server escapes it to the entity reference & # x C ; (spaced out because the eve-o forums own't let me type it otherwise ) but even in the escaped form this is not a valid character for an XML body. I've simply patched ntt's eveapi python module to nuke this character from any downloads before it hits the actual XML parser, but it still shouldn't be there.

You can load the corp's API for yourself right here and like the python library I use firefox rightly complains about the XML being bad.
Quote:

XML Parsing Error: reference to invalid character number
Location: https://api.eveonline.com/corp/CorporationSheet.xml.aspx?corporationID=320586188
Line Number 12, Column 2097:



See http://www.w3.org/TR/REC-xml/#charsets for reference.
Quote:

[2] Char ::= #x9 | #xA | #xD | [#x20-#xD7FF] | [#xE000-#xFFFD] | [#x10000-#x10FFFF]




Its not even hard to get the character into your description. Here is a corporation in TEST that I have director roles in. Whenever the API refreshes the corp information it should contain the character. LOLOL Not anyone it tanked our killboard and our IT guys made me take it out.
Desmont McCallock
#22 - 2013-02-02 20:05:35 UTC
This is not the first time I've seen this. And all this is happening because CCP refuses to use CData[] for user input text.
Karbowiak
Sacred Templars
Fraternity.
#23 - 2013-02-02 22:59:53 UTC
Desmont McCallock wrote:
This is not the first time I've seen this. And all this is happening because CCP refuses to use CData[] for user input text.


If that was the only problem with the API Lol
Desmont McCallock
#24 - 2013-02-03 07:33:54 UTC
Indeed, Karbowiak.
I was tempted to compile a list of all known API issues and sent it towards CCP illurkall (last known CCP's API guy), but then again... I don't get payed to do CCP's API debugging.
Marbin Drakon
Reboot Required
#25 - 2013-02-03 20:35:22 UTC
NUXI7 wrote:
Stuff


As far as I have seen, they are the only corp in an alliance in Eve that has that character. My git commit log has some unkind remarks about them.

Eve W-Space: Open source wormhole mapping and corporation management: https://forums.eveonline.com/default.aspx?g=posts&t=210073

Squizz Caphinator
The Wormhole Police
#26 - 2013-02-03 20:41:23 UTC
A document going into detail to explain the problem with KillLogs:

https://docs.google.com/document/d/16YfJwjhuH5A3cS4NTMDFDkprnOVKsvgtuRIKk8xjTM8/preview?pli=1&sle=true

Various projects I enjoy putting my free time into:

https://zkillboard.com | https://evewho.com

Caius Sivaris
Dark Nexxus
#27 - 2013-02-03 21:55:32 UTC
Writing that as a CEO, not a direct API user.

Is there a supported way for me to have my corp kill log fed to both BC and eve-kill without getting one alt with director rights to create the API for the second killboard?

If Squizz, is right, and I've little doubt he is, and if I understand him correctly, that's not possible.

Is that acceptable, I don't think so, kill feed is the poster child for something a memcached deployment would improve drastically and those limitation don't shed a good light on CCP Engineering.
Silk daShocka
Greasy Hair Club
#28 - 2013-02-04 08:33:06 UTC
I've read teh document that was linked to on the Eve-Kill site.

EveMon has an option in settings that appears to access the "Combat Log" every hour (I believe that's the default setting). Could this be causing EveKill to be getting errors as well? I have turned this option off entirely, since I don't use it myself.

If this setting in Evemon is causing issues for the killboards perhaps the EveMon developers could be contacted to have this feature removed since I don't think anyone really uses it.

Anyways, I really hope this can be sorted out so that Eve-Kill can operate as it has to this day.
CCP Prism X
C C P
C C P Alliance
#29 - 2013-02-04 08:40:14 UTC
Squizzy is, sadly, right. And so are most other complaints here.

There are numerous eccentricities with the return XMLs of the API. The lack of typeIDs returned with ownerIDs forcing extra calls, that may or may not error depending on the call and owner type and a complete lack of KillMail caching are probably the two most problematic issues at the moment.

There are also numerous idiocies in the backend. There is no caching on erronous input so that every erronous request incurrs the full overhead of raising an exception, over and over again. Just the fact that the application keeps raising errors is a very bad decision that needs to be rethrough.

And this is not acceptable for a live service that provides our customers with useful tools we do not seem to find the resources to develop. Sadly we're now in the situation, again, where we cannot find resources to maintain the API either. There is a short meeting later today where we'll go over the situation and I will be pushing for this situation to be amended rather than swept under the rug. Hopefully something will come out of that.

And we are terribly sorry for the excessive disruption over the past few days. Sometimes you just need to rock the boat a fair bit more than usual to draw attention to the leak. Blink
Karbowiak
Sacred Templars
Fraternity.
#30 - 2013-02-04 09:21:57 UTC
CCP Prism X wrote:
Squizzy is, sadly, right. And so are most other complaints here.

There are numerous eccentricities with the return XMLs of the API. The lack of typeIDs returned with ownerIDs forcing extra calls, that may or may not error depending on the call and owner type and a complete lack of KillMail caching are probably the two most problematic issues at the moment.

There are also numerous idiocies in the backend. There is no caching on erronous input so that every erronous request incurrs the full overhead of raising an exception, over and over again. Just the fact that the application keeps raising errors is a very bad decision that needs to be rethrough.

And this is not acceptable for a live service that provides our customers with useful tools we do not seem to find the resources to develop. Sadly we're now in the situation, again, where we cannot find resources to maintain the API either. There is a short meeting later today where we'll go over the situation and I will be pushing for this situation to be amended rather than swept under the rug. Hopefully something will come out of that.

And we are terribly sorry for the excessive disruption over the past few days. Sometimes you just need to rock the boat a fair bit more than usual to draw attention to the leak. Blink


Here's to building an entirely new boat, instead of plugging the holes! Cool

Anyways, hope you guys can figure something out, and hope you can keep us informed of the result Big smile
Marcel Devereux
Aideron Robotics
Aideron Robotics.
#31 - 2013-02-04 11:38:36 UTC
Karbowiak wrote:
CCP Prism X wrote:
Squizzy is, sadly, right. And so are most other complaints here.

There are numerous eccentricities with the return XMLs of the API. The lack of typeIDs returned with ownerIDs forcing extra calls, that may or may not error depending on the call and owner type and a complete lack of KillMail caching are probably the two most problematic issues at the moment.

There are also numerous idiocies in the backend. There is no caching on erronous input so that every erronous request incurrs the full overhead of raising an exception, over and over again. Just the fact that the application keeps raising errors is a very bad decision that needs to be rethrough.

And this is not acceptable for a live service that provides our customers with useful tools we do not seem to find the resources to develop. Sadly we're now in the situation, again, where we cannot find resources to maintain the API either. There is a short meeting later today where we'll go over the situation and I will be pushing for this situation to be amended rather than swept under the rug. Hopefully something will come out of that.

And we are terribly sorry for the excessive disruption over the past few days. Sometimes you just need to rock the boat a fair bit more than usual to draw attention to the leak. Blink


Here's to building an entirely new boat, instead of plugging the holes! Cool

Anyways, hope you guys can figure something out, and hope you can keep us informed of the result Big smile


A new boat? I think you meant spaceship ;-)
Desmont McCallock
#32 - 2013-02-04 18:08:52 UTC  |  Edited by: Desmont McCallock
Silk daShocka wrote:
I've read teh document that was linked to on the Eve-Kill site.

EveMon has an option in settings that appears to access the "Combat Log" every hour (I believe that's the default setting). Could this be causing EveKill to be getting errors as well? I have turned this option off entirely, since I don't use it myself.

If this setting in Evemon is causing issues for the killboards perhaps the EveMon developers could be contacted to have this feature removed since I don't think anyone really uses it.

Anyways, I really hope this can be sorted out so that Eve-Kill can operate as it has to this day.

If I, by adding the "Combat Log" monitor in EVEMon, am the cause of all this, then I should be praised and not be blamed, for exposing the faults of the 'accursed' API call.
And I really don't understand your logic of 'arm hurts, cut arm', on blaming EVEMon and asking for the feature to be removed, when those issues are common knowledge to all killboard site devs and are happening for years.
To tell you the truth I avoided adding that feature for years now, knowing the side effects it may cause. I finally decided to add it, under CCP's concept that we are the 'Enablers' and that CCP should adjust to our needs and not we to CCP's.


It's about time CCP gives some love to the API.

Edit: Currently, I have 'high esteem ' only for CCP PrismX among the CCP staff. (Although I admit that CCP Puncturis is my second favorite).
Squizz Caphinator
The Wormhole Police
#33 - 2013-02-04 18:42:42 UTC
Desmont,

I noticed when you added the combat log to EveMon and I remember thinking "oh yay, more 119's!", but I don't fault you for it. You're only adding a feature that your users would like to see :)

Silk daShocka,

EveMon very well could be causing some of the errors, and that would explain why I've noticed a pattern in 119's on an hourly basis: Higher during EU and US prime timezones, whereas Chinese/Russian prime timezones tend to have fewer 119's. I can only speculate that more people (e.g. English speakers) have their machines turned on with EveMon running which then forces competition for the KillLog request during those hours where 119 errors are peaking.

The biggest competition, however, isn't with EveMon. It is between zKB/Eve-Kill and Battleclinic. Users who have provided their keys to each killboard merely wants to see their kills in each website. Can't really blame the user for this as they expect the API to "Just Work (TM)."

Various projects I enjoy putting my free time into:

https://zkillboard.com | https://evewho.com

Lorren Canada
High is the Way - But all eyes are Upon the Ground
#34 - 2013-02-05 11:11:48 UTC
Fucking nerds
Squizz Caphinator
The Wormhole Police
#35 - 2013-02-05 14:13:32 UTC
Lorren Canada wrote:
Fucking nerds


damn right!

Various projects I enjoy putting my free time into:

https://zkillboard.com | https://evewho.com

Barakkus
#36 - 2013-02-06 03:34:58 UTC
I'll fix it. :)

http://youtu.be/yytbDZrw1jc

Latino lover
Gebeleizis Legion
#37 - 2013-02-06 06:22:38 UTC
ok , I'm a noob , I don't understand why are they banned... I didn't change my api or done anything with it since you introduced api custimizsation ... and now aftere 2 years i'm trying to introduce it on a killboard and a I get this
"Your IP address has been temporarily blocked because it is causing too many errors. See the cacheUntil timestamp for when it will be opened again. IPs that continually cause a lot of errors in the API will be permanently banned, please take measures to minimize problematic API calls from your application."
Tonto Auri
Vhero' Multipurpose Corp
#38 - 2013-02-10 12:29:24 UTC
CCP Prism X wrote:
Squizzy is, sadly, right. And so are most other complaints here.

There are numerous eccentricities with the return XMLs of the API. The lack of typeIDs returned with ownerIDs forcing extra calls, that may or may not error depending on the call and owner type and a complete lack of KillMail caching are probably the two most problematic issues at the moment.

There are also numerous idiocies in the backend. There is no caching on erronous input so that every erronous request incurrs the full overhead of raising an exception, over and over again. Just the fact that the application keeps raising errors is a very bad decision that needs to be rethrough.

And this is not acceptable for a live service that provides our customers with useful tools we do not seem to find the resources to develop. Sadly we're now in the situation, again, where we cannot find resources to maintain the API either. There is a short meeting later today where we'll go over the situation and I will be pushing for this situation to be amended rather than swept under the rug. Hopefully something will come out of that.

And we are terribly sorry for the excessive disruption over the past few days. Sometimes you just need to rock the boat a fair bit more than usual to draw attention to the leak. Blink

Told you, just set up nginx in front of your API servers. You won't need to worry about caching any longer. At least the continuous feeds, that are not time-critical, like kill logs, can be safely front-ended by it, or other caching server.

Two most common elements in the universe are hydrogen and stupidity. -- Harlan Ellison

Aineko Macx
#39 - 2013-03-02 10:56:34 UTC
cachedUntil is a misnomer, an euphemism at best, until CCP actually start caching (and returning cached) results.
Galmas
United System's Commonwealth
#40 - 2013-03-02 17:23:40 UTC
Caius Sivaris wrote:
Writing that as a CEO, not a direct API user.

Is there a supported way for me to have my corp kill log fed to both BC and eve-kill without getting one alt with director rights to create the API for the second killboard?

If Squizz, is right, and I've little doubt he is, and if I understand him correctly, that's not possible.

Is that acceptable, I don't think so, kill feed is the poster child for something a memcached deployment would improve drastically and those limitation don't shed a good light on CCP Engineering.



Sorry for spamming this in here but i havnt found any really 100% proper resource that explains how to overcome the "two killboards one corp/alliance" problem.

Does that alt with killlog rights need to be on a different account or doest it work when it is just a different character's api key of the same game account?

Thanks, Regards
Gal
Previous page123Next page