These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

 
  • Topic is locked indefinitely.
Previous page12
 

Authed CREST rate limits

First post
Author
Previous Topic Next Topic
Pete Butcher
The Scope
Gallente Federation
#21 - 2015-04-08 11:34:55 UTC
Is 503 possible to be returned in some other circumstances, eg. downtime?

http://evernus.com - the ultimate multiplatform EVE trade tool + nullsec Alliance Market tool
CCP FoxFour
C C P
C C P Alliance
#22 - 2015-04-08 12:05:12 UTC
Pete Butcher wrote:
Is 503 possible to be returned in some other circumstances, eg. downtime?


I believe downtime gives a 502 or 503 depending on the state of the proxy. Completely offline should be a 502 and starting up should be 503.

@CCP_FoxFour // Technical Designer // Team Tech Co

Third-party developer? Check out the official developers site for dev blogs, resources, and more.
Aineko Macx
#23 - 2015-04-09 08:01:33 UTC
CCP FoxFour wrote:
As an FYI, and this will be explained in an upcoming dev blog, the authed rate limit will be lowered to about 100 req/s, possibly less. Public CREST will be bumped to 150 req/s.

Are you moving the market orders endpoint over to public CREST? Otherwise having authed CREST more restricted makes no sense at all.

iveeCore 3.0: The PHP engine for industrial activities and CREST library
Ortho Loess
Escalated.
OnlyFleets.
#24 - 2015-04-09 21:06:08 UTC
That's the plan.

Turned out that authed crest routes all your requests through the same server, so it can't handle having a higher rate.

It's on public crest on sisi now, along with the 150/s rate limit.
Pete Butcher
The Scope
Gallente Federation
#25 - 2015-04-10 05:41:08 UTC
Ortho Loess wrote:
That's the plan.

Turned out that authed crest routes all your requests through the same server, so it can't handle having a higher rate.

It's on public crest on sisi now, along with the 150/s rate limit.


Hmm, that suggests authed CREST has some state stored server side. A bit strange approach since REST is supposed to be stateless to avoid exactly these issues.

http://evernus.com - the ultimate multiplatform EVE trade tool + nullsec Alliance Market tool
CCP FoxFour
C C P
C C P Alliance
#26 - 2015-04-10 09:11:05 UTC
Pete Butcher wrote:
Ortho Loess wrote:
That's the plan.

Turned out that authed crest routes all your requests through the same server, so it can't handle having a higher rate.

It's on public crest on sisi now, along with the 150/s rate limit.


Hmm, that suggests authed CREST has some state stored server side. A bit strange approach since REST is supposed to be stateless to avoid exactly these issues.


Very true, however that just the way it has to be due to the EVE server. /welp

@CCP_FoxFour // Technical Designer // Team Tech Co

Third-party developer? Check out the official developers site for dev blogs, resources, and more.
Pete Butcher
The Scope
Gallente Federation
#27 - 2015-04-28 15:05:17 UTC
Any news when this switch to public CREST will take place?

http://evernus.com - the ultimate multiplatform EVE trade tool + nullsec Alliance Market tool
CCP FoxFour
C C P
C C P Alliance
#28 - 2015-04-28 16:00:02 UTC
Pete Butcher wrote:
Any news when this switch to public CREST will take place?


There will be a dev blog about it.

@CCP_FoxFour // Technical Designer // Team Tech Co

Third-party developer? Check out the official developers site for dev blogs, resources, and more.
Gordon Newman
Deep Core Mining Inc.
Caldari State
#29 - 2015-06-21 14:55:27 UTC
I'm sorry for bringing this up again - is there any progress? I'm running into the same issues while building a new backend for EMDR (CREST-based instead of cache scraping) and I can't get stable connectivity at over 100 requests/second. I always get those EVE Proxy errors for ~50% of my requests which is unsustainable. After analysing Element43's database I figured that there are around 380k markets (region/type combinations) with orders in the market. If we were to fetch those every 5 minutes (x2 because ask and bid are two different endpoints) and fetch markets without any orders at a reduced rate like every 60 minutes, we'd still end up with way over 3k requests per second. Again - over three thousand OAuth'ed SSL-Encrypted requests per second. With some clever software engineering this might be achievable with a single server, however at the moment CREST is barely able to deliver a 30th of the needed volume. Maybe we'd all be better off with an aggregated market data endpoint e.g. all orders per type or region. This would greatly reduce the number of requests and the unnecessary overhead they generate, even with connection pooling. Or, even better, an official market data stream.

Co-Founder of element43 - Join our team! | EVE Live | EMDRMon | accounts-eve
Pete Butcher
The Scope
Gallente Federation
#30 - 2015-06-21 20:50:36 UTC
Gordon Newman wrote:
Maybe we'd all be better off with an aggregated market data endpoint e.g. all orders per type or region. This would greatly reduce the number of requests and the unnecessary overhead they generate, even with connection pooling. Or, even better, an official market data stream.


^ this. Getting stable market replies is a lottery now. I bet a keg of beer even those 100r/s will be too much if enough applications start polling the data via the same proxy. Again - please give us a simple versioned WADL along with (instead of?) HATEOAS and everyone will benefit from it - less work for us and less load for servers.

http://evernus.com - the ultimate multiplatform EVE trade tool + nullsec Alliance Market tool
Gordon Newman
Deep Core Mining Inc.
Caldari State
#31 - 2015-08-05 07:23:26 UTC
Any progress?

Co-Founder of element43 - Join our team! | EVE Live | EMDRMon | accounts-eve
Kali Izia
GoomWaffe
#32 - 2015-08-05 07:38:55 UTC
The endpoints are available on public CREST on TQ now.
I'm not sure if the rate limits have been increased yet or what they've been set to.
Pete Butcher
The Scope
Gallente Federation
#33 - 2015-08-05 07:42:43 UTC
Kali Izia wrote:
The endpoints are available on public CREST on TQ now.
I'm not sure if the rate limits have been increased yet or what they've been set to.


I think it's best to wait for that official dev blog. I have a feeling things might change a couple of times.

http://evernus.com - the ultimate multiplatform EVE trade tool + nullsec Alliance Market tool
Previous page12