These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

 
  • Topic is locked indefinitely.
 

Devsite blog: Public CREST Rate Limits

First post
Author
Previous Topic Next Topic
CCP Phantom
C C P
C C P Alliance
#1 - 2014-11-18 14:14:30 UTC  |  Edited by: CCP Logibro
With Phoebe we improved the CREST caching backend so that the caching of requests could be fine-tuned and optimized. If you wonder about the effect of those improvements, read on!

Additionally with we have adjusted the request rate to 30 per second and the burst size to 100.

Check out CCP FoxFour latest devsite blog Public CREST Rate Limits for more information!

CCP Phantom - Senior Community Developer
Querns
Science and Trade Institute
Caldari State
#2 - 2014-11-18 16:20:14 UTC
TODO added to my public crest client: "implement rate limiting." :V

Thanks for the heads-up on this.

This post was crafted by the wormhole expert of the Goonswarm Economic Warfare Cabal, the foremost authority on Eve: Online economics and gameplay.
CCP FoxFour
C C P
C C P Alliance
#3 - 2014-11-18 16:23:20 UTC
Querns wrote:
TODO added to my public crest client: "implement rate limiting." :V

Thanks for the heads-up on this.


You're welcome. :D

@CCP_FoxFour // Technical Designer // Team Tech Co

Third-party developer? Check out the official developers site for dev blogs, resources, and more.
Arronicus
State War Academy
Caldari State
#4 - 2014-11-18 17:41:49 UTC
Would it be possible to change the background colour of the blog to anything other than eye-bleed white? I'm on Chrome, if that makes any difference, and with the stark white background with thin black font, it makes this genuinely painful for me to read.
Alternatively, if there is some setting for me to change this, I would LOVE to know how.

Thank-you
CCP FoxFour
C C P
C C P Alliance
#5 - 2014-11-18 17:44:09 UTC
Arronicus wrote:
Would it be possible to change the background colour of the blog to anything other than eye-bleed white? I'm on Chrome, if that makes any difference, and with the stark white background with thin black font, it makes this genuinely painful for me to read.
Alternatively, if there is some setting for me to change this, I would LOVE to know how.

Thank-you


Will pass the feedback along.

@CCP_FoxFour // Technical Designer // Team Tech Co

Third-party developer? Check out the official developers site for dev blogs, resources, and more.
Gynax Gallenor
Conquering Darkness
#6 - 2014-11-19 08:24:17 UTC
R is an amazing tool. If you like that, you should definitely check out the package ggplot2.

I use R all the time so fell free to hit me up for help with it if you start using it more.

Fly Reckless, cos flying safe is no damn fun!

http://flyreckless.com/newsite/
Mixu Paatelainen
Eve Refinery
#7 - 2014-11-19 09:39:53 UTC
+1 for R being amazing, glad to see you guys playing with it!
Alejandro Rebenga
#8 - 2014-11-19 14:18:39 UTC
I can see it in my head, three devs using R. They were like ARRRRRRRRR!!!!

dev1: '...but how can we..'
dev2: '..no no thats for the.. how did you manage to do that?'
dev3: 'Can't we just use Paint for this or something?'
dev1: 'ARRRRRR, I give up! im going out for a beer..'
dev2&dev3: 'ARRRRRR!'
SpaceSaft
Almost Dangerous
Wolves Amongst Strangers
#9 - 2014-11-20 08:33:40 UTC
Why write in R when you can use matplotlib with python?
Pak Narhoo
Splinter Foundation
#10 - 2014-11-20 14:14:47 UTC
I'm anything but a developer but this blog, black letters on a white background is sooo much easier on my ageing eyes then the other blog style, you have no idea. Cry
Ortho Loess
Escalated.
OnlyFleets.
#11 - 2014-11-20 22:37:57 UTC
Forgive me if this is a stupid question, but the latest blog said the rate limits are much higher for authenticated CREST. Does it say anywhere what this higher limit is?
CCP FoxFour
C C P
C C P Alliance
#12 - 2014-11-21 09:53:00 UTC
Ortho Loess wrote:
Forgive me if this is a stupid question, but the latest blog said the rate limits are much higher for authenticated CREST. Does it say anywhere what this higher limit is?


No as we are still working on adjusting what that number will be. Suffice it to say however that it is substantially higher.

@CCP_FoxFour // Technical Designer // Team Tech Co

Third-party developer? Check out the official developers site for dev blogs, resources, and more.
Nerdz Rool
Royal Amarr Institute
Amarr Empire
#13 - 2014-11-21 21:56:33 UTC
CCP FoxFour wrote:
Arronicus wrote:
Would it be possible to change the background colour of the blog to anything other than eye-bleed white? I'm on Chrome, if that makes any difference, and with the stark white background with thin black font, it makes this genuinely painful for me to read.
Alternatively, if there is some setting for me to change this, I would LOVE to know how.

Thank-you


Will pass the feedback along.


Lol a black dev skin would be awesome. Dark grey background with Lime green text FTW!
Parkeur
Science and Trade Institute
Caldari State
#14 - 2015-02-01 08:01:56 UTC
Forgive the stupid question, but what does burst size mean?
Zifrian
The Frog Pond
Ribbit.
#15 - 2015-03-24 00:19:14 UTC  |  Edited by: Zifrian
Parkeur wrote:
Forgive the stupid question, but what does burst size mean?

I'd like to know as well.

Also, I'm still getting 3600 for max-age from the market history endpoint. Am I doing something wrong?

Maximze your Industry Potential! - Download EVE Isk per Hour!

Import CCP's SDE - EVE SDE Database Builder
Kali Izia
GoomWaffe
#16 - 2015-03-24 04:38:39 UTC
Zifrian wrote:
Parkeur wrote:
Forgive the stupid question, but what does burst size mean?

I'd like to know as well.

If you implement a token bucket/leaky bucket algorithm your code should be able to conform with no problems, but basically:

You can send an occasional burst of 100 requests all at once. If you do, you'll hit the rate limit once you try to send your 101st request unless you wait.

Your bucket refills at a rate of 1 per 1/30th of a second. So if you send 100 requests at once, you need to wait 3.33 seconds before you can send another 100 requests. Or if you only wait 2 seconds you can send another 60 etc.

Or you can send a constant 30 requests every 1 second instead.
Zifrian
The Frog Pond
Ribbit.
#17 - 2015-03-24 12:42:50 UTC
Kali Izia wrote:
Zifrian wrote:
Parkeur wrote:
Forgive the stupid question, but what does burst size mean?

I'd like to know as well.

If you implement a token bucket/leaky bucket algorithm your code should be able to conform with no problems, but basically:

You can send an occasional burst of 100 requests all at once. If you do, you'll hit the rate limit once you try to send your 101st request unless you wait.

Your bucket refills at a rate of 1 per 1/30th of a second. So if you send 100 requests at once, you need to wait 3.33 seconds before you can send another 100 requests. Or if you only wait 2 seconds you can send another 60 etc.

Or you can send a constant 30 requests every 1 second instead.

Excellent response, thank you.

Maximze your Industry Potential! - Download EVE Isk per Hour!

Import CCP's SDE - EVE SDE Database Builder
Ortho Loess
Escalated.
OnlyFleets.
#18 - 2015-03-24 12:58:54 UTC
Zifrian wrote:
Also, I'm still getting 3600 for max-age from the market history endpoint. Am I doing something wrong?


Market history is only updated every 24 hrs iirc, so cache time is set long.
Zifrian
The Frog Pond
Ribbit.
#19 - 2015-03-24 13:04:09 UTC
Ortho Loess wrote:
Zifrian wrote:
Also, I'm still getting 3600 for max-age from the market history endpoint. Am I doing something wrong?


Market history is only updated every 24 hrs iirc, so cache time is set long.

Yeah every day at midnight. Based on this blog though I should get a cache time in seconds until midnight. I'm a bit of a novice on this stuff so I assume I'm not reading/doing something right. The other endpoints return times as advertised.

Maximze your Industry Potential! - Download EVE Isk per Hour!

Import CCP's SDE - EVE SDE Database Builder