These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

 
  • Topic is locked indefinitely.
 

Can we have GZIP enabled on the API?

First post
Author
Previous Topic Next Topic
xHjfx
The Legion of Spoon
Curatores Veritatis Alliance
#1 - 2012-10-31 14:28:06 UTC
Hi, in order to minimize the data transfer between the API, can you please consider activating GZIP on the API?

You guys save a ton of bandwidth and our API retrieval times speed up because it's downloading smaller files i.e.

AllianceList will compress to around 80% of its original size.

Just wondering if we can have a developer response on this please and here's two previous posts on the subject:

https://forums.eveonline.com/default.aspx?g=posts&t=62922

https://forums.eveonline.com/default.aspx?g=posts&t=13719
CCP illurkall
C C P
C C P Alliance
#2 - 2012-10-31 14:58:04 UTC
Hey

I'll look into it. It makes perfect sense to me to enable compression. Need to dig around though if there are some crazy reasons why this wasn't done in the past.
xHjfx
The Legion of Spoon
Curatores Veritatis Alliance
#3 - 2012-10-31 15:08:49 UTC
Thanks for your quick reply :)

Would it be possible to get a reply with your results when you get around to having a look?

Cheers
CCP illurkall
C C P
C C P Alliance
#4 - 2012-10-31 15:13:46 UTC
I should have something to report next week. There are a few housekeeping tasks I'm looking into for the API that I hope to deploy next week.
Sentient Blade
Crisis Atmosphere
Coalition of the Unfortunate
#5 - 2012-10-31 15:53:03 UTC
CCP illurkall wrote:
I should have something to report next week. There are a few housekeeping tasks I'm looking into for the API that I hope to deploy next week.


Pleaseee let one of these be JSON output.
Desmont McCallock
#6 - 2012-10-31 17:51:57 UTC
Sentient Blade wrote:
CCP illurkall wrote:
I should have something to report next week. There are a few housekeeping tasks I'm looking into for the API that I hope to deploy next week.


Pleaseee let one of these be JSON output.
Which is quite easy on ASP.NET if CCP switches to .NET 4.
CCP illurkall can correct me If am wrong but I believe that the API is build on .NET 2.
Turk Fezzik
Sons of Seyllin
Pirate Lords of War
#7 - 2012-10-31 21:37:10 UTC
Sentient Blade wrote:
CCP illurkall wrote:
I should have something to report next week. There are a few housekeeping tasks I'm looking into for the API that I hope to deploy next week.


Pleaseee let one of these be JSON output.


I read that JSON provides around 40% data "compression" over XML on its own (sorry couldn't find the source of that claim). If you have to choose between JSON or gzip I'd vote for JSON.
Sentient Blade
Crisis Atmosphere
Coalition of the Unfortunate
#8 - 2012-10-31 22:33:08 UTC
Turk Fezzik wrote:
Sentient Blade wrote:
CCP illurkall wrote:
I should have something to report next week. There are a few housekeeping tasks I'm looking into for the API that I hope to deploy next week.


Pleaseee let one of these be JSON output.


I read that JSON provides around 40% data "compression" over XML on its own (sorry couldn't find the source of that claim). If you have to choose between JSON or gzip I'd vote for JSON.


Depends how you format it. Attribute heavy structures (like EVE) would mean that there is not much difference, if you use named nodes for every little thing then yeah, JSON would be much more efficient.

The main benefit to JSON is it's self representative of the underlying data structures. An array is an array, an object is an object. No jumping through hoops.

The downside is you can't validate it against a schema. But who gives a damn.
CCP illurkall
C C P
C C P Alliance
#9 - 2012-11-01 11:53:55 UTC
The deployment next week is about housekeeping. Replacing our internal caching layer, cleaning up some unused/unnecessary HTTP headers. Hopefully enabling gzip compression, just need to test that out a little first.
No new features at this time.
The API is running on .net 4.0 but it is unfortunately not a small task to enable json output for all the endpoints based on how the code is structured.
June Ting
Full Broadside
Deepwater Hooligans
#10 - 2012-11-01 14:10:08 UTC
Awesome, this is fantastic to hear!

I fight for the freedom of my people.
Desmont McCallock
#11 - 2012-11-01 14:17:53 UTC  |  Edited by: Desmont McCallock
If you end up enabling gzip I highly recommend in using ICSharpCode.SharpZibLib and not the .NET GZipStream or DeflateStream (the latest produce other headers than the ZipLib uses and you will end up with platforms incompatibilities).
xHjfx
The Legion of Spoon
Curatores Veritatis Alliance
#12 - 2012-11-06 17:14:29 UTC
Bumps
Cyerus
University of Caille
Gallente Federation
#13 - 2012-11-06 20:05:18 UTC
Desmont McCallock wrote:
If you end up enabling gzip I highly recommend in using ICSharpCode.SharpZibLib and not the .NET GZipStream or DeflateStream (the latest produce other headers than the ZipLib uses and you will end up with platforms incompatibilities).


I agree with Desmont, have had experience with this happening at my old employer.
Karbowiak
State War Academy
Caldari State
#14 - 2012-11-10 00:27:53 UTC
CCP illurkall, did you get around to fixing gzip ?
MJ Maverick
Hybrid Industrial
The Curatores Veritatis Auxiliary
#15 - 2012-11-11 17:44:36 UTC
xHjfx
The Legion of Spoon
Curatores Veritatis Alliance
#16 - 2012-11-12 09:20:36 UTC
MJ Maverick wrote:
Also, I refer to my thread about AllianceList.xml queries without downloading.

https://forums.eveonline.com/default.aspx?g=posts&m=1272204#post1272204


Can't say we've had that issue using chromium bleeding edge builds or curl..
Wacktopia
Fleet-Up.com
Keep It Simple Software Group
#17 - 2012-11-12 22:55:36 UTC
CCP illurkall wrote:
The deployment next week is about housekeeping. Replacing our internal caching layer, cleaning up some unused/unnecessary HTTP headers. Hopefully enabling gzip compression, just need to test that out a little first.
No new features at this time.
The API is running on .net 4.0 but it is unfortunately not a small task to enable json output for all the endpoints based on how the code is structured.


Gzip would be a massive help.

Kitchen sink? Seriousy, get your ship together -  Fleet-Up.com
CCP illurkall
C C P
C C P Alliance
#18 - 2012-11-13 11:11:29 UTC
Let's discuss here