EVE Technology Lab

It's trivial to support POST and PUT on my side at EMDR, which I can do. Technically, PUT is the "correct" way to upload market data, in this case.

That said, most well-designed libraries should be equally easy for PUT and POST. Here's a good Python example:

http://docs.python-requests.org/en/latest/api/#requests.head

You're basically just changing requests.post to requests.put. It shouldn't be any more difficult than that, especially if we're not form encoding.

As a note, when form encoding, please remember to actually form encode the contents. While most characters are safe in the message, the + in the timestamps gets decoded to a space, leading to an invalid date/time.

As for what key? Why not data. Its a nice four letter word, and I don't have to change anything

EMDR will continue accepting uploads without form encoding, so if you're doing that, keep on keeping on. I'll get around to adding support for the silly form-encoded POSTs sometime.

Quoting famous words:

If you have to use key/value pairs, and you're sending key/value data, (this next bit should be obvious) why not use the top level keys for the POST?

Granted, the existing format isn't conducive to that, but maybe that's food for thought. (I never was a fan of rowset/row structures in XML/JSON dumps.)

I'm not really a stakeholder at this point, so take my posts as musings, not requirements.

Been looking into what I need to do to make Yapeal into an uploader for the MarketOrder API and had some questions for the people involved in receiving the data. Because of how the API works it would be nice to send all the market items together in a single POST request just with multiple key-value pairs. I know there was some talk about there being no reason it can't be done from a format point of view but I was wondering if any of you have thought about allowing it and processing them that way. I'll try to make it a little clear how the POST would look:
UUDIF={JSON ...}&UUDIF={JSON ...}

Each one would be for a single TypeID so like the first one is TypeID=123 and the second one would be for TypeID=456 etc. so each is a complete dataset they would just be sent together in the same POST.

Is this something you think you'd support or will I need to look at sending a POST for each one? I can see this saving a lot of network bandwidth for everyone if it was supported.

I strongly encourage using Content-Encoding: gzip or Content-Encoding: deflate (same algorithm, different headers). If you know what you're doing, its easy to support (aka, I learned by integrating with EMUU ;)).

EVE-Central already supports gzip on the outbound data, which is a helpful bandwidth saver.

Guess I need to remember to hit refresh once in a while on that page so I see the changes Ok looks good and that does answer my question Thanks to whomever took the time for the updating on the wiki.

Setting the header saying you accept or would like to send your data compressed it doesn't make it required it just let's the server know that you are connecting to you can and if the server is setup to do so it usually will but it does take both ends agreeing to do so at the start of the connect to happen. If you are using something like cURL there are settings you can use to sent the correct headers and Apache etc have settings to turn it off and on as well.

Edit: The Implementations page was also a good idea

Another tip for those using PHP clients and servers.

PHP Client:
In order to encode for 'Content-Encoding: gzip' you will need to use "gzencode()" function.
For the 'Content-Encoding: deflate' you will need to use "gzcompress()" function.

PHP Server:
On the server side there an issue with decoding.
In order to decode for 'Content-Encoding: gzip' you will need to use "gzdecode()" function. But this function is not implemented even in version 5.3.13 (haven't tried 5.4.3). However the code for this function can be found at: http://pastebin.com/v8MJnNjH
For the 'Content-Encoding: deflate' you will need to use "gzuncompress()" function.

While we are at it.

Python Client:
For 'Content-Encoding: gzip' I have currently no knowledge, although looking at Python's documentation (http://docs.python.org/library/zlib.html#module-zlib), it's possible to construct the crc32 checksum and a gzip header is always '1f,8b,08,00' (in hex).
For 'Content-Encoding: deflate' use the zlib library "zlib.compress()".

Python Server:
You can support both 'Content-Encoding: gzip' and 'Content-Encoding: deflate' just by using "zlib.decompress(data, 15 + 32)", which will enable zlib to auto-check the compression header and do the appropriate decompression.

.NET Client:
For 'Content-Encoding: gzip' use a .NET zlib library (http://www.icsharpcode.net/opensource/sharpziplib/) and method "GZipOutputStream". Alternatively you could use .NET native "GZipStream" although it produces a slightly bigger compressed stream than zlib.
For 'Content-Encoding: deflate' use the zlib method "DeflaterOutputStream". The use of .NET native "DeflateStream" is not advised because it's not compatible with Scala's Spray decodeRequest(deflate) or Python's zlib.decompress() (works with PHP gzuncompress() though).

.NET Server:
Using .NET zlib library is mandatory here in order to support all type of clients.
For 'Content-Encoding: gzip' use method "GZipInputStream".
For 'Content-Encoding: deflate' use method "DeflaterInputStream".
Alternatively you could use a hack and strip the first 2 bytes (header) and last 4 bytes (checksum) and inflate the rest of the stream with .NET native "DeflateStream" method.