These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

 
  • Topic is locked indefinitely.
 

CREST experimental access for the Alliance Tournament

First post First post
Author
Ferria
Outsourced Manufacturing
#61 - 2013-07-25 14:15:12 UTC
So when can I get this kind of functionality for Dust514? why you gotta taunt us like this?
Seismic Stan
Freebooted Junkworks
#62 - 2013-07-25 14:20:53 UTC
CCP QC wrote:
Hi all.

Since this stuff is going rather well, me and CCP Veritas though it would be nice to push this to 11. So here we go:

The vnd.ccp.eve.TournamentMatch-v1 resource now has two new optional fields:
* firstReplayFrame
* lastReplayFrame


Both of these link to a TournamentRealtimeMatchFrame-v1 containing a pretty amazing amount of info =)

Because it's easier to see for yourselves, just hit:
http://public-crest-duality.testeveonline.com/tournaments/3/series/5/matches/2/realtime/0/


A few notes on this:
1) Is this real life?
yep
2) Will this persist post match/tournament/downtime?
yes/yes/yes
3) Will this be updated in real-real time during the AT?
These endpoints will update more or less in synch with the twitch stream, so in real-time, but delayed by 60 seconds to not leak too much information to participants
4) what happens during real time action if I GET a frame that doesn't exists yet?!
I'll hold your request until it does. just make the GET and when it returns you should have your frame. Assuming all goes well.
4) Why?! OH GOD WHY?!
I like a good challenge.


o7

NINJA EDIT:
these endpoints are under HEAVY dev right now, expect change on them.


This sounds like the beginning of something amazing. For a non-techie, could you explain what the applications of this are?

For instance:

Can it be used to build a battlerecorder which would play back the events in some kind of simulation?

Is CCP developing a graphical interface to provide post-match analysis during the tournament?

Can this be applied to data from engagements in EVE proper for the general use of PvPers?
CCP Veritas
C C P
C C P Alliance
#63 - 2013-07-25 14:32:51 UTC
There are *significant* hurdles in the way of applying this technology to Eve in general. The tournament provides us a very bounded environment - 15 minutes max, 24 ships max, constrained location - which makes this sort of thing possible.

CCP Veritas - Technical Director - EVE Online

GizzyBoy
I N E X T R E M I S
Tactical Narcotics Team
#64 - 2013-07-25 14:38:20 UTC
making me login to eve forums :P

no sleep for me to night,


Curses you devs you! ima try and bring so much extra alcomahole next fan fest, you can write off the the rest of MAY.
GizzyBoy
I N E X T R E M I S
Tactical Narcotics Team
#65 - 2013-07-25 14:48:36 UTC
CCP Veritas wrote:
There are *significant* hurdles in the way of applying this technology to Eve in general. The tournament provides us a very bounded environment - 15 minutes max, 24 ships max, constrained location - which makes this sort of thing possible.



no the game data for your current pvp engagement couldnt be streamed in real time as the crest system has no way of knowing what pvp engagement your talking about.

What id like them to do however, is save the incoming stream of data pre tick that your client receives, and be able manipulate that data via one of the following.

You can save your engagements in a compressable small file size file.
You can play the game in low quality settings, yet the stream will be able to be re rendered in any setting you wish.
You can produce a video from the stream in highest video settings straight to file,
You can direct the data stream to another rendering service in your local network to produce a completely rendered view, sans chat / possible system information, which can be passed directly to twitch / 3rd party streaming site.
Peter Powers
Terrorists of Dimensions
#66 - 2013-07-25 14:52:10 UTC
nice demo :)

glad to see a bit more about CREST.
i wish i would have had a bit more time at hand to toy around with it a bit before the AT :(

3rdPartyEve.net - your catalogue for 3rd party applications

Peter Powers
Terrorists of Dimensions
#67 - 2013-07-25 14:54:50 UTC
CCP Veritas wrote:
There are *significant* hurdles in the way of applying this technology to Eve in general. The tournament provides us a very bounded environment - 15 minutes max, 24 ships max, constrained location - which makes this sort of thing possible.


well if you would provide an interface where i post a kill id, and it gives me the last 300 ticks of the grid where that kill occured, that would already allow for some amazing applications.

but for that you would need to record and store all ticks per grid, which probably is way too much data.

3rdPartyEve.net - your catalogue for 3rd party applications

Makari Aeron
Imperial Shipment
Amarr Empire
#68 - 2013-07-25 15:06:08 UTC
CCP FoxFour wrote:
WEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE!

Gonads and Strife?



Now if only CREST existed in tandem with SSO and allowed for all the current API endpoints ;) Keep up the good work!

CCP RedDawn: Ugly people are just playing life on HARD mode. Personally, I'm playing on an INFERNO difficulty.

CCP Goliath: I often believe that the best way to get something done is to shout at the person trying to help you. http://goo.gl/PKGDP

jonnykefka
Adhocracy Incorporated
Adhocracy
#69 - 2013-07-25 16:04:57 UTC
CCP Veritas wrote:
There are *significant* hurdles in the way of applying this technology to Eve in general. The tournament provides us a very bounded environment - 15 minutes max, 24 ships max, constrained location - which makes this sort of thing possible.


Understandable, but could we look mid-range future at a configurable on/off switch, the same way there's advanced camera tools for filmmakers? Playback is obviously it's own nasty beast but seriously the potential of this thing is tremendous. We have a couple of really great movie-makers in our corp, if we could record a fight (in w-space, so often following similar parameters to an AT match) and they had the tools to go back and re-shoot some of it with better camera angles and less UI?

Look, just from a marketing perspective this thing has amazing potential. What would you rather watch, a bunch of colored brackets at minimum visual settings or a ship blazing through combat with all of the art team's work on full display? You wouldn't even need to make trailers anymore! We would make them for you!

Obviously it's going to be a challenge to make it widely available, I fully expect that this is something that would take a year plus to make right, but consider this one more voice calling for you to put the work into it.
CCP Veritas
C C P
C C P Alliance
#70 - 2013-07-25 16:48:47 UTC
We put some more mildly ridiculous data at this endpoint:
http://public-crest-duality.testeveonline.com/tournaments/3/series/6/matches/0/

CCP Veritas - Technical Director - EVE Online

Zael Jun
Garoun Investment Bank
Gallente Federation
#71 - 2013-07-25 17:50:20 UTC  |  Edited by: Zael Jun
CCP QC wrote:
Luigi Thirty wrote:
Why are you using your own MIME type instead of application/json? Why are you using nginx 1.2 which has more holes than a golf course?


We use our own mime type to support being asked for different version of a resource by the clients. By sending in an accept header with the correct mimetype, say "application/vnd.eve.ccp.Tournament-v1" the server knows to return you that particular version. It allows us to release new versions of these resource and your client will keep working as long as we support the version you want. Asking the server with an accept header that doesn't match returns you the latest version of a particular resource.


This is not standard practice and I'll hope you reconsider. If you are returning json, your mime-type is application/json. If you need to version your API, as most APIs do, you should either change the resource identifier (/v1/endpoint to /v2/endpoint) or create your own header (e.g. API-Version: 0.00.1-build5290 if you must be so granular). I specifically say to use API-Header because, even though it is not a standard header, it is a header in common use for exactly this purpose. Further, RFC 6648 deprecates the use of X- headers in favor of all headers being treated like they could become acceptable standards in the future (as the API-Version header is very likely to do).
Nicen Jehr
Subsidy H.R.S.
Xagenic Freymvork
#72 - 2013-07-25 18:01:46 UTC  |  Edited by: Nicen Jehr
Zael Jun wrote:
CCP QC wrote:
We use our own mime type... say "application/vnd.eve.ccp.Tournament-v1" the server knows to return you that particular version.
This is not standard practice and I'll hope you reconsider. If you are returning json, your mime-type is application/json... create your own header (e.g. API-Version: 0.00.1-build5290 if you must be so granular). I specifically say to use API-Header because, even though it is not a standard header, it is a header in common use for exactly this purpose.
I am new to the dev/CREST party, if CCP put the versioning info in a header and set mime type to application/json - how would applications request a particular version?

Would my application send a GET request with a API-Version header? And the CREST response would contain its own API-Version header, hopefully the same as what I requested, and if not, the most recent version?

If that's an option it sounds like a better way to do it than repurposing MIME type. Isn't the entire purpose of headers to provide metadata about the content?
CCP QC
C C P
C C P Alliance
#73 - 2013-07-25 18:12:02 UTC  |  Edited by: CCP QC
I don't want to get into an argument over this, as the system will stay this way, but I disagree with the following:

Quote:
If you are returning json, your mime-type is application/json


While I know that many use application/json as the "json" mimetype, I think this is wrong.

See, Json is not what I'm returning. Json is how I encode what I'm returning. The representation of the resource you are transferring is represented in a mimetype, say application/vnd.ccp.eve.TournamentTeam-v1, and the encoding of that representation is +json.

Should we decide to support xml in the future, you'd request application/vnd.ccp.eve.TournamentTeam-v1+xml

In any event, this is not going to change. If you don't like it, that's ok. But I think if you give it a try, you might find that this way is quite nice to use.

TL;DR: no special header, no url /vX/ style. Give me an Accept header and if I can match it you get exactly that, if not, you get the latest available representation
metiskafelez
Space Autism LLC.
#74 - 2013-07-25 18:55:25 UTC  |  Edited by: metiskafelez
This is really cool! I can't wait to for this to go live on TQ, and see what streamers can do with it!

Also, to be a bit more on topic with where this thread is currently at. As a matter of preferred practice I like that the Mimetypes are open ended exactly for the above mentioned reason. The object that you are getting is JSON encoded, but not necessarily able to be consumed as a JSON object.

Specifying versioning allows for applications created to handle one version to still be functional as newer versions are implemented.

Me Gusta.
Zael Jun
Garoun Investment Bank
Gallente Federation
#75 - 2013-07-25 19:07:16 UTC  |  Edited by: Zael Jun
Nicen Jehr wrote:
I am new to the dev/CREST party, if CCP put the versioning info in a header and set mime type to application/json - how would applications request a particular version?

Would my application send a GET request with a API-Version header? And the CREST response would contain its own API-Version header, hopefully the same as what I requested, and if not, the most recent version?

If that's an option it sounds like a better way to do it than repurposing MIME type.


Current best practice when using an API-Version header is to make it a request header. Some people say it should be required, but I think this limits the ability to RESTfully request without manually creating the request (e.g. if I access /endpoint and have authenticated in some fashion does version really matter? I think a default is okay and should be noted by the server in some fashion) and REST is supposed to be as simple, and logical, as possible.

Other options I've seen and like, but have little practice, is adjusting the Accept header, which supports arbitrary parameter. You could then send a request like this:
Accept: application/json; version=2.0, application/json; q=0.1; version=1.0

That request would mean, "my application is compatible with 2 versions of the API, but I prefer the new one." the server would then respond with
Content-Type: application/json; version: 1

This says, "yeah, you tried to access the resource using the new version, but this particular resource doesn't support the new version yet." (See RFC 2616 about the accept header and additional parameters; here's express, a node.js framework, supporting additional information on the accept header as an example of practice: https://github.com/visionmedia/express/pull/1563)

All of these options have drawbacks, however. I don't know if the drawbacks are as bad as a non-semantic mimetype that changes with every version, especially when the mimetype is basically implementing the last example I gave in a very ugly manner. Here's the same example as above using CCP mimes using the same scenario as above:

Accept: application/vnd.eve.ccp.Tournament-v2+json, application/vnd.eve.ccp.Tournament-v1+json; q=0.1
Content-Type: application/vnd.eve.ccp.Tournament-v2+json

CCP QC wrote:
I don't want to get into an argument over this

I'm not arguing, I'm discussing the disadvantages of your implementation vs. the other options that are becoming widely accepted as best practices.

Quote:
While I know that many use application/json as the "json" mimetype, I think this is wrong.

See, Json is not what I'm returning. Json is how I encode what I'm returning. The representation of the resource you are transferring is represented in a mimetype, say application/vnd.ccp.eve.TournamentTeam-v1, and the encoding of that representation is +json.


I'd have to disagree with your definition of a mimetype, but I can understand why you made your decision based on that definition. A mimetype identifies the encoding of a resource so that an application using the resource knows how to manage it (See original definitions and uses born out of STMP). By your definition, text/html is not a reasonable mimetype as, since it is my website, I should really be saying that the resource is of type text/vnd.zael-jun+html (after all, the HTML on my page is of a completely different format than any other page, even assuming i only use standard tags). This position is not reasonable. The resources is /zael-juns-site/endpoint, the encoding is text/html but, you know what, I also give my resources in application/json for the cool kids. The only thing that matters is that you know how to parse and use the resource, which you can do because I've used standard mimetypes.

Further, as I said above in response to Nicen Jehr, this implementation is really just smashing the Accept header parameters into one mimetype instead of using the ability to set parameters within the Accept header. I can see why you might be doing it for ease of use, and I'm sure you'll get along fine if you don't change it, I was merely pointing out more acceptable and standard ways of handling such problems. While this is your application, being such a big and powerful player out there I think it is a good thing to set good examples for those that will follow in your footsteps.
Marcel Devereux
Aideron Robotics
Aideron Robotics.
#76 - 2013-07-25 19:19:01 UTC  |  Edited by: Marcel Devereux
CCP QC wrote:
I don't want to get into an argument over this, as the system will stay this way, but I disagree with the following:

Quote:
If you are returning json, your mime-type is application/json


While I know that many use application/json as the "json" mimetype, I think this is wrong.

See, Json is not what I'm returning. Json is how I encode what I'm returning. The representation of the resource you are transferring is represented in a mimetype, say application/vnd.ccp.eve.TournamentTeam-v1, and the encoding of that representation is +json.

Should we decide to support xml in the future, you'd request application/vnd.ccp.eve.TournamentTeam-v1+xml

In any event, this is not going to change. If you don't like it, that's ok. But I think if you give it a try, you might find that this way is quite nice to use.

TL;DR: no special header, no url /vX/ style. Give me an Accept header and if I can match it you get exactly that, if not, you get the latest available representation


Why does the return type need to be application/json?
l0rd carlos
the king asked me to guard the mountain
#77 - 2013-07-25 19:19:04 UTC
CCP QC wrote:
Hi all.

Since this stuff is going rather well, me and CCP Veritas though it would be nice to push this to 11.


Why don't you just make ten louder and make ten be the top number and make that a little louder?

Youtube Channel about Micro and Small scale PvP with commentary: Fleet Commentary by l0rd carlos

Marcel Devereux
Aideron Robotics
Aideron Robotics.
#78 - 2013-07-25 19:57:46 UTC
CCP Veritas wrote:
There are *significant* hurdles in the way of applying this technology to Eve in general. The tournament provides us a very bounded environment - 15 minutes max, 24 ships max, constrained location - which makes this sort of thing possible.


So you mean those thousands of clients that currently connect to your system is using magic to get the physics data of objects into the client?

;-)
Slvrsmth
Native Freshfood
Minmatar Republic
#79 - 2013-07-25 20:12:48 UTC
Long ago, in a discussion far far away, there were magical words CREST and OAuth used in the same sentence. Any news on this, or have I been dreaming?

I'm asking because due to lack of information I've been starting to implement my own 'middleman' service (essentially, a OAuth-enabled API proxy, allowing small apps access to auth-protected data without having to manage API keys themselves) for EvE-related projects.

I started development due to lack of new CREST-related information, but if the project is still being worked on and will have such abilities (will it?) I'd be more than happy to stop and wait - a CCP-run service would be more 'legit'. And I wouldn't have to maintain it myself :)

So, is it feasible to expect such API functionality deployed against TQ data before, say, winter expansion?

PS Please use standard mime types - in my experience it's much easier for API consumers to work with additional header rather than merge that information into content-type.

Carebearium - find the best solar system for you!

CCP QC
C C P
C C P Alliance
#80 - 2013-07-25 20:14:11 UTC
Marcel Devereux wrote:
CCP Veritas wrote:
There are *significant* hurdles in the way of applying this technology to Eve in general. The tournament provides us a very bounded environment - 15 minutes max, 24 ships max, constrained location - which makes this sort of thing possible.


So you mean those thousands of clients that currently connect to your system is using magic to get the physics data of objects into the client?

;-)


Well, that's partially the magic of Eve. The actual clients don't get x,y,z except when they or someone else joins a bubble. A part from that, they only get the "Dude X started orbiting Dude Y" or "Dude A started shooting dude B" events and the simulation is then synchronized. If no one clicks anything, even a 2000 man fleet fight produces just about no traffic.

The method we use here needs to store all x,y,z and send them down all the time, no local simulation. THAT doesn't scale well to 2000 ship fleet.