These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

 
  • Topic is locked indefinitely.
 

Developer Documentation

Author
Japhia Shebna
Pangolin Dynamics
#1 - 2013-10-29 06:45:52 UTC
I'm a newbie to Eve, but one of the things I researched before seriously considering buying it was the API. And boy, has that been quite the search.

While all my Eve friends tell me the API is an important part of Eve (and every player knows what an API is), there's absolutely no mention of it on the primary website. None, zilch, nada, not even in the deep recesses. Even the knowledge base, which has an API section, is empty.

If you find your way onto (the right part of) the forums, you'll find a sticky that claims to have links to the appropriate information. But it links to an unofficial spam-ridden page on a wiki with a single inactive administrator, and a broken link to self-admittedly incomplete official documentation. (This thread is of course locked, so no one can make suggestions here.) In the old version of the thread (which I haven't a clue how I found), there is a comment by a regular member that the official documentation has moved to the official Eve wiki (information that is never edited into the original post, despite an Eve dev posting later in the thread). That page of course states that it's out of date by several years. Both these wiki pages contain a number of links to relevant devblog posts that generally 404 (e.g. http://community.eveonline.com/devblog.asp?a=blog&bid=848 ).

From what I can tell (remember, I'm a newbie, so I don't have the benefit of seeing changes as they come in), changes to the API are being made - bugs are being fixed and features are being added. But when they are, there's only a haphazard attempt at documenting the changes, if any attempt at all. Even the documentation for CREST, which I gather is the in-development API that will be all awesome and stuff, is pretty sparse, despite being auto-generated (presumably via comment-extraction from the source).

When people ask for documentation, the common answer seems to be to use EVEDev. But, even aside from the aforementioned issues with it, EVEDev is not as active as claimed. More importantly, how are users supposed to document changes when they don't know what has changed, or more importantly, why it's changed?

Earlier this year new development help spawned creation of a roadmap for the API, but documentation did not make it on the list. While I'm sure you're nice, caring people, this makes me feel like you don't respect or care at all about third-party developers - which makes me sad, because I was hoping to be one. It's important to note that your actual feelings on the subject aren't really that important - it's how they get transmitted to the public that matters. You can take heart in knowing you're not alone; this is a problem that's rampant throughout companies, even software companies that should know better.

So. Here's where we make this post more than a rant (which is prohibited by forum rule #3 ;) ) into something constructive - how can we help this situation?

While pointing out shortcomings (as above) is easy, it's only helpful to a point. And perhaps the other developers don't share my gripes. :)

Obviously allowing community involvement is a good idea. Also obviously, you can't provide us with access to the Eve VCS diffs to write patch notes ourselves. :)

While wikis are great at allowing ease of editing, ease of editing by everyone is not necessarily what we want - as proven by the spammers. Personally, I think Github pull requests are an excellent tool for this sort of thing, particularly since they finished up the ability to do the whole process in-browser. Pull requests provide transparency, give a chance for others to comment on potential improvements, add a gatekeeper process to prevent spam (or even just low-quality changes), and hook into a service many developers are already using.

There are many formats for documentation, including plain ol' html, but one I've recently become a fan of is reStructuredText through Sphinx (btw, A beginners guide to writing documentation from the same site is a good starting read). And the best place to put your Sphinx-generated docs is Read the Docs, which handles all the infrastructure stuff so you don't have to. BTW, not only does will it build automatically from Github and provide Github edit links directly on the page, it [url=https://read-the-docs.readthedocs.org/en/latest/alternate_domains.html#cname-support]has CNAME support, so the project could live at, say, developers.eveonline.com. ;)

[continued on next page]
Japhia Shebna
Pangolin Dynamics
#2 - 2013-10-29 06:46:24 UTC  |  Edited by: Japhia Shebna
Since a feature suggestion is worth practically nothing without something to back it up, here's my offer: if you decide a project like the one I've described above is a good idea, I'll do the initial Sphinx setup, work with you to get a script that will make it automatic for you to commit doc changes to the repo when you commit the source changes to your internal system, and flesh out some basic user documentation. While I haven't received any awards for my docs, I kickstarted an internal high-level documentation project at work (successful, in that others are now writing pages without me having to bother them), and have worked on docs for both my own small projects and a variety of others' (through Github). I think documentation is dreadfully important, and I'm willing to stake my time on it.

One thing you, the Eve developers, could do that would be enormously helpful would be to "bless" a particular documentation project - whether it's one of the existing ones, or a new one, it doesn't matter too much, as long as there's one canonical source for information, one place new developers know to be both accurate and expansive, one place contributors can focus their efforts on improving. This is probably something that merits discussion, so I don't expect you to do it right now. :)

So - what do you say? Shall we do this?
Steve Ronuken
Fuzzwork Enterprises
Vote Steve Ronuken for CSM
#3 - 2013-10-29 08:38:55 UTC
https://wiki.eveonline.com/en/wiki/EVE_API_Functions

#eve-dev on irc.coldfront.net is a reasonable place to go looking for people to talk to.

Woo! CSM XI!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter

Peter Powers
Terrorists of Dimensions
HORSE-KILLERS
#4 - 2013-10-29 11:50:40 UTC
the basic idea itself i like.

however, i'd largely prefer Markdown over reStructuredText, to be more exact,
Github Flavored Markdown. https://help.github.com/articles/github-flavored-markdown

That would go nicely with a lot of us already using github, so learning gfm is something most have already done,
also documents in gfm will be rendered by github (so no need for an additional documentation system),
and parsers for markdown are widely available in all languages.
Also we could extend the documentation with code examples in several languages, and GFM features code highlighting for most.

The Pull-Requests idea is nice, however that requires someone actively taking care of managing those. Someone whos not only there for a few month and then leaves (the game and the community).

3rdPartyEve.net - your catalogue for 3rd party applications

Tseehn Marhn
Rat Duck
#5 - 2013-10-29 16:42:52 UTC  |  Edited by: Tseehn Marhn
I came into this game exclusively to program tools for it. It's the biggest reason I joined, and pretty much all I do here.

I can see your point of view. While I think it would be nice to have one place where all the tools are available, along with detailed instructions on how to use them, I don't think it's really a deal-breaker.

For any 3rd party tool, there are only four possible sources of data: API, CCP DBs, cache scraper, and EMDR. The API is really the only one of those things that could benefit from documentation, and a very reasonable job was done at the site Steve linked to. The other three are a simple SQL backup file (CCP DBs), an unofficial source that will likely never be documented (cache), and a user created stream of market data (EMDR) which is already documented well enough.

I'm not saying your idea isn't a good one. However in my experience, the lack of documentation for some things was more an annoyance than a problem. The API isn't hard to learn how to use, and the rest of the data sources are already documented, or need no introduction.

I think what happened here is you've experienced your first run-in with one of CCP's mantras: New Features - Coming Soon™. An improved API has been on the table for quite some time, along with a list of other things developers would like to see. The rub of it is dev tools are not sexy - which is why you couldn't find anything about the API on EVE's website. Tools by and large don't sell subscriptions. "A static data dump, and an API for game data access?!? SHUT UP AND TAKE MY MONEY", isn't something most gamers say. Because of this, CCP will likely only work on things like the API in fits and starts, and even then, probably won't include niceties like documentation.

In the mean time, should you have any questions on development things, feel free to EveMail me, or make a post here. I'm sure anyone here would be happy to help.
Japhia Shebna
Pangolin Dynamics
#6 - 2013-10-30 08:02:47 UTC
Steve Ronuken wrote:
https://wiki.eveonline.com/en/wiki/EVE_API_Functions


Yes, I'm aware of this page. To enumerate the problems listed above:

1. It's not linked to from any easily-findable source on the primary part of the website.
2. It's not linked to from the resources sticky in this subforum.
3. It states (in a very prominent banner) that it's several years out-of-date.
4. It links at the top to a 404ing devblog post, supposedly about information that's not on the wiki.
5. Mindshare seems to be split between it and the unofficial wiki.

Quote:
#eve-dev on irc.coldfront.net is a reasonable place to go looking for people to talk to.


I'll take a look, thanks.

Peter Powers wrote:
the basic idea itself i like.

however, i'd largely prefer Markdown over reStructuredText, to be more exact,
Github Flavored Markdown. https://help.github.com/articles/github-flavored-markdown

That would go nicely with a lot of us already using github, so learning gfm is something most have already done,
also documents in gfm will be rendered by github (so no need for an additional documentation system),
and parsers for markdown are widely available in all languages.
Also we could extend the documentation with code examples in several languages, and GFM features code highlighting for most.


Yeah, I agree on Markdown being preferable to reST. Unfortunately, I haven't run across a Markdown-based documentation system I like. If we're avoiding doing docstring-extraction, it may be feasible to just link together a series of markdown pages, perhaps with the assistance of Jekyll. I dunno, what do you think?

Quote:

The Pull-Requests idea is nice, however that requires someone actively taking care of managing those. Someone whos not only there for a few month and then leaves (the game and the community).


A very good point indeed.

Ideally, the Eve developers, PR folks, and whoever else would have administrative access (unlike eve-id.net), so replacements could be made. I doubt they wish to gain yet another job task, however, so this should usually be taken care of by some trusted set of users (in a similar fashion to the in-game Eve help groups). I'm not in a position of being able to suggest who this group would comprise of.

As long as there's at least one person to oversee patches (but not necessarily create any themselves), the system would sustain.
Japhia Shebna
Pangolin Dynamics
#7 - 2013-10-30 08:03:07 UTC

Tseehn Marhn wrote:
I came into this game exclusively to program tools for it. It's the biggest reason I joined, and pretty much all I do here.

I can see your point of view. While I think it would be nice to have one place where all the tools are available, along with detailed instructions on how to use them, I don't think it's really a deal-breaker.


Given that you have much more experience using these data access points, I defer to your judgement. I should also point out that ever since my first run-in with the Django docs, I've held very high standards for documentation.

Quote:
I think what happened here is you've experienced your first run-in with one of CCP's mantras: New Features - Coming Soon™. An improved API has been on the table for quite some time, along with a list of other things developers would like to see. The rub of it is dev tools are not sexy - which is why you couldn't find anything about the API on EVE's website. Tools by and large don't sell subscriptions. "A static data dump, and an API for game data access?!? SHUT UP AND TAKE MY MONEY", isn't something most gamers say. Because of this, CCP will likely only work on things like the API in fits and starts, and even then, probably won't include niceties like documentation.


Yeah, as much as I'd like to think the world revolves around programmers (and me specifically), it doesn't. Bummer.

But thinking about documentation as just a nicety is, I feel, a flaw in your software development process. Just as with automated tests, appropriate documentation forces you to think more about the code you're writing, structure it in a way that makes sense, think about how the changes you're making affect others, and generally produce a higher-quality product in the long run. But that's a very long topic we shouldn't get into, because it's absolutely derail any sort of constructive conversation.

Quote:
In the mean time, should you have any questions on development things, feel free to EveMail me, or make a post here. I'm sure anyone here would be happy to help.


Thanks! I've been pleasantly surprised in my short time here at the overall maturity and helpfulness of the community - you guys are great.
Peter Powers
Terrorists of Dimensions
HORSE-KILLERS
#8 - 2013-10-30 10:21:39 UTC
Japhia Shebna wrote:

Yeah, I agree on Markdown being preferable to reST. Unfortunately, I haven't run across a Markdown-based documentation system I like. If we're avoiding doing docstring-extraction, it may be feasible to just link together a series of markdown pages, perhaps with the assistance of Jekyll. I dunno, what do you think?

I don't know if we really need to do this in separate pages, a well organized document might be enough.
also, writing a script that builds a set of html pages from markdown i trivial, so that's fair enough as well.

Japhia Shebna wrote:
Ideally, the Eve developers, PR folks, and whoever else would have administrative access (unlike eve-id.net), so replacements could be made. I doubt they wish to gain yet another job task, however, so this should usually be taken care of by some trusted set of users (in a similar fashion to the in-game Eve help groups). I'm not in a position of being able to suggest who this group would comprise of.

I'd love to see CCP take some action and help organizing this, but i doubt it would happen. As for the Volunteers, i had a chat with one of 'em earlier this year about this forum, and he told me that most of 'em try to stay away from third party stuff, mostly because they lack to understand it.
So i think this initiative must come from us (the community).

Japhia Shebna wrote:

As long as there's at least one person to oversee patches (but not necessarily create any themselves), the system would sustain.

To oversee the patches you kinda need at least some basic understanding of its contents.

I can offer to help out with this (just not alone),
and i can offer to host a build version of the documentation at 3rdpartyeve.net or a subdomain.

3rdPartyEve.net - your catalogue for 3rd party applications

islador
Antigen.
#9 - 2013-11-05 21:46:30 UTC
eve-dev was recently defaced. I was using it yesterday without trouble and now see it as a spam filled useless wiki. Fortunately the wayback machine has preserved the documentation! http://web.archive.org/web/20130811195618/http://wiki.eve-id.net/Main_Page
Peter Powers
Terrorists of Dimensions
HORSE-KILLERS
#10 - 2013-11-06 10:20:44 UTC
islador wrote:
eve-dev was recently defaced. I was using it yesterday without trouble and now see it as a spam filled useless wiki. Fortunately the wayback machine has preserved the documentation! http://web.archive.org/web/20130811195618/http://wiki.eve-id.net/Main_Page

i've mirrored the api docs a while ago http://3rdpartyeve.net/eveapi,

also i've started on an initial version of a markdown based api doc (will make it public as soon as it is usefull, just running html2markdown over it is obviously not enough.

3rdPartyEve.net - your catalogue for 3rd party applications

Hel O'Ween
Men On A Mission
#11 - 2013-11-06 17:32:35 UTC
Ah, good to hear, Peter. Was thinking of cloning/forking the wiki myself, but I (and others, I'Ve learned meanwhile) tried to reach the domain owner first. Without luck, I need to add.

EVEWalletAware - an offline wallet manager.

raylu D
HELLSINKER
#12 - 2013-11-12 02:56:58 UTC
Peter Powers wrote:

i've mirrored the api docs a while ago http://3rdpartyeve.net/eveapi,

also i've started on an initial version of a markdown based api doc (will make it public as soon as it is usefull, just running html2markdown over it is obviously not enough.



FYI, your site is borked.

Can we see the API doc that you've written so far? Perhaps we could contribute to it in whatever form it's in; it doesn't have to look nice.
Peter Powers
Terrorists of Dimensions
HORSE-KILLERS
#13 - 2013-11-14 10:24:54 UTC
raylu D wrote:

FYI, your site is borked.

what exactly? if you mean the right hand links, yes, thats because those are generated from the names it had in the wiki... going to change.


raylu D wrote:

Can we see the API doc that you've written so far? Perhaps we could contribute to it in whatever form it's in; it doesn't have to look nice.

i'm planning to put it on a public github next weekend. Also, i haven't written it, so far i'm trying to convert what i extracted from the old wiki.

3rdPartyEve.net - your catalogue for 3rd party applications