EVE Technology Lab

Greetings to the EVE community.

I am currently working on a web-based fitter, and together with Pyfa/aurora developers we decided the whole community (including us) could benefit from a loadout format that doesn't suck. Before blowing my own horn, here's a quick rundown of the existing formats, and why they're f*cked up beyond repair:

1. EVE XML loadout export: it doesn't include typeIDs, so if an expansion changes names of some modules, you're screwed.
2. ShipDNA: it's nice enough for simple stuff, and integrates nicely with the client, but it supports nowhere enough functionality (like charges, or module state) to be good as a more general format. Also, it's not at all extendable.
3. EFT format: probably the worst of all. Only typenames and no clear parsing rules. It is ambiguous in many ways (such as: character encoding, line terminations, ordering, etc.), its extendability is very poor and not correctly specified.

So, we came up with a new format: the Common Loadout Format (CLF). There is a draft of the specification available now, and if you're a developer interested in this format, I'd like you to take a couple of minutes, read the draft and share your feedback with us.

Here's the draft of the CLF, version 1: http://artefact2.com/clf/draft-common-loadout-format-01

Here are some questions to help you put your impressions into words:

• Is the specification clear enough? Did you have to read certain parts multiple times to understand them? Did you find some sentences ambiguous/unclear?
• Are the provided examples clear and useful enough? (Have they helped you understand the format?)
• Have you spotted inconsistencies, mistakes, or contradictions inside the spec. that should be corrected?
• Do you have any ideas of what we might have forgotten to add in the format?

Once the draft is considered good and mature enough, it will be released as the "official" standard for CLF version 1, and reference validator implementations will be made available shortly (in PHP and possibly Python).

Also, keep in mind: this document is a draft, and it is inappropriate to rely on it as it is still a work in progress.

Thanks.

I think it's just the word "fitting" meaning different things to different people here. As it is defined in the draft, it is "A given ship with information about its state, including fitted modules, fitted charges, drones, implants, etc. and possibly variations of those.". The key word here being variation — a preset is nothing more than a variation of the same fitting. Presets were intended just for slight variations of the same idea, like a pimped version, a T2 version and a T1 version. The format was not designed to store multiple, unrelated loadouts in one JSON object.


That's a fair point. Although, even if it's in most cases pointless to not have a preset, I can't see why we should forbid it. The format could also be used just for its drone presets, for example.

There's actually a valid use case against this. Consider a freighter loadout, where you cannot fit any modules, but you still want to save the implants (like agility implant, and warp speed implant).

The movitation for this was to make it easier for entites not wanting to support charge presets. If it ends up creating more confusion/problems, it will probably be removed in favor of the solution you just described.

(Cont. in the next post)

RFC 2822 is intended for email messages and the date format there is likewise. The ISO 8601 Date Time format is simpler to deal with when converting from a string to a datetime. ISO 8601

Field names using the lowercase_with_underscores naming convention are slightly more readable than with simply the lowercase convention:
creationdate vs creation_date.

Consider removing the extraneous spaces in the JSON shown in the specification, while keeping spaces that are normal when writing mathematical expressions or simply writing in English. For example:
No: { "typeid": 8105, "charge": { "typeid": 209 } }
Yes: {"typeid": 8105, "charge": {"typeid": 209}}

Read the Pet Peeves section even if you're not writing in Python. Python Coding Style as much of it applies regardless of programming language.

JSON is a subset of YAML. YAML is meant to be easy for both humans and programs to read - relative to XML in particular. It is the direction that CCP is going as well. Why not give it a try? I created the following by simply copy pasting the JSON from the spec (removing the ... at the end) and fed into the following code:

import yaml

json_expression = 'copy pasted JSON'
print yaml.dump(yaml.load(json_expression))

presets:
- chargepresets:
- {id: 0, name: Scourge missiles}
modules:
- charge: {typeid: 209}
typeid: 8105
- charge: {typeid: 209}
typeid: 8105
- charge: {typeid: 209}
typeid: 8105
- charge: {typeid: 209}
typeid: 8105
- charge: {typeid: 209}
typeid: 8105
- charge: {typeid: 209}
typeid: 8105
- charge: {typeid: 209}
typeid: 8105
presetname: Meta 4 launchers
- chargepresets:
- {id: 0, name: Scourge missiles}
- {id: 1, name: Scourge fury missiles}
- {id: 2, name: Scourge precision missiles}
modules:
- charges:
- {typeid: 209}
- {cpid: 1, typeid: 2629}
- {cpid: 2, typeid: 24513}
typeid: 2410
- charges:
- {typeid: 209}
- {cpid: 1, typeid: 2629}
- {cpid: 2, typeid: 24513}
typeid: 2410
- charges:
- {typeid: 209}
- {cpid: 1, typeid: 2629}
- {cpid: 2, typeid: 24513}
typeid: 2410
- charges:
- {typeid: 209}
- {cpid: 1, typeid: 2629}
- {cpid: 2, typeid: 24513}
typeid: 2410
- charges:
- {typeid: 209}
- {cpid: 1, typeid: 2629}
- {cpid: 2, typeid: 24513}
typeid: 2410
- charges:
- {typeid: 209}
- {cpid: 1, typeid: 2629}
- {cpid: 2, typeid: 24513}
typeid: 2410
- charges:
- {typeid: 209}
- {cpid: 1, typeid: 2629}
- {cpid: 2, typeid: 24513}
typeid: 2410
presetname: Tech II launchers
ship: {typeid: 24698, typename: Drake}

XML has its advantages though provided humans won't be reading the output often. Name spaces are built in and xpath expressions can be used to extract items of interest declaratively while with JSON / YAML / Hashes / Python Dicts the only way is iterative looping unless you know exactly where in the structure the items are located - and even then to fetch multiple means looping.

The evefit team approves of this format.

I've been providing input via IRC and will naturally be providing more.

I'd add a mandatory field with the current game version of the data used to create the fitting. That way programs could try to determine changed IDs or simply use older tables (unless I missed a note it's there; I think there's only a value for the format version?).

How about the SDE version?

That should do the trick. The more frequently occurring problem though is CCP devs deciding to simply delete type ids altogether. Look at what happened with the old scanner probes: Quest etc - their type ids are just gone - I think the same thing happened with mines but then they were brought back in if I'm not mistaken.

So what to do if a fitting references type ids that no longer exist? Just drop them out I suppose or map them to whatever they were switched to as was done for probes.

That's a big, big no-no. If you want to go back to only using typenames, then you might as well keep using the EFT or the EVE XML format. Typenames change far, far more often than typeids (for example, propulsion mods, invulnerability fields, missiles and torps, etc.). If CCP decides to change typeids, I think they would also break all the personal and corporation fittings (from the in-game fitting browser) using them by doing so, so it's unlikely they'll do it often. With your method an entity would need to have pretty much all the SDEs ready in your database to parse typenames. Typeids tend to change much less, so hardcoding the (few) changes and only keeping the latest SDE could work. I could even see a small file regrouping all the typeid changes (and the build number of when it happened) being maintained for entities to use.

Charge preset IDs are here to avoid repeating the preset name over and over, to make it easier (if someone is manually editing the file) to change the names. It's also better for parsing, because you immediately know how many charge presets there are.

This is undeniably more appealing to the eye and more consise than the current CLF, however, this comes at a great cost:

• Where are the preset descriptions? And the charge preset names/descriptions?
• How would you handle modules which don't have a charge fitted in some charge presets? (Like, a sensor booster with a script in charge preset A, but not in charge preset B) You could use nulls in your array, but that's a bit hackish and you're essentially back to using implicit cpids.
• Also, you thew extendability out of the window. What if you ever want to add a field to the format? With arrays and no keys, you hide a lot of the logic in the parser (that has to make a lot of assumptions about where to find information), so by touching the format a little bit (like by adding a column), you're breaking all parsers out there. That's terrible. Using keys is better for backwards and forwards compatibility (old parsers will just ignore the new key).

I hope you're not taking my criticism personally, I appreciate the feedback ;)

As far as I know Gripen's ways (EFT), he's not going to support CLF, not until he's forced to.
Talking to MrCue (BC admin), supporting CLF depends on its popularity.
From EVEMon's side, I intent to support CLF as long as it is used by an app.

Edit: Currently EVEMon supports XML, DNA, EFT formats.

Converting from the CLF to another format is easy. The other way is not so easy, but can still be done with a few assumptions. Once the format stabilizes a bit, I will make a reference validator/parser in PHP for the CLF, and I guess I could also include a little conversion script to juggle between formats. Maybe I could use it in a simple webpage where users paste their loadouts, select an output format and they get the result, and host it somewhere. (The code would be most likely public domain.)

Regarding adoption, only time will tell us. But I think that if the format is good, and gets used by some popular applications, it will grow popularity on its own. (And I'm sure application developers will be glad to have a solid specification and reference implementation to code their parsers against.) Anyway, I am definitely sure I will primarily use the CLF in the project I'm working on (a browser-based fitter), if that counts.

I will update the draft soon to make a few changes based on the feedback I got here in the last few days. After that, we'll have to decide on an expiration date for the draft: after this date, the draft either becomes the "official" specification (and becomes immutable, except for trivial changes such as typos), or a second draft is published (with a new expiration date).