These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

 
  • Topic is locked indefinitely.
 

MSSQL vs. YAML
Author
Previous Topic Next Topic
Valda Abia
Science and Trade Institute
Caldari State
#1 - 2013-11-27 15:44:08 UTC
This has been bothering me for some time: What's the difference between the MSSQL data and the YAML data? Why are the two segregated in the toolkit?

I get that YAML is a different format, but it seems really odd to have data, some of which overlaps, output in two different formats. Plus there's no great easy way to integrate the YAML data, from what I can see?

Why include those files? Is there some technical reason that I'm not aware of that causes it to have to be done that way? Is there a better way for me, a novice, to deal with the data each time it's updated?
Elmore Jones
New Eden Mining Organisation
The Craftsmen
#2 - 2013-11-27 18:53:31 UTC
Changes to Toolkit Export Devblog

Read this :) It might be what you want to know.

+++ Reality Error 404 - Reboot Cosmos +++
Valda Abia
Science and Trade Institute
Caldari State
#3 - 2013-11-27 23:05:16 UTC
So what do people use to access this data? Do you really scan through a flat text file every time you need to supplement a SQL query with data from the YAML files? That seems... suboptimal... similar to when you end up with a massive JSON array that you need to deal with.

Why move this direction? Sure it's human readable... does the EVE Client and DB talk better with things that have been moved to YAML? From a technical standpoint, I'm probably (certainly) biased toward databases; with a good set of indexes and keys, accessing large data sets is a snap. I just don't see how YAML can match that performance, but then again, I know little about YAML.

Is it fast? Stable? Or is the human readability the only reason to go to it?

Also, if there's a conflict between the YAML and the DB, I'm assuming the YAML is the source of truth? For example, some of the typeIDs in the latest YAML dump have a SoundID; the corresponding typeID in the DB has that set to NULL.
Steve Ronuken
Fuzzwork Enterprises
Vote Steve Ronuken for CSM
#4 - 2013-11-28 09:50:21 UTC
The yaml is mostly because it's quite difficult to version control a database. version controlling a text file, on the other hand, is incredibly simple.

As for how you use the yaml, many people stick it into a database of some sort. Some of us convert it into a relational form and stick it in a table, others stick it directly into a database which supports that kind of data (mongodb, for example)

I would normally pay attention to the yaml, over the database.

Woo! CSM XI!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter