These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

 
  • Topic is locked indefinitely.
 

Requesting CCP To Provide The 'Complete' MSSQL Datadump and YAML

First post
Author
Previous Topic Next Topic
Lai Rai
Perkone
Caldari State
#1 - 2015-07-10 23:58:53 UTC  |  Edited by: Lai Rai
To CCP devs related to third-party development,

In addition to the YAML files, can you please continue to provide MSSQL datadump like you used to i.e. with invTypes still on the datadump.

For me, it is a lot easier to perform query things that I need from the database such as what marketGroup of items I need rather than parsing from the YAML file. The level of specificity that I can do using the database dump is much more convenient right now. It will also take time to change my underlying code to reflect the changes.

Thank you for your consideration,

Lai Rai
Kali Izia
GoomWaffe
#2 - 2015-07-11 00:51:43 UTC
The thing is that the data doesn't natively exist in MSSQL anymore. CCP is moving their own data out of databases and into YAML to better support versioning.

So either they would have to convert it, you can convert it (using something like https://forums.eveonline.com/default.aspx?g=posts&t=324435&find=unread), or use one of the conversions that already exists (like https://www.fuzzwork.co.uk/dump/)

There was a discussion in #devfleet earlier and what might happen is that once everything is fully in YAML, CCP will help to start a community tool to convert the YAML files to various database formats.
Lai Rai
Perkone
Caldari State
#3 - 2015-07-11 16:56:31 UTC
Quote:
The thing is that the data doesn't natively exist in MSSQL anymore. CCP is moving their own data out of databases and into YAML to better support versioning.


Ahhh, that's understandable. Time for me to work on converting them :3

Thank you for the explanation.
Steve Ronuken
Fuzzwork Enterprises
Vote Steve Ronuken for CSM
#4 - 2015-07-11 17:33:13 UTC
I'm working on a 'generic' migrator (using python, with sqlalchemy) to move the yaml database into whatever database you might want (which is supported by sqlalchemy. postgres, mysql, mssql, sqlite, all possible)

https://github.com/fuzzysteve/SDE-loaders is the results so far.

There's a strong chance that CCP will be moving everything into yaml, leading to it just being the yaml files in the SDE. Which is fully understandable from their POV. Just a little work to get it into other formats.

Woo! CSM XI!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter
Lai Rai
Perkone
Caldari State
#5 - 2015-07-13 23:15:35 UTC
Thank you for sharing the link about your migrator. At the moment, I won't be using it as I have changed a bit of the database structure and I prefer turning the yaml to csv format :3
Zifrian
The Frog Pond
Ribbit.
#6 - 2015-07-14 01:58:53 UTC
Steve Ronuken wrote:
I'm working on a 'generic' migrator (using python, with sqlalchemy) to move the yaml database into whatever database you might want (which is supported by sqlalchemy. postgres, mysql, mssql, sqlite, all possible)

https://github.com/fuzzysteve/SDE-loaders is the results so far.

There's a strong chance that CCP will be moving everything into yaml, leading to it just being the yaml files in the SDE. Which is fully understandable from their POV. Just a little work to get it into other formats.

I'd like an actual plan for this transition other than what happened this last patch. As much as I absolutely hate YAML, I'd rather have all the SDE data in that format than the hodgepodge system we have now.

Your converter idea is a good one though.

Maximze your Industry Potential! - Download EVE Isk per Hour!

Import CCP's SDE - EVE SDE Database Builder
Desmont McCallock
#7 - 2015-07-14 10:18:59 UTC
Just in case anyone is interested into it, I plan to port the SDEExternalsToSql tool to DNX Core 5 ones the stable 1.0.0 version is out (according to MS this will happen with the release of VS 2015 on 20th July 2015). That means that the tool will be cross-platform for starters and additional support for other SQL databases will be possible. This of course will not happen over night, but is just an expression of interest.