These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

 
  • Topic is locked indefinitely.
 

Mysql (and excel) conversion of Crucible 1.5 Static Data Dump
Author
Previous Topic Next Topic
Steve Ronuken
Fuzzwork Enterprises
Vote Steve Ronuken for CSM
#1 - 2012-03-15 00:27:55 UTC  |  Edited by: Steve Ronuken
Edit:

See https://forums.eveonline.com/default.aspx?g=posts&t=100797 for Escalation





While I'm /pretty/ certain it came through ok, caveat emptor. Some tables have fewer rows in them, but at least some of those (like invTypes) have fewer in the actual dump, too. looks like some cleanup has been done. Filesize of the dump is comparble.

That's a first pass conversion of the static datadump to mysql. Tablenames are camel case, for those on case sensitive platforms.

http://www.fuzzwork.co.uk/dump/

in mysql55-cru15.sql.bz2, you have a full database dump, with drop table statements. MyISAM, as you don't need transactional support for lookup tables. no database creation/drop
in the directory cru15/ you have the sql dumps, and the xls dumps (csv doesn't handle new lines well)




if you have any trouble, let me know and I'll see what I can do.

Woo! CSM XI!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter
Zeta Zhul
Preemptive Paranoia
#2 - 2012-03-15 04:02:07 UTC
Hey thanks!
Desmont McCallock
#3 - 2012-03-15 07:49:08 UTC
I confirm that clean up has been done in DB. Also there is a new category ID [(350001, "Catma")?].
6ie
The Kiwis
#4 - 2012-03-15 11:49:36 UTC  |  Edited by: 6ie
Nice, thanks. I don't use MyISAM, so will be importing the sql one by one.

Edit: There are 2 versions of each of these. Guessing the capitalized one is the real one.

[ ] warCombatZoneSystems.sql.bz2 15-Mar-2012 00:09 1.2K Bzip2 compressed SQL dump
[ ] warcombatzonesystems.sql.bz2 15-Mar-2012 00:09 1.2K Bzip2 compressed SQL dump

[ ] warCombatZones.sql.bz2 15-Mar-2012 00:09 971 Bzip2 compressed SQL dump
[ ] warcombatzones.sql.bz2 15-Mar-2012 00:09 959 Bzip2 compressed SQL dump
Thebriwan
LUX Uls Xystus
#5 - 2012-03-15 12:05:38 UTC
Thank you very much!
Steve Ronuken
Fuzzwork Enterprises
Vote Steve Ronuken for CSM
#6 - 2012-03-15 12:54:53 UTC
6ie wrote:
Nice, thanks. I don't use MyISAM, so will be importing the sql one by one.

Edit: There are 2 versions of each of these. Guessing the capitalized one is the real one.

[ ] warCombatZoneSystems.sql.bz2 15-Mar-2012 00:09 1.2K Bzip2 compressed SQL dump
[ ] warcombatzonesystems.sql.bz2 15-Mar-2012 00:09 1.2K Bzip2 compressed SQL dump

[ ] warCombatZones.sql.bz2 15-Mar-2012 00:09 971 Bzip2 compressed SQL dump
[ ] warcombatzones.sql.bz2 15-Mar-2012 00:09 959 Bzip2 compressed SQL dump



Oops. I thought I'd tossed those lower case ones.

Yes, the camel case one is the right one.

The reason I keep them in MyISAM, is because it's generally the faster table form, when you're not having to use transactions. Sure, you can't put in foreign key relationships between them, but that shouldn't be important when it's read only. Or at least, that's what the case used to be, speed wise. Having done a little looking, it's looking like that might not be the case any more. Ugh

You /could/ get the big file, and run a global search and replace against it, replacing the MyISAM's with INNODBs. Should be pretty safe to do.


I'll probably put future conversions into Inno db.

Woo! CSM XI!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter
MrWhitei God
Sebiestor Tribe
Minmatar Republic
#7 - 2012-03-15 13:35:54 UTC
just an FYI

the data dump has Trauma missiles in as Scourge.
All other missile names seem fine.

Thats the only issue ive come across so far
Blake Armitage
The Scope
Gallente Federation
#8 - 2012-03-15 14:00:26 UTC
MrWhitei God wrote:
just an FYI

the data dump has Trauma missiles in as Scourge.
All other missile names seem fine.

Thats the only issue ive come across so far


I think this is a valid entry as far too many people complained and CCP backed off on the Trauma change for all the Drake flying minions.
MrWhitei God
Sebiestor Tribe
Minmatar Republic
#9 - 2012-03-15 19:18:28 UTC
Blake Armitage wrote:
MrWhitei God wrote:
just an FYI

the data dump has Trauma missiles in as Scourge.
All other missile names seem fine.

Thats the only issue ive come across so far


I think this is a valid entry as far too many people complained and CCP backed off on the Trauma change for all the Drake flying minions.


i just had to confirm this ingame.
Market search for scourge came up empty.
Market search for Trauma returned all the missiles.

Even in the DB dump the description references Trauma however lists the name as Scourge
NUXI7
Sebiestor Tribe
Minmatar Republic
#10 - 2012-03-15 22:57:22 UTC
Blake Armitage wrote:
MrWhitei God wrote:
just an FYI

the data dump has Trauma missiles in as Scourge.
All other missile names seem fine.

Thats the only issue ive come across so far


I think this is a valid entry as far too many people complained and CCP backed off on the Trauma change for all the Drake flying minions.


Pffft they should give my Hound back it's bane torpedos then.
Steve Ronuken
Fuzzwork Enterprises
Vote Steve Ronuken for CSM
#11 - 2012-03-15 23:18:22 UTC
The SDD itself has all the Scourge references, so it's not an artefact of the conversion.

Woo! CSM XI!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter
MrWhitei God
Sebiestor Tribe
Minmatar Republic
#12 - 2012-03-15 23:25:49 UTC
yeh, seems the SDD is messed up with regards to Launchers.

Missile launchers were to be renamed but that didnt go in the patch.
But the SDD has the new names for all the missile launchers and the Scourge/Trauma ammo.

Hopefully they either make the name changes in-game or update the datadump to reflect actual item names
Zeta Zhul
Preemptive Paranoia
#13 - 2012-03-16 00:56:14 UTC
Trauma? Scourge?

Oh for the love of .......
Crestor Markham
Sebiestor Tribe
Minmatar Republic
#14 - 2012-03-28 21:56:37 UTC
this is beyond handy. thanks!

for everyone's info, he already has crucible 1.6 up in that linked directory as well Big smile
LifeHatesMe
LifeHatesUsAll
#15 - 2012-03-29 06:17:24 UTC
Name changes... why is it dust 514 has normal, advanced, and prototype items. But we get stuck with normal afterburners, "experimental" afterburners, and "prototype" ones? :S

I wonder if this means they will change the names of Drones too.
Vex Kaycal
Hedion University
Amarr Empire
#16 - 2012-06-12 19:37:57 UTC
Hello Steve, thanks for making the excels :p , I've just checked them out, though I have notice that the dgmTypeAttributes.xls has reached the maximum rows, and can't fit anymore data? is it maybe because it .xls?
Steve Ronuken
Fuzzwork Enterprises
Vote Steve Ronuken for CSM
#17 - 2012-06-12 19:57:00 UTC
Probably. It'll be a side effect of how I'm exporting them most likely.

I've just re-exported all the tables with more than 63000 rows, as csv instead of excel. Might work better.

Woo! CSM XI!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter
Vex Kaycal
Hedion University
Amarr Empire
#18 - 2012-06-13 04:05:07 UTC
Wow! that was fast. great! Thanks Steve. Your stuffs is really helpful Cool