These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

Forum Index

EVE Forums » EVE Technology and Research Center » EVE Technology Lab » The SDE for Galatea 1.0 is now available

Topic is locked indefinitely.

The SDE for Galatea 1.0 is now available First post First post
Author

Previous Topic Next Topic

C C P C C P Alliance 44
CCP Creber Cattus C C P C C P Alliance Likes received: 44	#1 - 2015-08-24 14:32:58 UTC 1 https://developers.eveonline.com/resource/static-data-export ( http://cdn1.eveonline.com/data/Galatea_1.0_114618_db.zip )

655
Desmont McCallock Likes received: 655	#2 - 2015-08-24 14:59:42 UTC \| Edited by: Desmont McCallock Thanks for the heads up. Lets hope that this time the groupIDs and categoryIDs yaml files are in there (yeap they are ). Ahhh also new yaml file... tournamentRuleSets.yaml.

State War Academy Caldari State 657
Uppsy Daisy State War Academy Caldari State Likes received: 657	#3 - 2015-08-24 15:35:37 UTC I have been away from this for a few months. The invTypes table is not in the database. Has it been replaced by the YAML file?


Desmont McCallock Likes received: 655	#4 - 2015-08-24 15:36:26 UTC Uppsy Daisy wrote: I have been away from this for a few months. The invTypes table is not in the database. Has it been replaced by the YAML file? Since Aegis.


Desmont McCallock Likes received: 655	#5 - 2015-08-24 15:47:54 UTC @Steve Ronuken Do you have any idea what's up with this tournamentRuleSets.yaml file?

Fuzzwork Enterprises Vote Steve Ronuken for CSM 6,759
Steve Ronuken Fuzzwork Enterprises Vote Steve Ronuken for CSM Likes received: 6,759	#6 - 2015-08-24 17:32:50 UTC I've not looked at it yet That'll come when I get home. Then I'll run my regular (new and improved, using python) conversion process and poke around at anything new. (yay for security guards not turning up until late, and having to stick around because I'm a keyholder) Woo! CSM XI! Fuzzwork Enterprises Twitter: @fuzzysteve on Twitter

Steve Ronuken

Fuzzwork Enterprises

Vote Steve Ronuken for CSM

Likes received: 6,759

#7 - 2015-08-24 21:03:26 UTC | Edited by: Steve Ronuken

Having looked at it, I'd just ignore it. It's very niche data. Banned ships (totally banned. like the Nestor), and the costs for ships by group or typeid, in the current AT.

I'm working up the conversion now, and should have a release soon. It's based off my new python conversion process (including the universe data) so there may be artefacts which need corrected. The main advantage is, once it's smoothed out, it's a case of 'load all data into the MSSQL db, then convert to mysql. done.' Smile

Increases the bus factor, which is always a good thing. (The bus factor is why I started converting in the first place. I was depending on someone else, who got slower and slower after releases. So I worked up my own process, so I didn't need to wait.)

Woo! CSM XI!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter

Steve Ronuken

Fuzzwork Enterprises

Vote Steve Ronuken for CSM

Likes received: 6,759

#8 - 2015-08-25 00:31:49 UTC

mysql
https://www.fuzzwork.co.uk/dump/mysql56-galatea-1.114618.tbz2
sqlite
https://www.fuzzwork.co.uk/dump/galatea-1.114618/eve.db.bz2
Postgres
https://www.fuzzwork.co.uk/dump/galatea-1.114618/postgres-galatea-1.114618.dmp.bz2
MS SQL
https://www.fuzzwork.co.uk/dump/galatea-1.114618/galatea-1.114618.bak.bz2
other stuff
https://www.fuzzwork.co.uk/dump/galatea-1.114618/

Conversion was done with:
https://github.com/fuzzysteve/SDE-loaders

The mysql migration toolkit
A mysql > sqlite conversion script
pgloader (from mysql)

It's getting there. Smile

Some datatypes may have changed a bit, as the conversion process is different. Just be aware. Translation data is loaded back into trnTranslations.

The scripts on github load everything into an SQL server DB. (that's why there's an mssql db backup there now)

Woo! CSM XI!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter

Desmont McCallock

Likes received: 655

#9 - 2015-08-25 06:39:49 UTC

Steve Ronuken wrote:

1. I'm ignoring it also.
2. Ah yes the bus factor.One of the things I constantly was nagging to my former boss, when he was trying to assign me more responsibilities.

Manda Nudis 17
Mona Kasenumi Manda Nudis Likes received: 17	#10 - 2015-08-25 09:36:31 UTC \| Edited by: Mona Kasenumi I tried to download it a few times but the file seem to be corrupted. I'm getting "unexpected EOF" and I can't unpack it (at least using winrar or windows explorer).

Mona Kasenumi

Manda Nudis

Likes received: 17

#11 - 2015-08-25 10:56:18 UTC

Just found out the issue: CDN is ending the download prematurely for some reason.
I had to run wget multiple times to get a working version.

wget http://cdn1.eveonline.com/data/Galatea_1.0_114618_db.zip
converted 'http://cdn1.eveonline.com/data/Galatea_1.0_114618_db.zip' (ANSI_X3.4-1968) -> 'http://cdn1.eveonline.com/data/Galatea_1.0_114618_db.zip' (UTF-8)
--2015-08-25 05:49:46-- http://cdn1.eveonline.com/data/Galatea_1.0_114618_db.zip
Resolving cdn1.eveonline.com (cdn1.eveonline.com)... 54.230.0.73, 54.230.0.107, 54.230.0.24, ...
Connecting to cdn1.eveonline.com (cdn1.eveonline.com)|54.230.0.73|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 114416535 (109M) [application/zip]
Saving to: 'Galatea_1.0_114618_db.zip.4'

Galatea_1.0_114618_db.zi 5%[> ] 6.05M 14.5MB/s in 0.4s

2015-08-25 05:49:47 (14.5 MB/s) - Connection closed at byte 6344412. Retrying.

--2015-08-25 05:49:48-- (try: 2) http://cdn1.eveonline.com/data/Galatea_1.0_114618_db.zip
Connecting to cdn1.eveonline.com (cdn1.eveonline.com)|54.230.0.73|:80... connected.
HTTP request sent, awaiting response... 416 Requested Range Not Satisfiable

The file is already fully retrieved; nothing to do.


Steve Ronuken Fuzzwork Enterprises Vote Steve Ronuken for CSM Likes received: 6,759	#12 - 2015-08-25 11:42:25 UTC Probably related to the other CDN problems CCP's been suffering from yesterday and today. One alternate option you have is to grab the 'updated' backup from mine (has all the yaml and sqlite db rolled into it) Woo! CSM XI! Fuzzwork Enterprises Twitter: @fuzzysteve on Twitter


Desmont McCallock Likes received: 655	#13 - 2015-08-25 11:46:43 UTC \| Edited by: Desmont McCallock Indeed there are issues with the CDN. Posting mirror link.

Ydnari

Estrale Frontiers

Likes received: 435

#14 - 2015-08-25 21:07:45 UTC

Steve Ronuken wrote:

Hmm - industryActivityMaterials.consume appears to have disappeared.

Steve Ronuken

Fuzzwork Enterprises

Vote Steve Ronuken for CSM

Likes received: 6,759

#15 - 2015-08-25 22:16:14 UTC

Ydnari wrote:

Steve Ronuken wrote:

Hmm - industryActivityMaterials.consume appears to have disappeared.

It stopped being in the SDE a while ago. I just had it auto fill with 1.

With the change in process, I didn't bother filling it in again.

Woo! CSM XI!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter

Garoun Investment Bank Gallente Federation 38
salacious necrosis Garoun Investment Bank Gallente Federation Likes received: 38	#16 - 2015-08-26 02:43:31 UTC Hey Steve, Looks like none of the skin tables are populated this time around (although the schemas still exist). On purpose? It's not difficult to populate from the YAML, just curious if you intended this. sal Use EveKit ! - Tools for EVE Online 3rd party development

Steve Ronuken

Fuzzwork Enterprises

Vote Steve Ronuken for CSM

Likes received: 6,759

#17 - 2015-08-26 12:17:28 UTC

salacious necrosis wrote:

Hey Steve,

Looks like none of the skin tables are populated this time around (although the schemas still exist). On purpose? It's not difficult to populate from the YAML, just curious if you intended this.

sal

Nope. Turns out with an edit I did because the materials file changed, it didn't actually commit anything (Not sure why, tbh. should have done).

I'll be releasing those files again tonight. (not a full release though)

Woo! CSM XI!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter

Golden Gnu

Lobach Inc.

Likes received: 212

#18 - 2015-08-27 15:11:04 UTC

@Steve Ronuken
Looks like there is UTF-8 errors in your MySQL conversion:
Ex.:
invTypes->TypeID: 34457
Should be: 末日沙场级YC117年特别版"
Is: ?????YC117????
My database is: utf8_general_ci (I don't know if that matters)
Imported with: mysql.exe on win10 (Also, I don't know if that matters - always worked before)

As, always, thanks for the conversion. One hell of a timesaver o7

Creator of jEveAssets - the asset manager

"Download is the meaning of life, upload is the meaning of intelligent life"


Desmont McCallock Likes received: 655	#19 - 2015-08-27 15:44:50 UTC Steve, typeName and description are nvarchar not just varchar. I made the same mistake with my tool.


Steve Ronuken Fuzzwork Enterprises Vote Steve Ronuken for CSM Likes received: 6,759	#20 - 2015-08-27 16:41:04 UTC Desmont McCallock wrote: Steve, typeName and description are nvarchar not just varchar. I made the same mistake with my tool. bah. Thought I'd caught all of that. I'll see about kicking it some more, and see what I can get done. Woo! CSM XI! Fuzzwork Enterprises Twitter: @fuzzysteve on Twitter

12Next page

EVE Technology Lab

C C P

C C P Alliance

State War Academy

Caldari State

Fuzzwork Enterprises

Vote Steve Ronuken for CSM

Manda Nudis

Estrale Frontiers

Garoun Investment Bank

Gallente Federation

Lobach Inc.