These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

 
  • Topic is locked indefinitely.
 

Aegis SDE conversion

First post
Author
Previous Topic Next Topic
Steve Ronuken
Fuzzwork Enterprises
Vote Steve Ronuken for CSM
#1 - 2015-07-09 20:40:18 UTC
Sorry for the delay folks:

mysql https://www.fuzzwork.co.uk/dump/mysql56-aegis-1.1.1-114255.tbz2
postgres https://www.fuzzwork.co.uk/dump/aegis-1.1.1-114255/postgres-aegis-1.114255.sql.bz2
sqlite https://www.fuzzwork.co.uk/dump/aegis-1.1.1-114255/eve.db.bz2
everything else https://www.fuzzwork.co.uk/dump/aegis-1.1.1-114255/

invGroups, invCategories, invTypes, trnTranslations have been repopulated from the yaml. no descriptions for invGroups and invCategories.

Woo! CSM XI!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter
Steve Ronuken
Fuzzwork Enterprises
Vote Steve Ronuken for CSM
#2 - 2015-07-09 20:54:45 UTC
bah. looks like there's a screwup in the invtypes load. I'll look into it.

Woo! CSM XI!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter
Desmont McCallock
#3 - 2015-07-09 21:03:03 UTC
This might be related.
Steve Ronuken
Fuzzwork Enterprises
Vote Steve Ronuken for CSM
#4 - 2015-07-09 21:33:51 UTC
Desmont McCallock wrote:
This might be related.



Nope. It was a screwup with published, and a silent fail on insert. (no, you can't insert "false" into a tinyint Sad )

On the other hand, all sorted now.


I've just updated it in place.

Woo! CSM XI!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter
Icy One
The Executives
#5 - 2015-07-09 22:15:44 UTC
The published field still seems wrong. There are published items but I think the 1 and 0 are the wrong way round from my initial digging.
Selaria Unbertable
Bellator in Capsulam
#6 - 2015-07-11 11:49:26 UTC
o/ Steve,

your efforts are appreciated as usual.
I think you forgot to include the invVolumes table you added in your last conversion though.
Steve Ronuken
Fuzzwork Enterprises
Vote Steve Ronuken for CSM
#7 - 2015-07-11 17:36:06 UTC
Selaria Unbertable wrote:
o/ Steve,

your efforts are appreciated as usual.
I think you forgot to include the invVolumes table you added in your last conversion though.



Sad

https://github.com/fuzzysteve/yamlloader/blob/master/invVolumes2.sql

I rebuilt it a few times, due to issues with the invTypes conversion. This is the SQL to build the table.

oops.

Woo! CSM XI!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter
Steve Ronuken
Fuzzwork Enterprises
Vote Steve Ronuken for CSM
#8 - 2015-07-11 17:37:00 UTC
Icy One wrote:
The published field still seems wrong. There are published items but I think the 1 and 0 are the wrong way round from my initial digging.



This was the case (ish.)

It was corrected an hour or so after your post.

Woo! CSM XI!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter
Golden Gnu
Lobach Inc.
#9 - 2015-07-13 12:57:42 UTC  |  Edited by: Golden Gnu
Looks like invCategories are missing some entries.
All the entries with "published = false" are not included.
They're included in the yaml file.
Ex.
0 = System
3 = Station
etc.

An updated version would be very much appreciated.

Thank you for doing all the work for all of us. :)

Creator of jEveAssets - the asset manager

"Download is the meaning of life, upload is the meaning of intelligent life"
Steve Ronuken
Fuzzwork Enterprises
Vote Steve Ronuken for CSM
#10 - 2015-07-13 17:24:34 UTC
Golden Gnu wrote:
Looks like invCategories are missing some entries.
All the entries with "published = false" are not included.
They're included in the yaml file.
Ex.
0 = System
3 = Station
etc.

An updated version would be very much appreciated.

Thank you for doing all the work for all of us. :)



There's an updated invCategories.sql file in the directory.

( https://github.com/fuzzysteve/SDE-loaders/blob/master/importCategories.py may be of interest too, as an alternate way to populate it, that I'm seriously considering for the next release)

Woo! CSM XI!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter
Thales Severasse
Strategic Exploration and Development Corp
Silent Company
#11 - 2015-07-14 16:51:53 UTC
Steve,

First of all, thanks! Much appreciated.

Just one thing I found with the MySQL version.

Several of the names (such as regionName in mapRegions and solarSystemName in mapSolarSystems) are set to LONGTEXT.

This causes an error with some versions of MySQL, when using mysql/mysqli to acces them. It tries to allocate all the possible memory it could possibly need for a LONGTEXT (even though it only needs a few bytes)... which is more than PHP has allocated to it. And then crashes.

PDO works fine, though.

Also, changing the type to TEXT works as well. I can't imagine anything would have a name longer than would fit in TEXT.

Not a huge problem, just thought I'd mention it.
Steve Ronuken
Fuzzwork Enterprises
Vote Steve Ronuken for CSM
#12 - 2015-07-14 17:17:15 UTC
the long text bit is an artefact of the conversion process. I'm using the mysql workbench migration tool to pull it over, and it only kinda supports sqlite. It gets confused by bit, and long.

As for the driver, if you can use PDO, I highly recommend using it. Mostly for the ability to use parameters. They don't completely eliminate sql injection, but they come damn close.

Woo! CSM XI!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter
Aineko Macx
#13 - 2015-07-14 17:58:23 UTC
Mysqlnd (supersedes mysqli and default since PHP 5.4) isn't affected by this LONGTEXT issue. And even with vanilla mysqli you can do prepared statements, circumventing the problem.

iveeCore 3.0: The PHP engine for industrial activities and CREST library