EVE Technology Lab

This version is deprecated. Use version 3.

Many of you know the first version of the SDEExternalsToSql tool.
With the amount of changes that are being made to the SDE and the knowledge gained by building the first version and handling the SDE, the need to improve the tool arose. As a result of those circumstances I rewrote it, optimized and generalized it, so it can handle more use cases, much faster.

As usual the source code is available at EVEMon's repository as the tool is embedded in it (look into the Tools folder).

The compiled version can be found for downloading here.

The latest version is: 2.0.1

Direct dl link: EVEMonSDEExternalsToSql-2.0.1.zip

Notice: The SQL data dump, yaml and SQLite db files are not distributed along with the tool. You will have to get them from the community toolkit page.

-- Features --

* Restores the data dump, yaml and sqlite db files back into an SQL Server (no need to restore the data dump via SSMS).
* Imports any SDE (literally any, but in case you find an SDE with issues please let me known)
* Supports command line arguments (use -help to see the list)
* Cleaner config file (it now only contains the needed connection info to your SQL Server, and it's the only thing that you may need to modify)
* Ulta-High Speed (it now takes less than 5 minutes to import the entire SDE, depending on your machine's cpu power)
* One place for the SDE file(s) (just drop the SDE zip file as is or the files contained in the SDE zip file, into the 'SDEFiles' folder)

Supported OS: Windows XP, Vista, 7, 8, 8.1, 10

Requirements

* .NET 4.0 or greater
* SQL Server instance

Yaml files mapping:

blueprints.yaml -> invBlueprintTypes, ramTypeRequirements
categoryIDs.yaml -> invCategories, translationTables, trnTranslationColumns, trnTranslations
certificates.yaml -> crtClasses, crtCertificates, crtRecommendations, crtRelationships
graphicIDs.yaml -> eveGraphics
groupIDs.yaml -> invGroups, translationTables, trnTranslationColumns, trnTranslations
iconIDs.yaml -> eveIcons
skinLicenses.yaml -> sknLicenses
skinMaterials.yaml -> sknMaterials
skins.yaml -> sknSkins
tournamentRuleSets.yaml -> ignored
typeIDs.yaml -> invTypes, dgmMasteries, dgmTypeMasteries, dgmTraits, dgmTypeTraits, translationTables, trnTranslationColumns, trnTranslations

Sqlite files mapping:

universeDataDx.db -> mapCelestialStatistics, mapConstellationJumps, mapConstellations, mapDenormalize, mapJumps, mapLandmarks, mapLocationScenes, mapLocationWormholeClasses, mapRegionJumps, mapRegions, mapSolarSystemJumps, mapSolarSystems

Known Issues:

None (so far)


Enjoy responsibly.

Not empty quoting.

Minor bug (if you can call it a bug):
If you run the tool with a working directory that is not the one with the exe file, it will fail to import universeDataDx.db, but, everything else works. It's super easy to work around ofc. :)

Run the tool with a batch file (win):
---
C:
CD \
C:\EVEMonSDEExternalsToSql-2.0.0\EVEMonSDEExternalsToSql.exe
---
That will make the working directory c: and it will fail to import universeDataDx.db (but, everything else works)

if you do:
---
C:
CD \
CD EVEMonSDEExternalsToSql-2.0.0
EVEMonSDEExternalsToSql.exe
---
It works without problems.

Release of EVEMonSDEExternalsToSql 2.0.1 that fixes the above mentioned behavior.

Hi Desmont,

Great tool. Unfortunately with the new Vanguard database I have a problem. During "import mapDenormalize" it runs to timeout.
The process is at 99%, CPU usage is still high, so the system works, it's doing somehing, just the program throws a timeout.
Can you increase the timeout in the program? Or put the timeout value in the config file, so it's possible to increase it manually?

Thanks,
Hamster

I tried to setup both
connection timeout
and
timeout

values. Tried to enter high number like 50000, and 0 (as I read 0 means infinite) the program ignored it. it took the same amount of time to timeout. (it takes about 100 sec)

There is nothing wrong with the SQL server, it's just not a normal PC, it's a VM, where I have an SQL 2012 installed. As in the end I need the database in SQL 2008 format. (Which is on my normal PC, with SSD, 8gb ram etc) There the SQLite import works like a charm.

So in the end I could put it together, on the VM importing the DB / yaml, and on the physical machine doing the import for SQLite.

Here is the console output:

Connection to SQLite 'main' Database: Successful
Importing mapCelestialStatistics... 100% in 0:01:17.7587737
Importing mapConstellationJumps... 100% in 0:00:00.3244939
Importing mapConstellations... 100% in 0:00:00.3488424

Unable to import mapDenormalize
Reason was: Timeout expired. The timeout period elapsed prior to completion of
the operation or the server is not responding.


In the trace file, there is nothing interesting:

Native library pre-loader is trying to load native SQLite library "d:\Import2.01\x64\SQLite.Interop.dll"...
SQLite error (1): no such table: __MigrationHistory
SQLite error (1): no such table: __MigrationHistory
SQLite error (1): no such table: EdmMetadata


However I have restarted the server, (and added 2gb ram (sum: 6gb) and now it works.

Holy spiced unicorn balls this tool is amazing

I just spent most of today pondering the best way to parse those abominable YAML files!

Very much appreciate both the work you've put in and the time you've saved me, will be perusing the source to see how you managed to get this thing to run so fast.

-Shady-