These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

 
  • Topic is locked indefinitely.
 

EVESDEToSQL (v3.5.0)
Author
Previous Topic Next Topic
Desmont McCallock
#21 - 2015-12-21 12:29:49 UTC
Hel O'Ween wrote:
And surely enough, that fixed it. Big smile

Thanks, Desmont.
Thanks for confirming. Smile
salacious necrosis
Garoun Investment Bank
Gallente Federation
#22 - 2015-12-24 18:14:27 UTC
Hi Des,

Used your exporter for the first time today. Everything worked well except for SQLite import. To get that to work I had to copy x86/SQLite.Interop.dll to the main EVESDEToSQL directory. Otherwise, the importer fails with a trace.txt like this:

Quote:

EVESDEToSQL: Started
Native library pre-loader is trying to load native SQLite library "C:\Users\mark_000\projects\sdeimport\EVESDEToSQL\x86\SQLite.Interop.dll"...
Native library pre-loader is trying to load native SQLite library "C:\Users\mark_000\projects\sdeimport\EVESDEToSQL\x86\SQLite.Interop.dll"...
System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.DllNotFoundException: Unable to load DLL 'SQLite.Interop.dll': The specified module could not be found. (Exception from HRESULT: 0x8007007E)
at System.Data.SQLite.UnsafeNativeMethods.sqlite3_config_none(SQLiteConfigOpsEnum op)
at System.Data.SQLite.SQLite3.StaticIsInitialized()
at System.Data.SQLite.SQLiteLog.Initialize()
at System.Data.SQLite.SQLiteConnection..ctor(String connectionString, Boolean parseViaFramework)
at System.Data.SQLite.SQLiteConnection..ctor(String connectionString)
--- End of inner exception stack trace ---
at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at EVESDEToSQL.Providers.DbConnectionProvider.CreateConnection(Type connectionType, String nameOrConnectionString) in EVESDEToSQL\Providers\DbConnectionProvider.cs:line 97
at EVESDEToSQL.Providers.SqliteConnectionProvider..ctor(String nameOrConnectionString) in EVESDEToSQL\Providers\SqliteConnectionProvider.cs:line 22
at EVESDEToSQL.Importer.ImportSqlite() in EVESDEToSQL\Importer.cs:line 185
at EVESDEToSQL.Importer.ImportSDEFiles(String[] args) in EVESDEToSQL\Importer.cs:line 135
at EVESDEToSQL.Importer.Start(String[] args) in EVESDEToSQL\Importer.cs:line 90

System.DllNotFoundException: Unable to load DLL 'SQLite.Interop.dll': The specified module could not be found. (Exception from HRESULT: 0x8007007E)
at System.Data.SQLite.UnsafeNativeMethods.sqlite3_config_none(SQLiteConfigOpsEnum op)
at System.Data.SQLite.SQLite3.StaticIsInitialized()
at System.Data.SQLite.SQLiteLog.Initialize()
at System.Data.SQLite.SQLiteConnection..ctor(String connectionString, Boolean parseViaFramework)
at System.Data.SQLite.SQLiteConnection..ctor(String connectionString)


My box is x64 but it didn't try to load from that directory at all. Anyway, happy to help debug further if you provide instructions.

Use EveKit ! - Tools for EVE Online 3rd party development
Desmont McCallock
#23 - 2015-12-25 13:13:10 UTC  |  Edited by: Desmont McCallock
Although I suspect it's mostly an SQLite issue I'll surely look into it.

It doesn't matter if your box is x64. The app is set to run as x86 (to provide support fro MS Access mdb files).

Never the less, I would much appreciate if you could tell me what OS your box is running on.
Desmont McCallock
#24 - 2015-12-25 13:14:22 UTC
Is there any interest in providing an SQL CE (SQL Server Compact 4) dump?
salacious necrosis
Garoun Investment Bank
Gallente Federation
#25 - 2015-12-25 18:34:07 UTC
Desmont McCallock wrote:
Although I suspect it's mostly an SQLite issue I'll surely look into it.

It doesn't matter if your box is x64. The app is set to run as x86 (to provide support fro MS Access mdb files).

Never the less, I would much appreciate if you could tell me what OS your box is running on.


Running Windows 8.1 Pro, with .NET 4.0.30319

I don't have any of the SQLite dll's in my path except for the those included in EVESDEToSQL. I DO have the sqlite3 CLI binary in my path, but I don't think that should affect anything.

Use EveKit ! - Tools for EVE Online 3rd party development
Desmont McCallock
#26 - 2015-12-26 14:08:12 UTC  |  Edited by: Desmont McCallock
So I checked the tool again and I can't find anything wrong with it.

The only way I could reproduce the exact exception you are reporting, is to rename the 'x64' folder to 'x86', to fool the app so it tries to load the 64bit version of the interop library.

So for some reason on your machine the tool although it says it's looking into 'x86' it tries to load the 64bit dll or for some unknown reason to me the interop dlls got mixed up.

I suggest you delete the folder where the tool currently resides and re-download it.

Just for the reference here is the Dynamic-Link Library Search Order that Windows uses.

  1. The directory from which the application loaded.
  2. The system directory. Use the GetSystemDirectory function to get the path of this directory.
  3. The 16-bit system directory. There is no function that obtains the path of this directory, but it is searched.
  4. The Windows directory. Use the GetWindowsDirectory function to get the path of this directory.
  5. The current directory.
  6. The directories that are listed in the PATH environment variable. Note that this does not include the per-application path specified by the App Paths registry key.
Hel O'Ween
Men On A Mission
#27 - 2015-12-28 11:10:30 UTC
I'm very new to this .NET stuff, so I might be totally of here, but I remember something about assemblies being registered in the GAC ... might something like this cause the lookup order to not working as expected?

EVEWalletAware - an offline wallet manager.
Desmont McCallock
#28 - 2015-12-28 12:41:42 UTC
Hel O'Ween wrote:
I'm very new to this .NET stuff, so I might be totally of here, but I remember something about assemblies being registered in the GAC ... might something like this cause the lookup order to not working as expected?
You are correct Hel O'Ween. It's a possibility.
Desmont McCallock
#29 - 2016-02-05 11:46:08 UTC  |  Edited by: Desmont McCallock
Released version 3.2.0.

ChangeLog
Updated SQLite nuget package.
Updated YamlDotNet nuget package.
Included missing SMO assembly (was though available in GAC if SQL Server was installed on the machine).
Grenval Truvaki
Center for Advanced Studies
Gallente Federation
#30 - 2016-03-01 18:14:26 UTC
Hi Desmont,

This tool seems really useful to me! However i've run into a bit of an issue.

Reason was: System.Data.SqlClient.SqlError: Cannot open backup device 'C:\Users\
Simon\Downloads\EVESDEToSQL-3.2.0\SDEFiles\DATADUMP201602091443.bak'. Operating
system error 3(The system cannot find the path specified.).

Press any key to exit.

Any idea's to what I've done wrong? haha
Desmont McCallock
#31 - 2016-03-01 20:11:10 UTC  |  Edited by: Desmont McCallock
Grenval Truvaki wrote:
Hi Desmont,

This tool seems really useful to me! However i've run into a bit of an issue.

Reason was: System.Data.SqlClient.SqlError: Cannot open backup device 'C:\Users\
Simon\Downloads\EVESDEToSQL-3.2.0\SDEFiles\DATADUMP201602091443.bak'. Operating
system error 3(The system cannot find the path specified.).

Press any key to exit.

Any idea's to what I've done wrong? haha

Have you followed instructions from OP?
See also https://bitbucket.org/Desmont_McCallock/evesdetosql
Grenval Truvaki
Center for Advanced Studies
Gallente Federation
#32 - 2016-03-01 21:18:09 UTC
Desmont McCallock wrote:
Grenval Truvaki wrote:
Hi Desmont,

This tool seems really useful to me! However i've run into a bit of an issue.

Reason was: System.Data.SqlClient.SqlError: Cannot open backup device 'C:\Users\
Simon\Downloads\EVESDEToSQL-3.2.0\SDEFiles\DATADUMP201602091443.bak'. Operating
system error 3(The system cannot find the path specified.).

Press any key to exit.

Any idea's to what I've done wrong? haha

Have you followed instructions from OP?
See also https://bitbucket.org/Desmont_McCallock/evesdetosql


Yes I have, however I think my database is missing some tables now. I'm doing a fresh run of the SDE on my MS SQL server.

Right now i've managed to get this far:

C:\Users\Simon>\Downloads\EVESDEToSQL-3.2.0\EVEsdetosql import -no
restore
Unzipping SDE zip file: Successful

Connection to SQL Server 'DB_9F457E_projectDB' database: Successful

Parsing categoryIDs.yaml...
Unable to execute SQL command: UPDATE translationTables SET tcGroupID = 4, tcID
= 6, sourceTable = 'inventory.categoriesTx', destinationTable = 'dbo.invCategori
es', translatedKey = 'categoryName' WHERE sourceTable = 'inventory.categoriesTx'
AND translatedKey = 'categoryName'
Invalid object name 'translationTables'.

Press any key to exit.
Grenval Truvaki
Center for Advanced Studies
Gallente Federation
#33 - 2016-03-01 22:29:12 UTC
I should also mention when I try to apply the bak file on my server I'm given this error in response:

Error: The media family on device 'c:\db_temp\datadump201602091443.bak' is incorrectly formed. SQL Server cannot process this media family.
Desmont McCallock
#34 - 2016-03-02 08:38:20 UTC  |  Edited by: Desmont McCallock
Grenval Truvaki wrote:
I should also mention when I try to apply the bak file on my server I'm given this error in response:

Error: The media family on device 'c:\db_temp\datadump201602091443.bak' is incorrectly formed. SQL Server cannot process this media family.

Ahh yes. CCP said that the SDE MSSQL backup file is created with SQL 2014. SQL instances prior to v2014 will not be able to restore the backup file.

Edit: I updated the OP requirements so it's clearer.
Grenval Truvaki
Center for Advanced Studies
Gallente Federation
#35 - 2016-03-02 13:45:43 UTC  |  Edited by: Grenval Truvaki
Desmont McCallock wrote:
Grenval Truvaki wrote:
I should also mention when I try to apply the bak file on my server I'm given this error in response:

Error: The media family on device 'c:\db_temp\datadump201602091443.bak' is incorrectly formed. SQL Server cannot process this media family.

Ahh yes. CCP said that the SDE MSSQL backup file is created with SQL 2014. SQL instances prior to v2014 will not be able to restore the backup file.

Edit: I updated the OP requirements so it's clearer.


Thank you for the help :) Don't suppose you know of any way around this? I'm stuck using MS SQL 2012.

EDIT: I've managed to update my server after talking to my server provider, however, nothing is every easy heh. I've been given another error now :(

_________________________________________________________________________
Unable to import trnTranslations
Reason was: Timeout expired. The timeout period elapsed prior to completion of
the operation or the server is not responding.

Press any key to exit.
_________________________________________________________________________
Desmont McCallock
#36 - 2016-03-02 16:15:11 UTC  |  Edited by: Desmont McCallock
Set the connection timeout in the connectionstring in the config ( http://www.connectionstrings.com/all-sql-server-connection-string-keywords/ ).
Grenval Truvaki
Center for Advanced Studies
Gallente Federation
#37 - 2016-03-02 16:28:55 UTC  |  Edited by: Grenval Truvaki
Desmont McCallock wrote:
Set the connection timeout in the connectionstring in the config ( http://www.connectionstrings.com/all-sql-server-connection-string-keywords/ ).


I gave a two minute timeout on the connectionstring, it's still hanging at 50% when trying to import typeIDs.yaml

UPDATE:

So after giving the string a whole 20 mins before it would timeout :P it's finally done, thank you very much for your help and for this tool!
Desmont McCallock
#38 - 2016-03-03 12:10:03 UTC
Grenval Truvaki wrote:
Desmont McCallock wrote:
Set the connection timeout in the connectionstring in the config ( http://www.connectionstrings.com/all-sql-server-connection-string-keywords/ ).


I gave a two minute timeout on the connectionstring, it's still hanging at 50% when trying to import typeIDs.yaml

UPDATE:

So after giving the string a whole 20 mins before it would timeout :P it's finally done, thank you very much for your help and for this tool!
From what I understood your SQL Server is cloud based. So depending on your upload speed you had to adjust the connection timeout accordingly.
m Light
I N E X T R E M I S
Tactical Narcotics Team
#39 - 2016-03-16 22:39:28 UTC
o7
ty

Importing SQL data dump to 'EveStaticData' database... 100 % in 0:00:44.285446

Connection to SQL Server 'EveStaticData' database: Successful

Importing categoryIDs.yaml... 100 % in 0:00:00.4530976
Importing groupIDs.yaml... 100 % in 0:00:00.356952
Importing graphicIDs.yaml... 100 % in 0:00:00.3333244
Importing iconIDs.yaml... 100 % in 0:00:00.1151018
Importing skins.yaml... 100 % in 0:00:00.1637418
Importing skinMaterials.yaml... 100 % in 0:00:00.2591509
Importing skinLicenses.yaml... 100 % in 0:00:05.6676665
Importing typeIDs.yaml... 100 % in 0:00:44.0759559
Importing certificates.yaml... 100 % in 0:00:00.73331
Importing blueprints.yaml... 100 % in 0:00:05.4793214

Connection to SQLite 'main' database: Successful

Importing mapCelestialStatistics... 100 % in 0:00:34.3261924
Importing mapConstellationJumps... 100 % in 0:00:00.0502679
Importing mapConstellations... 100 % in 0:00:00.048732
Importing mapDenormalize... 100 % in 0:02:07.2610279
Importing mapJumps... 100 % in 0:00:00.1278917
Importing mapLandmarks... 100 % in 0:00:00.0177527
Importing mapLocationScenes... 100 % in 0:00:00.0160912
Importing mapLocationWormholeClasses... 100 % in 0:00:00.0200616
Importing mapRegionJumps... 100 % in 0:00:00.0173105
Importing mapRegions... 100 % in 0:00:00.0236341
Importing mapSolarSystemJumps... 100 % in 0:00:00.2487416
Importing mapSolarSystems... 100 % in 0:00:00.3020079

Disconnection from SQLite 'main' database: Successful

Reorganizing indexes of database 'EveStaticData'... 100 % in 0:00:36.4269115

Shrinking database 'EveStaticData'... in 0:00:13.9365718

Disconnection from SQL Server 'EveStaticData' database: Successful


Importing files completed in 0:05:15.5000529

100% == 100 mil ISK ty
Desmont McCallock
#40 - 2016-03-17 08:58:42 UTC
Glad you found it useful. Thank you.