These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

 
  • Topic is locked indefinitely.
 

December release SDE upfront?

First post First post
Author
Previous Topic Next Topic
Desmont McCallock
#1 - 2015-12-04 14:39:56 UTC
Looking at the patch notes for the release of Dec 8th, I realized that it's crucial that we get the SDE (at least a preview) by Dec 7th, in order to work on the changes that have be done on our tools and apps.

In hope that someone from CCP staff hear my plead.
CCP FoxFour
C C P
C C P Alliance
#2 - 2015-12-04 23:20:45 UTC
Desmont McCallock wrote:
Looking at the patch notes for the release of Dec 8th, I realized that it's crucial that we get the SDE (at least a preview) by Dec 7th, in order to work on the changes that have be done on our tools and apps.

In hope that someone from CCP staff hear my plead.


Steve pointed this out to me just a few hours ago but it's a bit late today for me to get it (need to poke a few people) so I will try and get it out ASAP on Monday.

@CCP_FoxFour // Technical Designer // Team Tech Co

Third-party developer? Check out the official developers site for dev blogs, resources, and more.
Desmont McCallock
#3 - 2015-12-05 07:58:12 UTC
CCP FoxFour wrote:
Desmont McCallock wrote:
Looking at the patch notes for the release of Dec 8th, I realized that it's crucial that we get the SDE (at least a preview) by Dec 7th, in order to work on the changes that have be done on our tools and apps.

In hope that someone from CCP staff hear my plead.


Steve pointed this out to me just a few hours ago but it's a bit late today for me to get it (need to poke a few people) so I will try and get it out ASAP on Monday.
Thanks for your concern.
CCP FoxFour
C C P
C C P Alliance
#4 - 2015-12-07 09:07:15 UTC
Frostline SDE is up: https://developers.eveonline.com/resource/resources

@CCP_FoxFour // Technical Designer // Team Tech Co

Third-party developer? Check out the official developers site for dev blogs, resources, and more.
Altrue
Exploration Frontier inc
Tactical-Retreat
#5 - 2015-12-07 10:26:50 UTC
CCP FoxFour wrote:
Frostline SDE is up: https://developers.eveonline.com/resource/resources


So the "December Release" will be called "Frostline"! Heard it there first folks! (Secret so well guarded that the Patch notes itself doesn't mention the name, and therefore doesn't appear in the drop-down menu)

Signature Tanking Best Tanking

[Ex-F] CEO - Eve-guides.fr

Ultimate Citadel Guide - 2016 EVE Career Chart
Evanova Android
Traquenard Labs
#6 - 2015-12-07 12:57:36 UTC
CCP FoxFour wrote:
Frostline SDE is up: https://developers.eveonline.com/resource/resources


That is really awesome. Fuzz's data conversions. has also been updated already.

It really helps to have the SDE in advance. Thanks a lot for that.

Evanova - The Android App for Eve Online
CCP FoxFour
C C P
C C P Alliance
#7 - 2015-12-07 13:42:42 UTC
Have been informed about a problem with the invFlags table. Will update the SDE when a fix is submitted by the appropriate team.

@CCP_FoxFour // Technical Designer // Team Tech Co

Third-party developer? Check out the official developers site for dev blogs, resources, and more.
Hel O'Ween
Men On A Mission
#8 - 2015-12-07 17:51:31 UTC
While your at it, could you bring the ZIP's name in line with the previous SDE dumps if you put up the new one? It's missing the "_db" at the end.

EVEWalletAware - an offline wallet manager.
Zifrian
The Frog Pond
Ribbit.
#9 - 2015-12-08 00:07:39 UTC  |  Edited by: Zifrian
Link isn't working for me. What's the correct address? I tried different versions and it didn't work.

Edit: Assume this is why it's down for me?

CCP FoxFour wrote:
Have been informed about a problem with the invFlags table. Will update the SDE when a fix is submitted by the appropriate team.


Assuming this won't be updated until tomorrow morning. Anyone got a copy saved somewhere that I can download? What?

Maximze your Industry Potential! - Download EVE Isk per Hour!

Import CCP's SDE - EVE SDE Database Builder
Steve Ronuken
Fuzzwork Enterprises
Vote Steve Ronuken for CSM
#10 - 2015-12-08 01:41:17 UTC
Zifrian wrote:
Link isn't working for me. What's the correct address? I tried different versions and it didn't work.

Edit: Assume this is why it's down for me?

CCP FoxFour wrote:
Have been informed about a problem with the invFlags table. Will update the SDE when a fix is submitted by the appropriate team.


Assuming this won't be updated until tomorrow morning. Anyone got a copy saved somewhere that I can download? What?



I don't have a clean copy of it. But there is https://www.fuzzwork.co.uk/dump/frostline-1.0-116241/Frostline.bak.bz2 which is the one I loaded up with the other tables?

Woo! CSM XI!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter
Zifrian
The Frog Pond
Ribbit.
#11 - 2015-12-08 02:12:15 UTC
Steve Ronuken wrote:
Zifrian wrote:
Link isn't working for me. What's the correct address? I tried different versions and it didn't work.

Edit: Assume this is why it's down for me?

CCP FoxFour wrote:
Have been informed about a problem with the invFlags table. Will update the SDE when a fix is submitted by the appropriate team.


Assuming this won't be updated until tomorrow morning. Anyone got a copy saved somewhere that I can download? What?



I don't have a clean copy of it. But there is https://www.fuzzwork.co.uk/dump/frostline-1.0-116241/Frostline.bak.bz2 which is the one I loaded up with the other tables?

Thanks Steve but your schema is different than mine on the industry stuff from yaml. I'm one of those weirdos that made his own importer heh.

Appreciate the link though, I'll just get it up tomorrow.

Maximze your Industry Potential! - Download EVE Isk per Hour!

Import CCP's SDE - EVE SDE Database Builder
Desmont McCallock
#12 - 2015-12-08 08:12:07 UTC
The problem with invFlags isn't hard to fix on your own. IDs 157-163 have the text in place of the name and vices versa. It's just a matter of data consistency.
Hel O'Ween
Men On A Mission
#13 - 2015-12-08 09:23:56 UTC  |  Edited by: Hel O'Ween
Desmont, I've encountered a strange issue when importing the SDE: using the current version of EVESDEToSQL, when processing mapDenormailze it proceeds up to 100% hangs there for a while and then throws an error

"Unable to import mapDenormalize. Reason was: Timeout expired. The timeout has been exceeded before the end of the current operation or the server doesn't respond" (Translated from the German error message).

Using the last available version of EVEMonSDEExternalsToSql does the job just fine.

[Added]
Just spotted the file trace.txt in the folder. Seems to be related:

Native library pre-loader is trying to load native SQLite library "C:\EVE\EVEMonSDEExternalsToSql\x64\SQLite.Interop.dll"...
SQLite error (1): no such table: __MigrationHistory
SQLite error (1): no such table: __MigrationHistory
SQLite error (1): no such table: EdmMetadata

EVEWalletAware - an offline wallet manager.
Desmont McCallock
#14 - 2015-12-08 10:00:13 UTC  |  Edited by: Desmont McCallock
Are you using a VM? Can you increase the memory? EVESDEToSQL runs as 32bit while the old EVEMonSDEExternalsToSQL run as 64bit. It all comes down to memory allocation limit.
Hel O'Ween
Men On A Mission
#15 - 2015-12-08 12:05:43 UTC
Desmont McCallock wrote:
Are you using a VM?


No

Quote:
Can you increase the memory? EVESDEToSQL runs as 32bit while the old EVEMonSDEExternalsToSQL run as 64bit. It all comes down to memory allocation limit.


Erhm ...I'm running it on Win7 64-Bit with 8GB RAM ... not sure how to increase RAM for a 32-Bit process there ...

P.S. Sorry for posting in the wrong thread, BTW. My bad. Oops

EVEWalletAware - an offline wallet manager.
Desmont McCallock
#16 - 2015-12-08 12:20:11 UTC  |  Edited by: Desmont McCallock
Hel O'Ween wrote:
Desmont McCallock wrote:
Are you using a VM?


No

Quote:
Can you increase the memory? EVESDEToSQL runs as 32bit while the old EVEMonSDEExternalsToSQL run as 64bit. It all comes down to memory allocation limit.


Erhm ...I'm running it on Win7 64-Bit with 8GB RAM ... not sure how to increase RAM for a 32-Bit process there ...

P.S. Sorry for posting in the wrong thread, BTW. My bad. Oops
Check that your SQL Server instance allows enough memory to be allocated. (Right click on instance > Properties > Memory > Max server memory).

It's OK, this is a Desmont thread so questions to Desmont are allowed. Big smile
Hel O'Ween
Men On A Mission
#17 - 2015-12-08 17:44:14 UTC
Desmont McCallock wrote:
Hel O'Ween wrote:
Desmont McCallock wrote:
Are you using a VM?


No

Quote:
Can you increase the memory? EVESDEToSQL runs as 32bit while the old EVEMonSDEExternalsToSQL run as 64bit. It all comes down to memory allocation limit.


Erhm ...I'm running it on Win7 64-Bit with 8GB RAM ... not sure how to increase RAM for a 32-Bit process there ...

P.S. Sorry for posting in the wrong thread, BTW. My bad. Oops
Check that your SQL Server instance allows enough memory to be allocated. (Right click on instance > Properties > Memory > Max server memory).


Well, that value is set as 2,147,483,647 MB. What irritates me is that it's counting up to 100%:

Unzipping SDE zip file: Successful

Importing SQL data dump to 'EVEDB' database... 100 % in 0:00:11.6315352

Connection to SQL Server 'EVEDB' database: Successful

Importing categoryIDs.yaml... 100 % in 0:00:00.5456483
Importing groupIDs.yaml... 100 % in 0:00:00.6222568
Importing graphicIDs.yaml... 100 % in 0:00:00.4650456
Importing iconIDs.yaml... 100 % in 0:00:00.1408501
Importing skins.yaml... 100 % in 0:00:00.2094414
Importing skinMaterials.yaml... 100 % in 0:00:00.0937472
Importing skinLicenses.yaml... 100 % in 0:00:00.3318846
Importing typeIDs.yaml... 100 % in 0:00:35.7834965
Importing certificates.yaml... 100 % in 0:00:01.635666
Importing blueprints.yaml... 100 % in 0:00:04.1642298

Connection to SQLite 'main' database: Successful

Importing mapCelestialStatistics... 100 % in 0:00:23.0074472
Importing mapConstellationJumps... 100 % in 0:00:00.1544097
Importing mapConstellations... 100 % in 0:00:00.1264534
Importing mapDenormalize... 100 %


It hangs there for a couple of seconds (as it does with mapCelestialStatistics) and then goes

Unable to import mapDenormalize
Reason was: Timeout abgelaufen. Das Zeitlimit wurde vor dem Beenden des Vorgangs überschritten oder der Server reagiert nicht.


I played with adding the Connection Timeout attribute to the connection string, uping the default 15 seconds timeout to 60 - to no avail.

The server is a locally installed SQLExpress 2012, which :

Microsoft SQL Server 2012 - 11.0.5343.0 (X64)   May  4 2015 19:11:32   Copyright (c) Microsoft Corporation  Express Edition (64-bit) on Windows NT 6.1 (X64) (Build 7601: Service Pack 1)

EVEWalletAware - an offline wallet manager.
Desmont McCallock
#18 - 2015-12-08 19:28:52 UTC
To save you the hassle divide the operation in two parts.

1. EVESDEToSQL -nosqlite
2. EVESDEToSQL -norestore -noyaml

and you will be fine.
Medusa The Gorgon
Temple of the Serpent
The Gorgon Empire
#19 - 2015-12-08 21:48:26 UTC
Official typeIDs.yaml lacks first ' in ‘Micro' Remote Shield Booster lost first '. Can be that all first ' are vanished.
Hel O'Ween
Men On A Mission
#20 - 2015-12-09 17:35:29 UTC
Desmont McCallock wrote:
To save you the hassle divide the operation in two parts.

1. EVESDEToSQL -nosqlite
2. EVESDEToSQL -norestore -noyaml

and you will be fine.


Thanks, Desmont. Will try that when time permits.

I admittedly haven't taken the time to look/think through all the options you're providing with this one. Oops

EVEWalletAware - an offline wallet manager.