These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

 
  • Topic is locked indefinitely.
123Next pageLast page
 

Hyperion 1.0 SDE is out.

First post
Author
Previous Topic Next Topic
Steve Ronuken
Fuzzwork Enterprises
Vote Steve Ronuken for CSM
#1 - 2014-07-22 14:11:28 UTC  |  Edited by: CCP Phantom
Dunk Dinkle
Brave Newbies Inc.
Brave Collective
#2 - 2014-07-22 14:26:03 UTC
bzip2 compression - my favorite!

Thanks! ;)
Steve Ronuken
Fuzzwork Enterprises
Vote Steve Ronuken for CSM
#3 - 2014-07-22 14:34:15 UTC
Dunk Dinkle wrote:
bzip2 compression - my favorite!

Thanks! ;)



I'd use gzip, but web browsers have an unfortunate tendency to want to unpack that.

Woo! CSM XI!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter
Legedric Striker
State War Academy
Caldari State
#4 - 2014-07-22 19:06:26 UTC
Excellent work!

However, I am new to EVE 3rd party development and I am especially interested in EVE's skill data. So I downloaded several database dumps, MS Sql, mySql and SqLite in case I would miss something...

But on none of those dumps I could find any table containing skill or ship information. Did I take a wrong turn somewhere or am I just too blind to see the obvious?

I mean there are tools like EvEMon which are using this kind of information but I don't know where they got it from.

Any help is much appreciated!

EVE-Skillplan.net - Plan your pilot skill training online on PC, Mac, tablet or smartphone!
Steve Ronuken
Fuzzwork Enterprises
Vote Steve Ronuken for CSM
#5 - 2014-07-22 19:17:05 UTC
Legedric Striker wrote:
Excellent work!

However, I am new to EVE 3rd party development and I am especially interested in EVE's skill data. So I downloaded several database dumps, MS Sql, mySql and SqLite in case I would miss something...

But on none of those dumps I could find any table containing skill or ship information. Did I take a wrong turn somewhere or am I just too blind to see the obvious?

I mean there are tools like EvEMon which are using this kind of information but I don't know where they got it from.

Any help is much appreciated!



It's a little less than obvious.

The data you want is in the dgm tables.

invTypes will give you the typeID for a ship.
dgmAttributeTypes has information on what an attribute means (by attributeID)
dgmTypeAttributes links between the two, and give your the value.


So
select * from invTypes where typename = 'Rifter';
tells you that a rifter has typeid 587

Then you can use
select attributeName,description,coalesce(valueInt,valueFloat) from dgmTypeAttributes join dgmAttributeTypes on (dgmTypeAttributes.attributeID=dgmAttributeTypes.attributeID) where typeid=587;

to get the attributes.

It may seem a little unwieldy, but it allows for an arbitrary number of attributes per thing.

(it's not enough to create a ship fitting tool, but that's because of how complex the stacking can be)


Woo! CSM XI!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter
Legedric Striker
State War Academy
Caldari State
#6 - 2014-07-22 19:39:49 UTC
Ahhh thanks alot! I will see what I can find in those tables ;)

EVE-Skillplan.net - Plan your pilot skill training online on PC, Mac, tablet or smartphone!
Nicals Enderas
Science and Trade Institute
Caldari State
#7 - 2014-07-23 05:17:32 UTC
I think I didn't understand well the content of those db...

I downloaded the sqlite3 dump and started importing it to be use with Django's ORM that is very strict on relations. It seems that some relations broken:

Quote:
sqlite> select count(*) from crpNPCCorporations where factionID not in (select factionID from chrFactions);
1


So one NPCCorp is related to a faction that don't exist. Is this normal ?

Thanks.
Legedric Striker
State War Academy
Caldari State
#8 - 2014-07-23 05:57:06 UTC  |  Edited by: Legedric Striker
Yes, it's the "Arkombine" to be precise.

IIRC it was a faction that wasn't related or fighting for any empire but more for all empire corps. I don't know what exactly happened to them but I think they were removed from the game but their databse record is still there.

Try manually setting the factionID to 0 and you may be fine?

EVE-Skillplan.net - Plan your pilot skill training online on PC, Mac, tablet or smartphone!
Nicals Enderas
Science and Trade Institute
Caldari State
#9 - 2014-07-23 06:17:53 UTC
Ok. So if I encounter such a case, it won't be a problem if I simply drop the data ?
Legedric Striker
State War Academy
Caldari State
#10 - 2014-07-23 06:22:11 UTC  |  Edited by: Legedric Striker
Well regarding relations you may encounter other "problems" if you drop the data completely as there may be other data related to your above mentioned example.

Instead I would just "kill" the relation by setting the factionID to 0 in this case or create a dummy faction with the related ID and name it "unused" to fix the missing relation.

EVE-Skillplan.net - Plan your pilot skill training online on PC, Mac, tablet or smartphone!
Muscaat
EVE Markets
#11 - 2014-07-23 10:12:39 UTC
Compressed ore blueprints still appear in blueprints.yaml even though they've been removed from the game.

For example, Compressed Veldspar (blueprintID=28493), which still claims to want 166500 ordinary Veldspar to produce. (Hi Chribba.)

They still appear in invTypes too, so this might have been deliberate?
Chribba
Otherworld Enterprises
Otherworld Empire
#12 - 2014-07-23 10:36:44 UTC  |  Edited by: Chribba
They're not removed from the game, only from pilots (most of them...). They are still items and real. But yes, they absolutely are horrible for their purpose now, wants a ton of Veld to compress into something that then gets refined into 415 units of Trit haha

★★★ Secure 3rd party service ★★★

Visit my in-game channel 'Holy Veldspar'

Twitter @ChribbaVeldspar
Pfaeron
Imperial Academy
Amarr Empire
#13 - 2014-07-25 02:45:09 UTC  |  Edited by: Pfaeron
So.... is this problem reported? I just ran into it myself.
The old database, we used invTypeMaterials to lookup what an item would refine into.
This table is still included in the new crius database. But its data is wrong.


From the official database dump [Crius_1.0_100038_db.zip] (and fuzz's as well),
it is now claimed that both Veldpar and Compressed Velspar both have 415 tritanium in them.

typeID typeName materialTypeID quantity
1230 Veldspar 34 415
28432 Compressed Veldspar 34 415

Veldspar is correct.
Compresed Veldspar ... that is not right...and quite important little detail.
I am now suspicious of the entire table.

The old data... not correct anymore, but you can see how it used to be correct.. showing properly that compresed veldspar contained an amount of tritanium equivalent to 500 Veldspar.

typeID typeName materialTypeID quantity
1230 Veldspar 34 1000
28432 Compressed Veldspar 34 500000

Quote:
SELECT [invTypeMaterials].[typeID]
,invTypes.typeName
,[materialTypeID]
,[quantity]
FROM [[invTypeMaterials]
inner join [invTypes] on [invTypes].[typeID] = [invTypeMaterials].[typeID]
WHERE [invTypeMaterials].typeID = 28432 OR [invTypeMaterials].typeID=1230
Pfaeron
Imperial Academy
Amarr Empire
#14 - 2014-07-25 06:55:29 UTC  |  Edited by: Pfaeron
Ok.. I take the previous statement back.. The data is as I said.. 415 in both veldpsar and compresed veldspar.
But this is not erroneous data.

Veldspar has 415 tritanium in it.. but takes 100 units to refine (the portion size) so realy its 4.15 units of tritanium per veldpsar
where conpressed veldspar also has the same amount of tritanium in it.. 415. but its portion size is 1.. so it has 100 times more tritanium. I see.. got it... no problem.
Airi Cho
Dark-Rising
Wrecking Machine.
#15 - 2014-07-26 02:17:55 UTC  |  Edited by: Airi Cho
Steve Ronuken wrote:
Postgres
https://www.fuzzwork.co.uk/dump/crius-1.0-100038/postgres-crius-1.0-100038.dmp.bz2

Postgres dump is now done in postgres format, for restoration with pg_restore. excludes privileges.


This dump assumes there is an user "eve" in the DB.

Quote:

pg_restore: [archiver (db)] Error from TOC entry 247; 1259 67221 TABLE trnTranslations eve
pg_restore: [archiver (db)] could not execute query: ERROR: role "eve" does not exist
Command was: ALTER TABLE public."trnTranslations" OWNER TO eve;


can we get plain SQL too?

seems pg_restore --no-owner does the trick
Louis Vitton
Viziam
Amarr Empire
#16 - 2014-07-26 11:26:27 UTC
Steve invTraits was empty on the recent conversions for mysql is this meant to be so?
Desmont McCallock
#17 - 2014-07-26 15:50:34 UTC  |  Edited by: Desmont McCallock
Does anyone have any idea why those (30187, 30558,30562, 30582, 30586, 30588, 30599, 30600, 30605, 30614, 30615, 30618, 30628, 30632, 30633, 30752, 30753, 30754) are inside the blueprints.yaml file? They aren't even blueprints but items that can be reverse engineered and in the yaml file they only have the reverse engineering activity. Any reasonable explanation is much appreciated.
Steve Ronuken
Fuzzwork Enterprises
Vote Steve Ronuken for CSM
#18 - 2014-07-26 16:19:33 UTC
Desmont McCallock wrote:
Does anyone have any idea why those (30187, 30558,30562, 30582, 30586, 30588, 30599, 30600, 30605, 30614, 30615, 30618, 30628, 30632, 30633, 30752, 30753, 30754) are inside the blueprints.yaml file? They aren't even blueprints but items that can be reverse engineered and in the yaml file they only have the reverse engineering activity. Any reasonable explanation is much appreciated.



I'd assume it's because the name of the file is a little misleading. blueprints.yaml really should be called industrialActivities.yaml or similar.

It's a somewhat relevant place to put the data, which fits the same format as all the rest.

Woo! CSM XI!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter
Steve Ronuken
Fuzzwork Enterprises
Vote Steve Ronuken for CSM
#19 - 2014-07-26 16:25:06 UTC
Louis Vitton wrote:
Steve invTraits was empty on the recent conversions for mysql is this meant to be so?



umm. nope.

Screwup on my part. forgot to update the database name that script was working with. And I've made a note to point them all at a single configuration file.

I've updated the individual dump files. so people can load in just that data.

Woo! CSM XI!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter
Desmont McCallock
#20 - 2014-07-26 16:25:43 UTC
Well I guess that I have to include them in the invBlueprints table, although technically they aren't blueprints and I will have to introduce another column called reverseEngeneeringTime as they have a time factor for that activity.

No one said that this would be easy, importing the yaml files back to a DB, that is.
123Next pageLast page