These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

 
  • Topic is locked indefinitely.
 

Questions about the SDE

First post
Author
guigui lechat
the no fock given
#1 - 2017-03-27 02:41:59 UTC
I'm so tired about having to find everything by guess in the SDE …
Is there something that presents the links between the tables ?
I need to which minerals
- can spawn in HS belts
- can spawn in HS anomalies

So far
- I look in fsd/categoryIDs.yaml to get the id of the category with english name | description equal to "Asteroid"
- I get all groupID from fsd/groupIDs.yaml for which the categoryID is matching.
- I get ALL the types ids from fsd/typeIDs.yaml whith a group corresponding.

How can I guess after that if that type is minable from eg HS asteroid belt ?
eg Compressed Arkonor's ID is 28367 ; I can't find a "typeID: 28367" in dgmTypeEffects.yaml and attributes linked in dgmTypeAttributes.yaml refer to the skills required.

grep "typeID: 28367" */*.yaml
bsd/dgmTypeAttributes.yaml:    typeID: 28367
bsd/dgmTypeAttributes.yaml:    typeID: 28367
bsd/invTypeMaterials.yaml:    typeID: 28367
bsd/invTypeMaterials.yaml:    typeID: 28367
bsd/invTypeMaterials.yaml:    typeID: 28367
fsd/blueprints.yaml:                typeID: 28367


invTypeMaterials says what materal I can extract from ore, blueprints describe the production in industry.

blueprints.yaml says "Compressed Arkonor" is a product of "Compressed Arkonor Blueprint" (ORLY ?)

dgmTypeAttributes.yaml gives two attributes :
-   attributeID: 790
    typeID: 28367
    valueInt: 12180
-   attributeID: 182
    typeID: 28367
    valueInt: 3386


looking the attributeIDs in dgmAttributeTypes.yaml:

-   attributeID: 790
    attributeName: reprocessingSkillType
-   attributeID: 182
    attributeName: requiredSkill1


None of them seems to be relate to spawning and now I am in a dead-end.

So far I found I can hardcode the groups which can spawn in HS, in HS anoms, in LS, in LS anoms. But in that case I may as well hardcode the asteroids - and their corresponding volumes.

The code I'm trying right now ( getting the ore isk/m3 with the variants) is at
https://github.com/guiguilechat/EveOnline/blob/master/Database/src/test/java/fr/guiguilechat/eveonline/database/ShowJitaOrePrice.java
Blacksmoke16
Resilience.
#2 - 2017-03-27 03:09:36 UTC  |  Edited by: Blacksmoke16
You would just have to do it manually, far as i know the only place that says it is the description.

EX for Jaspet:

Quote:
Jaspet has three valuable mineral types, making it easy to sell. It has a large portion of mexallon plus some nocxium and zydrine.

Available primarily in 0.4 security status solar systems or lower.
Steve Ronuken
Fuzzwork Enterprises
Vote Steve Ronuken for CSM
#3 - 2017-03-27 08:53:03 UTC
Unfortunately, blacksmoke16 is correct.

There's some things which exist only in the game logic, rather than in the SDE. And that's unlikely to change (as it'd be a total PITA for CCP to keep up to date)

I'd suggest you may wish to use a conversion of the SDE, rather than the yaml files. I have mysql, postgres and sqlite versions on my site.


The volume of the ore is in invTypes. Along with the portionSize, which is how many units you need to be able to reprocess.

Woo! CSM XI!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter

guigui lechat
the no fock given
#4 - 2017-03-27 11:39:49 UTC
Thank you.

well I also have issues with ore being extracted from missions… and which belong to eg arkonor group. The only way I can deal with them is by requesting eve central if their BO @ Jita is 0.

I will group the groupIDs by HS/LS/NS then.

On a side note, I found errors in the SDE, where can I report them ?
Steve Ronuken
Fuzzwork Enterprises
Vote Steve Ronuken for CSM
#5 - 2017-03-27 12:27:17 UTC
guigui lechat wrote:
Thank you.

well I also have issues with ore being extracted from missions… and which belong to eg arkonor group. The only way I can deal with them is by requesting eve central if their BO @ Jita is 0.

I will group the groupIDs by HS/LS/NS then.

On a side note, I found errors in the SDE, where can I report them ?



What were the errors?

Woo! CSM XI!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter

guigui lechat
the no fock given
#6 - 2017-03-27 13:25:51 UTC
One is that the decryptors' ME, TE, maxrun attributes are stored in the valueFloat instead of the valueInt (while they are int) in the dgmTypeAttributes.yaml .

I also have concerns with attributes naming
- "rate of fire" attribute for turrets and launchers should be "delay of fire" ( I think I don't have to explain that one - https://en.wikipedia.org/wiki/Rate_of_fire in case)
- the shield recharger II 's 15% bonus to 'shield recharge rate' is actually a reduction to shield recharge time - meaning a 17.65% increase in shield recharge rate.
- same for the "rate of fire" of eg gyro (it's a delay of fire reduction)
- same for the "shield recharge bonus" of shield recharger II, purgers (it's a delay of recharge reduction)
- 30% damage reduction of invul II should not be negative : the bonus is a straight 30% reduction to incoming damage.
But I think that should go on another topic (where ?).

I also found typos but forgot them.
Althalus Stenory
Flying Blacksmiths
#7 - 2017-03-27 14:56:47 UTC
For ore, you can check invTypeMaterials where only refinable ore/modules are. Missions ore are not listed here (because you can't refine them), so it allows you to exclude them easily.

EsiPy - Python 2.7 / 3.3+ Swagger Client based on pyswagger for ESI

guigui lechat
the no fock given
#8 - 2017-03-27 15:04:22 UTC
nice, thank you !
Tonto Auri
Vhero' Multipurpose Corp
#9 - 2017-04-01 17:00:58 UTC
guigui lechat wrote:
One is that the decryptors' ME, TE, maxrun attributes are stored in the valueFloat instead of the valueInt (while they are int) in the dgmTypeAttributes.yaml .

This is not a bug per se. You have to coalesce both columns to get the real value, regardless of the intent.

Quote:
I also have concerns with attributes naming

Legacy issue. Can't be helped.
Quote:
I also found typos but forgot them.

Typos actually must be properly reported through usual channels, once you've confirmed them existent in game.

Two most common elements in the universe are hydrogen and stupidity. -- Harlan Ellison

Steve Ronuken
Fuzzwork Enterprises
Vote Steve Ronuken for CSM
#10 - 2017-04-01 18:42:25 UTC
guigui lechat wrote:
One is that the decryptors' ME, TE, maxrun attributes are stored in the valueFloat instead of the valueInt (while they are int) in the dgmTypeAttributes.yaml .

I also have concerns with attributes naming
- "rate of fire" attribute for turrets and launchers should be "delay of fire" ( I think I don't have to explain that one - https://en.wikipedia.org/wiki/Rate_of_fire in case)
- the shield recharger II 's 15% bonus to 'shield recharge rate' is actually a reduction to shield recharge time - meaning a 17.65% increase in shield recharge rate.
- same for the "rate of fire" of eg gyro (it's a delay of fire reduction)
- same for the "shield recharge bonus" of shield recharger II, purgers (it's a delay of recharge reduction)
- 30% damage reduction of invul II should not be negative : the bonus is a straight 30% reduction to incoming damage.
But I think that should go on another topic (where ?).

I also found typos but forgot them.



As this is stuff generated from eve, I don't really see it changing. I mean, it's labeled that way inside eve too.

the ME/TE/max run, sure, but tbh, anyone using the SDE should be coalescing the valueint/valuefloat anyway. There are other places it's different.


These aren't _SDE_ problems. They're from the core DB from Eve.

Woo! CSM XI!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter

guigui lechat
the no fock given
#11 - 2017-04-20 21:30:34 UTC  |  Edited by: guigui lechat
Some typeIDs are present in the blueprint.yaml but not in the typeIDs.yaml:

34188, 34189, 37262, 37263, 37264, 37009, 37265, 37010, 37266, 37267, 37013, 37017, 37018, 37019, 37276, 37277, 37278, 37279, 35875, 35876, 37286, 35879, 35883, 35884, 35885, 35887, 37439, 37440, 37059, 37830, 37831, 37070, 37200, 37201, 37841, 37842, 3924, 37082, 37362, 37363, 35956, 37236, 37237, 37238, 37239, 37240, 37241, 37242, 37243, 35964, 37244, 37245, 37247


is there an elegant solution to know what is last sde version ? I can download https://developers.eveonline.com/resource/resources and regexp the url but it seems kinda weird.
This error comes from last 2017-03-30 version (was also present before I think)


Some of those typeIDs are related to clone grade. The corresponding clone grades are "published: false" but the blueprints do not have this information. I did not check every missing typeID
RJ Lane
School of Applied Knowledge
Caldari State
#12 - 2017-05-23 00:20:40 UTC
guigui lechat wrote:


is there an elegant solution to know what is last sde version ? I can download https://developers.eveonline.com/resource/resources and regexp the url but it seems kinda weird.
This error comes from last 2017-03-30 version (was also present before I think)



+1 to this question
Blacksmoke16
Resilience.
#13 - 2017-05-23 00:26:42 UTC
RJ Lane wrote:
guigui lechat wrote:


is there an elegant solution to know what is last sde version ? I can download https://developers.eveonline.com/resource/resources and regexp the url but it seems kinda weird.
This error comes from last 2017-03-30 version (was also present before I think)



+1 to this question


Not 100% how this works, Steve can confirm, but could you use https://www.fuzzwork.co.uk/dump/ and then store and compare the md5 value from like mysql-latest.tar.bz2.md5. If they are the same then SDE is at the latest version, else it is not.
guigui lechat
the no fock given
#14 - 2017-05-23 10:42:55 UTC
this is functionally the same as dl/ling the download page, albeit relying on an additionnal component (fuzzwork) thus increasing potential bugs.

What I feel weird is that I rely on a function that is not expressely described by devs, so my code may become broken without a reason.
Steve Ronuken
Fuzzwork Enterprises
Vote Steve Ronuken for CSM
#15 - 2017-05-23 12:01:45 UTC
guigui lechat wrote:
this is functionally the same as dl/ling the download page, albeit relying on an additionnal component (fuzzwork) thus increasing potential bugs.

What I feel weird is that I rely on a function that is not expressely described by devs, so my code may become broken without a reason.



All the information is directly available from the devs. It's in the yaml file.

All I'm doing is shifting it into a different format(s), because yaml isn't particularly useful for some of the purposes for which it's being used. (It is great as a data exchange format.)

Before it was yaml, it was an sql server backup file. Which was a total pita to use in anything else.

Woo! CSM XI!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter

guigui lechat
the no fock given
#16 - 2017-05-23 12:30:35 UTC  |  Edited by: guigui lechat
"All the information is directly available from the devs. It's in the yaml file."
Which information ?

just to clarify. I need to keep up-to-date with the sde. So I keep track of last version of SDE and only download it again when a newer one is available.
If I need to download it to know if I must download it, this makes no sense.

what I ask for is a : should I download it again ? function. Which would not rely one something more than the data provided by devs.
Althalus Stenory
Flying Blacksmiths
#17 - 2017-05-24 06:14:18 UTC
1. https://www.fuzzwork.co.uk/dump/ and get the latest XXXX md5 (mysql, sqlite, whatever), so you have a link with "latest", you don't care about specific version, and you can do it "brainless", it'll always work.

2. https://developers.eveonline.com/resource/resources as there are no "latest" using yaml SDE, you have to give the link manually, so either you know when to give it, or you'll have to dl it, then check something (md5 checksum or something that does the same might be what you want) and decide what to do.

As a result:
1. depends on a third party dev, but don't dl unnecessary stuff
2. always dl but only check to use it or not.

EsiPy - Python 2.7 / 3.3+ Swagger Client based on pyswagger for ESI

guigui lechat
the no fock given
#18 - 2017-05-24 23:24:56 UTC
read the above answers, please. As I said I already parse the url from the dev page to get the latest link. This is a BAD solution because I assume many things about the download page and those things are not ensured by dev ; still the best solution I got so far.