These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

 
  • Topic is locked indefinitely.
 

Is there a way to hand in "bugs" in the static data dump?

First post
Author
Previous Topic Next Topic
Altessa Post
Midnight special super sexy
#1 - 2014-12-07 16:30:26 UTC
Yeah, yeah, the in game bug tool. However, I feel uncomfortable to hand in a bug for the game when it "only" concerns developers. Let us face it, it is less serious.

There are a few things strange in the static data dump: Blueprints without market group. (Which explains to me why the ships have another hierarchy in the market browser than their blueprints, lol)
The yaml file has some bizarre entries: blueprint with invention but no invention product (id 25923). Another one with no manufacturing section. I can code around these issues but it feels wrong.

Is there a way to hand in such issues?

On the internet, you can be whatever you want to be. It is amazing that so many people chose to be stupid.
Zifrian
The Frog Pond
Ribbit.
#2 - 2014-12-07 17:16:47 UTC
Altessa Post wrote:
Yeah, yeah, the in game bug tool. However, I feel uncomfortable to hand in a bug for the game when it "only" concerns developers. Let us face it, it is less serious.

There are a few things strange in the static data dump: Blueprints without market group. (Which explains to me why the ships have another hierarchy in the market browser than their blueprints, lol)
The yaml file has some bizarre entries: blueprint with invention but no invention product (id 25923). Another one with no manufacturing section. I can code around these issues but it feels wrong.

Is there a way to hand in such issues?

Not that I know of. You aren't alone though. I make due with what we get. I've either modified the yaml files or copied lost data (one particular table in Phoebe) with older DBs.

Some provide data dumps that correct issues, if you want to use them.

FoxFour has been probably one of the best 3rd party dev guys since I got involved with this but I still believe this is all collateral jobs for him. So it's not a priority that CCP will address most likely for a handful of people.

Maximze your Industry Potential! - Download EVE Isk per Hour!

Import CCP's SDE - EVE SDE Database Builder
CCP FoxFour
C C P
C C P Alliance
#3 - 2014-12-08 10:21:03 UTC
Can you give me a detailed list of things that you believe are wrong with the SDE?

@CCP_FoxFour // Technical Designer // Team Tech Co

Third-party developer? Check out the official developers site for dev blogs, resources, and more.
Altessa Post
Midnight special super sexy
#4 - 2014-12-09 19:53:42 UTC
CCP FoxFour wrote:
Can you give me a detailed list of things that you believe are wrong with the SDE?

Thanks, I shall create a small list of my issues with an explanation. When you have time, you can then consider whether it is worth the effort. However, it will probably take until the weekend.

On the internet, you can be whatever you want to be. It is amazing that so many people chose to be stupid.
Altessa Post
Midnight special super sexy
#5 - 2014-12-14 11:11:15 UTC
Well, as I do not have a model or a schema, I cannot claim that there are “bugs“. Yet, with some reasonable assumption, I think there are things which are probably not as intended.

Static data dump:
Assumption: Tradable items (thinks which can be bought and sold on the market without contract) are supposed to have a market group.

Unfortunately,. there are tradable items where the marketGroupID is NULL.
The market group is used for their hierarchical representation in the market viewer. This works nicely for example for ships. Yet, the blueprints of ships are sorted differently. This is partially because some artificial market groups which exist for ships (e.g. “Standard Frigates”, [EVE_rhea].[dbo].[invMarketGroups].[marketGroupID] =5) are missing. And worse, this is because some blueprints do not have a market group at all! Here, I assume this is wrong.

When I start working with the static DB dump, I artificially insert market groups for T2/T3 ships which are missing in my opinion:
  • "Marauder blueprints"
  • "Blackops blueprints"
  • "HICs blueprints"
  • "Jump freighters blueprints"
  • "Electronic attack frigates"
  • "Strategic cruisers"
  • "Expedition frigates"

And I assign the respective ship blueprints their artificial market ID (a “Paladin Blueprint” then receives the marketgroupID of "Marauder blueprints" etc.).

On the internet, you can be whatever you want to be. It is amazing that so many people chose to be stupid.
Altessa Post
Midnight special super sexy
#6 - 2014-12-14 11:17:57 UTC
YAML File
The yaml file is actually worse. Several entries feel wrong. Once again my assumptions and the explanation why this leads to problems.

Assumption: A blueprint is supposed to have an entry in its “activities”.

We could even be stricter and claim that a blueprint must have either a “manufacturing” activity or an “invention” activity. Because if a blueprint cannot be used for manufacturing or for invention then why have it at all?

Yet, we have “blueprints” which are technically useless because they do not allow manufacturing nor invention (I also wrote whether the item is "published" according to the DB or not. Although, this should not matter):
blueprint 935 (reaper blueprint) not published
blueprint 943 (impairor blueprint) not published
blueprint 948 (ibis blueprint) not published
blueprint 953 (velator blueprint) not published
blueprint 2864 (primae blueprint) published
blueprint 3533 (echelon blueprint) published
blueprint 27657 (Foundation Upgrade Platform Blueprint) not published
blueprint 27659 (Pedestal Upgrade Platform Blueprint) not published
blueprint 27661 (Monument Upgrade Platform Blueprint) not published
blueprint 27663 (Amarr Basic Outpost Factory Platform Blueprint) not published
blueprint 27665 (Amarr Advanced Outpost Factory Platform Blueprint) not published
blueprint 27667 (Amarr Outpost Factory Platform Blueprint) not published
blueprint 27938 (Caldari Basic Outpost Factory Platform Blueprint) not published
blueprint 27940 (Gallente Basic Outpost Factory Platform Blueprint) not published
blueprint 27942 (Minmatar Basic Outpost Factory Platform Blueprint) not published
blueprint 27958 till 28064 (various upgrade and outpost Blueprints) not published


Assumption: A blueprint with an “invention” activity is supposed to have a “products” section in the invention activity.

Yes, it could be for free to do invention (no materials section) however, if you can do invention then there has to be an outcome. So, we need a products section.

Here, too, we have blueprints breaking this assumption:
blueprint 25923 (Large EW Drone Range Augmentor I Blueprint) not published
blueprint 32066 (Small EW Drone Range Augmentor I Blueprint) published
blueprint 32068 (Medium EW Drone Range Augmentor I Blueprint) published
blueprint 34267 (Small Higgs Anchor I Blueprint) published
blueprint 34269 (Medium Higgs Anchor I Blueprint) published
blueprint 34307 (Large Higgs Anchor I Blueprint) published
blueprint 34309 (Capital Higgs Anchor I Blueprint) published

I can program around these things by dropping my assumptions. However, I can imagine that in a year or two when you tinker around with your yaml parser you might trip over such things yourself. And the debugging is not so funny...

On the internet, you can be whatever you want to be. It is amazing that so many people chose to be stupid.
Zifrian
The Frog Pond
Ribbit.
#7 - 2014-12-14 16:34:34 UTC
Altessa Post wrote:
Well, as I do not have a model or a schema, I cannot claim that there are “bugs“. Yet, with some reasonable assumption, I think there are things which are probably not as intended.

Static data dump:
Assumption: Tradable items (thinks which can be bought and sold on the market without contract) are supposed to have a market group.

Unfortunately,. there are tradable items where the marketGroupID is NULL.
The market group is used for their hierarchical representation in the market viewer. This works nicely for example for ships. Yet, the blueprints of ships are sorted differently. This is partially because some artificial market groups which exist for ships (e.g. “Standard Frigates”, [EVE_rhea].[dbo].[invMarketGroups].[marketGroupID] =5) are missing. And worse, this is because some blueprints do not have a market group at all! Here, I assume this is wrong.

When I start working with the static DB dump, I artificially insert market groups for T2/T3 ships which are missing in my opinion:
  • "Marauder blueprints"
  • "Blackops blueprints"
  • "HICs blueprints"
  • "Jump freighters blueprints"
  • "Electronic attack frigates"
  • "Strategic cruisers"
  • "Expedition frigates"

And I assign the respective ship blueprints their artificial market ID (a “Paladin Blueprint” then receives the marketgroupID of "Marauder blueprints" etc.).

You can't buy those bps on the market. Back in the day you could buy and sell T2 BPOs that people got from the lottery so people would sell them and they had market groups. Those BPs you listed were added after the lottery was removed and invention added. So the data that shows T2 BPOs with market ids is really a legacy thing mainly (although some people have t2bpos and could use the market, but typically do private trades now).

Maximze your Industry Potential! - Download EVE Isk per Hour!

Import CCP's SDE - EVE SDE Database Builder
Zifrian
The Frog Pond
Ribbit.
#8 - 2014-12-14 16:52:22 UTC
Altessa Post wrote:
YAML File
The yaml file is actually worse. Several entries feel wrong. Once again my assumptions and the explanation why this leads to problems.

Assumption: A blueprint is supposed to have an entry in its “activities”.

We could even be stricter and claim that a blueprint must have either a “manufacturing” activity or an “invention” activity. Because if a blueprint cannot be used for manufacturing or for invention then why have it at all?

Yet, we have “blueprints” which are technically useless because they do not allow manufacturing nor invention (I also wrote whether the item is "published" according to the DB or not. Although, this should not matter):
blueprint 935 (reaper blueprint) not published
blueprint 943 (impairor blueprint) not published
blueprint 948 (ibis blueprint) not published
blueprint 953 (velator blueprint) not published
blueprint 2864 (primae blueprint) published
blueprint 3533 (echelon blueprint) published
blueprint 27657 (Foundation Upgrade Platform Blueprint) not published
blueprint 27659 (Pedestal Upgrade Platform Blueprint) not published
blueprint 27661 (Monument Upgrade Platform Blueprint) not published
blueprint 27663 (Amarr Basic Outpost Factory Platform Blueprint) not published
blueprint 27665 (Amarr Advanced Outpost Factory Platform Blueprint) not published
blueprint 27667 (Amarr Outpost Factory Platform Blueprint) not published
blueprint 27938 (Caldari Basic Outpost Factory Platform Blueprint) not published
blueprint 27940 (Gallente Basic Outpost Factory Platform Blueprint) not published
blueprint 27942 (Minmatar Basic Outpost Factory Platform Blueprint) not published
blueprint 27958 till 28064 (various upgrade and outpost Blueprints) not published


Assumption: A blueprint with an “invention” activity is supposed to have a “products” section in the invention activity.

Yes, it could be for free to do invention (no materials section) however, if you can do invention then there has to be an outcome. So, we need a products section.

Here, too, we have blueprints breaking this assumption:
blueprint 25923 (Large EW Drone Range Augmentor I Blueprint) not published
blueprint 32066 (Small EW Drone Range Augmentor I Blueprint) published
blueprint 32068 (Medium EW Drone Range Augmentor I Blueprint) published
blueprint 34267 (Small Higgs Anchor I Blueprint) published
blueprint 34269 (Medium Higgs Anchor I Blueprint) published
blueprint 34307 (Large Higgs Anchor I Blueprint) published
blueprint 34309 (Capital Higgs Anchor I Blueprint) published

I can program around these things by dropping my assumptions. However, I can imagine that in a year or two when you tinker around with your yaml parser you might trip over such things yourself. And the debugging is not so funny...

Since I've been working with the SDE, those bps are typical. Usually I assume it's a testing thing they are doing or just left over from some other issue. Basically yeah, you need to only choose those that are published (there are three published fields - one in each invTypes, invGroups, and invCategories). Basically, if a bp isn't published, then it isn't in game and therefore not something you need to pay attention to.

Maximze your Industry Potential! - Download EVE Isk per Hour!

Import CCP's SDE - EVE SDE Database Builder
Altessa Post
Midnight special super sexy
#9 - 2014-12-14 17:48:36 UTC
Zifrian wrote:

You can't buy those bps on the market. Back in the day you could buy and sell T2 BPOs that people got from the lottery so people would sell them and they had market groups. Those BPs you listed were added after the lottery was removed and invention added. So the data that shows T2 BPOs with market ids is really a legacy thing mainly (although some people have t2bpos and could use the market, but typically do private trades now).

You are right.
Especially blueprints for newer T2 ships (e.g. Expedition Frigate) just do not exist. This is unpleasant for the way I internally handle blueprints in my code (always following the market groups) but that is my own problem.
I withdraw this issue. The data dump is correct.

On the internet, you can be whatever you want to be. It is amazing that so many people chose to be stupid.
Altessa Post
Midnight special super sexy
#10 - 2014-12-14 17:57:03 UTC
Zifrian wrote:

Since I've been working with the SDE, those bps are typical. Usually I assume it's a testing thing they are doing or just left over from some other issue. Basically yeah, you need to only choose those that are published (there are three published fields - one in each invTypes, invGroups, and invCategories). Basically, if a bp isn't published, then it isn't in game and therefore not something you need to pay attention to.

Some of the blueprints are published.
Plus, if you extract the blueprint info from the database and put it in a file then I think the file should be consistent in itself. This allows using a parser which has to know nothing about the DB.


BTW, I just stumbled over the bowhead blueprint which allows you to invent an Anshar Blueprint. Why that? It breaks another of my assumptions :-) (a T2 blueprint is the result of invention from exactly one T1 blueprint)

On the internet, you can be whatever you want to be. It is amazing that so many people chose to be stupid.
Zifrian
The Frog Pond
Ribbit.
#11 - 2014-12-14 20:33:31 UTC
Altessa Post wrote:
Zifrian wrote:

Since I've been working with the SDE, those bps are typical. Usually I assume it's a testing thing they are doing or just left over from some other issue. Basically yeah, you need to only choose those that are published (there are three published fields - one in each invTypes, invGroups, and invCategories). Basically, if a bp isn't published, then it isn't in game and therefore not something you need to pay attention to.

Some of the blueprints are published.
Plus, if you extract the blueprint info from the database and put it in a file then I think the file should be consistent in itself. This allows using a parser which has to know nothing about the DB.

BTW, I just stumbled over the bowhead blueprint which allows you to invent an Anshar Blueprint. Why that? It breaks another of my assumptions :-) (a T2 blueprint is the result of invention from exactly one T1 blueprint)

Check the published field on the group and category tables. It might not be published there. As far as the rigs you say that should have invention, but it looks like that's not implemented yet. Finally, on the bowhead, this is a bug that was fixed in the patch the other day but the SDE hasn't been updated yet (the SDE is built from the current game build).

Basically, if you want a perfect SDE, you probably aren't going to get one. There is always some quirk or data that isn't as you want it. You just have to work around it. Like I posted earlier, FF is pretty awesome for all the work he's doing but it's not his full time job. So you have to accept some imperfections in the data. I have special processing to ignore the Primae bp for instance. I've done stuff like this for a few years now.

Maximze your Industry Potential! - Download EVE Isk per Hour!

Import CCP's SDE - EVE SDE Database Builder