These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Information Portal

 
  • Topic is locked indefinitely.
123Next pageLast page
 

CSV dump of all PVP kills between Mar 1-Apr 4

First post
Author
Previous Topic Next Topic
CCP Quant
C C P
C C P Alliance
#1 - 2016-04-07 11:19:48 UTC  |  Edited by: CCP Phantom
Hey guys,

We've noticed that a bunch of players are trying to go through kill data, especially now during the war. This can be very tedious to scrape and stitch together from the killboards, so we're looking into ways to accommodate. Provided below is a link to all PVP kills made in the period Mar 1 to Apr 4, with the condition that the victim is a player character and the final blow is made by a player character. Keep in mind that the isk lost / destroyed values for supers are lower than the killboard ones since they dont have any market prices behind them for the hulls and some "exotic" modules, only base costs. You will have to adjust for that.

http://cdn1.eveonline.com/community/quant/WWB_killdump.zip

I've denormalized the data for you quite extensively, e.g. added solarSystemName, regionName, type names, and so forth. This might not be necessary but it allows people to make analysis on the data as-is without the need to join in other eve datasets.
characterIDs are also not included at this point, since we're not comfortable with including them at this point.


If this turns out successful, and only good things come out of this, we are more likely to move into a direction where we can provide kill data in bulk, be it through CSV dumps like this one (e.g. in the monthly economic report data dump) or through CREST.

Happy crunching!

Edit: Another thing I should've noted as it's a bit confusing, dropped = lost - destroyed. So the total is "iskLost", what didn't drop is iskDestroyed.
woooooooooooo
Center for Advanced Studies
Gallente Federation
#2 - 2016-04-07 11:23:20 UTC
Thanks!
Zappity
New Eden Tank Testing Services
#3 - 2016-04-07 11:25:27 UTC
Thank you. It will be interesting to see how accurate the killboards are compared to this. I think the time has come for crest kills.

Zappity's Adventures for a taste of lowsec and nullsec.
Dreamer Targaryen
KarmaFleet
Goonswarm Federation
#4 - 2016-04-07 12:59:20 UTC
Thank you, especially for already joining the data with names for corps, alliances and systems.

For my fellow analysts:
Keep in mind, that the column "victimCoalition" is not an ingame-attribute and as such has to be treated as corrupted data.
An example is the following line, which lists "Circle-Of-Two" as a member of "The Imperium" in april, where in reality they left the glorious empire a week prior to that:
98261786,The Clown Shoe Crew,Circle-Of-Two,The Imperium,98343305,Run and Gun Mercenary Corps,FETID,,670,Capsule,Capsule,2016-04-03 03:06:16,30002719,Miroitem,10000032,Sinq Laison,1022826005.05,1022826005.05,204565201.01
Anthar Thebess
#5 - 2016-04-07 13:27:39 UTC
Can we get this kind of information auto-generated at the end of each month?
This will save every one server time and bandwidth.

Stop discrimination, help in a fight against terrorists

Show your support to The Cause!
CCP Quant
C C P
C C P Alliance
#6 - 2016-04-07 13:49:40 UTC
Dreamer Targaryen wrote:
Thank you, especially for already joining the data with names for corps, alliances and systems.

For my fellow analysts:
Keep in mind, that the column "victimCoalition" is not an ingame-attribute and as such has to be treated as corrupted data.
An example is the following line, which lists "Circle-Of-Two" as a member of "The Imperium" in april, where in reality they left the glorious empire a week prior to that:
98261786,The Clown Shoe Crew,Circle-Of-Two,The Imperium,98343305,Run and Gun Mercenary Corps,FETID,,670,Capsule,Capsule,2016-04-03 03:06:16,30002719,Miroitem,10000032,Sinq Laison,1022826005.05,1022826005.05,204565201.01


Good point I forgot to mention, it uses http://rischwa.net/api/coalitions/current to assign coalitions. I might try to get that into our ETL processes so that we can report coalitions more accurately, given that this provider is reliable Smile
DiceyQ
Black Omega Security
Minmatar Fleet Alliance
#7 - 2016-04-07 17:00:13 UTC  |  Edited by: DiceyQ
CCP Quant wrote:
Hey guys,

We've noticed that a bunch of players are trying to go through kill data, especially now during the war. This can be very tedious to scrape and stitch together from the killboards, so we're looking into ways to accommodate. Provided below is a link to all PVP kills made in the period Mar 1 to Apr 4, with the condition that the victim is a player character and the final blow is made by a player character. Keep in mind that the isk lost / destroyed values for supers are lower than the killboard ones since they dont have any market prices behind them for the hulls and some "exotic" modules, only base costs. You will have to adjust for that.

http://cdn1.eveonline.com/community/quant/WWB_killdump.zip

I've denormalized the data for you quite extensively, e.g. added solarSystemName, regionName, type names, and so forth. This might not be necessary but it allows people to make analysis on the data as-is without the need to join in other eve datasets.
characterIDs are also not included at this point, since we're not comfortable with including them at this point.


If this turns out successful, and only good things come out of this, we are more likely to move into a direction where we can provide kill data in bulk, be it through CSV dumps like this one (e.g. in the monthly economic report data dump) or through CREST.

Happy crunching!


nice, can we have a csv dump of POS mod contents that died between mar 1 and april 7?
Zappity
New Eden Tank Testing Services
#8 - 2016-04-07 19:27:00 UTC
DiceyQ wrote:
CCP Quant wrote:
Hey guys,

We've noticed that a bunch of players are trying to go through kill data, especially now during the war. This can be very tedious to scrape and stitch together from the killboards, so we're looking into ways to accommodate. Provided below is a link to all PVP kills made in the period Mar 1 to Apr 4, with the condition that the victim is a player character and the final blow is made by a player character. Keep in mind that the isk lost / destroyed values for supers are lower than the killboard ones since they dont have any market prices behind them for the hulls and some "exotic" modules, only base costs. You will have to adjust for that.

http://cdn1.eveonline.com/community/quant/WWB_killdump.zip

I've denormalized the data for you quite extensively, e.g. added solarSystemName, regionName, type names, and so forth. This might not be necessary but it allows people to make analysis on the data as-is without the need to join in other eve datasets.
characterIDs are also not included at this point, since we're not comfortable with including them at this point.


If this turns out successful, and only good things come out of this, we are more likely to move into a direction where we can provide kill data in bulk, be it through CSV dumps like this one (e.g. in the monthly economic report data dump) or through CREST.

Happy crunching!


nice, can we have a csv dump of POS mod contents that died between mar 1 and april 7?

Didn't you know that they were all empty?

Zappity's Adventures for a taste of lowsec and nullsec.
Masao Kurata
Perkone
Caldari State
#9 - 2016-04-07 20:02:23 UTC
Excuse me? Some people like to keep certain kills and losses secret, this is a big change in policy.
CCP Quant
C C P
C C P Alliance
#10 - 2016-04-07 21:26:42 UTC  |  Edited by: CCP Quant
Masao Kurata wrote:
Excuse me? Some people like to keep certain kills and losses secret, this is a big change in policy.


We excluded characterID for that purpose. If that is still giving away too much info we could move(from corporationID as the lowest identifier) up to allianceID, but yeah this sentiment is the reason kills aren't accessible in bulk through CREST. However, one of the biggest arguments for hiding kills has been to hide fitting, these kill logs don't show fitting. Also, it's only PVP kills where a player character kills a player character.
Hugh Jorgan
Pandemic Horde Inc.
Pandemic Horde
#11 - 2016-04-08 01:02:28 UTC
Thanks, this is awesome! Already used for this:
https://pay.reddit.com/r/Eve/comments/4du3im/slopes_for_the_slope_throne_isk_lost_in_pvp_per/

Might have to put it in R and see what falls out.
Stamin Primer
Echelon Research
Goonswarm Federation
#12 - 2016-04-08 01:48:07 UTC
I think this is going to be a fantastic tool going forward, especially during this massive war and the fights around New Eden. It is better than going through battle reports/kill reports which can be tedious to open multiple links. I honestly think that most pilots would be more comfortable not having their character name released in relation to the reports. Thanks for compiling!
Robbert Smith
Center for Advanced Studies
Gallente Federation
#13 - 2016-04-08 02:40:23 UTC
Great stuff. Thanks for posting.

Personally, I think it needs something to identify individual pilots. That way we can tell how many individual pilots are involved. I think it would be very enlightening. The identifying ID needn't be the characterID available from the API, it could just be random and uniquely identifying. Admittedly, you could probably match those IDs up with a public killboard.

But, as CCP Quant mentioned, there are no fits, so people can just suck it up.

On the other hand, I could see a good argument for excluding wormhole systems.
Masao Kurata
Perkone
Caldari State
#14 - 2016-04-08 05:01:45 UTC
CCP Quant wrote:
Masao Kurata wrote:
Excuse me? Some people like to keep certain kills and losses secret, this is a big change in policy.


We excluded characterID for that purpose. If that is still giving away too much info we could move(from corporationID as the lowest identifier) up to allianceID, but yeah this sentiment is the reason kills aren't accessible in bulk through CREST. However, one of the biggest arguments for hiding kills has been to hide fitting, these kill logs don't show fitting. Also, it's only PVP kills where a player character kills a player character.


Okay, that was a knee jerk reaction. Upon actually reviewing the columns included and considering the matter, this level of data is.... mostly fine? It is possible in some cases to trivially get the kill hash from the data (when both the victim and killers are in one or two man corporations), and getting a range of possible kill ids based on the timestamp is easy, providing enough information to query public CREST for the full kill details. I imagine that if one were to run a script over the kills querying on the assumption that a CEO killed a CEO, that alone would yield many killmails previously not pushed to any killboards.
Masao Kurata
Perkone
Caldari State
#15 - 2016-04-08 05:03:29 UTC
Robbert Smith wrote:
Personally, I think it needs something to identify individual pilots. That way we can tell how many individual pilots are involved. I think it would be very enlightening. The identifying ID needn't be the characterID available from the API, it could just be random and uniquely identifying. Admittedly, you could probably match those IDs up with a public killboard.


Unfortunately that would actually make getting kill hashes very easy simply by comparing timestamps to published kills to deobfuscate the non-characterid unique identifiers for anyone who has any published kills or losses.
Petrified
Old and Petrified Syndication
#16 - 2016-04-08 05:06:22 UTC
Very nice. Thank you for the data compilation. Looking forward to seeing if this becomes a normal practice.

Cloaking is the closest thing to a "Pause Game" button one can get while in space.

Support better localization for the Japanese Community.
Rattman
Deep Core Mining Inc.
Caldari State
#17 - 2016-04-08 07:41:06 UTC
No pos mods.

Any reason for that would like to know how many SCAA have died in the north
CCP Quant
C C P
C C P Alliance
#18 - 2016-04-08 09:01:42 UTC
Masao Kurata wrote:
CCP Quant wrote:
Masao Kurata wrote:
Excuse me? Some people like to keep certain kills and losses secret, this is a big change in policy.


We excluded characterID for that purpose. If that is still giving away too much info we could move(from corporationID as the lowest identifier) up to allianceID, but yeah this sentiment is the reason kills aren't accessible in bulk through CREST. However, one of the biggest arguments for hiding kills has been to hide fitting, these kill logs don't show fitting. Also, it's only PVP kills where a player character kills a player character.


Okay, that was a knee jerk reaction. .


I didn't want to dance around the subject Smile no knee jerk reaction intended! I'm biased here as If this were completely up to me I'd say we open up the killfeed in crest and the 0.1% of pilots who don't want kills shown will just have to suck it up Pirate However, and luckily for the 0.1% the decision is not solely in my hands.
ChromeStriker
Sebiestor Tribe
Minmatar Republic
#19 - 2016-04-08 09:10:03 UTC
CCP Quant wrote:
Masao Kurata wrote:
CCP Quant wrote:
Masao Kurata wrote:
Excuse me? Some people like to keep certain kills and losses secret, this is a big change in policy.


We excluded characterID for that purpose. If that is still giving away too much info we could move(from corporationID as the lowest identifier) up to allianceID, but yeah this sentiment is the reason kills aren't accessible in bulk through CREST. However, one of the biggest arguments for hiding kills has been to hide fitting, these kill logs don't show fitting. Also, it's only PVP kills where a player character kills a player character.


Okay, that was a knee jerk reaction. .


I didn't want to dance around the subject Smile no knee jerk reaction intended! I'm biased here as If this were completely up to me I'd say we open up the killfeed in crest and the 0.1% of pilots who don't want kills shown will just have to suck it up Pirate However, and luckily for the 0.1% the decision is not solely in my hands.


I would argue the benifits outway the cons... get it up on crest so we can really get our teeth into it... its like your catering to the 0.1% people still running "windows 95" ...

No Worries
Lokyar Brightmane
Imperial Academy
Amarr Empire
#20 - 2016-04-08 09:27:00 UTC
CCP Quant wrote:
Masao Kurata wrote:
CCP Quant wrote:
Masao Kurata wrote:
Excuse me? Some people like to keep certain kills and losses secret, this is a big change in policy.


We excluded characterID for that purpose. If that is still giving away too much info we could move(from corporationID as the lowest identifier) up to allianceID, but yeah this sentiment is the reason kills aren't accessible in bulk through CREST. However, one of the biggest arguments for hiding kills has been to hide fitting, these kill logs don't show fitting. Also, it's only PVP kills where a player character kills a player character.


Okay, that was a knee jerk reaction. .


I didn't want to dance around the subject Smile no knee jerk reaction intended! I'm biased here as If this were completely up to me I'd say we open up the killfeed in crest and the 0.1% of pilots who don't want kills shown will just have to suck it up Pirate However, and luckily for the 0.1% the decision is not solely in my hands.

As I understand the conversation, he meant that he made a knee jerk reaction and actually thinks it's a good idea, now he's had time to review it further.
123Next pageLast page