These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

 
  • Topic is locked indefinitely.
Previous page12
 

External killmail link hashing algorithm

First post First post
Author
Previous Topic Next Topic
Dragonaire
Here there be Dragons
#21 - 2014-08-11 21:05:43 UTC
I know they are worried their 'super secret' fitting will get out because no one else could figure it out right Blink Hate to break it to you but everybody already knows about it and it not that good Big smile

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.
Masao Kurata
Perkone
Caldari State
#22 - 2014-08-12 20:25:20 UTC
Squizz Caphinator wrote:
Masao Kurata wrote:
The fact that it's possible to do at all should be concerning. Scraping kills is not intended to be possible. And I picked the example of a highsec ganker for a reason, there are only 5 or 6 truly good ganking ships.


Unless you already have the information as outlined above, you can't scrape killmails. If you do have that information, you already know everything but the killID - which is only of marginal value to those gathering intel.


Yes you can, you didn't read my post.

And of course it's not really all that harmful but this API is not meant to be possible to scrape, yet it is under certain circumstances.
Squizz Caphinator
The Wormhole Police
#23 - 2014-08-12 21:43:39 UTC
Masao Kurata wrote:


Yes you can, you didn't read my post.

And of course it's not really all that harmful but this API is not meant to be possible to scrape, yet it is under certain circumstances.


http://www.amazon.com/Aluminum-Foil-Deflector-Beanie-Protection/dp/1581603762

Various projects I enjoy putting my free time into:

https://zkillboard.com | https://evewho.com
Mazaron
Caldari Provisions
Caldari State
#24 - 2014-08-13 07:46:07 UTC
Didn't CCP say that since zkillboard has like 99.7% of all killmails they were going to make them public anyway?
Joshua Foiritain
Coreli Corporation
Pandemic Legion
#25 - 2014-08-13 08:30:55 UTC
Mazaron wrote:
Didn't CCP say that since zkillboard has like 99.7% of all killmails they were going to make them public anyway?

No that plan was cancelled because the owners of the remaining mails did not want their failures to become public Blink

The Coreli Corporation is recruiting.
Steve Ronuken
Fuzzwork Enterprises
Vote Steve Ronuken for CSM
#26 - 2014-08-13 12:05:44 UTC
95%.

While I personally support making them all public, I can understand the reasons for not doing so. And I wouldn't want them immediately public.

There are people who don't put their kills on public boards, because it leaks useful information. Like when they're active.

Sure, the loss mail might show up, but there are some targets who don't have killmails set up on a public killboard.

Woo! CSM XI!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter
Masao Kurata
Perkone
Caldari State
#27 - 2014-08-13 16:09:24 UTC
Okay, here's an anecdotal statistic for you: the alt I pvp on most has to go back 21 kills to find one that's on zkillboard.
Aquila Sagitta
Blue-Fire
#28 - 2014-08-19 18:28:00 UTC
And how many kills total does your alt have? 21 kills isn't much in the grande scheme of things thats like a week or less of pvp

Blue-Fire Best Fire
Masao Kurata
Perkone
Caldari State
#29 - 2014-08-19 21:19:57 UTC
The point is that none of those are on any killboard. This is fairly typical, I don't have an actual statistic for how many of his victims have registered APIs on killboards because of complications like kills I intentionally posted, fleet kills and such but it's very low. Potentially exposing kills by scraping is not a non-issue.
Joshua Foiritain
Coreli Corporation
Pandemic Legion
#30 - 2014-08-20 14:14:40 UTC
Masao Kurata wrote:
The point is that none of those are on any killboard. This is fairly typical, I don't have an actual statistic for how many of his victims have registered APIs on killboards because of complications like kills I intentionally posted, fleet kills and such but it's very low. Potentially exposing kills by scraping is not a non-issue.

Scraping kills was already possible before. Theres a limited number of API Id's and a limited number of possible Hashes.

The Coreli Corporation is recruiting.
Masao Kurata
Perkone
Caldari State
#31 - 2014-08-20 14:33:19 UTC
Joshua Foiritain wrote:
Scraping kills was already possible before. Theres a limited number of API Id's and a limited number of possible Hashes.


No it wasn't, you'd need to find the verification code for an API key, which is something in the region of 1500 bits I think. The universe will end before you get a kill that way.

On the other hand, I could get all loss mails in a specific class for a specific character on a specific day with a mere few thousand CREST API calls.
Karbowiak
Sacred Templars
Fraternity.
#32 - 2014-08-20 19:41:38 UTC
Steve Ronuken wrote:
95%.

While I personally support making them all public, I can understand the reasons for not doing so. And I wouldn't want them immediately public.

There are people who don't put their kills on public boards, because it leaks useful information. Like when they're active.

Sure, the loss mail might show up, but there are some targets who don't have killmails set up on a public killboard.


Could just make it opt-in from the client.

"I want to share all my mails publicly going back to the beginning of the universe!"
"I dont want to share all my mails, they're mine, **** you!"
Joshua Foiritain
Coreli Corporation
Pandemic Legion
#33 - 2014-08-21 15:14:55 UTC
Masao Kurata wrote:
Joshua Foiritain wrote:
Scraping kills was already possible before. Theres a limited number of API Id's and a limited number of possible Hashes.


No it wasn't, you'd need to find the verification code for an API key, which is something in the region of 1500 bits I think. The universe will end before you get a kill that way.

On the other hand, I could get all loss mails in a specific class for a specific character on a specific day with a mere few thousand CREST API calls.

There's a finite number of possible hashes and there's 41 million ID codes. Narrowing the ID code range down to less then 100.000 shouldn't be a problem if you're looking for a kill that happened at a specific time. After that its simply a matter of time, so yes; it was possible.

The Coreli Corporation is recruiting.
Squizz Caphinator
The Wormhole Police
#34 - 2014-08-21 15:46:38 UTC  |  Edited by: Squizz Caphinator
Joshua Foiritain wrote:

There's a finite number of possible hashes and there's 41 million ID codes. Narrowing the ID code range down to less then 100.000 shouldn't be a problem if you're looking for a kill that happened at a specific time. After that its simply a matter of time, so yes; it was possible.


In the zkillboard database there are over 4 million CREST verified kills and over 4 million unique hashes. There are some dupes, but those would be something like numerous shuttles all getting blown up at the same time by the same person.

Now, let us do some math. It's a 40 character hash that can consist of 0 through 9 or a through f. That's 16 possibilities for the first character. If you have 40 characters, that would then be 16^40 possible combinations.

That finite possible number of hashes is 1.46e+48.

If you think you can scrape, you must have a few million years available to you.

Various projects I enjoy putting my free time into:

https://zkillboard.com | https://evewho.com
Masao Kurata
Perkone
Caldari State
#35 - 2014-08-21 17:04:05 UTC
The key with scraping CREST is that you don't have to blindly check hashes, the hash formula was given earlier. To find a loss to npcs (concord, rats etc.) from a specific player all you need to do is change the time and ship type in that formula. Yes this is a very specific type of killmail, but you can scrape that specific type and that's a problem.
Joshua Foiritain
Coreli Corporation
Pandemic Legion
#36 - 2014-08-21 17:41:20 UTC
Squizz Caphinator wrote:
Joshua Foiritain wrote:

There's a finite number of possible hashes and there's 41 million ID codes. Narrowing the ID code range down to less then 100.000 shouldn't be a problem if you're looking for a kill that happened at a specific time. After that its simply a matter of time, so yes; it was possible.


In the zkillboard database there are over 4 million CREST verified kills and over 4 million unique hashes. There are some dupes, but those would be something like numerous shuttles all getting blown up at the same time by the same person.

Now, let us do some math. It's a 40 character hash that can consist of 0 through 9 or a through f. That's 16 possibilities for the first character. If you have 40 characters, that would then be 16^40 possible combinations.

That finite possible number of hashes is 1.46e+48.

If you think you can scrape, you must have a few million years available to you.

I never said it was a good or useful idea. I said it was possible. Blink

The Coreli Corporation is recruiting.
Ydnari
Estrale Frontiers
#37 - 2014-08-21 19:36:06 UTC
Masao Kurata wrote:
The key with scraping CREST is that you don't have to blindly check hashes, the hash formula was given earlier. To find a loss to npcs (concord, rats etc.) from a specific player all you need to do is change the time and ship type in that formula. Yes this is a very specific type of killmail, but you can scrape that specific type and that's a problem.
The non-CREST API has published rate limits including an exception rate limit. If CREST has anything similar you'll hit that in a second and be the proud owner of an (initially) temporary IP ban. If it doesn't, then people hitting it with tens or hundreds of thousands of failed requests to get this very narrow class of kills is going to get it added pretty quick.

This really isn't an issue.

--
Darkblad
Doomheim
#38 - 2014-08-31 08:04:59 UTC
Did anyone already create an online tool to convert posted kills (like a zKillboard kill URL) into public-crest.eveonline.com/killmails/ID/hash/ - and maybe even [url=killReport:ID:hash>Kill Report[/url> to get the ingame kill report? Got a Google sheets one, but that's lame. What?

NPE – ISD – RIP
Salvoxia
Free Carpenters Union
#39 - 2014-08-31 08:51:40 UTC
Hi,

may Squizz correct me if I'm wrong but judging from the zKillboard repo at github zKB already has a background-job for generating the CREST hash for kills that were posted via API, so that the CREST link is available for these kills, too.

Also, the latest version of EDK generates a CREST link for all kills it's possible for.

Regards,
Salvoxia
Darkblad
Doomheim
#40 - 2014-08-31 09:13:58 UTC
You're right. The "CREST verified" of a kill provides the URL Oops

NPE – ISD – RIP
Previous page12