These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE General Discussion

 
  • Topic is locked indefinitely.
 

Need MOAR Kill Mails!

First post
Author
Previous Topic Next Topic
Khorkrak
KarmaFleet
Goonswarm Federation
#1 - 2012-10-09 21:55:22 UTC  |  Edited by: Khorkrak
I'm testing the performance of our public, open source, free to use killboard system and would like to see how it fares with increased load - that is more kill mail data, API keys and simultaneous users.

It's still under development. Far from all of the features found in EDK / eve-kill and Battleclinic are present, however more is being added, often daily. Feel free to suggest enhancements as well. Some distinguishing aspects include the absence of all forms of advertising, quick page loads and reliable up-time: decloaked public stats page

Current stats as I write this:
85,797 kill mails
1,119,448 dropped / deleted item rows
714,925 attackers
76,518 characters
13,799 corporations
2,082 alliances
97 API Keys

PVP stats displayed here are fairly unique as well:
There's an overall PVP status so that you can see at a glance how things are going:
http://decloaked.com/pilot/Khorkrak/losses
Coding while farming ISK in null sec hasn't been one of my best moves obviously Oops

On the other side of the spectrum is something like this: http://decloaked.com/pilot/Mystical%20Might/stats You can click on those pie wedges for more detailed info - adding the ability to display the related kills soon as well.

Please check the site out and have fun.

We're on this channel if you'd like to chat: irc.coldfront.net #pykb

Developer of http://www.decloaked.com and http://sourceforge.net/projects/pykb/
Jim Era
#2 - 2012-10-09 21:56:53 UTC
wat


people take km's seriously? ahah

Wat™
Unsuccessful At Everything
The Troll Bridge
#3 - 2012-10-09 22:08:50 UTC
Your Norton Security Suite has detected a shameless plug. Do you wish to remove it? [Y]

Since the cessation of their usefulness is imminent, may I appropriate your belongings?
Marlona Sky
State War Academy
Caldari State
#4 - 2012-10-09 22:14:18 UTC
PvP Status: Winning

Nice. It looks like a solid start for an epic kill board.
Chribba
Otherworld Enterprises
Otherworld Empire
#5 - 2012-10-09 22:41:29 UTC
Unless you haven't already gotten a ton of old ones (2006-2010)...

Base: eve-files.com/chribba/
Files: podbase_data.part1.rar to podbase_data.part4.rar

★★★ Secure 3rd party service ★★★

Visit my in-game channel 'Holy Veldspar'

Twitter @ChribbaVeldspar
ISD Suvetar
ISD Community Communications Liaisons
ISD Alliance
#6 - 2012-10-09 22:49:15 UTC
Chribba wrote:
Unless you haven't already gotten a ton of old ones (2006-2010)...

Base: eve-files.com/chribba/
Files: podbase_data.part1.rar to podbase_data.part4.rar


Community services, you're doing it right Cool

Good luck with the new killboard.

[b]ISD Suvetar Captain/Commando Community Communication Liaisons (CCLs) Interstellar Services Department[/b]
Khorkrak
KarmaFleet
Goonswarm Federation
#7 - 2012-10-09 23:29:59 UTC
Chribba wrote:
Unless you haven't already gotten a ton of old ones (2006-2010)...

Base: eve-files.com/chribba/
Files: podbase_data.part1.rar to podbase_data.part4.rar


Wow that's incredibly cool and useful! Far more than I'd expected.

Developer of http://www.decloaked.com and http://sourceforge.net/projects/pykb/
Thor Kerrigan
Guardians of Asceticism
#8 - 2012-10-10 04:07:00 UTC
I love the pie charts, they look SO tasty :)
Andski
Science and Trade Institute
Caldari State
#9 - 2012-10-10 04:23:31 UTC
I like the fact that you use mod_rewrite to make it easier to look a character up rather than searching.

Twitter: @EVEAndski

"It's easy to speak for the silent majority. They rarely object to what you put into their mouths."    - Abrazzar
Xolve
GoonWaffe
Goonswarm Federation
#10 - 2012-10-10 07:27:54 UTC
Marlona Sky wrote:
PvP Status: Winning

Nice. It looks like a solid start for an epic kill board.


Pretty Much
Barakkus
#11 - 2012-10-10 11:38:56 UTC
Figure out a way to get my history up there without having to post every mail I have sitting on battleclinic and eve-kill :P

http://youtu.be/yytbDZrw1jc
Cede Forster
Deep Core Mining Inc.
Caldari State
#12 - 2012-10-10 11:42:34 UTC  |  Edited by: Cede Forster
I'd like to use this opportunity to congratulate you on the choice to allow the kills to be opened in a new tab.
There are a lot of awesome features and ideas as well but this is the most dire needed improvement.


I hope you keep this project up, ill be looking at it.

suggestion:

at /stats the display of gain/lost could be enhanced with a net gain/loss switch, showing the delta instead of the total
Lin-Young Borovskova
Doomheim
#13 - 2012-10-10 11:53:45 UTC
Khorkrak wrote:
I'm testing the performance of our public, open source, free to use killboard system and would like to see how it fares with increased load - that is more kill mail data, API keys and simultaneous users.

It's still under development. Far from all of the features found in EDK / eve-kill and Battleclinic are present, however more is being added, often daily. Feel free to suggest enhancements as well. Some distinguishing aspects include the absence of all forms of advertising, quick page loads and reliable up-time: decloaked public stats page

Current stats as I write this:
85,797 kill mails
1,119,448 dropped / deleted item rows
714,925 attackers
76,518 characters
13,799 corporations
2,082 alliances
97 API Keys

PVP stats displayed here are fairly unique as well:
There's an overall PVP status so that you can see at a glance how things are going:
http://decloaked.com/pilot/Khorkrak/losses
Coding while farming ISK in null sec hasn't been one of my best moves obviously Oops

On the other side of the spectrum is something like this: http://decloaked.com/pilot/Mystical%20Might/stats You can click on those pie wedges for more detailed info - adding the ability to display the related kills soon as well.

Please check the site out and have fun.

We're on this channel if you'd like to chat: irc.coldfront.net #pykb



Interesting.

brb
Inquisitor Kitchner
The Executives
#14 - 2012-10-10 12:56:52 UTC
Looks like a really good start. Interested to see if you manage to develop this a bit more into a mainstream killboard. Especially with EVE-Kill looking a bit dodgey.

"If an injury has to be done to a man it should be so severe that his vengeance need not be feared." - Niccolo Machiavelli
Lord Ryan
True Xero
#15 - 2012-10-10 13:02:00 UTC

Needs colums!

But cool!

Do not assume anything above this line was typed by me. Nerf the Truth, it's inconvenient.
Thorn Galen
Bene Gesserit ChapterHouse
The Curatores Veritatis Auxiliary
#16 - 2012-10-10 13:13:08 UTC
TBH, this is the best killboard I have seen to date. I really do hope that it becomes a permanent site. Good luck o7

COO - Bene Gesserit Chapterhouse  CEO - Sanctuary Pact Alliance
Lord Ryan
True Xero
#17 - 2012-10-10 13:20:00 UTC
I entered my API. Now how do I search the thing?

Do not assume anything above this line was typed by me. Nerf the Truth, it's inconvenient.
Khorkrak
KarmaFleet
Goonswarm Federation
#18 - 2012-10-10 13:34:31 UTC  |  Edited by: Khorkrak
Just type in some type in the search box in the upper right or enter the relevant info as a url (these are case sensitive though). So for example: http://decloaked.com/pilot/Lord%20Ryan/kills You don't typically have to use %20 instead of a space in most browsers - just type a space Smile Your killmails should be updated soon™.

I just checked the fetch log and see that it tried however there were none available - this can occur if the same API Key used that's in place on another killboard is entered here (CCP please change this as it's really lame and annoying). I remember there being an issue a while back where it was this way across an entire account but I think that's been fixed.

Developer of http://www.decloaked.com and http://sourceforge.net/projects/pykb/
Khorkrak
KarmaFleet
Goonswarm Federation
#19 - 2012-10-10 13:57:04 UTC  |  Edited by: Khorkrak
Barakkus wrote:
Figure out a way to get my history up there without having to post every mail I have sitting on battleclinic and eve-kill :P


We can extract kill mails from EDK killboards provided that the owner is fine with us doing so. As for pulling them from eve-kill, that's doable too but it will take a long time because we'll only pull data for a single entity at a time so that's 1 (pilot / corp / alliance) and adhere to their rule of 1 extract of 50 killmails per half hour.

If you have a dump of text or xml killmails that can work as well - wouldn't be nice if:

  1. You could fetch all past kill log history from the EVE API - even if limited to doing so once a week or whatever.
  2. At least be able to extract your kill mails locally from the EVE Client into a file.


maybe one day....

Developer of http://www.decloaked.com and http://sourceforge.net/projects/pykb/
Khorkrak
KarmaFleet
Goonswarm Federation
#20 - 2012-10-10 14:07:25 UTC  |  Edited by: Khorkrak
Andski wrote:
I like the fact that you use mod_rewrite to make it easier to look a character up rather than searching.


Yep that's the idea - make the urls self explanatory "semantic" as it was referred to by some other bright developers that have offered some sound advice recently.

mod_rewrite was a good guess but the site is not running on Apache. Here's the setup:

CloudFlare CDN -> Gigenet -> Varnish -> Nginx -> uWSGI -> Flask (Python) -> uWSGI Cache -> PostgreSQL

Data aggregation is offloaded to a separate process that only adjusts the summarized data for changes e.g no need to recalculate the entire mountain of stats. It's just a simple cron job that runs every 2 minutes for now.

The server is a modest 2 CPU, 2 GB RAM, 40 GB disk (SAN), 1 IP, cloud server running Ubuntu - no behemoth. The key is caching and spreading the load via the CDN along with using a far better DBMS than MySQL. This particular CDN also filters out some common hacking related noise so that's a nice plus. I may split it into two servers - one for the database and another for the web site if load becomes an issue over time. So far it's fine.

Developer of http://www.decloaked.com and http://sourceforge.net/projects/pykb/