These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

 
  • Topic is locked indefinitely.
 

Paying 50mil for a simple SQL query

Author
Teebling
Garoun Investment Bank
Gallente Federation
#1 - 2013-01-26 00:37:57 UTC  |  Edited by: Teebling
Hi Tech & Research,

I've been trying to use Grismar's EvE Explorer and DOTLAN in conjunction in order to find a system that:

  • Is a 0.1 or 0.2 system
  • Has 1 or more stations
  • Has a factory service
  • Is < 10 jumps from either Amarr, Dodixie, Hek or Jita
  • Is > 4j from high sec
  • Has less than 200j a day on average
  • Is a dead end system

Grismar + DOTLAN offer me the luxury of being able to find out systems that meet all of these prerequisites with the exception of the last one (a dead end system). It's fairly important that it's a dead end system, hence why I've come here looking for aid.

I found an SQL query that goes through the latest static data dump to find 'low sec dead-end systems bordering high sec', which can be found here http://wiki.eve-id.net/Low_sec_dead-end_systems_bordering_high_sec and is the following:

SELECT a.solarSystemName AS lowsec, ra.regionName AS region
FROM (
SELECT m.`fromSolarSystemID` AS fromID, m.`toSolarSystemID` AS toID
FROM mapSolarSystemJumps m
GROUP BY fromID
HAVING count(fromID)=1)
AS deadEnds,
mapSolarSystems a,
mapSolarSystems b,
mapRegions ra
WHERE a.solarSystemID=deadEnds.fromID
AND a.`security` < 0.45
AND a.`security` > 0
AND b.solarSystemID=deadEnds.toID
AND b.`security` >= 0.45
AND ra.regionID=a.regionID
ORDER BY a.solarSystemName;

This is good but not perfect :) I'd like one of you to modify this query or make your own, that uses a dynamic api data dump, to also meet all of the prerequisites that I've listed at the beginning of this post and eve mail me the list of systems produced.

Will pay 50mil to the first person who can EvEmail me the results,

Thanks,
Teebling
Louis Vitton
Viziam
Amarr Empire
#2 - 2013-01-27 11:28:52 UTC
I dont think everything you want can fit into a single SQL query.
I think that for less then 10 jumps you will need to do some route planning with a server side code to process it and calculate.
here is a topic on the route planning not sure if you have any interest in it.

https://forums.eveonline.com/default.aspx?g=posts&t=67791&find=unread

also jumps would come from the API so not in the database you would have to put them into a database prasing the api
Iiridayn
Cloning Bay 02477829
#3 - 2013-02-10 10:00:14 UTC
Would you mind clarifying what you mean by "dynamic api data dump"? I am curious to see if your question has an answer, but first I'll need to be certain that I understand the question well enough to check.
Tonto Auri
Vhero' Multipurpose Corp
#4 - 2013-02-10 12:03:16 UTC  |  Edited by: Tonto Auri
Iiridayn, libraries like Pheal offer API data as database tables (siply put, they fetch API feeds into tables ald let you work over them in a straightforward way.)

P.S.
And I agree with above poster, your request far from simple. At least number of jumps information is not readily available in any form, unless you use pre-rendered tables from external sources.

Two most common elements in the universe are hydrogen and stupidity. -- Harlan Ellison

Iiridayn
Cloning Bay 02477829
#5 - 2013-02-11 08:36:35 UTC  |  Edited by: Iiridayn
Tonto - thank you for the reference, but I'm not a big fan of adding layers to my architecture. In this case, I'm somewhat interested in the perverse joy of making it work with just stock T-SQL (a language I've used just once before). Honestly though, since I'm not the customer this time (see http://gaming.stackexchange.com/a/103876/9760), I wanted to get a better handle on the real customer's requirements before spending the next tens of hours building a solution that answers the question I thought was asked.

Teebling - Yes, we've established that this is not a "simple" SQL query. My most significant concern after digging into this further is that I might not be able to do this without a cron, storing historical "jumps in last hour' (http://wiki.eveonline.com/en/wiki/EVE_API_Maps_Jumps) into a table and using that table for the "simple" SQL query.

Therein lies the crux of the acceptance criteria issue - average jumps per day. I was going to offer dropping that one filter in favor of manual inspection - I don't expect a great many systems will match your other criteria. Unfortunately I was able to identify only a single public source for historical jump count records - DOTLAN. Unfortunately, the information is provided in jumps per hour, not jumps per day, and the average over whichever interesting time period would have to be calculated by hand. Additionally, I am not personally super interested in hosting historical API responses myself - I am ... not the most successful systems administrator on the planet. Plus, it would take some days just to gather a small representative sample.

SO - we have a couple of options at this point. One - you might be willing to run the cron and host the database locally. I'm not quite prepared to go there myself, but another player might be willing to help you using something like Pheal. I also might change my mind, but I doubt that this eventuality has a high probability. If you have an alliance, you might have alliance servers and sysadmins to help you out here, but you might have asked them first, and quite possibly gotten rebuffed as here.

Second - we might speak with Wollari about adding average jumps per day to his site. This information would likely be interesting to an audience broader than yourself, and he does already have the data (unlike any other public source a brief search turned up). Again, if you're in a big alliance, you might have a private source, but this seems improbable. I might suggest a small donation, a PLEX or two, might go a long way towards securing his cooperation.

Again though, if average daily jump count isn't critical, I have a decent idea how to put the rest of a (super ugly) "query" (udf) together, and I've checked that the information exists in the static data dump.

Edit: Just checked out your corp, looks like you might have a webmaster after all. If you want to go the own cron route, might want to speak with them first.
Teebling
Garoun Investment Bank
Gallente Federation
#6 - 2013-02-13 15:26:02 UTC
I got your in game message Ilridyan, thanks so much for the work you've put into this!

As for the number of jumps, that isn't as important since as you say, DOTLAN and manual reconnaissance can provide me with the data I need here.

However, do you think you can fit all the other features on the list into one query to be taken from the static dump? That kind of search functionality would still be very useful for me, and anything requiring dynamic API stuff can just be ignored (since the search will narrow results down a lot anyway; it won't take long to find the info I need on said systems myself using other tools such as Grismar's and DOTLAN).

I'm a webmaster myself but I only do front end/art stuff, anything beyond basic PHP scares the living daylights out of me :D
Tonto Auri
Vhero' Multipurpose Corp
#7 - 2013-02-13 16:36:47 UTC  |  Edited by: Tonto Auri
I do believe this should be possible.
Although, constructing list of station facilities from the DB proved to be convoluted, I don't see any apparent roadblocks.
You can start from staStations as your beginning point, and couple it with mapSolarsystems.security to fill first two criteria. 'Have a station' is a major criteria, as they are all listed in a separate table.

Quote:
SELECT * FROM (SELECT s.solarSystemID, s.stationID, COUNT(*) numStations, m.`security` FROM stastations s LEFT JOIN mapsolarsystems m ON s.solarSystemID = m.solarSystemID WHERE m.`security` > 0 AND m.`security` < 0.25 GROUP BY s.solarSystemID, s.stationID) t LEFT JOIN (SELECT j.fromSolarsystemID solarSystemID, COUNT(*) numJumps FROM mapsolarsystemjumps j GROUP BY j.fromSolarsystemID HAVING numJumps = 1) g ON t.solarSystemID = g.solarsystemID


12 systems. 19 stations. Anyone else? I didn't included the check for manufacturing facilities, though.

EDIT: Modified query to include station ID's for further buildup.

Two most common elements in the universe are hydrogen and stupidity. -- Harlan Ellison

Iiridayn
Cloning Bay 02477829
#8 - 2013-02-14 11:51:43 UTC  |  Edited by: Iiridayn
So, I took it this far:

Quote:
SELECT * FROM mapSolarSystems JOIN (
SELECT fromSolarSystemID FROM mapSolarSystemJumps GROUP BY fromSolarSystemID HAVING COUNT(fromSolarSystemID) = 1
) AS deadEndSolarSystems ON (mapSolarSystems.solarSystemID = deadEndSolarSystems.fromSolarSystemID)
WHERE security < 0.25 and security > 0 AND solarSystemID IN (
SELECT DISTINCT solarSystemID FROM staStations WHERE operationID IN (SELECT operationID FROM staOperationServices WHERE serviceID = 8192)
)


And discovered that there are exactly 4 systems that are dead end systems in 0.1 or 0.2 with factory services. Since my route distance calculator is ugly, I switched to a brief manual inspection, to see if building up the query would be worthwhile.

Of the 4 systems that matched the query, only one is more than 4 jumps from high sec, but Kuhri is 25 jumps from Amarr, the closest of the 4 systems you were interested in (Jita: 34, Rens: 36, Dodixie: 39).

Notoras is interesting though - exactly 4 jumps from high sec, and only 8 jumps to Jita (and 10 to Dodixie). Unsurprisingly, visual inspection reveals that daily jumps aren't as low as they could be, with (at the present) 411 jumps in the past 24 hours.

The other two systems are Camal (Rens: 13, Amarr: 20, Dodixie: 22, Jita: 28) with only 2 jumps to high sec, and Rorsins (Dodixie: 10, Rens: 11, Jita: 10, Amarr: 17) and only 3 jumps from high sec - and only 136 jumps in the past 24 hours.

I do not completely understand your requirements - but my own predilections as a high sec carebear would lead me to prefer Rorsins over Notoras due to the lower jump frequency, and the reduced low sec travel distance would not bother me ;).
Tonto Auri
Vhero' Multipurpose Corp
#9 - 2013-02-14 22:01:48 UTC
You shouldn't start from mapSolarSystems, as I mentioned earlier. It's inefficeint.

Two most common elements in the universe are hydrogen and stupidity. -- Harlan Ellison

Tonto Auri
Vhero' Multipurpose Corp
#10 - 2013-02-14 22:05:09 UTC
Iiridayn wrote:
I do not completely understand your requirements - but my own predilections as a high sec carebear would lead me to prefer Rorsins over Notoras due to the lower jump frequency, and the reduced low sec travel distance would not bother me ;).

Sudenly, the number of jumps in all mentioned systems spike ^_^

Two most common elements in the universe are hydrogen and stupidity. -- Harlan Ellison

Iiridayn
Cloning Bay 02477829
#11 - 2013-02-15 05:06:07 UTC
Tonto Auri wrote:
Sudenly, the number of jumps in all mentioned systems spike ^_^

Heh, with all of the two people who bother to read this :). I was worried at first about giving away Sivala as the center of high sec when I found it out - but realized shortly afterwards that nobody else cared ;).