These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

 
  • Topic is locked indefinitely.
 

Remote Mongo, MySQL and SQL Server access to CCP data dump (Crucible 1.5).
Author
Previous Topic Next Topic
Kouryusei
Keizai Inc
#1 - 2012-03-18 19:13:21 UTC  |  Edited by: Kouryusei
Do you want to access the data stored within the CCP static data dump without actually installing SQL Server? Do you want to connect to it via MySQL or Mongo instead? Well, now you can. I've put up a publically accessible version of the latest data dump (Crucible 1.5) with connectors available for Mongo, MySQL and SQL Server. These instances will be updated with the latest version of the data as and when it is released.

I have also put up the necessary files if you want to import directly in to your own Mongo or MySQL instance. If you're importing in to SQL Server, then use the damn files that CCP have provided. In the near future (next week), I will be providing an open API for people wanting to do a lookup on the database without actually having to connect to the database.

The below instances are all freely accessible and are in read-only mode. They're helping me use a tiny fraction of the excess capacity I have on my rather extensive AWS infrastructure.

Remote Access

Mongo

Currently setting up secure infrastructure, should be done by 2PM GMT on 19th March, 2012.

MySQL

Host: eve-mysql.develop.cx (develop.cdivhyo8z4rb.eu-west-1.rds.amazonaws.com)

Database: ccpdata_dbo
User: ccp
Pass: ccp

SQL Server

Host: eve-db.develop.cx (46.137.88.154)

User: ccp
Pass: ccp

Database Exports

These will be coming in the next 6 - 12 hours.
Leolie Aylet
The Cybernetic Consciousness
#2 - 2012-03-19 04:09:40 UTC
Great !

But on the mysql server, in the table invType there are missing entries isn't it ?
TypeID jumps at 17425 to 30118.

It's possible to have all the entries ?
LifeHatesMe
LifeHatesUsAll
#3 - 2012-03-20 07:48:52 UTC
So.. can we replicate from your remote server to our local one? (aka download)

And it says your export option will be done sometime 10 hours ago.. ??
Steve Ronuken
Fuzzwork Enterprises
Vote Steve Ronuken for CSM
#4 - 2012-03-20 09:38:50 UTC
LifeHatesMe wrote:
So.. can we replicate from your remote server to our local one? (aka download)

And it says your export option will be done sometime 10 hours ago.. ??


If you want a local copy in mysql, just grab from my site. (or check out the other threads, with the other ports that have been done)

Woo! CSM XI!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter
Kouryusei
Keizai Inc
#5 - 2012-03-20 10:50:23 UTC
Leolie Aylet wrote:
Great !

But on the mysql server, in the table invType there are missing entries isn't it ?
TypeID jumps at 17425 to 30118.

It's possible to have all the entries ?


Currently looking in to this for you.

LifeHatesMe wrote:
So.. can we replicate from your remote server to our local one? (aka download)

And it says your export option will be done sometime 10 hours ago.. ??


You're more than welcome to do that, and some stuff came up - only just got home!

Steve Ronuken wrote:
LifeHatesMe wrote:
So.. can we replicate from your remote server to our local one? (aka download)

And it says your export option will be done sometime 10 hours ago.. ??


If you want a local copy in mysql, just grab from my site. (or check out the other threads, with the other ports that have been done)


As per this post, if you need a local copy in MySQL - you can just grab it from someone else. The primary idea of these is just to be a remotely accessible variant for applications (or people) who don't want to (or can't) run their own database server. The Mongo version is about go up now I'm actually back, then I'll run the necessary exports.
Cyerus
University of Caille
Gallente Federation
#6 - 2012-03-20 12:21:14 UTC
Good idea, although I won't be using the feature.

I assume that all the access is read-only?
Kouryusei
Keizai Inc
#7 - 2012-03-20 12:57:38 UTC
Cyerus wrote:
Good idea, although I won't be using the feature.

I assume that all the access is read-only?


Indeed it is. I'm considering making on-demand read/write instances available for people, but I'm not sure what the demand would be like, if at all.
Leolie Aylet
The Cybernetic Consciousness
#8 - 2012-03-22 20:16:55 UTC
If it can help, on my local database, the table invType have 18708 rows.
On yours, the table only contains 7178 rows.

There are some missing entries i guess.
LifeHatesMe
LifeHatesUsAll
#9 - 2012-03-23 00:09:19 UTC
Leolie Aylet wrote:
If it can help, on my local database, the table invType have 18708 rows.
On yours, the table only contains 7178 rows.

There are some missing entries i guess.

I am +1'ing your post. Good job Leolie!! :D
Kouryusei
Keizai Inc
#10 - 2012-03-24 15:44:45 UTC
I'm making some changes to the way the system works, keep an eye on this thread - it'll now work by spinning up a new read/write instance of your own database. The current MySQL data is incorrect, as per posts above - currently resolving this.
LifeHatesMe
LifeHatesUsAll
#11 - 2012-03-26 00:40:32 UTC
Kouryusei wrote:
I'm making some changes to the way the system works, keep an eye on this thread - it'll now work by spinning up a new read/write instance of your own database. The current MySQL data is incorrect, as per posts above - currently resolving this.
When can I fix your database thingie?