These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

 
  • Topic is locked indefinitely.
 

Eve-Central XML

First post
Author
Previous Topic Next Topic
Mr Kidd
Center for Advanced Studies
Gallente Federation
#1 - 2011-10-25 01:12:42 UTC  |  Edited by: Mr Kidd
I've worked out the basics of importing data using the marketstat_import.xsl file from Eve-central. I've got it working in Openoffice thanks to the thread describing how to do it.

But, I have a few questions about typeid and how I can pull pricing.

Is there a list of typeid's that eve-central uses or do I have to look up each individual item on eve-central in order to get the typeid from the url?

The xsl file from eve-central has avg, max, min, median, volume which is good info, but what if I want prices from specific stations? Is there anyway to pull that information? I might not need detailed instructions. You could just point me in the right direction. I just don't know where to get all this information that I'm needing.

Don't ban me, bro!
Chevalleis
The Scope
Gallente Federation
#2 - 2011-10-25 04:34:44 UTC
Mr Kidd
Center for Advanced Studies
Gallente Federation
#3 - 2011-10-25 12:28:29 UTC
You da man! Thanks.

Don't ban me, bro!
Mr Kidd
Center for Advanced Studies
Gallente Federation
#4 - 2011-10-25 13:18:10 UTC
One more question.

The xsl from eve-central pulls volumes, avgs, median, highs/lows of prices. But I want to be able to pull the individual prices and perform my own analysis. How can I do that?

Don't ban me, bro!
Jenn Makanen
Doomheim
#5 - 2011-10-25 14:12:07 UTC
You can always grab the CSVs of all the uploads, and manage it yourself.

It's not a small volume of data however. And you'll need to merge the changes in. (as the order ids are for each order. as they get updated, the details changes)
CCP Phantom
C C P
C C P Alliance
#6 - 2011-10-25 14:42:58 UTC
Moved from Market Discussions.

CCP Phantom - Senior Community Developer
Mr Kidd
Center for Advanced Studies
Gallente Federation
#7 - 2011-10-25 15:02:30 UTC
Jenn Makanen wrote:
You can always grab the CSVs of all the uploads, and manage it yourself.

It's not a small volume of data however. And you'll need to merge the changes in. (as the order ids are for each order. as they get updated, the details changes)



Yeah, I think that's prolly more than I want to deal with. Thanks for the suggestion though.

CCP Phantom wrote:
Moved from Market Discussions.


Thank you so much for placing this in a forum that most market wiz's prolly don't even browse. CCP, you're WINNING!

Don't ban me, bro!
Jenn Makanen
Doomheim
#8 - 2011-10-25 15:38:18 UTC  |  Edited by: Jenn Makanen
For loading it into a mysql database (you'll need to create a table). you'll need to load a bunch of back-dated files too. linux shell script. You'll also want to slap a bunch of indexes on it, so it responds somewhere near usefully. and have the incarna DB dump loaded, to join against.

#!/bin/bash -l
DATE=`date -d "-1 day" +%Y-%m-%d`
cd /tmp/salesdata
wget http://eve-central.com/dumps/$DATE.dump.gz
gunzip $DATE.dump.gz
dos2unix $DATE.dump
chmod a+r $DATE.dump
mysql eve --execute "load data local infile '/tmp/salesdata/$DATE.dump' Replace into table salesdata Fields terminated by ',' ENCLOSED BY '\"' ignore 1 lines (orderid,regionid,systemid,stationid,typeid,bid,price,minvolume,volremain,volenter,issued,duration,rang,reportedby,@dt) set reportedtime=str_to_date(substring(@dt,1,19), '%Y-%m-%d %H:%i:%s');"



Edit:
You'll probably want to add a username and password to the mysql command. as this is running on a vm on my local machine, I didn't figure it to be a problem. Database called eve, table called salesdata
Mr Kidd
Center for Advanced Studies
Gallente Federation
#9 - 2011-10-25 18:04:00 UTC
Jenn Makanen wrote:
For loading it into a mysql database (you'll need to create a table). you'll need to load a bunch of back-dated files too. linux shell script. You'll also want to slap a bunch of indexes on it, so it responds somewhere near usefully. and have the incarna DB dump loaded, to join against.

#!/bin/bash -l
DATE=`date -d "-1 day" +%Y-%m-%d`
cd /tmp/salesdata
wget http://eve-central.com/dumps/$DATE.dump.gz
gunzip $DATE.dump.gz
dos2unix $DATE.dump
chmod a+r $DATE.dump
mysql eve --execute "load data local infile '/tmp/salesdata/$DATE.dump' Replace into table salesdata Fields terminated by ',' ENCLOSED BY '\"' ignore 1 lines (orderid,regionid,systemid,stationid,typeid,bid,price,minvolume,volremain,volenter,issued,duration,rang,reportedby,@dt) set reportedtime=str_to_date(substring(@dt,1,19), '%Y-%m-%d %H:%i:%s');"



Edit:
You'll probably want to add a username and password to the mysql command. as this is running on a vm on my local machine, I didn't figure it to be a problem. Database called eve, table called salesdata


I've looked at mysql in the past. Honestly, I just don't want to deal with it. I'll use what I can make work without re-inventing the wheel to get all this other stuff to work. It's going to mean installing linux, setting up mysql, etc, just to pull data. I don't need it that bad.

Don't ban me, bro!
Jenn Makanen
Doomheim
#10 - 2011-10-25 18:12:46 UTC  |  Edited by: Jenn Makanen
Well, you could do the entire thing using the windows version of mysql. But I know what you mean. I just had everything already to hand, just needing a few tweaks to get it working Big smile
Tonto Auri
Vhero' Multipurpose Corp
#11 - 2011-10-26 12:16:53 UTC
Mr Kidd wrote:
CCP Phantom wrote:
Moved from Market Discussions.


Thank you so much for placing this in a forum that most market wiz's prolly don't even browse. CCP, you're WINNING!

You're asking question about 3rd party application programming and complaining about your thread being moved to appropriate forum? Kidding, amirite?

Two most common elements in the universe are hydrogen and stupidity. -- Harlan Ellison