These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

 
  • Topic is locked indefinitely.
 

Real-time market data service: EVE Market Data Relay
Author
Previous Topic Next Topic
Muscaat
EVE Markets
#241 - 2012-11-02 14:32:55 UTC
Desmont McCallock wrote:
Good to know you are still around. Have you noticed that your stats page is broken?

Which page is this? Site seems fine to me...
Ilyk Halibut
Deep Core Mining Inc.
Caldari State
#242 - 2012-11-02 15:18:14 UTC
Vaerah Vahrokha wrote:
Muscaat wrote:
Vaerah Vahrokha wrote:
I'd love if you or some other experienced programmer in there could suggest me a cheap hosting service that lets me install such a permanently running application needed to listen to ZeroMQ messages.
I have checked Webfaction and Dreamhost but both are expensive for totally free EvE projects. Plus they really push into multi-year subscription purchases and I can't commit to such a budget.
All I needed to do is to listen for EMDR messages 24/7 and slap them in a database, I don't even need a web frontend (no I can't do it with a computer at home).

I'm in a similar position to you, Vaerah - my host doesn't want persistent processes but I'd like to collect EMDR data. I went for a cheap VPS - there are some really cheap packages around. http://www.lowendbox.com/ is a good place to start looking. We're talking down to about US$3/mo in some cases. Might be worth a try.


Thank you, much appreciated! I can shell $3 for something I won't ever get a nickel from, but $12 was a bit too steep (unless going for a lifelong pre-paid plan then it drops to $8).


The thing you'll want to be really careful of is bandwidth limits. A lot of cheaper VPS services will lure you in with a low price, but cut other things like bandwidth. EMDR can suck up a good bit. I don't know what consumption is like for a single consumer these days (someone else feel free to chime in), but if you pay $3/month, only to get $12+ in bandwidth overage fees, you haven't really saved any money.

Also, $12 isn't necessarily what you'd call an expensive VPS :)

EVE Market Data Relay - A real-time feed of EVE Market data http://www.eve-emdr.com
Muscaat
EVE Markets
#243 - 2012-11-02 16:49:40 UTC
Ilyk Halibut wrote:
I don't know what consumption is like for a single consumer these days (someone else feel free to chime in)


I'm looking at about 1GB/week on my consumer (roughly - that figure includes other traffic too).
Vaerah Vahrokha
Vahrokh Consulting
#244 - 2012-11-02 18:18:21 UTC
Ilyk Halibut wrote:

The thing you'll want to be really careful of is bandwidth limits. A lot of cheaper VPS services will lure you in with a low price, but cut other things like bandwidth. EMDR can suck up a good bit. I don't know what consumption is like for a single consumer these days (someone else feel free to chime in), but if you pay $3/month, only to get $12+ in bandwidth overage fees, you haven't really saved any money.

Also, $12 isn't necessarily what you'd call an expensive VPS :)


The $8 plan supported 300GB / month so it should be safe enough.

However it's not a VPS, VPS cost a lot more and at this point I find no justification for throwing in from $500 to $1200 a year for something that yields no revenue at all.


As for bandwidth usage, what about providing the ability to pick a family of markets? I am only interested in a small subset yet I'll have both to un-necessarily load a CPU and consume bandwidth to consume all the data and discard 95% of it and even then, only keep Jita prices out of that.

It'd be cool to construct a JSON or something request involving the desired items IDs and regions / stations and just pull the strictly needed data.

EvE Online portal | EvE markets tutorials
Ilyk Halibut
Deep Core Mining Inc.
Caldari State
#245 - 2012-11-02 19:17:51 UTC
Vaerah Vahrokha wrote:

The $8 plan supported 300GB / month so it should be safe enough.

However it's not a VPS, VPS cost a lot more and at this point I find no justification for throwing in from $500 to $1200 a year for something that yields no revenue at all.


Wat? There are plenty of similarly priced VPS solutions in the $10-$20 range. I suggest Rackspace ($16) for those on a budget, but wanting rock solid service. Linode has an awesome plan at $20 if you need a little more beef. You can go cheaper, but often at the expense of quality.

That said, I personally have zero interest in catering to those who can't run daemons. This is a fun hobby for me, and that just doesn't excite me. It'd probably involve writing an API, which is not what EMDR is. It's a dumb stream of data.

Vaerah Vahrokha wrote:

It'd be cool to construct a JSON or something request involving the desired items IDs and regions / stations and just pull the strictly needed data.


This is just not how EMDR works. It's not really even a technical possibility without me splicing and dicing feeds and finding relays for a number of differently flavored feeds.

On a related note, I wrote and currently maintain EMDR for fun, and I cater to a happy medium. I spend about $20/month to run EMDR, and I never see a dime back. But that doesn't matter, I enjoy the technical challenge. I think you'll need to ask yourself whether you're viewing this as a hobby (a $10-$20/month hobby is dirt cheap), or a burden that you're trying to cut losses from.

Just don't go out to eat one night a month and you've basically earned your costs back. Maybe skip on that expensive coffee a time or two.

EVE Market Data Relay - A real-time feed of EVE Market data http://www.eve-emdr.com
Desmont McCallock
#246 - 2012-11-02 20:16:10 UTC  |  Edited by: Desmont McCallock
Muscaat wrote:
Desmont McCallock wrote:
Good to know you are still around. Have you noticed that your stats page is broken?

Which page is this? Site seems fine to me...
Actually I have to bug Otaci about that. You both use the same words for your sites but with a twist (and that confused me).
Muscaat
EVE Markets
#247 - 2012-11-04 11:27:51 UTC
Desmont McCallock wrote:
Muscaat wrote:
Desmont McCallock wrote:
Good to know you are still around. Have you noticed that your stats page is broken?

Which page is this? Site seems fine to me...
Actually I have to bug Otaci about that. You both use the same words for your sites but with a twist (and that confused me).

Ha, thought it might have been that P
Gordon Newman
Deep Core Mining Inc.
Caldari State
#248 - 2012-11-07 15:16:44 UTC
Vaerah Vahrokha wrote:

It'd be cool to construct a JSON or something request involving the desired items IDs and regions / stations and just pull the strictly needed data.


What you're looking for is an API to fetch market data for your app. Over at element43 we're at the beginning of writing such an API so you don't have to deal with the hassles of acquiring, filtering and storing market data from EMDR, if all you want to do is just analyzing that data. Feel free to visit our IRC: #element43 at irc.coldfront.net We're always looking for feedback to improve our tool.
Alternatively you can just use our consumer code, which you can get over at GitHub. It should run on any decent linux VPS.

Co-Founder of element43 - Join our team! | EVE Live | EMDRMon | accounts-eve
Ilyk Halibut
Deep Core Mining Inc.
Caldari State
#249 - 2012-11-07 18:47:05 UTC
These Element 43 guys are doing great work. For those needing some "value-added" stuff, get in touch with them like Gordon is saying. They can hook you up, and we're all happy.

Big props for doing this all open source, too!

EVE Market Data Relay - A real-time feed of EVE Market data http://www.eve-emdr.com
Desmont McCallock
#250 - 2012-12-26 16:19:24 UTC
Presenting EMDR Monitor.