These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

 
  • Topic is locked indefinitely.
 

[RTM] Real time market data project

Author
Johnny Giuliano
Doomheim
#1 - 2016-01-10 05:36:55 UTC
Hello there,

I started working on a project based on EMDR stream data for market information, based on nodejs and redis cache at the moment. In the future when the project will grow i will move it to use MongoDB.

I would like to know for people that are really into market operations, what they would like to see from a realtime market data, this means in terms of general features, and also features that would be available when you login with Eve SSO and provide your API key for just market operations information.

Currently there are 2 pages developed, and in beta testing (hence the lack of items names, regions, stations and so on)

Image 1 - sell/buy orders view in this list the items are updated realtime, and categorized by the volume of transactions, with the following logic: if a listed item is there and we gather an update from EMDR, the new value is pushed for the item, if the item is not there and it qualifies for the category to be viewed, it will be pushed as a new item in the list. in this view you can find the station id, region id, and all information about the orders.

Image 2 - main page - list of items , in this view you will find the list of items that are the most transactioned at the moment, with their respective itemd (didn't pull out the name yet, this will be done after core functionalities will be coded) and how many total orders are for this item. Click on a specific item, will detail the sell and buy orders updated realtime for that specific item only, and where you can find it.

Showing this, i want some feedback from you about what functions you will like to use, see, because i am not so financial tycoon and for sure i will miss some features if i don't ask around

Thanks !
Pete Butcher
The Scope
Gallente Federation
#2 - 2016-01-10 08:15:13 UTC
Doesn't eve-central already do all this? You can check out their code.

http://evernus.com - the ultimate multiplatform EVE trade tool + nullsec Alliance Market tool

Johnny Giuliano
Doomheim
#3 - 2016-01-10 08:22:35 UTC
Pete Butcher wrote:
Doesn't eve-central already do all this? You can check out their code.


eve-central is too complex, here is a list of requirements:

Requirements (Scala core data API and upload ingester)

Java 7 or 8 (JDK)
SBT (Scala Simple Build Tool) 0.13.x
PostgreSQL 9.1+
Requirements (Python web UI stack)

Python 2.7 (3.x not supported)
psycopg2
Pyro (for the pathfinder server)
numpy (for statistic calculations)
CherryPy 3.2 (primary web server)
PostgreSQL 9.0+


my platform only uses 1 file and 2 softwares: nodejs + redis cache. all client side things are done in Angular.js, and it provides an JSON API for every output, in which form you want , so what pages will be populated on the frontend, that specific information will be available on the API.

i want to achieve a more robust, user friendly platform with latest html5 tech and programming , but not just the frontend, but with user login and personal features regarding the market. Maybe to select some favorite items to watch on your own dashboard rather than all, things like these.

That is why i would like to listen to you guys and drive me in a direction to implement features that currently are scattered around multiple market sites or don't exist at all.


Pete Butcher
The Scope
Gallente Federation
#4 - 2016-01-10 08:55:08 UTC
All I can say at the moment is - stay away from CREST. It's so unreliable and so slow, you won't get far with it. I doubt you can fetch the whole market withing a day.

http://evernus.com - the ultimate multiplatform EVE trade tool + nullsec Alliance Market tool

Desmont McCallock
#5 - 2016-01-10 09:06:07 UTC
Sorry to inform you but as of version 3 of EVEMon, we are removing our market uploader (EMUU).
EVEMon provides the 95% of EMDR's data.

You should switch to CREST instead.
Johnny Giuliano
Doomheim
#6 - 2016-01-10 09:16:32 UTC
Thanks for the notice, due to the nature of programming is very easy to switch to another source, as i use JSON to decode the EMDR data, and also the CREST response.

Pete Butcher
The Scope
Gallente Federation
#7 - 2016-01-10 09:32:17 UTC
Desmont McCallock wrote:
Sorry to inform you but as of version 3 of EVEMon, we are removing our market uploader (EMUU).
EVEMon provides the 95% of EMDR's data.

You should switch to CREST instead.


Hope they'll fix it before you turn it off.

http://evernus.com - the ultimate multiplatform EVE trade tool + nullsec Alliance Market tool