These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

 
  • Topic is locked indefinitely.
123Next page
 

Element43 - Next-Gen Markets and Industry - Vanguard 1.0

First post
Author
Gordon Newman
Deep Core Mining Inc.
Caldari State
#1 - 2013-06-30 20:40:32 UTC  |  Edited by: Gordon Newman
element-43.com

Element43 is an all-in-one site to track the markets, your industry jobs, trading, you name it. Our goal with Element43 is to provide you, the capsuleer, with everything you need to successfully manage your EVE-wide enterprises. We are an open source project available on GitHub at https://github.com/EVE-Tools/element43



The not so short version:

After nearly a year of constant work and close to 900 commits in our GitHub repository we feel that time has come to present you Element43. Element43 allows you to browse EVE market information in style. No more scrolling through endless lists of orders, no more hassling with plain XML. Access close to real-time market data and personalized financial analysis with ease. Want to know average prices and volumes of certain items all around the universe? Want to access historic pricing and volume information? No problem - chances are high that the answer to your questions is just a single click away. With this version of Element43 we want to lay the foundation of a community-driven, extendable and accessible all-in-one tool.

This post is divided into three sections representing the three major groups of users we‘d like to address:


  1. Players - The discerning players and traders/industrialists starting their business who look up prices now and then and might like some tracking of their assets and financial undertakings

  2. Advanced Players - Advanced industrialists who would like to feed realtime prices into their spreadsheets

  3. Developers - Developers of 3rd party tools - no matter what background you come from



Players

Element43 allows you to browse the EVE market in a clear and simple way. Your user experience is one of our top priorities - what is market data worth if you can’t access it in a convenient and intelligible way? Presentation of data is about assisting you in turning that data into decisions. We want to help you to spend less time digging through endless tables and actually get you making decisions and playing the game. We’re just at the beginning of the project, but you can already get a quick overview over the (past) prices in the main trade-hubs, as well as detailed regional analysis of a certain item. Furthermore you can already track your orders and your wallet, so you stay on top of your trading activities. You‘ll also want to try our manufacturing calculator, the tradefinder and the station ranking.

Get started now: https://element-43.com/

Advanced Players

Remember when you were reading through raw XML while all you wanted to get was the price for Tritanium in Jita? We offer a full, syntax-highlighted web-browseable API for conveniently accessing raw data in the format of your choice (XML, JSON and YAML). To support advanced custom calculations, we have implemented powerful API endpoints which allow you to query not only prices, but also price history (high, low, average, even volumes traded) and the full static data export provided by CCP. If you just want to evaluate our data, you can try the legacy marketstat API that should be compatible to EVE-Central‘s.

Visit: https://element-43.com/api/

Future versions will include advanced industrial tracking capabilities.

A guide on how to get our data working with your spreadsheets is in the works.

These endpoints should be most interesting:

itemRegionStat (Regional Item Statistics)
orderHistory (Price History for Regions)

Click the button labeled OPTIONS to get a list of filters you can apply.

Co-Founder of element43 - Join our team! | EVE Live | EMDRMon | accounts-eve

Gordon Newman
Deep Core Mining Inc.
Caldari State
#2 - 2013-06-30 20:41:30 UTC  |  Edited by: Gordon Newman
Developers

Preamble

Quote:
We have always been able (for some websites) to contribute to an existing website, but I have found that the difficulty with doing so is that you feel it is someone else's baby already. They might accept your code, or they might reject it for any number of (valid or invalid) reasons.

I know I have things I'd rather not see in EMK, but I've also seen lots of features on other websites, that I haven't been able to produce myself, or that I think would be awesome, if given a decent layout. I also have lots of ideas on how I would redo a market website if I would start from scratch again, and there are probably lots of developers out there whom would be able to give me very good suggestions on how to do it even better.

Therefore, I pose to you the question: do you think we would be able to build a common market website together, pooling development resources, and quite possibly building the best site yet?

Link to original Thread


This is what Element43 aims to be: the framework and “decent layout” for your ideas for the next generation of EVE market data and personal financial and industrial tracking applications. Element43 is not only supposed to be accessible for the average user, it is also designed to get developers started in a fast and easy way. We want to enable you to conveniently create beautiful and accessible apps. Wouldn‘t it be just awesome to work on a project together? We don‘t claim to be perfect, however one has to start somewhere. We tried to provide a solid, modular foundation, in order to enable you to create powerful EVE-related modules. This is some of the stuff we already have done for you:


  • Wrote two advanced consumers for EMDR market data
  • Mapped the entire SDE to objects for convenient access via the ORM
  • Stored over 30GB of market data in our main database
  • ~ 3.5M Active orders
  • ~ 70M Archived ordrs
  • ~ 69M History datapoints (OHLC + Volume)
  • + Lots of other statistics
  • Wrote documentation to get you started
  • Implemented a distributed task queue e.g. for API updates
  • Wrote a registration and authentication system for users and their API keys
  • Wrote lots of (boilerplate) code you can refer to
  • Created logos and a design, which is simple to use and extend - even for backend programmers :)
  • Deployed the code to a main instance hosted at element-43.com



If you want to start a project like this - you can‘t get around picking a programming language and a framework you‘ll build-up on. We have chosen Python + Django for Element43, since it is easy to learn, modular and well-tested. We know this is not going to fit everyone‘s preferences, but we hope that in the effort to create a great tool for the community (in the end this is what this is all about!) we can overcome our personal coding habits and try something new. In fact both the founding members came from completely different backgrounds (PHP and Ruby on Rails). Consider this our invitation to you to become a part of our team, so we can work together and create something lasting the entire EVE community will benefit from.

A brief introduction on how to get started as a developer can be found here: https://element43.readthedocs.org/en/latest/gettingstarted.html

Element43 is a true international open-source project available at: https://github.com/EVE-Tools/element43

Feel free to fork the repository and hack around. We‘re looking forward to merging-in your creations and welcoming you in our team.

If you have any questions or feedback drop us an e-mail at info@element-43.com or visit our IRC channel over at #element43 on irc.coldfront.net
You can also post your thoughts in our Google Group (https://groups.google.com/forum/#!forum/element43) or simply reply to this post.

A slide-in feedback form will be available shortly

Current Team

Project Leads / Core Devs:
Greg Oberfield / greg
Sebastian Wagner / zweizeichen (me)

Developers:
Jens Lauterbach / jens

Code, Support, Help:
Greg Taylor / gtaylor (Creator of EMDR)

System Admins and Hardware Sponsors:
zweizeichen

Snipa (will be back later this year)

Consultants:
Blake (http://k162space.com)

Co-Founder of element43 - Join our team! | EVE Live | EMDRMon | accounts-eve

Gordon Newman
Deep Core Mining Inc.
Caldari State
#3 - 2013-06-30 20:42:05 UTC
[placeholder]

Co-Founder of element43 - Join our team! | EVE Live | EMDRMon | accounts-eve

Risingson
#4 - 2013-06-30 21:19:25 UTC  |  Edited by: Risingson
got to check it out but heimatar is red and sinq is green for sure :)
Grant Blackman
Garoun Investment Bank
Gallente Federation
#5 - 2013-07-01 16:30:03 UTC
This isn't my usual toon, but I'm the guy that keeps EMDR running. For what it's worth, E43 has been developed the right way, and I really like the emphasis on style and ease of use. It's also one of the easiest market sites to contribute to.

Definitely worth a look, even for those that have existing sites they prefer.
Kid Delicious
Caldari Provisions
Caldari State
#6 - 2013-07-02 07:55:10 UTC
This looks really good. Some organization on the api page would go a long way, like the rest_api/urls.py file. Also maybe separating custom apis from sde tables.
Gordon Newman
Deep Core Mining Inc.
Caldari State
#7 - 2013-07-02 16:13:24 UTC
Thank you for your suggestions. Some proper API Documentation is in the works. When it's done, we'll release it on our GitHub page at http://element43.github.io. Improving the structure and speed of the individual views could take some time, though.

Co-Founder of element43 - Join our team! | EVE Live | EMDRMon | accounts-eve

Gordon Newman
Deep Core Mining Inc.
Caldari State
#8 - 2013-07-05 15:52:33 UTC
We're planning to introduce indices which will provide you with powerful indicators for certain groups of items (Minerals, Ices...) - join the discussion here and help shaping this feature: Topic

Co-Founder of element43 - Join our team! | EVE Live | EMDRMon | accounts-eve

ShadowandLight
Trigger Happy Capsuleers
#9 - 2013-07-26 02:52:46 UTC
is this software able to be locally installed on a personal server? or is the data put onto your server encrypted per pilot?
Erica Northman
Aliastra
Gallente Federation
#10 - 2013-07-26 09:15:46 UTC
ShadowandLight wrote:
is this software able to be locally installed on a personal server? or is the data put onto your server encrypted per pilot?


https://github.com/EVE-Tools/element43

Creator of evemanager. Project website.

Vaerah Vahrokha
Vahrokh Consulting
#11 - 2013-08-04 19:45:57 UTC
Hello,

this looks like the kind of project I always wanted to start and never had time to do.

Is there any provision to implement OHLC based candle charts?
Vaerah Vahrokha
Vahrokh Consulting
#12 - 2013-08-11 11:32:17 UTC  |  Edited by: Vaerah Vahrokha
Website seems only to work in the home page and little else, the rest shows "Connecting..." in the browser and stays like that forever.

Edit: not forever, after some minutes I got a "502 Bad gateway".
BarbersHuZ
M00nlight Shadows
#13 - 2013-08-12 19:44:58 UTC
The project get new servers and is down for a few days.
Gordon Newman
Deep Core Mining Inc.
Caldari State
#14 - 2013-09-22 13:41:07 UTC
Everything is back online with new hardware now - faster than ever before.
We also deployed a couple of important updates such as the Odyssey 1.1 SDE and various layout-related libraries. There might be a layout glitch here and there, we're currently working on fixing those. The portal's overall function should not be affected by this.

Vaerah Vahrokha wrote:

Is there any provision to implement OHLC based candle charts?

Try clicking the order's region in the quicklook view. This will give you the chart you're looking for.


Stay tuned for further updates Big smile

Co-Founder of element43 - Join our team! | EVE Live | EMDRMon | accounts-eve

Gordon Newman
Deep Core Mining Inc.
Caldari State
#15 - 2013-09-23 15:57:54 UTC  |  Edited by: Gordon Newman
Changelog for today:


  • Greatly improved our layout on most mobile devices (responsive layout + media queries)
  • Fixed the personal wallet chart of the dashboard
  • Restored continuity of API updates


As always: Feel free to send us your feedback and report any bugs you encounter! Big smile

Co-Founder of element43 - Join our team! | EVE Live | EMDRMon | accounts-eve

Vaerah Vahrokha
Vahrokh Consulting
#16 - 2013-09-24 06:58:52 UTC
Gordon Newman wrote:

Try clicking the order's region in the quicklook view. This will give you the chart you're looking for.


Stay tuned for further updates Big smile


Hello,

I have watched this PLEX chart.

It has something wrong, the market is not like that. If it was, it could not be swing traded.

Here's how the PLEX market really appeared for real and also the same "model" shown by Dr Ejyo at Fanfest 2013.

Don't look at the trend lines or horizontal lines but at the "candle" bars. As you see the market structure is visible and tradable.
Gordon Newman
Deep Core Mining Inc.
Caldari State
#17 - 2013-09-24 15:49:52 UTC
You're right - that chart is kind of broken. Probably this is is a data grouping and scaling issue. The data itself should be correct, as we're getting it straight from EVE's cache via EMDR and no special post processing is being applied to the numbers. Especially here it is really difficult to distinguish movements since there are a lot of extreme outliers which cause the entire graph to zoom out (just look at the scale -250M to +750MSmile).

A possible solution for that problem would be to group our candle bars - this would iron out most of those outliers. Or we could change the way we generate the data points for our graph. To be honest I'm not particularly happy with our current solution - it basically takes yesterday's mean value as 'open' value and today's mean as 'close' value. Probably the solution for this problem would involve a mixture of both approaches.

Do you have any specific suggestions on how that chart could be improved? AFAIK you've turned EVE's market data into candle bar charts for a while now Blink Also, do you know if there is a recording of Dr Ejyo's talk?

Co-Founder of element43 - Join our team! | EVE Live | EMDRMon | accounts-eve

Vaerah Vahrokha
Vahrokh Consulting
#18 - 2013-09-24 18:26:21 UTC  |  Edited by: Vaerah Vahrokha
Gordon Newman wrote:
You're right - that chart is kind of broken. Probably this is is a data grouping and scaling issue. The data itself should be correct, as we're getting it straight from EVE's cache via EMDR and no special post processing is being applied to the numbers. Especially here it is really difficult to distinguish movements since there are a lot of extreme outliers which cause the entire graph to zoom out (just look at the scale -250M to +750MSmile).

A possible solution for that problem would be to group our candle bars - this would iron out most of those outliers. Or we could change the way we generate the data points for our graph. To be honest I'm not particularly happy with our current solution - it basically takes yesterday's mean value as 'open' value and today's mean as 'close' value. Probably the solution for this problem would involve a mixture of both approaches.

Do you have any specific suggestions on how that chart could be improved? AFAIK you've turned EVE's market data into candle bar charts for a while now Blink Also, do you know if there is a recording of Dr Ejyo's talk?


As you might know, I have given an hand or 3 to implement EveMarketeer.com (now dead Cry) bar charts and also a Guild Wars 2 similar website, if you want I'll gladly do the same for your website as well.

For a starter, EvE provides HLC data and you need OHLC data. The "O" may be synthesized. But in your case, depending how you store the data, you might have all the 4 data points.
If you want to discuss about this and also about the rules to create financial software compliant bars (which in practice are good coding practices anyway) feel free to eve mail me. I could use a brief description about how you get the data points and when.

As for Dr Ejyo's talk yes it's been recorded and yes he has shown "somewhat inspired" candle charts P. If I find the video URL I'll post it.
Gordon Newman
Deep Core Mining Inc.
Caldari State
#19 - 2013-09-25 18:08:28 UTC  |  Edited by: Gordon Newman
Found the talks:

The one you mentioned

and

Another one about DUST514's economy

I hope that we can get our hands on official data soonish™

Co-Founder of element43 - Join our team! | EVE Live | EMDRMon | accounts-eve

Gordon Newman
Deep Core Mining Inc.
Caldari State
#20 - 2013-11-18 18:58:51 UTC
Our server is currently undergoing network-related maintenance by our hosting provider (OVH) - that's the price you pay for cheap servers Smile Everything should be back to normal in a couple of days.

Co-Founder of element43 - Join our team! | EVE Live | EMDRMon | accounts-eve

123Next page