These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

Market Discussions

 
  • Topic is locked indefinitely.
 

Experiment #01: RL finance analysis applied to EvE

First post First post First post
Author
roigon
TURN LEFT
#21 - 2011-12-25 23:57:41 UTC
Vaerah Vahrokha wrote:
...


I see, so to basically while the segmentation of the market in candles is arbitrary, the numbers they enclose are not, and by segmenting those we can get a clearer picture of what the market was doing in that segment of time. As such we can rely on the sequence of those candles to give us a picture of how the market has been evolving over time not only in high/low/avg/daucion(sp), but also in buy vs. sell giving a potentially clearer picture of movement vs. graph, because graph isn't segmented. (although I guess with daucian(sp) channel you could still have the "wick" sort of. But by enclosing that in segments/candles we can remove complexity by way of abstraction.

I guess I should dive into the older thread though, but it's all quite interesting. My current strategy has been to simply find stable fast moving products and set up buy and sell orders that get triggered in the normal weekly movement of the product price. My all time favorite item being DC2's that more often then not have a almost flat price with very little fluctuation. But still a workable margin between buy and sell so that I don't have to adjust my prices all that often. (why yes I'm lazy)

Although for the last 2 weeks I've been writing my own application to be able to rapidly automate analyzing of market items to find hidden gems. Gaps the market hasn't yet picked up on. Luckily being a developer of trade the building of this software was fun and easy. Unfortunately I have little market knowledge so felt a bit lost as to what to do with all this data that I can now so easily collect. Which incidentally is why I stumbled upon this post. I don't think the method you describe lends itself easily for automation, but it's interesting in it's own right.

I'm thinking about computing either the avg base production cost of items to see how much of a "bubble" the market value is vs. production cost, or maybe compute the reprocessing value vs. my own skills and see if I can pick up stuff that's worth more reprocessed then it's market value. (Which I've seen happen once or twice)

The easiest would probably be to pick two market hubs and do a comparative price check and either haul or hire a hauler to make my profit. But meh, that sounds too much like actual work.
Callean Drevus
Perkone
Caldari State
#22 - 2011-12-26 03:48:14 UTC
roigon wrote:
...


It actually lends itself pretty well to automation. I added it to EVE Marketeer in under two hours. Then again, I already had all the infrastructure built.

Most of the things you seem to be planning to do/build are already in EVE Marketeer. Not saying you shouldn't build them yourself anyway (private data), just mentioning, since it would be a shame to build everything yourself and then figure out it already existed. P

As far as I read the only thing it doesn't do yet is show the gems, at least, not publically, since I've not found a decent algorithm to determine them yet.

Developer/Creator of EVE Marketeer

Alain Kinsella
#23 - 2011-12-26 04:07:17 UTC  |  Edited by: Alain Kinsella
Welcome Back VV. I've got at least one corp member poking their head in this subforum now, so will point out this thread and the old one. I was actually preparing to re-post internally what I pulled from your site, when this thread re-surfaced.

After giving up on trying to re-synthesize the OHLC from CCP's history files, I started my own research for alternate TA scanning methods (with emphasis on something that leverages the existing Eve Graph). I may have actually found one, Shocked but won't go into detail until I read more and try it a bit.


Jeyson Vicious wrote:
This looks like it might be genius. But I would need +3 willpower/intelligence implants and a booster of the right sort to really sit down and absorb it.


As already mentioned, some have implemented the algo's involved already - But, the one 'public' product is web-based so there's some trust involved.

EDIT - The methodology I'm exploring would use the existing Eve data (for 'at a glance' scanning once you get used to it), but also requires you view the markets using the client (a dealbreaker for some, maybe most). I've already used a primitive form of this in a couple mid- to long-term trades, and profited from all of these so far.


@ Callean - screenshot please? Are you actually displaying candles, lines etc or just doing the OHLC synthesis?

"The Meta Game does not stop at the game. Ever."

Currently Retired / Semi-Casual (pending changes to RL concerns).

Vaerah Vahrokha
Vahrokh Consulting
#24 - 2011-12-26 07:25:00 UTC  |  Edited by: Vaerah Vahrokha
Callean Drevus wrote:
roigon wrote:
...


It actually lends itself pretty well to automation. I added it to EVE Marketeer in under two hours. Then again, I already had all the infrastructure built.

Most of the things you seem to be planning to do/build are already in EVE Marketeer. Not saying you shouldn't build them yourself anyway (private data), just mentioning, since it would be a shame to build everything yourself and then figure out it already existed. P

As far as I read the only thing it doesn't do yet is show the gems, at least, not publically, since I've not found a decent algorithm to determine them yet.


I LOVE your website. I went straight for the candle stick charts and they are awesome.

May I give you some little tips (what my group's trading teacher told us to do back at the time)?


1) Do not limit the history to 1 year, expecially in month view.

In that respect, if you want, I may send you about half year worth of OHLC data older than yours about: PLEX, Trit, Megacyte, Biomass, Mexallon, Nitrogen Isotopes, Technetium.


2) Please make the horizontal price lines optional (a checkbox) or at least thinner, because atm they interfere with the candle bodies.

3) The candles wicks should be of the color of the body. IE if the bullish candles were green, both wicks should be green. Alternatively (graphing engine limitations) wicks should be all light grey / white.

4) The fillets / borders around candles must either be optional or just 1 pixel wide. You can get incredibly bad results by using thick borders. By the way are borders drawn inside the candle body or outside? The former is the correct way.

5) Optional: provide a preference where you can set candles as green = close > open and red = close < open. Or at least an option to invert white with the other color. Using a somewhat red-looking color for bullish candles is confusing. Dojis (candles with no body) should be grey / white and so should be both of their wicks.

6) Optional: It would be a neat boon to provide placeable horizontal lines but it's not so important, one can just take a screenshot and do it.
Vaerah Vahrokha
Vahrokh Consulting
#25 - 2011-12-26 07:29:22 UTC
Alain Kinsella wrote:
Welcome Back VV. I've got at least one corp member poking their head in this subforum now, so will point out this thread and the old one. I was actually preparing to re-post internally what I pulled from your site, when this thread re-surfaced.

After giving up on trying to re-synthesize the OHLC from CCP's history files, I started my own research for alternate TA scanning methods (with emphasis on something that leverages the existing Eve Graph). I may have actually found one, Shocked but won't go into detail until I read more and try it a bit.


Jeyson Vicious wrote:
This looks like it might be genius. But I would need +3 willpower/intelligence implants and a booster of the right sort to really sit down and absorb it.


As already mentioned, some have implemented the algo's involved already - But, the one 'public' product is web-based so there's some trust involved.

EDIT - The methodology I'm exploring would use the existing Eve data (for 'at a glance' scanning once you get used to it), but also requires you view the markets using the client (a dealbreaker for some, maybe most). I've already used a primitive form of this in a couple mid- to long-term trades, and profited from all of these so far.


@ Callean - screenshot please? Are you actually displaying candles, lines etc or just doing the OHLC synthesis?



1) If you have lots of time you may even fetch open and close by carefully parsing EvE Central data's first / last orders of the day. It's approximate though but probably well OK for most actively traded items (> 30 transactions a day).

2) About your question to Callean, go to this URL and then click the "Trends" tab.
Callean Drevus
Perkone
Caldari State
#26 - 2011-12-26 17:35:20 UTC  |  Edited by: Callean Drevus
Vaerah Vahrokha wrote:
I LOVE your website. I went straight for the candle stick charts and they are awesome.

May I give you some little tips (what my group's trading teacher told us to do back at the time)?


1) Do not limit the history to 1 year, expecially in month view.

In that respect, if you want, I may send you about half year worth of OHLC data older than yours about: PLEX, Trit, Megacyte, Biomass, Mexallon, Nitrogen Isotopes, Technetium.


2) Please make the horizontal price lines optional (a checkbox) or at least thinner, because atm they interfere with the candle bodies.

3) The candles wicks should be of the color of the body. IE if the bullish candles were green, both wicks should be green. Alternatively (graphing engine limitations) wicks should be all light grey / white.

4) The fillets / borders around candles must either be optional or just 1 pixel wide. You can get incredibly bad results by using thick borders. By the way are borders drawn inside the candle body or outside? The former is the correct way.

5) Optional: provide a preference where you can set candles as green = close > open and red = close < open. Or at least an option to invert white with the other color. Using a somewhat red-looking color for bullish candles is confusing. Dojis (candles with no body) should be grey / white and so should be both of their wicks.

6) Optional: It would be a neat boon to provide placeable horizontal lines but it's not so important, one can just take a screenshot and do it.


1. History isn't limited to one year at all, currently, the maximum is about 5 years (eg. 1800 days). It is possible there is limited information though, and in that case it will only show information up to the limit available.

2-5: I agree completely, but this is using google charts, and I'm limited by what the graphing engine allows (which is pretty much nothing), at least inassofar there is documentation, in which I cannot find any way to change those specific properties. It was something that annoyed me as well. I might look into using a different graphing engine to change these properties.

6. Difficult, but not impossible, something that might be added later on (also dependent on graphing engine API).

EDIT: Awesome, after some searching I've found a lib which should allow me to make most of these issues a reality :)

Developer/Creator of EVE Marketeer

Vaerah Vahrokha
Vahrokh Consulting
#27 - 2011-12-27 00:26:19 UTC  |  Edited by: Vaerah Vahrokha
@Callean

You know, appetite comes with eating.
What if under Options and Tools you added the last tab data export: Trends?

The EOD (end of day) industry formats are several but this is the most common:


2011-12-23,2829.87,2839.97,2774.16,2794.00,18058104
2011-12-24,2794.00,2838.94,2732.11,2799.87,12132568
2011-12-25,2799.87,2800.00,2737.11,2799.82,10524017

where

First column is YYYY-MM-DD (0 padded ofc if needed)
Second is Open
Third is High
Fourth is Low
Fifth is Close
Sixth is volume (optional for some platforms).

There are also other standards I can give you if you so desire.


I have to say that your website is functional and fast, with no fluffy-sluggish garbage others had.

Since I have enough money to not know what to do with it, If you implement the changes + data export above, I am going to donate you 1B as a sign of appreciation.
Vaerah Vahrokha
Vahrokh Consulting
#28 - 2011-12-28 00:03:09 UTC  |  Edited by: Vaerah Vahrokha
After updating some of the previous posts, I decided to also create a second thread called How to read the in game EvE market charts .

That one is a sort of introduction to some of the advanced concepts here exposed.

A much better and illustrated version of that tutorial has been published on my website at this location.

At a certain point I'll start showing comparison charts and posts about what can be seen by using both approaches to the EvE markets.
Callean Drevus
Perkone
Caldari State
#29 - 2011-12-28 10:16:09 UTC
Vaerah Vahrokha wrote:
@Callean
...

Yup, will be done, the only thing that prevents it from being online right now is that the new lib I'm using isn't as fire and forget as google's API was, so the graphs are ugly now, and I need to do some polishing :)

Developer/Creator of EVE Marketeer

TheSmokingHertog
Julia's Interstellar Trade Emperium
#30 - 2011-12-28 11:20:54 UTC
Callean Drevus wrote:
Vaerah Vahrokha wrote:
@Callean
...

Yup, will be done, the only thing that prevents it from being online right now is that the new lib I'm using isn't as fire and forget as google's API was, so the graphs are ugly now, and I need to do some polishing :)


That explains why I could not find them :).

"Dogma is kind of like quantum physics, observing the dogma state will change it." ~ CCP Prism X

"Schrödinger's Missile. I dig it." ~ Makari Aeron

-= "Brain in a Box on Singularity" - April 2015 =-

Vaerah Vahrokha
Vahrokh Consulting
#31 - 2011-12-28 13:34:51 UTC
Callean Drevus wrote:
Vaerah Vahrokha wrote:
@Callean
...

Yup, will be done, the only thing that prevents it from being online right now is that the new lib I'm using isn't as fire and forget as google's API was, so the graphs are ugly now, and I need to do some polishing :)


Have you tried something like this free graphics package?
Callean Drevus
Perkone
Caldari State
#32 - 2011-12-28 15:34:27 UTC  |  Edited by: Callean Drevus
Vaerah Vahrokha wrote:
Callean Drevus wrote:
Vaerah Vahrokha wrote:
@Callean
...

Yup, will be done, the only thing that prevents it from being online right now is that the new lib I'm using isn't as fire and forget as google's API was, so the graphs are ugly now, and I need to do some polishing :)


Have you tried something like this free graphics package?


Brrr, flash charts, no thank you P I'd like it to work in the IGB as well Blink

And... Charts have been updated ^^ they're awesome in my opinion, and much better than anything I've had before. I also overhauled the history chart, which is now more akin to the ingame version. The best thing is you can zoom by dragging with your mouse (and reset zoom by double clicking the graph).

For your pleasure, I've also added OHLC export to the item information page.

Developer/Creator of EVE Marketeer

TheSmokingHertog
Julia's Interstellar Trade Emperium
#33 - 2011-12-28 19:05:42 UTC
Callean Drevus wrote:
....


The site here is very slow atm, the search querry was not loading on "enter" and a product is not loading beyond the second tabel heads.

Note: the site keeps giving a loading sign in the top corner though

"Dogma is kind of like quantum physics, observing the dogma state will change it." ~ CCP Prism X

"Schrödinger's Missile. I dig it." ~ Makari Aeron

-= "Brain in a Box on Singularity" - April 2015 =-

Vaerah Vahrokha
Vahrokh Consulting
#34 - 2011-12-28 20:51:06 UTC  |  Edited by: Vaerah Vahrokha
Callean Drevus wrote:
Vaerah Vahrokha wrote:
Callean Drevus wrote:
Vaerah Vahrokha wrote:
@Callean
...

Yup, will be done, the only thing that prevents it from being online right now is that the new lib I'm using isn't as fire and forget as google's API was, so the graphs are ugly now, and I need to do some polishing :)


Have you tried something like this free graphics package?


Brrr, flash charts, no thank you P I'd like it to work in the IGB as well Blink

And... Charts have been updated ^^ they're awesome in my opinion, and much better than anything I've had before. I also overhauled the history chart, which is now more akin to the ingame version. The best thing is you can zoom by dragging with your mouse (and reset zoom by double clicking the graph).

For your pleasure, I've also added OHLC export to the item information page.


Awesome, we are *almost* there!

The CSV export data are not correct for some reason.

IE Megacyte

2011-11-15;2713.07;2783.02;2713.15;2775;10893065

Open 2713.07 OK
High 2783.02 OK
Low 2713.15 => Wrong, it's above open and low must be always the lowest value instead
Close 2775 OK


2011-10-31;2747;2807.9699;2748.75;2749.02;10997942

Open 2747 OK
High 2807 OK
Low 2748 => Wrong
Close 2749 = OK

Low by definition is the lowest value, it cannot be higher than the open.

Here's how your graph looks like once imported (disregard the horizontal lines, it's what I use to trade):

Your Weekly graph
Notice how the candles are red when they should not and how the shadows are detached from body.

Your Daily graph
You can see how candles that should be green, are red (values inverted somewhere?)

This is how it looks in your website instead


Here's how Mega looks like with my own program:
Weekly graph

Daily graph

Edit: my examples are with my own software so the open / close may be different in absolute values. The colors are right though.

Edit2: there are also 2 other issues.
1) Data for Mega seems to stop 1 month ago (the web site screen shot shows that)
2) The display starts with monthly candles. I then switch to "day" and then click on "30" and the website hangs at "Loading content".
Callean Drevus
Perkone
Caldari State
#35 - 2011-12-29 09:45:07 UTC
1) Will see what is going wrong with the exported data format, though I think it might be possible to have a lower open than low due to the way the data is generated (I can certainly use a different way, but that will probably only work for minerals, so I'll have to provide both options).
2) Data for Megacyte might very well stop 30 days ago, I do not control the data is uploaded (or, well, I control it to a lesser extend).
3) It probably crashes (crashes? Or does it simply not show anything) because there isn't any candle data in the last 30 days (as you noted earlier, there is no data for those days).

Developer/Creator of EVE Marketeer

Vaerah Vahrokha
Vahrokh Consulting
#36 - 2011-12-29 13:03:07 UTC
Callean Drevus wrote:
1) Will see what is going wrong with the exported data format, though I think it might be possible to have a lower open than low due to the way the data is generated (I can certainly use a different way, but that will probably only work for minerals, so I'll have to provide both options).
2) Data for Megacyte might very well stop 30 days ago, I do not control the data is uploaded (or, well, I control it to a lesser extend).
3) It probably crashes (crashes? Or does it simply not show anything) because there isn't any candle data in the last 30 days (as you noted earlier, there is no data for those days).


1) It's possible to have weird data, because it's sanitized and because data gathering methods have their own quirks (i.e. mine is less accurate on close than it should).

Since 3rd party programs like trading platforms assume correct data, it's on our end to perform some basic checks, that could be like this:

a)

Low = min(O, H, L, C)
High = max(O, H, L, C)

b)

if (L > O) swap(L, O)
if (L > C) swap(L, C)
etc.

There will be some information loss but at least the output will be functional. I had to implement this consistency check myself.


2) Imho you should always make sure data for the most basic items (i.e. minerals) are updated, at least weekly. All it takes is to have them on the market shortcut bar and click on them.

Or, place a warning about data being not updated since a week for item XYZ so someone of the uploaders may decide to upload them (i.e. they are made aware that they should upload XYZ, else they don't know).

3) It does not "crash" but it stays hung with that circling arrows and Loading data... forever. Maybe if data = 0 => show "no data available for the specified time interval"?
Dramaticus
State War Academy
Caldari State
#37 - 2011-12-31 18:51:53 UTC
EnyoG would be impressed

The 'do-nothing' member of the GoonSwarm Economic Warfare Cabal

The edge is REALLY hard to see at times but it DOES exist and in this case we were looking at a situation where a new feature created for all of our customers was being virtually curbstomped by five of them

Callean Drevus
Perkone
Caldari State
#38 - 2012-01-01 16:56:34 UTC
Vaerah,

1) Just letting you know it is done, but not yet on the live EMK because it is in the same commit with some other stuff that needs to be worked out better first.

2) It may sound a bit as if I do not care, but updating the data is not my part of the development of EMK. I happily code, but I don't really feel like logging into EVE just to update data every week. That said, there is nothing that prevents you or anyone else from doing so.
But as you said, it might be a good idea to implement notifications of what should be uploaded across regions, especially for minerals or other items that are much traded/used.

3) Also done, but not on live yet (see 1).

Developer/Creator of EVE Marketeer

Vaerah Vahrokha
Vahrokh Consulting
#39 - 2012-01-03 00:12:48 UTC
Callean,

I see you have updated the website as I suggested to.

I did a brief consistency check between some of your data samples and mine, taken on those days were data was not clean:


Your gathered and sanitized data


Quote:
2011-11-21;2824.98;2877.9899;2771.2399;2854;19048427
2011-11-22;2854;2867.9699;2767.02;2771.21;18109912
2011-11-23;2771.21;2856.87;2767.16;2800;14071525
2011-11-24;2800;2807;2766.91;2803.7;10545829
2011-11-25;2803.7;2805.88;2766.41;2805.83;14353051
2011-11-26;2805.83;2805.84;2767.28;2803.85;15415996



My own gathered and sanitized data

Quote:
2011-11-21,2824.98,2877.99,2771.24,2854.00,19048427
2011-11-22,2854.00,2867.97,2767.02,2771.21,18109912
2011-11-23,2771.21,2856.87,2767.16,2800.00,14071525
2011-11-24,2800.00,2807.00,2766.91,2803.70,10545829
2011-11-25,2803.70,2805.88,2766.41,2805.83,14353051
2011-11-26,2805.83,2805.84,2767.28,2803.85,15415996


As you can see the differences (besides separator) are on the fourth decimal digit (you should round it to 2 digits imo, that's what EvE prices do), the match is basically perfect.


Notice how now the weekly candle bars graph looks completely clear, it is almost screaming "trade me now!"


Therefore, as I said above, I am going to send you 1 Billion as reward and tangible support for your outstanding job!
Vaerah Vahrokha
Vahrokh Consulting
#40 - 2012-01-03 01:33:11 UTC  |  Edited by: Vaerah Vahrokha
Some examples that show a full analysis. There are several factors involved that I will expose. Most of the times, a trader will not search for reasons to trade but for reasons NOT to trade. Only if there are very few reasons not to trade then it's prudent opening a position.

Therefore in the next graph I will show why I would NOT have traded at the several candles I will show but also what another trader who accepts the big risk of buying high for low reward would practically do.
(In my personal case, I am long on Mega since before I really stopped logging on (June / July).

I will first post all the graphs together then in the next days I'll describe them.


Megacyte - ticker MGCYT - The Forge prices


Monthly graph

Link to graph


Weekly graphs

Link to graph 1
Link to graph 2
Link to graph 3
Link to graph 4


Daily graph
Link to graph