These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

 
  • Topic is locked indefinitely.
 

Why are we not combining market data, player devs?
Author
Previous Topic Next Topic
Dragonaire
Here there be Dragons
#81 - 2011-12-01 23:27:19 UTC
I updated the wiki so everyone can take a look and see if they have any other changes on it.

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.
Garo Hertee
Eclipse Industrials
Quantum Forge
#82 - 2011-12-02 21:03:41 UTC
A good week for the Eve market sites...

Eve Central: 100% uptime
Eve Market Data: 100% uptime
Eve Marketeer: 98.2% uptime
Kaladr
Viziam
Amarr Empire
#83 - 2011-12-02 21:27:02 UTC  |  Edited by: Kaladr
Dragonaire wrote:
I updated the wiki so everyone can take a look and see if they have any other changes on it.


Looking good so far. I made one additional change: the inclusion of a generated_by map which lists the upload client name and version.

I am currently working on updates to Contribtastic (and creating Contribtastic Mac), and will look at making a version which can generate this upload format.

Creator of EVE-Central.com, the longest running EVE Market Aggregator
Dragonaire
Here there be Dragons
#84 - 2011-12-03 03:12:59 UTC
generated_by is a good addition don't know why I hadn't thought of it.
I added a history example for us to work with as well.

Quote:
I do like the idea of maybe using UUIDs but how would it be better than just using a MD5 or SHA1 hash of the URL with parameters? Also which parameters were you thinking about using/including in it?
Still wondering about this.

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.
Callean Drevus
Perkone
Caldari State
#85 - 2011-12-03 10:02:17 UTC  |  Edited by: Callean Drevus
Well, the UUID wouldn't be based on any fields, it would just be generated based on the natural randomness of the system (or using the fields, if you feel so inclined), the point is that the value of a UUID isn't based on an input that was given it, so it's almost guaranteed to be unique (much more so than MD5 for example, which could collide quite a bit more, even if you were using the unique URLS to generate it).

That at least, is the great plus of UUID in my opinion. Maybe someone else had some other good reasons to use it.

UPDATE: In addtion, if we are going to use that header row. I'd call it 'fields' or 'data_fields' instead of 'header' as it doesn't really have much to do with headers, and we might as well be semantically correct :)

Developer/Creator of EVE Marketeer
Dragonaire
Here there be Dragons
#86 - 2011-12-03 14:11:15 UTC
Quote:
UPDATE: In addtion, if we are going to use that header row. I'd call it 'fields' or 'data_fields' instead of 'header' as it doesn't really have much to do with headers, and we might as well be semantically correct :)

The reason I originally named it 'header' was I was viewing it like the header line of a CSV file but it could be named differently. Maybe 'field_names' or 'column_names' would be a better name.

The naming that really bothers me more is 'log'. I used that because the original uploader examples I looked at used that but IMHO naming it 'data', 'contents', or 'payload' would make more sense.

Quote:
... That at least, is the great plus of UUID in my opinion. Maybe someone else had some other good reasons to use it.

Actually UUID version 3 and 5 are MD5 or SHA1 of the URL so there isn't really much difference except for the formating that was why I asking the question since it would be possibly a extra step without really adding any extra value.

I've just been giving this a lot more thought and I think we might be going at this the wrong way. Instead of trying to figure out a common digest or checksum let's simple make sure to include enough metadata so that we know the receiver will have everything they would need to make one to insure loops don't happen and leave it up to the developers what they use. The network bandwidth and memory buffering differences is nil as long as it's less than a single TCP packet worth of data or at most two for locations with a lot of data rows.

Looking at it from the network bandwidth angle it might be a good idea to think a little more about how stuff is named but we don't want to go to far trying to make it smaller and end up make it human unfriendly either. I'll look at changing some of the names on the wiki and see what everyone thinks.

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.
Desmont McCallock
#87 - 2011-12-03 14:21:54 UTC
Dragonaire wrote:
Quote:
UPDATE: In addtion, if we are going to use that header row. I'd call it 'fields' or 'data_fields' instead of 'header' as it doesn't really have much to do with headers, and we might as well be semantically correct :)

The reason I originally named it 'header' was I was viewing it like the header line of a CSV file but it could be named differently. Maybe 'field_names' or 'column_names' would be a better name.

The naming that really bothers me more is 'log'. I used that because the original uploader examples I looked at used that but IMHO naming it 'data', 'contents', or 'payload' would make more sense.
I would go for :
1. header -> columns
2. log -> rowset
Dragonaire
Here there be Dragons
#88 - 2011-12-03 14:29:25 UTC
Would bring it more in line with API XML which might have some benefits on developer understanding and code reuse maybe.

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.
Desmont McCallock
#89 - 2011-12-03 14:31:16 UTC
Dragonaire wrote:
Would bring it more in line with API XML which might have some benefits on developer understanding and code reuse maybe.

Exactly my point.
Dragonaire
Here there be Dragons
#90 - 2011-12-03 14:36:39 UTC  |  Edited by: Dragonaire
Should we go the rest of the way and call it 'result' and use currentTime too?

Edit: The next question becomes does the Eve client cache provide the same data as the MarketOrder API? I know it doesn't have charID, orderState, accountKey, or escrow but isn't everything else the same just different naming?

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.
Dragonaire
Here there be Dragons
#91 - 2011-12-03 17:01:08 UTC
Ok after looking at both the Eve cache export data and Eve API MarketOrder columns I revised the 'orders' JSON format to use the same names etc in the wiki. This should allow developers from both areas to use it with minimal amount of data conversion. I used the same order for the columns as given by cache export data.

In the 'history' I decided to use the same order of columns as shown in game.

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.
Dragonaire
Here there be Dragons
#92 - 2011-12-04 00:50:47 UTC  |  Edited by: Dragonaire
I'm really sorry about the triple post Oops but I thought this was important enough and unrelated enough to my others to use one.

I'll direct you to a new forum thread on a new project those developers reading this thread might find interesting.
https://forums.eveonline.com/default.aspx?g=posts&t=41697

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.
Desmont McCallock
#93 - 2011-12-04 08:07:55 UTC
Linky leads to.... WH 404?
Dragonaire
Here there be Dragons
#94 - 2011-12-04 08:36:21 UTC
Sorry I miss edited it it should work now.

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.
Callean Drevus
Perkone
Caldari State
#95 - 2011-12-04 09:56:39 UTC
Ok, so I just found out I have lost the source code to the EMK uploader in a crash last week. Unless I can decompile the exe, I'll be stuck having to build it all again (including threaded support of course)...

Developer/Creator of EVE Marketeer
Dragonaire
Here there be Dragons
#96 - 2011-12-04 10:07:45 UTC
Was it in hg? If so you should still have it in one of the other copies out there. I might even have an older one floating around somewhere Blink

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.
Callean Drevus
Perkone
Caldari State
#97 - 2011-12-04 12:26:16 UTC
I wish :P that's the problem. I totally forgot to ever put that thing in version control.

Developer/Creator of EVE Marketeer
Tonto Auri
Vhero' Multipurpose Corp
#98 - 2011-12-04 16:17:58 UTC
Callean Drevus wrote:
I wish :P that's the problem. I totally forgot to ever put that thing in version control.

O.o
This is usually the second step I take for my own projects. I have a server next to me, with Subversion running over mirror raid.

Two most common elements in the universe are hydrogen and stupidity. -- Harlan Ellison
Desmont McCallock
#99 - 2011-12-04 18:08:00 UTC  |  Edited by: Desmont McCallock
Callean Drevus wrote:
Ok, so I just found out I have lost the source code to the EMK uploader in a crash last week. Unless I can decompile the exe, I'll be stuck having to build it all again (including threaded support of course)...


I thought I had a copy but I was wrong. As you said you never put it in VCS so I couldn't have one.
Dragonaire
Here there be Dragons
#100 - 2011-12-04 18:54:45 UTC
So been think on the JSON order feed stuff a little more and wondering if it really needs to include solarSystemID since that can be inferred from the stationID. The same could be said for the regionID but since it's only used once in the metaData it's less of an issue IMHO.

Anyone else have a feeling about this one way or the other?

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.