EVE Technology Lab

Nope. the dump has a wee bit more info.

Planet names do appear in the cache but only after you've visited a system (and then only for as long as it takes for the entry to get overwritten by something else with the same hash)

There's probably some logic to the planetID vs names (ascending order maybe) but that still leaves a few named planets as special case though.

Just a heads-up everyone.

For the next release of Reverence I will be doing the following:

- EVE client compatibility mode will be removed.
This mode used RecordSets for the row data (matching what the EVE client uses). This change should not affect anyone as the default container classes in Reverence offer a larger set of features anyway.

- Minimum supported EVE version will be Crucible 1.5 or even 1.6
Older versions that use the same table scheme may still work

- Support for old table names and old attribute names will be removed.
Things like cfg.ramtypematerials being cfg.invtypematerials, and attribute aliases such as celestialID when you should be using itemID instead.

- Possibly some other changes that may require updates to applications using Reverence.

The reason is that with the changes in table layout over the years, and the recent addition of tables with fields composed by the localization subsystem, I was going to have to maintain 3-4 different code paths per loader for certain tables, which is a bloody mess and prone to cause issues in the long run.

Hi there

Ok sorry for the walls of text, however I wanted to make a clear descriptions of what I'm seeing

I've been working on an application that uses the cache market order data that reverence provides

I'm using "autodumper" to pull it automatically, but autodumper is just a cache monitor and cache to market order file converter. I think the problem is with the reverence library and not autodumper. (it's the same autodumper that is mentioned in the Market scanner thread)

So what I'm seeing is occasionally the jump data has wrong values in it

I look up the item in the market, then export it manually, then compare what autodumper and the manually exported files say, and the jumps are different

Thanks to your debugging string in "blue" I was able to print to the screen the DBrow that comes directly from blue.marshal.load()

And from what I've been able to see that it's coming out of the blue.marshal.load() incorrectly

The call in autodumper looks like this


Now I can't see anywhere "obj" is referenced until it is used to pull data like thi


Here is an example (I've sorted by OrderID to get some alignment between them)

This is the export from Eve

"price","volRemaining","typeID","range","orderID","volEntered","minVolume","bid","issueDate","duration","stationID","regionID","solarSystemID","jumps"
2.8,643561,2512,32767,2379503173,2500000,1,"True","2012-03-27 21:04:19.000",90,60004588,10000030,30002510,0
10,24748,2512,32767,2409627628,96349,1,"False","2012-01-17 02:48:28.000",90,60014833,10000030,30002527,2
1.46,48500,2512,3,2463275078,50000,1,"True","2012-03-06 23:39:16.000",90,60015041,10000030,30012505,5
7,110858,2512,32767,2464714096,181000,1,"False","2012-03-08 13:15:43.000",90,60014791,10000030,30002531,8
4.15,385109,2512,-1,2466518598,1000000,1,"True","2012-04-06 00:10:44.000",90,60004588,10000030,30002510,0
2.81,31703,2512,0,2475932287,35000,1,"True","2012-04-04 22:13:57.000",90,60010048,10000030,30002543,5
4.11,24718,2512,0,2476071310,50000,1,"True","2012-04-05 04:57:07.000",90,60004588,10000030,30002510,0
4,29178,2512,-1,2482936322,30000,1,"True","2012-03-25 19:38:39.000",90,60004588,10000030,30002510,0
9.99,699695,2512,32767,2483543767,1000000,1,"False","2012-03-26 09:53:05.000",90,60004588,10000030,30002510,0
8,668156,2512,32767,2485094991,701956,1,"False","2012-03-27 21:04:08.000",90,60004615,10000030,30002526,1
10.5,22500,2512,32767,2487298606,30000,1,"False","2012-03-30 03:41:42.000",90,60011413,10000030,30002546,5
9,4900,2512,32767,2493494716,4900,1,"False","2012-04-04 13:37:49.000",90,60004675,10000030,30002564,13
4.14,437665,2512,-1,2494487679,500000,1,"True","2012-04-05 12:59:06.000",3,60004588,10000030,30002510,0
9.95,50000,2512,32767,2494627775,50000,1,"False","2012-04-05 15:03:20.000",90,60004753,10000030,30002548,6

Here is what Autodumper says

"price","volRemaining","typeID","range","orderID","volEntered","minVolume","bid","issueDate","duration","stationID","regionID","solarSystemID","jumps"
2.8,643561,2512,32767,2379503173,2500000,1,"True","2012-03-27 21:04:19",90,60004588,10000030,30002510,0
10,24748,2512,32767,2409627628,96349,1,"False","2012-01-17 02:48:28",90,60014833,10000030,30002527,5
1.46,48500,2512,3,2463275078,50000,1,"True","2012-03-06 23:39:16",90,60015041,10000030,30012505,0
7,110858,2512,32767,2464714096,181000,1,"False","2012-03-08 13:15:43",90,60014791,10000030,30002531,8
4.15,385109,2512,-1,2466518598,1000000,1,"True","2012-04-06 00:10:44",90,60004588,10000030,30002510,0
2.81,31703,2512,0,2475932287,35000,1,"True","2012-04-04 22:13:57",90,60010048,10000030,30002543,0
4.11,24718,2512,0,2476071310,50000,1,"True","2012-04-05 04:57:07",90,60004588,10000030,30002510,0
4,29178,2512,-1,2482936322,30000,1,"True","2012-03-25 19:38:39",90,60004588,10000030,30002510,0
9.99,699695,2512,32767,2483543767,1000000,1,"False","2012-03-26 09:53:05",90,60004588,10000030,30002510,5
8,668156,2512,32767,2485094991,701956,1,"False","2012-03-27 21:04:08",90,60004615,10000030,30002526,4
10.5,22500,2512,32767,2487298606,30000,1,"False","2012-03-30 03:41:42",90,60011413,10000030,30002546,8
9,4900,2512,32767,2493494716,4900,1,"False","2012-04-04 13:37:49",90,60004675,10000030,30002564,13
4.14,437665,2512,-1,2494487679,500000,1,"True","2012-04-05 12:59:06",3,60004588,10000030,30002510,0
9.95,50000,2512,32767,2494627775,50000,1,"False","2012-04-05 15:03:20",90,60004753,10000030,30002548,0

Look at the second row of data (orderID: 2409627628), you can see Jumps is inconsistent

I know it could be Autodumper, but I can't see how at this point. By printing "order" I believe I'm getting what blue.marshal.load() returned

Any chance you can take a look at this one?

I would like to use Python 3.2. Are you able to provide a windows binary?

Hi Entity, awesome library! For education''s sake, would you be willing to share a bit of the development process of your project, specifically decoding the table formats?

is this working for the new expansion?

It is.

I have a little issue.

I have created a quite large script that dumps the items price history on file (in various ways including OHLC format).

I use CachedMethodCalls => GetOldPriceHistory + GetNewPriceHistory as in the following small snippet


eve = blue.EVE(EVEROOT)
cfg = eve.getconfigmgr()
cachemgr = eve.getcachemgr()
cmc = cachemgr.LoadCacheFolder("CachedMethodCalls")

print "Deleting old records... \n"
for root, dirs, files in os.walk(OUTPATH, topdown=False):
for name in files:
os.remove(os.path.join(root, name))

for key, obj in cmc.iteritems():
for CurrentItemToDump in ItemsToDump:
if key[1]=="GetOldPriceHistory":
item = cfg.invtypes.Get(key[3])
region = cfg.evelocations.Get(key[2])

...

if key[1]=="GetNewPriceHistory":
item = cfg.invtypes.Get(key[3])
region = cfg.evelocations.Get(key[2])

...

I think the above should cover every price record.

But the behavior is not the expected.

On some markets, like Nitrogen Isotopes I get an output like:

1) GetOldPriceHistory => All old prices data up to 2 days ago
2) GetNewPriceHistory => Yesterday price.

Example:

...
2012-05-02 00:00:00, 1002.80, 1002.86, 1002.85, 47161814, 284
2012-05-03 00:00:00, 1003.43, 1039.97, 1004.96, 59085004, 330
2012-05-04 00:00:00, 1007.11, 1037.75, 1010.70, 80109798, 439
2012-05-05 00:00:00, 932.47, 969.96, 969.69, 83358331, 402
2012-05-06 00:00:00, 966.18, 1015.99, 998.76, 82448391, 370
2012-05-07 00:00:00, 998.47, 1005.60, 1001.90, 64987299, 372

As you see it ends May 7 and not 8 (today).


Other markets, like Isogen instead behave different:

2012-05-03 00:00:00, 86.00, 88.50, 88.00, 408729569, 1606
2012-05-04 00:00:00, 88.33, 88.69, 88.58, 309280884, 1509
2012-05-05 00:00:00, 87.43, 89.88, 87.87, 384421318, 1747
2012-05-06 00:00:00, 80.51, 88.25, 85.16, 522793936, 1973
2012-05-08 00:00:00, 30.00, 94.00, 83.65, 133281182, 464

In game I can see May 7 is all set to zero, the export as you can see does not include May 7 at all.

Anything that can be done to sanitize this situation?

Hey - love Reverence, it's a great tool! :)

Having a bit of an issue with mine, though. It requires ce99.cache to be in the cache folder to run, and my client doesn't seem to generate one on its own - not a huge deal, though, I got an old copy of it and manually putting it in the cache folder before running worked fine as a workaround.

However, since Crucible, this has made my market analysis tool increasingly lacking, as it's unable to recognize any items added more recently than the version of the cache file I have. Using the following code:

cmc = cachemgr.LoadCacheFolder("CachedMethodCalls")
key, obj = cmc.iteritems()
if key[1] == "GetOldPriceHistory":
itemid=key[3]
item = cfg.invtypes.Get(itemid)

returns None (as the value for 'item') for items added since Crucible.

I noticed in an earlier response, you were advocating not using the LoadCacheFolder command at all - I assume I'm doing something obvious hilariously wrong, and there's a better (if not necessarily simpler) way to do this. I only saw the ce99.cache issue mentioned a couple times in my research, and in those it seemed to be relating to setting the wrong directory path - mine reads the cache fine so long as ce99 is in there, but doesn't seem to generate it like it's presumably supposed to.

I would appreciate any input :)

It turns out I actually wasn't running the latest Reverence (due to being very not good with Git - I'd downloaded a newer version manually, but hadn't done it through Git so I believe I was still using the old one), but I've now fixed that (updating to the version released April 4 2012) and the problem remains.

My cache>MachoNet folder only has one folder (named for the tranquility IP, "87.237.38.200"), which I've cleared of everything but the most recent subfolder (329). This occurs whether I set EVEROOT to the Users/[username]/AppData chain ending in c_program_files_(x86)_ccp_eve_tranquility, or to the main CCP/EVE directory in Program Files (it correctly finds the cache in both cases, but requires ce99 to be placed in the CachedObjects subfolder manually to avoid crashing, and still has the missing-items problem mentioned above).

I'll continue my search to see if another cache folder's hiding somewhere, but haven't found anything yet (at least, there's no duplicate MachoNet folders anywhere - the only other place the name even cropped up was in my backups of the cache in a completely different directory, which I deleted just to make sure, to no effect). I will note that my EVE install and the Reverence files are on separate drives, though the odds of that actually mattering seem slim-to-none.

My code is available for inspection if taking a look at it would be helpful, though I'd prefer not to post it in thread :)

Thanks for the quick reply!