EVE Technology Lab

Is this still a thing? I've run scans for hours at a time with a 4s delay between queries, and as far as I can tell it kept right on -- at least, the market window kept updating to new items, although I didn't manually check if new cache files were still being created.

I tried running reverence on Sisi, and got this error:
"UnmarshalError: find_global failed to resolve: carbon.common.script.sys.crowset.CRowset"

A call to "cfg.invtypematerials.iteritems()" generates that error.

"cfg" is defined as: cfg = eve.getconfigmgr()

"eve" is defined as: eve = blue.EVE(eve_root, server)

where eve_root = r"C:\Program Files (x86)\CCP\EVE_Buck", ad server = "Singularity"



The same code works just fine on Tranquility.

Perhaps the cache format has changed in Sisi?

Also, a Windows 64-bit installer that incorporates the changes from 3 months ago would be great. Otherwise Visual Studio 2008 related errors crop up.

Thx for the hard work Entity.
The current hickup (which won't be the last, thanks to the evolving nature of eve) is a reason to once more call for an official market data feed from CCP: https://forums.eveonline.com/default.aspx?g=posts&t=274786

Sorry for the late update but aside from not being in the country and figuring out the FSD stuff, I had enough to do :P

Anyway, without further ado:

Reverence 1.6.0 (aka the Odyssey 1.1 update) has been pushed to github

Source distribution, 32bits and 64bits binaries are available here:
Downloads

Notable changes:

- The only officially supported python version is now Python 2.7.
- This update is NOT BACKWARDS COMPATIBLE with older EVE client versions.
- Massively improved FSD support: Higher performance, less RAM usage.
- the cfg.fsdTypeOverrides table is now correctly used by cfg.invtypes.


The examples on github are still out of date. Sorry about that.

EDIT: it appears a syntax error snuck by! it's fixed. if you downloaded 1.6.0 before this edit, download it again :)

I found an issue using the new version, the same scripts that used to work before the update now do nothing.

OS is Windows 7 Professional, python = 2.7.3.

This is the offending source code (the same that used to work):

EVEROOT = r"D:/Program Files/CCP/EVE"
OUTPATH = r"D:/Temp/EvEDumps/"
...
import time
import os
from reverence import blue
from time import sleep
...
eve = blue.EVE(EVEROOT)
cfg = eve.getconfigmgr()
cachemgr = eve.getcachemgr()
cmc = cachemgr.LoadCacheFolder("CachedMethodCalls")

I think this code comes from some of the provided examples and worked for years.

What stopped working is that now cmc = {} while it used to load whole collections.

Now I am not a Python guy so I just write down what I have seen debugging the calls:

I went to the object initialization

File: Cache.py

class CacheMgr:
def __init__(self, root, servername="Tranquility...

it finds the correct cache path and then cycles several Machonet folders till it gets to

u'C:\\\\Users\\\\user_name...\\\cache\\\\bulkdata\\\\379

Protocol found says 379 as well

then

appdatapath = u'C:\\\\Users\\\\user_name...m_files_ccp_eve_tranquility'

then it assigns some members in this bit of code:

if protocol > self.machoVersion:
self.machoVersion = protocol
self.root = root
self.appdatapath = appdatapath
self.cachepath = _join(appdatapath, "cache")
self.settingspath = _join(appdatapath, "settings")
self.machocachepath = _join(machopath, str(protocol))
self.BULK_SYSTEM_PATH = _join(root, 'bulkdata')
self.BULK_CACHE_PATH = _join(appdatapath, 'cache', 'bulkdata', str(protocol))
return

I can't see the self data members value, I don't know if it can be done with IDLE.

Later on the program gets to my

cmc = cachemgr.LoadCacheFolder("CachedMethodCalls")

call and enters this function:

def LoadCacheFolder(self, name, filter="*.cache"):
"""Loads all .cache files from specified folder. Returns a dict keyed on object name."""

# Note that this method is used mainly for debugging and testing,
# and is subject to change without notice.
crap = {}
for filename in glob.glob(_join(name, filter)):
what, obj = blue.marshal.Load(_readfile(filename))
crap[what] = obj
return crap

In there it starts the for cycle with name = "CachedMethodCalls" but it never gets to the inner loop and immediately returns crap, which of course is set to {} and thus my cmc collection is {} as well.


I have this folder with these contents:

C:\Users\user_name\AppData\Local\CCP\EVE\d_program_files_ccp_eve_tranquility\cache\MachoNet\87.237.38.200\379

CachedMethodCalls
CachedObjects
MethodCallCachingDetails


CachedMethodCalls in turn has a lot of *.cache files, like:

1ddd.cache
1ebb.cache
2b41.cache

These files were modified today (I refreshed the whole client cache).


Any idea about what I could do to fix this?

Yeah just found out that the glob mechanism called by the offending function:

def glob1(dirname, pattern):
if not dirname:
dirname = os.curdir
if isinstance(pattern, unicode) and not isinstance(dirname, unicode):
dirname = unicode(dirname, sys.getfilesystemencoding() or
sys.getdefaultencoding())
try:
names = os.listdir(dirname)
except os.error

it fails os.listdir(dirname) with error 3: "Directory not found".


I wonder how it worked for so long without ever specifying the absolute path O_o. Even stranger, there are dozens of people who used this script and no one had it break due to the relative path.


Since I am totally ignorant about Python, what should I write to get the correct Machonet absolute path? I suppose I have to put a method call or data member from the cfg manager or the eve object?

Thank you now it works ^^

Another colorful error. The script now works fine.

I need to bundle the script plus assorted Python runtime in an executable.

I setup pyinstaller and it creates a "dist" folder with the generated executable inside.

But when I start it I get:

File "string", line 111, in module
File "E:\Python27\Scripts\build\Blah\out00-PYZ.pyz\reverence.cache", line 259, in LoadCacheFolder
UnmarshalError: find_global failed to resolve: carbon.common.script.net.objectCaching.CachedMethodCallResult

What's that?

In PHP I could fix all by myself but Python seems to love cryptic messages I don't understand.

In the folder there are reverence._blue.pyd and reverence._pyFSD.pyd.

I have no problem compiling stuff (I have Visual Studio and compiled Reverence "the hard way"), but I don't know how to learn which file is missing.

I mean, this Carbon.common.script.net.objectCaching.CachedMethodCallResult has to be implemented somewhere, doesn't it?

If I understand correctly, I have to find a .PYD file. Compiling Reverence with Visual Studio creates two of those files: reverence._blue.pyd and reverence._pyFSD.pyd.

So, which PYD file implements Carbon.common.script.blah.blah? Is there a python tool that shows the dependencies?

Thank you for your continued support!

The solution you have shown is exactly what used to work for years. I found it out the hard way (I did not read that options post of yours) so I had this setup.py:

from distutils.core import setup
import py2exe

setup(
console=['EvEMarketHistory.py'],
options={
'py2exe':
{
'packages': ['reverence'],
'includes': ['reverence', 'os'],
}
}
)

This setup worked perfectly, till the last Reverence release.
Now by using the same build py2exe batch it runs for a while and at the end it fails with:

The following modules appear to be missing
['Carbon', 'Carbon.Files', 'blue', 'fsdSchemas.binaryLoader']


So I switched to another executable compiler. It compiles my script perfectly but when I start the compiled script:

Traceback (most recent call last):
File "string", line 111, in module
File "E:\Python27\Scripts\build\EvEMarketHistory\out00-PYZ.pyz\reverence.cache", line 259, in LoadCacheFolder
UnmarshalError: find_global failed to resolve: carbon.common.script.net.objectCaching.CachedMethodCallResult
Exception TypeError: 'expected string or Unicode object, NoneType found' in module 'threading' from 'E:\Python27\Scripts\dist\EVEMAR~1\threading.pyc' ignored


So even with the option to include reverence and even trying 2 different compilers it still has something ugly going on with Carbon.