These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

 
  • Topic is locked indefinitely.
 

Reverence - 100% compatible EVE cache library for Python

First post
Author
Previous Topic Next Topic
Halika Androm
Aliastra
Gallente Federation
#181 - 2014-02-25 09:12:19 UTC
Thanks for the speedy fix! Everything seems to be back to normal, will let you know if it borks itself heh :)
K Kerryngktonn
General Mechanics Ltd.
#182 - 2014-03-17 09:35:26 UTC
Hey, ntt,

Thanks for the powerful library.

Just wanted to say that as I seem to be using this "really freakin' bizarre install" of Linux, reverence doesn't work out of the box for me. The reason is that I'm using the EVE folder from a mounted Windows partition, so it's actually a root-owned directory, not the local user's. So I had to tweak cache.py a little :(

I suppose the double-OS install like mine is not a rare one, so you may wish to expand on the appdata path detector a little.

Appreciate your work on this wonderful thingie!
Vaerah Vahrokha
Vahrokh Consulting
#183 - 2014-04-16 18:21:51 UTC
Hello,

my software which worked perfectly with Reverence 1.7.0 stopped working with this error:

Traceback (most recent call last):
File "EvEMarketHistoryDumpOHLC.py", line 111, in module
cmc = cachemgr.LoadCacheFolder(cmc_path)
File "E:\Python27\lib\site-packages\reverence\cache.py", line 277, in LoadCacheFolder
what, obj = blue.marshal.Load(_readfile(filename))
UnmarshalError: find_global failed to resolve: utillib.KeyVal
Exception TypeError: 'expected string or Unicode object, NoneType found' in module 'threading' from 'E:\Python27\lib\threading.pyc' ignored


Is this something you can reproduce or shall I just to sock my head against a wall? Sad

EvE Online portal | EvE markets tutorials
Entity
X-Factor Industries
Synthetic Existence
#184 - 2014-04-16 18:58:39 UTC
Vaerah Vahrokha wrote:
Hello,

my software which worked perfectly with Reverence 1.7.0 stopped working with this error:

Traceback (most recent call last):
File "EvEMarketHistoryDumpOHLC.py", line 111, in module
cmc = cachemgr.LoadCacheFolder(cmc_path)
File "E:\Python27\lib\site-packages\reverence\cache.py", line 277, in LoadCacheFolder
what, obj = blue.marshal.Load(_readfile(filename))
UnmarshalError: find_global failed to resolve: utillib.KeyVal
Exception TypeError: 'expected string or Unicode object, NoneType found' in module 'threading' from 'E:\Python27\lib\threading.pyc' ignored


Is this something you can reproduce or shall I just to sock my head against a wall? Sad


Nah it's ccp being silly and still referencing stuff by GUID instead of their proper class name. I'll add a workaround.

╦......║...╔╗.║.║.╔╗.╦║.╔╗╔╦╗╔╗

║.╔╗╔╗╔╣.╔╗╠..╠ ╠╗╠╝.║╠ ╠╝║║║╚╗

╩═╚╝║.╚╝.╚╝║..╚╝║║╚╝.╩╚╝╚╝║.║╚╝

Got Item?
Vaerah Vahrokha
Vahrokh Consulting
#185 - 2014-04-16 19:06:26 UTC  |  Edited by: Vaerah Vahrokha
Entity wrote:
Vaerah Vahrokha wrote:
Hello,

my software which worked perfectly with Reverence 1.7.0 stopped working with this error:

Traceback (most recent call last):
File "EvEMarketHistoryDumpOHLC.py", line 111, in module
cmc = cachemgr.LoadCacheFolder(cmc_path)
File "E:\Python27\lib\site-packages\reverence\cache.py", line 277, in LoadCacheFolder
what, obj = blue.marshal.Load(_readfile(filename))
UnmarshalError: find_global failed to resolve: utillib.KeyVal
Exception TypeError: 'expected string or Unicode object, NoneType found' in module 'threading' from 'E:\Python27\lib\threading.pyc' ignored


Is this something you can reproduce or shall I just to sock my head against a wall? Sad


Nah it's ccp being silly and still referencing stuff by GUID instead of their proper class name. I'll add a workaround.


Thank you a lot!

Please make the update in the Windows installer, ATM I have issues compiling the library with Visual Studio.

EvE Online portal | EvE markets tutorials
Entity
X-Factor Industries
Synthetic Existence
#186 - 2014-04-16 20:02:20 UTC
Vaerah Vahrokha wrote:
[quote=Entity][quote=Vaerah Vahrokha]
Thank you a lot!

Please make the update in the Windows installer, ATM I have issues compiling the library with Visual Studio.


Done.

╦......║...╔╗.║.║.╔╗.╦║.╔╗╔╦╗╔╗

║.╔╗╔╗╔╣.╔╗╠..╠ ╠╗╠╝.║╠ ╠╝║║║╚╗

╩═╚╝║.╚╝.╚╝║..╚╝║║╚╝.╩╚╝╚╝║.║╚╝

Got Item?
cent ech
Pator Tech School
Minmatar Republic
#187 - 2014-04-17 17:39:40 UTC
is there a working example for market data dumps?
i just can not get it working. the example on github seems to be years old.
Vaerah Vahrokha
Vahrokh Consulting
#188 - 2014-04-20 20:37:53 UTC
Entity wrote:
Vaerah Vahrokha wrote:
[quote=Entity][quote=Vaerah Vahrokha]
Thank you a lot!

Please make the update in the Windows installer, ATM I have issues compiling the library with Visual Studio.


Done.


Thank you so much! It's working!

Have an happy Easter!

EvE Online portal | EvE markets tutorials
bongono
Arc Energy Inc
#189 - 2014-04-21 11:08:06 UTC
Is it possible to get the the volume history for items on the market through the cache, any pointers would be appreciated.
Awkward Pi Duolus
Pator Tech School
Minmatar Republic
#190 - 2014-04-22 03:30:51 UTC
Entity, just wanted to note how awesome you are to share this superb tool with us, and to keep it updated through all of CCP's whimsical gyrations!!
stoicfaux
#191 - 2014-04-23 12:53:00 UTC
Isn't cache scraping "illegal" now? http://community.eveonline.com/support/policies/third-party-policies/ specifically the first and last paragraphs.

Quote:
Cache Scraping
Our EULA does not permit the scraping of the EVE Online cache, as per EULA Paragraph 9.C

You may not reverse engineer, disassemble or decompile, or attempt to reverse engineer or derive source code from, all or any portion of the Software, or from any information accessible through the System (including, without limitation, data packets transmitted to and from the System over the Internet), or anything incorporated therein, or analyze, decipher, "sniff" or derive code (or attempt to do any of the foregoing) from any packet stream transmitted to or from the System, whether encrypted or not, or permit any third party to do any of the same, and you hereby expressly waive any legal rights you may have to do so. If the Software and/or the System contains license management technology, you may not circumvent or disable that technology.

We recognize that some players have engaged in cache scraping in the past, and we want to be clear this practice is not permitted. That said, unless there is an extreme case (i.e., cache scraping combined with other EULA violations), we will not penalize players who have engaged in this practice prior to 15 April 2013. Now that we have made our intent and policy clear, we may, in our sole discretion, deliver appropriate penalties for players that engage in cache scraping after 15 April 2013 (including temporary or permanent bans). In addition, we also may consider eliminating the cache to eliminate this practice and for performance reasons.


On a side note, can anyone actually find "paragraph 9.C" in the EULA?

Pon Farr Memorial: once every 7 years, all the carebears in high-sec must PvP or they will be temp-banned.
Entity
X-Factor Industries
Synthetic Existence
#192 - 2014-04-24 13:33:47 UTC
stoicfaux wrote:
Isn't cache scraping "illegal" now? http://community.eveonline.com/support/policies/third-party-policies/ specifically the first and last paragraphs.

Quote:
Cache Scraping
Our EULA does not permit the scraping of the EVE Online cache, as per EULA Paragraph 9.C

*snip*



Well, nobody is telling you to scrape the cache with this library. I personally only use it to access the game's bulkdata for the item definitions and map hierarchy etc, which isn't cache Cool

And even then, that EULA section is really just referring to the scraping of cache for the purpose of botting the market and other things

If you're just doing market data analysis and not doing anything naughty, you should be fine. (Especially considering there is no way for CCP to actually see what you are doing with a bunch of files on your own computer ;)

╦......║...╔╗.║.║.╔╗.╦║.╔╗╔╦╗╔╗

║.╔╗╔╗╔╣.╔╗╠..╠ ╠╗╠╝.║╠ ╠╝║║║╚╗

╩═╚╝║.╚╝.╚╝║..╚╝║║╚╝.╩╚╝╚╝║.║╚╝

Got Item?
Entity
X-Factor Industries
Synthetic Existence
#193 - 2014-06-15 00:49:56 UTC
Just a post to avoid topic getting locked :)

Looks like everything still works anyway.

╦......║...╔╗.║.║.╔╗.╦║.╔╗╔╦╗╔╗

║.╔╗╔╗╔╣.╔╗╠..╠ ╠╗╠╝.║╠ ╠╝║║║╚╗

╩═╚╝║.╚╝.╚╝║..╚╝║║╚╝.╩╚╝╚╝║.║╚╝

Got Item?
Vaerah Vahrokha
Vahrokh Consulting
#194 - 2014-07-08 16:48:03 UTC
Entity wrote:
Just a post to avoid topic getting locked :)

Looks like everything still works anyway.


Confirming that as of today, the library is still working!

EvE Online portal | EvE markets tutorials
DeviousMerc
Caldari Provisions
Caldari State
#195 - 2014-08-31 17:47:53 UTC
Do you know why chrome is blocking your dropbox install files as malware?
Someone asked before about examples could you answer this?
Git version of software is different than zipped on your dropbox - which is the latest and why are they different?
thanks
Entity
X-Factor Industries
Synthetic Existence
#196 - 2014-09-08 09:54:08 UTC
DeviousMerc wrote:
Do you know why chrome is blocking your dropbox install files as malware?
Someone asked before about examples could you answer this?
Git version of software is different than zipped on your dropbox - which is the latest and why are they different?
thanks


- no idea on the dropbox. probably because they are executables, and you should always be wary about running stuff from the internets? *shrug*
- what examples? there's a bunch in the distribution and they're out of date :P
- git version is always latest. I'm just not building new installers every time I change one letter of code. (read: I'm lazy).


╦......║...╔╗.║.║.╔╗.╦║.╔╗╔╦╗╔╗

║.╔╗╔╗╔╣.╔╗╠..╠ ╠╗╠╝.║╠ ╠╝║║║╚╗

╩═╚╝║.╚╝.╚╝║..╚╝║║╚╝.╩╚╝╚╝║.║╚╝

Got Item?
Mr Mac
Dark Goliath
#197 - 2014-09-13 14:05:15 UTC  |  Edited by: Mr Mac
I installed. Tried with implants, it worked but datadump not working. I updated datadump.py but still not working.
I changed input/output path. Am I missing something?

EDIT: I think I figured out but i got error
"IOError: [Errno 2] No such file or directory: u'C:\\Users\\#\\AppData\\Local\\CCP\\EVE\\g_eve_tranquility\\cache\\MachoNet\\87.237.38.200\\403\\CachedMethodCalls\\53c0.cache"
Entity
X-Factor Industries
Synthetic Existence
#198 - 2014-11-01 02:09:53 UTC
Mr Mac wrote:
I installed. Tried with implants, it worked but datadump not working. I updated datadump.py but still not working.
I changed input/output path. Am I missing something?

EDIT: I think I figured out but i got error
"IOError: [Errno 2] No such file or directory: u'C:\\Users\\#\\AppData\\Local\\CCP\\EVE\\g_eve_tranquility\\cache\\MachoNet\\87.237.38.200\\403\\CachedMethodCalls\\53c0.cache"


Yeah the examples are hopelessly out of date (and so is this response). Sorry for that :P

╦......║...╔╗.║.║.╔╗.╦║.╔╗╔╦╗╔╗

║.╔╗╔╗╔╣.╔╗╠..╠ ╠╗╠╝.║╠ ╠╝║║║╚╗

╩═╚╝║.╚╝.╚╝║..╚╝║║╚╝.╩╚╝╚╝║.║╚╝

Got Item?
ISD Dorrim Barstorlode
ISD Community Communications Liaisons
ISD Alliance
#199 - 2015-03-24 20:59:33 UTC
Unlocked at polite request by OP.

ISD Dorrim Barstorlode

Senior Lead

Community Communication Liaisons (CCLs)

Interstellar Services Department
Entity
X-Factor Industries
Synthetic Existence
#200 - 2015-03-24 21:22:02 UTC
(Thanks Dorrim)

It seems the new Download on Demand client removed all trace of the EmbedFS virtual filesystem.

This seems like as good as any opportunity to clean up the project a bit, and also go forward with my plan to remove support for multiple simultaneous clients (there's really only one usecase for it; generating diffs, and that's easily done by just doing an intermediate export step).

I'll see about fixing the current version to at least be able to load the FSD data again.

Might take a while though, because I've got less time to work on pet projects at the moment, sorry for the inconvenience :3

╦......║...╔╗.║.║.╔╗.╦║.╔╗╔╦╗╔╗

║.╔╗╔╗╔╣.╔╗╠..╠ ╠╗╠╝.║╠ ╠╝║║║╚╗

╩═╚╝║.╚╝.╚╝║..╚╝║║╚╝.╩╚╝╚╝║.║╚╝

Got Item?