These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

 
  • Topic is locked indefinitely.
 

EVECacheParser v1.0.2 - (EVE Online Cache File Parser - the .NET way)
Author
Previous Topic Next Topic
Desmont McCallock
#1 - 2012-03-19 17:07:03 UTC  |  Edited by: Desmont McCallock
SUMMARY
EVECacheParser is an EVE Online cache/bulkdata file parser library, written in C# and targeting the .NET Framework 4.
It is distributed under GPL v2 (see license.txt that is included with the distribution).


WHAT IT CAN DO
Reads and decodes cache files found in cache and bulkdata folders.


WHAT IT CAN'T DO
Nothing more than what it can do.


SOURCE CODE
EVECacheParser is an open source project hosted at https://bitbucket.org/Desmont_McCallock/evecacheparser.
A zip file containing the library and documentation can be found at the download section.


USAGE
Documentation on how to use can be found at the projects wiki.
If you are familiar with Entity's 'Reverence' you won't find it hard to use as the returned data are in the same format.


DISCLAIMER
This product does not modify in any way any file associated with the EVE client or writes files that change the EVE client behavior, therefore does not violate CCP's EVE Online EULA.


SPECIAL THANKS
At this point I would like to give special thanks to the following people, who without their work this product wouldn't be made possible.

- 'Entity' for providing and having the 'Reverence' code open sourced.
Also for being the insperation and the pioneer on the field.

- 'Kaladr' for providing and having the 'libevecache' code open sourced.
Also for supporting the project.

- 'Jay Wareth' for providing and having the 'EVECache .NET' code open sourced.
Without his work I wouldn't know where to start from.

- 'Vessper' for supporting and being the beta tester.


LATEST VERSION: 1.0.2

--- Change Log ---

v1.0.2
Fixed an issue where parsing a bulkdata cache file returned only the last entry as value.

v1.0.1
Added internal conversion for currency type data. User had to convert any currency data from Int64 to Decimal by dividing with 10000.0. EVECacheParser does that now internally.
Callean Drevus
Perkone
Caldari State
#2 - 2012-03-21 09:32:30 UTC
Friggin awesome! Even if I can not think of anything to use this for now, this is one of the best things that has been released in months.

Developer/Creator of EVE Marketeer
Shangra Mu'zgob
Parasitic Trading Systems
#3 - 2012-03-21 10:03:46 UTC
Excellent work ! I was too lazy to bother with Python to create my own lib for cache reading. I've been using Process.Start to invoke 3rd party cache scrapers in my C# market app, but you'we made by day :-).
Once again - well done Sir.
Trading for living / living for trading.
Desmont McCallock
#4 - 2012-04-20 21:55:31 UTC
Releasing EVECacheParser v1.0.1.
Packtu'sa
Nabaal Construction and Industrials Corp
Nabaal Syndicate
#5 - 2012-04-30 21:26:33 UTC
I'm looking forward to playing with this. However, can I ask why all the entry points are static? Is there a particular reason for this restriction?
Desmont McCallock
#6 - 2012-05-01 05:34:13 UTC  |  Edited by: Desmont McCallock
Instance creation is done internally. Also I wanted to make it as easy to use as can be (for the newbies, fewer code lines).
Ideki
Wolf Brothers INC
United Neopian Federation
#7 - 2012-05-17 12:37:54 UTC  |  Edited by: Ideki
Hi,
I am trying your parser, and it looks like there are some new data that you are not parsing correctly.

I got an exception in the Dumper.CheckResult

When you switch between the id, you do not handle "config", "rewardMgr", "GetMaterialsForTypeWithActivity" so they throw an argumentOutOfRangeException

When you receive a "marketProxy", "GetNewPriceHistory" is not handled and throws an argumentOutOfRangeException

When you receive a "stationSvc", "GetAllianceSystems" is not handled and throws an argumentOutOfRangeException

When you receive a "facWarMgr", an InvalidCastException is thrown

I know it is just a sample program, but it would be nice if it was working all the time. Smile

Creator of The EVE Planetary Planner
Desmont McCallock
#8 - 2012-05-17 12:45:32 UTC
As you said CheckResult() method is just sample hint code to show you people how the casting to proper type is done.
It's not hard to add the proper code to handle any, not handled by my code, cache methods.
Ideki
Wolf Brothers INC
United Neopian Federation
#9 - 2012-05-17 12:47:18 UTC
Desmont McCallock wrote:
As you said CheckResult() method is just sample hint code to show you people how the casting to proper type is done.
It's not hard to add the proper code to handle any, not handled by my code, cache methods.


No problem.

Creator of The EVE Planetary Planner
Thart
Caldari Provisions
Caldari State
#10 - 2012-06-21 08:04:43 UTC
Desmont, do you plan to extend the license to LGPL?

EVE Mentat - true trade tool
Desmont McCallock
#11 - 2012-06-21 09:59:10 UTC
Thart wrote:
Desmont, do you plan to extend the license to LGPL?
Unfortunately for you, no. My philosophy is that any free-software should be open-source (no offence). But there is a window in the GPL license that may allow you to use it.
Quote:
GPL wiki
"An alternative method of satisfying the copyleft is to provide a written offer to provide the source code on a physical medium (such as a CD) upon request."
Thart
Caldari Provisions
Caldari State
#12 - 2012-06-21 13:01:46 UTC
Desmont McCallock wrote:
Thart wrote:
Desmont, do you plan to extend the license to LGPL?
Unfortunately for you, no. My philosophy is that any free-software should be open-source (no offence). But there is a window in the GPL license that may allow you to use it.
Quote:
GPL wiki
"An alternative method of satisfying the copyleft is to provide a written offer to provide the source code on a physical medium (such as a CD) upon request."


Sounds like weird and illegitimate workaround for me.

In general it's a good illustration when the politics (yes, licensing - it's rather politics!) becomes the obstacle to developing high quality software. Licensing hysteria significantly limits the number of possible working combinations of good components.

P.S. No offence, of course ;)

EVE Mentat - true trade tool
Desmont McCallock
#13 - 2012-08-03 17:34:05 UTC
At the moment EVE Cache Parser provides the result of the parsing in the format of KeyValuePair.

Is there any interest for providing the result in any other format like XML, JSON, YAML?
If so, what would be your order of choice?
Dragonaire
Here there be Dragons
#14 - 2012-08-04 04:35:20 UTC
I'd start with JSON I think since most things seem to be moving to it. It has less overhead than XML which is nice and somehow just don't think YAML would be that useful but that's just IMHO Blink

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.
Vessper
Dark Mason Society
#15 - 2012-08-10 09:47:40 UTC
I've been looking at the bulk files and it looks like only a single result is being returned in the KeyValuePair. No problems with the Machonet files so just wandering if it's something I've missed or the result isn't returning everything?
Desmont McCallock
#16 - 2012-08-10 10:13:20 UTC
Lol I had the same idea as you (exporting the SDE from the bulkdata) but hadn't found the time to dive into it.
Indeed it seems that instead of returning the whole table, it only returns the last record.
I'll look into why, in the next days.
Vessper
Dark Mason Society
#17 - 2012-08-10 10:20:23 UTC
I figured with the changeover in the SDE from SQL to YAML, it would just be easier to get the data from the cache files rather than mess around combining them during the transition period (which may take years!).

Looking forward to the changes!
Desmont McCallock
#18 - 2012-08-10 10:41:13 UTC  |  Edited by: Desmont McCallock
Exactly what I thought. Not to mention that this way you can have the SDE by the time you log in the first account after a patch.
I'm planning on creating a tool that extracts the SDE from the cache files to YAML format, so to be compatible with any YAMLToSQL converters that will pop up in the future.

Edit: YamlToSQL by Jognu.
Desmont McCallock
#19 - 2012-08-10 19:15:44 UTC  |  Edited by: Desmont McCallock
Releasing EVECacheParser v1.0.2.
Vessper
Dark Mason Society
#20 - 2012-08-11 14:35:42 UTC
Thanks, works great!