These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

 
  • Topic is locked indefinitely.
 

[EveLib] A .NET library for EveXML, CREST, EveCentral, and more

First post
Author
Laura Shimaya
Companion Cube Corp.
#361 - 2016-03-19 22:08:21 UTC  |  Edited by: Laura Shimaya
Yes, the follwing method calls never return:

crest.GetRoot().Query(x => x.ItemTypes).Query(x => x.Single(r => r.Name == "Medium Remote Armor Repairer II"))
crest.GetRoot().Query(x => x.ItemTypes).AllItems();
crest.GetRoot().Query(x => x.MarketTypes).AllItems();


Update:

Even if the ItemType is in the first page of https://public-crest.eveonline.com/types/ the method call does not return.

Example:

crest.GetRoot().Query(x => x.ItemTypes).Query(x => x.Single(r => r.Name == "720mm Howitzer Artillery I"));


Update 2:

This works because the ItemType is on the first Crest page and thereby in the "Items" collection:

crest.GetRoot().Query(r => r.MarketTypes).Items.Single(i => i.Type.Name == "720mm Howitzer Artillery I");


This gives an exception (InvalidOperationException) because the ItemType is not on the first page:

crest.GetRoot().Query(r => r.MarketTypes).Items.Single(i => i.Type.Name == "Medium Remote Armor Repairer II");
Laura Shimaya
Companion Cube Corp.
#362 - 2016-03-20 02:22:58 UTC
Found another problem:


EveCrest crest = new EveCrest();

var root = crest.GetRoot();
var regions = root.Query(r => r.Regions).Items;

foreach(var re in regions)
{
Console.WriteLine(re.Href);
var reg = crest.Load(re);
}

After iterating over 0 - 9 regions the Load method never returns.

Output in Visual Studio:

Quote:

EveLib Verbose: 0 : JsonSerializer.Deserialize:Start
EveLib Verbose: 0 : JsonSerializer.Deserialize:Complete
EveLib Verbose: 0 : EveLibFileCache.LoadAsync:Start
EveLib Verbose: 0 : EveLibFileCache:CacheRegisterLookupUri: https://public-crest.eveonline.com/regions/11000007/?
EveLib Verbose: 0 : EveLibFileCache:CacheRegisterLookupHash: 9DD5315EC48497CB991B692B93CD4E701F82885E
EveLib Verbose: 0 : EveLibFileCache:CacheRegisterHit: False
EveLib Verbose: 0 : EveLibFileCache.LoadAsync:Complete
EveLib Error: 0 : Initiating Request: https://public-crest.eveonline.com/regions/11000007/?
EveLib Information: 0 : Response status: OK, OK
EveLib Verbose: 0 : From cache: False
EveLib Verbose: 0 : EveLibFileCache.StoreAsync:Start
EveLib Verbose: 0 : EveLibFileCache:Uri: https://public-crest.eveonline.com/regions/11000007/?
EveLib Verbose: 0 : EveLibFileCache:Cache Expiry: 03/20/2016 03:20:06
EveLib Verbose: 0 : EveLibFileCache:Writing cache data to disk: https://public-crest.eveonline.com/regions/11000007/?
EveLib Verbose: 0 : EveLibFileCache:Writing cache register to disk
EveLib Verbose: 0 : EveLibFileCache.StoreAsync:Complete
EveLib Verbose: 0 : JsonSerializer.Deserialize:Start
EveLib Verbose: 0 : JsonSerializer.Deserialize:Complete
EveLib Verbose: 0 : EveLibFileCache.LoadAsync:Start
EveLib Verbose: 0 : EveLibFileCache:CacheRegisterLookupUri: https://public-crest.eveonline.com/regions/11000008/?
EveLib Verbose: 0 : EveLibFileCache:CacheRegisterLookupHash: 5568F356C3045BDEC492F47C36EFE87F1C116679
EveLib Verbose: 0 : EveLibFileCache:CacheRegisterHit: False
EveLib Verbose: 0 : EveLibFileCache.LoadAsync:Complete
EveLib Error: 0 : Initiating Request: https://public-crest.eveonline.com/regions/11000008/?
EveLib Information: 0 : Response status: OK, OK
EveLib Verbose: 0 : From cache: False
EveLib Verbose: 0 : EveLibFileCache.StoreAsync:Start
EveLib Verbose: 0 : EveLibFileCache:Uri: https://public-crest.eveonline.com/regions/11000008/?
EveLib Verbose: 0 : EveLibFileCache:Cache Expiry: 03/20/2016 03:20:06
EveLib Verbose: 0 : EveLibFileCache:Writing cache data to disk: https://public-crest.eveonline.com/regions/11000008/?
EveLib Verbose: 0 : EveLibFileCache:Writing cache register to disk
The thread 0x2788 has exited with code 259 (0x103).
The thread 0x1be8 has exited with code 259 (0x103).
The thread 0x16e0 has exited with code 259 (0x103).
The thread 0x2dcc has exited with code 259 (0x103).
The thread 0x2cb4 has exited with code 259 (0x103).


So it seems that it tries to write something to the cache file. After that all threads get terminated. Hope that helps!
Icahmura Hasaki
Perkone
Caldari State
#363 - 2016-03-20 09:47:23 UTC
Try updating to EveCrest v3.3.3 and let me know if you still have any issues.

Developer of EveLib and EveAuthUtility

Icahmura Hasaki
Perkone
Caldari State
#364 - 2016-03-20 09:51:58 UTC  |  Edited by: Icahmura Hasaki
Quote:
Update 2:

This works because the ItemType is on the first Crest page and thereby in the "Items" collection:

crest.GetRoot().Query(r => r.MarketTypes).Items.Single(i => i.Type.Name == "720mm Howitzer Artillery I");


This gives an exception (InvalidOperationException) because the ItemType is not on the first page:

crest.GetRoot().Query(r => r.MarketTypes).Items.Single(i => i.Type.Name == "Medium Remote Armor Repairer II");



This is intended, and is why AllItems() exists. I could overwrite Items but I would rather let users have the option to limit the linq query to the current page if they wish to do so.

edit:
On the same note, this should work if automatic paging is enabled, even if the item isn't on the first page. So the API might be a slightly confusing, I'll try to make it a bit cleaner and clarify some things in the next major version.
crest.GetRoot().Query(r => r.MarketTypes).Query(r => r.Items.Single(i => i.Type.Name == "Medium Remote Armor Repairer II"));

Developer of EveLib and EveAuthUtility

Laura Shimaya
Companion Cube Corp.
#365 - 2016-03-20 14:49:12 UTC  |  Edited by: Laura Shimaya
I updated EveCrest to V3.3.3.0 via NuGet.

It seems that nothing changed. The following calls still timeout / never return:

crest.GetRoot().Query(x => x.ItemTypes).AllItems();

Quote:

EveLib Error: 0 : Initiating Request: https://public-crest.eveonline.com/types/?page=3&
EveLib Information: 0 : Response status: OK, OK
EveLib Verbose: 0 : From cache: False
EveLib Verbose: 0 : EveLibFileCache.StoreAsync:Start
EveLib Verbose: 0 : EveLibFileCache:Uri: https://public-crest.eveonline.com/types/?page=3&
EveLib Verbose: 0 : EveLibFileCache:Cache Expiry: 03/20/2016 15:40:36
EveLib Verbose: 0 : EveLibFileCache:Writing cache data to disk: https://public-crest.eveonline.com/types/?page=3&
EveLib Verbose: 0 : EveLibFileCache:Writing cache register to disk
The thread 0x1710 has exited with code 259 (0x103).
The thread 0xdc4 has exited with code 259 (0x103).
The thread 0x1008 has exited with code 259 (0x103).
The thread 0x12e0 has exited with code 259 (0x103).
The thread 0x17e8 has exited with code 259 (0x103).
The thread 0xb0 has exited with code 259 (0x103).


crest.GetRoot().Query(x => x.MarketTypes).AllItems();

Quote:

EveLib Error: 0 : Initiating Request: https://public-crest.eveonline.com/market/types/?page=3&
EveLib Information: 0 : Response status: OK, OK
EveLib Verbose: 0 : From cache: False
EveLib Verbose: 0 : EveLibFileCache.StoreAsync:Start
EveLib Verbose: 0 : EveLibFileCache:Uri: https://public-crest.eveonline.com/market/types/?page=3&
EveLib Verbose: 0 : EveLibFileCache:Cache Expiry: 03/20/2016 14:47:16
EveLib Verbose: 0 : EveLibFileCache:Writing cache data to disk: https://public-crest.eveonline.com/market/types/?page=3&
EveLib Verbose: 0 : EveLibFileCache:Writing cache register to disk
The thread 0x14b4 has exited with code 259 (0x103).
The thread 0x1a8 has exited with code 259 (0x103).
The thread 0x160c has exited with code 259 (0x103).
The thread 0x1008 has exited with code 259 (0x103).
The thread 0x12e0 has exited with code 259 (0x103).
The thread 0x126c has exited with code 259 (0x103).
The thread 0xc60 has exited with code 259 (0x103).


This gives me between 3 to 5 regions:

EveCrest crest = new EveCrest();

var root = crest.GetRoot();
var regions = root.Query(r => r.Regions).Items;

foreach(var re in regions)
{
Console.WriteLine(re.Href);
var reg = crest.Load(re);
}

Quote:

EveLib Error: 0 : Initiating Request: https://public-crest.eveonline.com/regions/10000019/?
EveLib Information: 0 : Response status: OK, OK
EveLib Verbose: 0 : From cache: False
EveLib Verbose: 0 : EveLibFileCache.StoreAsync:Start
EveLib Verbose: 0 : EveLibFileCache:Uri: https://public-crest.eveonline.com/regions/10000019/?
EveLib Verbose: 0 : EveLibFileCache:Cache Expiry: 03/20/2016 15:44:04
EveLib Verbose: 0 : EveLibFileCache:Writing cache data to disk: https://public-crest.eveonline.com/regions/10000019/?
EveLib Verbose: 0 : EveLibFileCache:Writing cache register to disk
The thread 0xa5c has exited with code 259 (0x103).
The thread 0xbec has exited with code 259 (0x103).
The thread 0x468 has exited with code 259 (0x103).
The thread 0xc4 has exited with code 259 (0x103).
The thread 0x6ec has exited with code 259 (0x103).


Quote:

This is intended, and is why AllItems() exists. I could overwrite Items but I would rather let users have the option to limit the linq query to the current page if they wish to do so.

Yes, the defference between Items and AllItems is pretty clear. The first one gives the results from the current Crest page the later should iterate over all pages.

This gives an error as "r" does not have a property "Items":

crest.GetRoot().Query(r => r.MarketTypes).Query(r => r.Items.Single(i => i.Type.Name == "Medium Remote Armor Repairer II"));

Thanks for your help!
Icahmura Hasaki
Perkone
Caldari State
#366 - 2016-03-21 09:25:30 UTC
Oh yes, the MarketTypes endpoint doesn't have the Items property. My point was, if you use Items enumeration within a Query() method, it will request all the pages in the collection, but if you just want to retrieve the full collection, you have to use AllItems().

My tests here are fine with the 3.3.3 version. You can disable the cache by setting the crest.RequestHandler.CacheLevel property, while I look for a solution.

Developer of EveLib and EveAuthUtility

Icahmura Hasaki
Perkone
Caldari State
#367 - 2016-03-21 18:18:01 UTC
If you are experiencing any issues with deadlocking, ie. methods never returns, please update to Core v3.0.5.

Developer of EveLib and EveAuthUtility

Laura Shimaya
Companion Cube Corp.
#368 - 2016-03-21 18:47:40 UTC
Quote:

If you are experiencing any issues with deadlocking, ie. methods never returns, please update to Core v3.0.5.


Core 3.0.5 fixed all of my problems! Tremendous work good sir!


Your caching mechanism is excenellent! Queries return nearly instantly even when iterating over all pages:

var item = crest.GetRoot().Query(x => x.ItemTypes).Query(x => x.Single(r => r.Name == "Light Neutron Blaster II"));

This query over 29 Crest pages takes around 1200 ms with a populated cache.
Paranid Warrier
Imperial Shipment
Amarr Empire
#369 - 2016-03-21 20:35:53 UTC
Hello,
Thank you for your tremendous efforts on creating this .NET library !!!
It is very useful.
One question, when are you going o implement fetching Buy and Sell region orders ?
I see that currently the maximum I can get is the URI of market orders.

Thanks !
Icahmura Hasaki
Perkone
Caldari State
#370 - 2016-03-22 22:27:29 UTC
Paranid Warrier wrote:
Hello,
Thank you for your tremendous efforts on creating this .NET library !!!
It is very useful.
One question, when are you going o implement fetching Buy and Sell region orders ?
I see that currently the maximum I can get is the URI of market orders.

Thanks !


Orders are supported.

https://forums.eveonline.com/default.aspx?g=posts&m=6348742#post6348742



Developer of EveLib and EveAuthUtility

Icahmura Hasaki
Perkone
Caldari State
#371 - 2016-03-23 15:11:11 UTC
Some neat new features coming up.

All CollectionResource objects implement IEnumerable, which works with automatic pagination. That is, any operation defined on IEnumerable will work across all pages in the collection by default. The CollectionResource.Items property will still only hold the local items for the specific page the object represents.

The pagination logic is also improved, only loading the next page when required. LINQ queries will be smarter and can perform better if you pay attention to how you write them. Operations such as Where(), ToList() and Single() will still enumerate the complete collection, while First() will stop and return the first match it finds. Using First() on collections with many pages can provide huge performance benefits and decrease the amount of requests considerably.

This also makes the code slightly neater, and is backwards compatible.
Previously, the Query() API let you omit the .Items property and would use automatic pagination to operate on all items in the collection. However, outside the Query() methods, you would have to get the Items property explicitly, and all operations were on the local items unless AllItems() were used.

Now, all operations on the collection object default to all items, adhering to general LINQ guidelines, and works similarly to ie. Entity Framework. The default implementation of ToList() and my ToListAsync() mostly replace AllItems() and AllItemsAsync(), with the exception that if Automatic Pagination is disabled, ToList() only returns the local list, while AllItems() always return the full collection.

Examples:

// Inside a Query statement, the API remains the same
Quote:
var alliances = _crest.GetRoot().Query(r => r.Alliances);
var alliance = alliances.Query(r => r.First(a => a.ShortName == "MZR"));


The difference is that previously all the pages in the collection would be queried. Now we query one page at a time until we find a match, and return that.

// Outside a query statment
var alliances = _crest.GetRoot().Query(r => r.Alliances);
var old = alliances.AllItems().First(a => a.ShortName == "MZR");
var now = alliances.First(a => a.ShortName == "MZR");


Now, operating directly on the Collection object is similar to operating on AllItems() previously, except we now have the smarter IEnumerable implementation. Using .Items is still the same as before, only operating on local items.

Another new feature is simpler query parameters. You no longer need to specify the parameter name, all tho that option remains. Resources that can be used as parameters are bound to a default parameter name, and you can pass the whole Entity object as a parameter.

Examples:


var itemLink = _crest.GetRoot().Query(r => r.ItemTypes).First();
var itemResource = crest.Load(itemLink);
var region =  _crest.GetRoot()
                .Query(r => r.Regions)
                .Query(r => r.Items.First());
var old = region.Query(r => r.MarketBuyOrders, "type", itemLink.Uri);
var now = region.Query(r => r.MarketBuyOrders, itemLink);
var now2 = region.Query(r => r.MarketBuyOrders, itemResource);


As you can see, this works with both a LinkedEntity object, and a CrestResource object.

Developer of EveLib and EveAuthUtility

Icahmura Hasaki
Perkone
Caldari State
#372 - 2016-03-23 18:27:03 UTC
Xiaou Bijoun
Imperial Academy
Amarr Empire
#373 - 2016-03-23 21:15:22 UTC  |  Edited by: Xiaou Bijoun
Haven't updated in a while, but this version broke some stuff.

Today I went from...
Core 3.0.3.0 to 3.0.5.0
EveCrest 3.2.4.0 to 3.3.3.0
EveXml 3.0.5.0 to 3.0.6.0
NewtonSoft.Json 8.0.2 to 8.0.3

Currently using .net 4.5.2 in case that has relevance.

This call now causes an exception but I am guessing it has to do with some of the updates shown above.

var sellordercollection = await toolbox.eveCrest.LoadAsync(region.MarketSellOrders, "type", item.type);

item.type = "https://public-crest.eveonline.com/types/27912/"

I get "Invalid URI: The URI scheme is not valid."

EDIT: Also the exception being thrown is 'System.UriFormatException'. I have a try catch but for AggregateException, which I though that you mentioned somewhere is how you throw exceptions from inside your library.
Icahmura Hasaki
Perkone
Caldari State
#374 - 2016-03-23 21:33:57 UTC  |  Edited by: Icahmura Hasaki
You should be on Core 3.0.6 and EveCrest 3.4.0. If you use async/await, you do not need to worry about AggregateException. Just catch exceptions as normal, those are only thrown if you're using the blocking (non async) methods.

I'll have a look at that right away, but it might already be fixed if you get the latest updates.

edit:
The code you showed works fine on the latest versions, so try updating. Let me know if you're still having issues.

Developer of EveLib and EveAuthUtility

Xiaou Bijoun
Imperial Academy
Amarr Empire
#375 - 2016-03-23 23:16:59 UTC  |  Edited by: Xiaou Bijoun
So I updated to 3.0.6 core and 3.4.0 crest but still does not work. I can fall back to a previous version and it works.

I tried to copy the example that you had in earlier post but the line

var old = region.Query(r => r.MarketBuyOrders, "type", itemLink.Uri);

gives error that itemLink.Uri does not exist and had to change to itemLink.Href.Uri.

Also that code appears to look for MarketBuyOrders of type #System.

I am guessing that is just an example, but it does work.

UPDATE: The problem was on my end, apparently either the Crest formats for some of the endpoints changed or the models used in the library changed or both. In order to reduce the need to access some of the endpoints (regions, markettypes, etc) due to the time necessary to access them all, I gather them once and store them locally in a file. I have not implemented a way for my program to automatically determine that it needs to get the data again so I just delete the file and it will get it again. Is there a way to determine if the information in a linked entity has changed (i.e. a version property)? I had thought of checking to see if the item count had changed but they could update an item for any reason and not change the count.

So the issue with the link is that parsing of the old data did not line up with the new data and caused properties to contain incorrect values. So the endpoint for the regions market orders contained something that was not even a Uri. After downloading the data again the problem has gone away.
Hel O'Ween
Men On A Mission
#376 - 2016-03-24 16:56:52 UTC  |  Edited by: Hel O'Ween
Xiaou Bijoun wrote:
So I updated to 3.0.6 core and 3.4.0 crest but still does not work. I can fall back to a previous version and it works.

I tried to copy the example that you had in earlier post but the line

var old = region.Query(r => r.MarketBuyOrders, "type", itemLink.Uri);

gives error that itemLink.Uri does not exist and had to change to itemLink.Href.Uri.

Also that code appears to look for MarketBuyOrders of type #System.

I am guessing that is just an example, but it does work.


Without knowing EveLib, my guess is that "type" in r.MarketBuyOrders, "type", itemLink.Uri) is meant to be replaced with the item's real typeID (from table invTypes), because #System has typeID 0, to which "type" would be converted if treated as a numeric value. I.e. the all to popular Tritanuim example has the typeID 34.

EVEWalletAware - an offline wallet manager.

Xiaou Bijoun
Imperial Academy
Amarr Empire
#377 - 2016-03-24 20:48:13 UTC
The "type" is a named parameter that actually gets appended to the Uri used to access the market order endpoint.

Here is the console output of the actual command generated for a concussion bomb from Domain.

https://public-crest.eveonline.com/market/10000043/orders/sell/?type=https://public-crest.eveonline.com/types/27912/&

You can see the endpoint for sell orders from domain

https://public-crest.eveonline.com/market/10000043/orders/sell/

has a type parameter

?type=

which is set to the Concussion Bomb (Type.ID = 27912) endpoint's Uri

https://public-crest.eveonline.com/types/27912/&

Is the & at the end necessary? or an artifact of possibly adding more parameters?
Xiaou Bijoun
Imperial Academy
Amarr Empire
#378 - 2016-04-21 17:08:25 UTC  |  Edited by: Xiaou Bijoun
Has this forum thread died out? I have been trying to post a question and it doesn't seem to be working.

EDIT: Okay it is working.

I am having an issue with the ZKillboard constructor.

It throws an exception.

"Field not found: 'eZet.EveLib.Core.Config.CacheFactory'."

It works in the evelib-master project in both the ZKillboard_Test and directly create my own ZKillboard object.

My project has many other libraries used including Newsoft.Json. Is it possible there is some conflict? I was wondering if it was using statements but it would not compile if the correct using statements were missing right?

My thought is to start a new project and only use the ZKillboard object to begin with to make sure it works, then start adding in other libraries until I can duplicate the problem.

Any other thoughts?
Icahmura Hasaki
Perkone
Caldari State
#379 - 2016-04-22 13:20:02 UTC
If the nuget version isnt working, but the master branch is, I suspect I just didnt update the nuget version correctly, so it's referring to an older version of Core, which causes that error. I'm 99% sure that's the reason, I'll have a look later.
ZKillboard currently isn't able to parse all the data from the statistics endpoint they provide, I ran into some issues with parsing some of the more complex data.

About your previous question, the & at the end of the url isn't necessary.

Developer of EveLib and EveAuthUtility

Icahmura Hasaki
Perkone
Caldari State
#380 - 2016-04-23 17:09:39 UTC
I've updated ZKillboard on nuget, please let me know if you still have any issues.

Developer of EveLib and EveAuthUtility