These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

 
  • Topic is locked indefinitely.
 

Yapeal PHP API library (version 1.0.5)

First post
Author
Previous Topic Next Topic
Satis Iqulenax
Free Imperial Vikings
#221 - 2012-01-24 00:52:10 UTC
Gabba Cyno wrote:
I was talking about that but since there is a starter guide, figured that might be something good to add to it also.
We do appreciate your suggestion. However, I don't know if it will be included in the Starter Guide.
I will however add something in the wiki but it will be when I get the time for it. Maybe I'll add a link to it in the Starter Guide so people would know where to look if they went into the Starter Guide to check it.

I can't tell right now when that would be. It could be today or maybe in the next couple of weeks.

I'll keep you posted in here about it when I get the chance to look into it.

Kind regards
Satis

Committer of Yapeal for Eve API. Developer of XMLSchema. A database schema tool for PHP
No La
Greased Lightning Engineering
#222 - 2012-01-24 08:47:10 UTC  |  Edited by: No La
Dragonaire wrote:
It should try to fill it back in for you as it always tries to walk back as far as it can.


Apparently it doesn't work. I'm still missing data (for two days now, because i deleted everything up to the 19th, in the hope that it would just fetch everything from then to now.)

I did manage to manually retrieve a xml file from the api, that seems to contain everything i'm missing.

Is there a way to let yapeal process that xml file, or do i have to write some parser myself?

thanks,
No La
Dragonaire
Here there be Dragons
#223 - 2012-01-24 15:40:06 UTC
Might try deleting the cache files and what's in you utilXmlCache so it tries getting everything again as Yapeal is designed to always walk back to get all the data as far back as the API allows.

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.
No La
Greased Lightning Engineering
#224 - 2012-01-25 08:08:41 UTC
I tried that. I deleted all market journal cache files, deleted all entries from the charWalletJournal table from the 19th till now.
After running yapeal, i end up with no data for the 20th, 21th and 22e.
For the 23rd to now the data is there.

Seems yapeal only fetches the last 3 days.
Dragonaire
Here there be Dragons
#225 - 2012-01-25 21:31:31 UTC
No La wrote:
I tried that. I deleted all market journal cache files, deleted all entries from the charWalletJournal table from the 19th till now.
After running yapeal, i end up with no data for the 20th, 21th and 22e.
For the 23rd to now the data is there.

Seems yapeal only fetches the last 3 days.
I just tried Yapeal on my test system and Yapeal was able to retrieve new transaction and journal entries for more than 2 weeks worth of backlogged data since it had last been run include ones for the days you are having problems with. I did identify some other problems that might be causing problems for everyone which is do to both changes CCP made to the APIs and a bug I introduced in the install/util.xml file.

When I made some changes to util.xml so it could work without having to make changes to everyone's my.cnf file to allow larger allowed packets. Some extra commas got missed when breaking up the single insert into multiple inserts which caused the SQL to have errors in it. I've now fixed the problem.

Additionally the bogus class/api/corp/corpCalendarEventAttendees.php API file that had somehow been added in error has been deleted. Everyone should insure they read the CHANGELOG when update Yapeal so the file is deleted completely and no longer causes errors.

version 12.025.1327

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.
No La
Greased Lightning Engineering
#226 - 2012-01-26 14:26:44 UTC
Thanks for trying to reproduce the problem.

I already solved it by importing the manually downloaded WalletJournal.xml file (with rowCount=2560).

Yapeal has been updating without issues after that.
Astraeus Deltrynon
EveTech Enterprises
#227 - 2012-01-30 20:48:18 UTC
Love your library, outstanding. I'm having a problem though, it installed perfectly but when I try to use "$regKey->store()" to insert a new API key, it gives me "Registered new API key" even if I give it a fake one, and it won't show up in the utilRegisteredKeys table.

On a related note, if I try "$char->store()" to insert a new registered character, it goes in but the character name comes up as "null".

It seems almost like it's not getting data from the EVE API properly...
Satis Iqulenax
Free Imperial Vikings
#228 - 2012-01-31 01:37:06 UTC  |  Edited by: Satis Iqulenax
Astraeus Deltrynon wrote:
Love your library, outstanding. I'm having a problem though, it installed perfectly but when I try to use "$regKey->store()" to insert a new API key, it gives me "Registered new API key" even if I give it a fake one, and it won't show up in the utilRegisteredKeys table.
I'm not sure what the problem is here. Maybe Dragonaire knows about it.

Astraeus Deltrynon wrote:
On a related note, if I try "$char->store()" to insert a new registered character, it goes in but the character name comes up as "null".

It seems almost like it's not getting data from the EVE API properly...
This is something I know about.

The problem is, that after you have added a CAK info to the utilRegisteredKeys table you will need to run yapeal.php to let Yapeal update the account* tables so when you are adding a character with the RegisteredCharacter tool, it can get the character name from those tables.
If you followed the Starter Guide, I wrote this right before the section class RegisteredCharacter:
Quote:
Before moving on, you should manually run yapeal.php from CLI to populate the account* tables
Note: even it the character name is not added in the utilRegisteredCharacter table, Yapeal will still be able to pull the data from that character anyway.

Kind regards
Satis

Committer of Yapeal for Eve API. Developer of XMLSchema. A database schema tool for PHP
Dragonaire
Here there be Dragons
#229 - 2012-01-31 05:34:27 UTC
Astraeus Deltrynon wrote:
Love your library, outstanding. I'm having a problem though, it installed perfectly but when I try to use "$regKey->store()" to insert a new API key, it gives me "Registered new API key" even if I give it a fake one, and it won't show up in the utilRegisteredKeys table.
Actually it will store it in the utilRegisteredKey table but it in no way does Yapeal at the time you do $regKey->store() tries to validate what you add beyond insuring the required columns have something in them and are of a compatible type (integer, string, etc). As Satis Iqulenax pointed out you have to run the main yapeal.php file for it to try pulling anything from the APIs at which time it tries using your fake info and will report the error sent back from the API server and that Yapeal is deactivating that key in to the error logs.

Astraeus Deltrynon wrote:
On a related note, if I try "$char->store()" to insert a new registered character, it goes in but the character name comes up as "null".

It seems almost like it's not getting data from the EVE API properly...
The name is going to be blank if you haven't set it like Satis Iqulenax said above as well.

To put it simply Yapeal works on the GIGO (Garbage In = Garbage Out) principal but with the twist that it'll let you know by an error in the logs and and then deactivates things when it can figure it out there's some wrong with the info so does not keep trying to use the same GI over and over again. It's kind of like a 3 year old: It'll take what ever you give it and try what you said once then tells you it can't figure it out and want you to make it work right P

Hope that the above helps and reading or re-reading the starter guide helps you understand how Yapeal works a little better.

If you have any more questions after the above please let me know.

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.
Overhead
The Scope
Gallente Federation
#230 - 2012-01-31 23:33:19 UTC  |  Edited by: Overhead
Is there some best practice to do some pre- and post processing of data before/after yapeal.php runs?

I know there is a preparetables function I could (ab)use to do what I have in mind, but that means I cannot simply update yapeal without editting all the files in the classes.

I've no clue how to do post processing without figuring out which data has been updated without going trough the cacheduntil table which my guts tell me isnt best practice as well.

Would love if you guys could share your view on this.
malaire
#231 - 2012-02-07 15:28:37 UTC  |  Edited by: malaire
I just installed Yapeal and got it working with one account. However I have few questions about API keys:

Reading http://code.google.com/p/yapeal/wiki/UtilDatabaseTableDependences ...

It says that both "Account" and "Character" keys are used for utilRegisteredKey. What is Account key? Do you mean character key which retrieves information for all characters? And why is corporation key not mentioned?

Another question: If I want to get all information, I think I need to insert one all-characters key per account and one corporation key per corporation. What bits should I remove from activeAPIMask of corporation key to avoid duplicated data. Documentation mentions accountStatus but do I need to remove anything else?

EDIT: It seems corporation key doesn't actually have accountStatus bit. Is there any duplicated information between corporation key and all-characters key?

New to EVE? Don't forget to read: The Manual * The Wiki * The Career Options * and everything else
Dragonaire
Here there be Dragons
#232 - 2012-02-07 16:26:46 UTC
malaire wrote:
Reading http://code.google.com/p/yapeal/wiki/UtilDatabaseTableDependences ...

It says that both "Account" and "Character" keys are used for utilRegisteredKey. What is Account key? Do you mean character key which retrieves information for all characters? And why is corporation key not mentioned?
Yes in the APIKeyInfo API there's a 'type' field which returns 'account' which is known as 'all' when making or editing a key Roll

Quote:
Another question: If I want to get all information, I think I need to insert one all-characters key per account and one corporation key per corporation. What bits should I remove from activeAPIMask of corporation key to avoid duplicated data. Documentation mentions accountStatus but do I need to remove anything else?
You are correct you'll only need one key to get information for all the characters on one account but one per corporation for the corp info. AccountStatus in managed internally to Yapeal and you don't need to worry about it. I'm not really sure where you got that you might need to turn off some of the APIs to avoid duplicate data between account, character and corporation keys. There can be a small bit of duplicate info in a couple of the database tables from the account section but as explained in the page it doesn't cause any problems or extra calls to the APIs etc so you really don't need to be concerned with it.

Quote:
EDIT: It seems corporation key doesn't actually have accountStatus bit. Is there any duplicated information between corporation key and all-characters key?
No there is no duplicate APIs between char and corp keys. Many APIs have both corp and char versions for example AssetList but the corp and char info does not overlap in any way.

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.
malaire
#233 - 2012-02-07 16:36:10 UTC
Dragonaire wrote:
I'm not really sure where you got that you might need to turn off some of the APIs to avoid duplicate data between account, character and corporation keys. There can be a small bit of duplicate info in a couple of the database tables from the account section but as explained in the page it doesn't cause any problems or extra calls to the APIs etc so you really don't need to be concerned with it.

The paragraph "Note that both 'Account' and 'Character' type keys are used for this table ..." in http://code.google.com/p/yapeal/wiki/UtilDatabaseTableDependences ends with "... but there will still be some duplication that shouldn't cause any problems."

That "shouldn't cause" just made me a bit uneasy since it sounds like there is possibility of problems. What?

New to EVE? Don't forget to read: The Manual * The Wiki * The Career Options * and everything else
Dragonaire
Here there be Dragons
#234 - 2012-02-07 16:45:45 UTC  |  Edited by: Dragonaire
Ok guess I maybe should have added the rest of that thought which is ...any problems for anyone using it.
Yapeal knows what it's doing and it isn't a problem for it but could cause some confusion for anyone trying to use it or readding that page it seems Blink

EDIT: I've updated the page to make it a little clearer what I meant. I also wrote that when much of the CAK stuff was very new and thought there might be some other underlaying issues I hadn't worked out yet which I'm glad to say turned out to be non-issues and me just not having worked with them enough to understand some things Smile

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.
malaire
#235 - 2012-02-07 17:35:08 UTC
What is best place to suggest new features?

e.g. I would like to get "currentTime" from XML added to utilcacheduntil-table so that I could easily check how old any data in various tables is. Or would that allways be same as cachedUntil minus appropriate interval from utilcachedinterval-table?

New to EVE? Don't forget to read: The Manual * The Wiki * The Career Options * and everything else
Dragonaire
Here there be Dragons
#236 - 2012-02-07 18:33:04 UTC
Since the CachedUntil time Yapeal uses is based on the cache interval it's what you need to use. The currentTime is used to figure out when to get the APIs since by and large CCP does things with the cachedUntil time reported by the API server that make it largely useless. There was I time when Yapeal used it but I got tired of all the bug reports and everyone got tired of all the APIs errors that were returned when Yapeal actually did what it said Roll So now it uses the currentTime and cache interval to calculate then new cachedUntil time that end up in the table.

Just a bit of a warning don't think that the cachedUntil time has any bearing on how 'fresh' the data is. It something everybody that deals with the API has to learn is that there is no way to tell from the API when the actual data was updated it only tells you when you get it and when you can again.

I'll try to give you a quick example so you understand what's going on. Say two directors for a corp are pull the same API using two installs of Yapeal. One of them pulls say the WalletJournals and since it hasn't been request before CCP actually goes and grabs the data but they then cache it. Let's now say the other director request the WalletJournals as well but 10 minutes later. CCP sees they have the data cached so they return it from there but they only return the currentTime and when you should ask for the data again in cachedUntil based on the caching interval at best. They don't give you the actual time they well be refreshing their cache so as to prevent stampeding for it between the two applications requesting it. This is very important for server load for them especially with their DT which tends to force all the API calls to bunch up. So that means even though the second director's Yapeal could actually get new data 10 minutes soon if it knew it isn't give that knowledge so has to just use the cache interval.

I wouldn't even try to show any of that to the users of your application or would at most do what EveMon does and show when it will be updating it.

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.
malaire
#237 - 2012-02-07 19:12:57 UTC
ok, so it is not possible to correlate data from different API calls (e.g. charassetlist + corpassetlist + characcountbalance + corpaccountbalance = total asset value at certain moment) Sad

Well, anyway really nice library - I just need to learn what I can do and what I can't do with it. Big smile

New to EVE? Don't forget to read: The Manual * The Wiki * The Career Options * and everything else
Dragonaire
Here there be Dragons
#238 - 2012-02-08 00:09:01 UTC
you can do your selects from the table and limit them to the date of the last transaction you want to look at in all the tables and say that was the balance then but you would need to run at least one cache interval behind to be absolutely accurate. Since asset list is only every 6 hours you have to be at least that far behind but you could try rolling in any transactions from the information in WalletJournal/transactions as will but still be limited to their cache intervals

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.
Dragonaire
Here there be Dragons
#239 - 2012-02-13 15:41:03 UTC
Just an update to let everyone know I've starting the process to migrate the Yapeal project over completely to SourceForge and as part of that process I've decided it's time to update the project on SourceForge to the new Beta style.
https://sourceforge.net/apps/trac/sourceforge/ticket/24305

Work is underway to convert the wiki as it's the biggest issue to be solved but I've been experimenting with a process that seems to work. The process has some manual work to it so it'll take time. For now everything will continue to be on both sites but over the next few weeks expect more and more things to appear on SourceForge and to start see pages redirecting you from Google code to the new stuff on SourceForge.

Needless to say actual development work and bug fixing will be minimal until after the move is complete as there is only one of me and a part time helper working on the project right now Blink

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.
Eduardo'o
Ministry of War
Amarr Empire
#240 - 2012-02-14 20:52:44 UTC
Dragonaire,

I have been using yaeal for a little while now developing a corporation management tool.

I now want to get my yapeal upto the latest version. I have no idea though how to do the upgrade. I guess replaceing the files is easy, but how does one upgrade an exising yapeal database to a newer version without loosing data?

PS: Yapeal is AWESOME

Eduardo'o