These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

 
  • Topic is locked indefinitely.
 

Yapeal PHP API library (2.0)

Author
Dragonaire
Here there be Dragons
#181 - 2014-11-19 21:14:07 UTC  |  Edited by: Dragonaire
Battle On - More like that feature of using config outside of Yapeal directory was never implemented but I'll look at fixing that so it will look up and over to config directory for a config file as well like it says in the CONFIG.md.

https://github.com/Dragonrun1/yapeal/issues/83
Pushed an update that should work now if you can give it a try Battle On and let me know if it seems to work for you.

Just to make it clear how this works if you have both
myApp/vendor/yapeal/config/yapeal.yaml
and
myApp/config/yapeal.yaml
files then any settings in the later file overwrites any matching setting from the first file and they both overwrite any defaults set from
Wiring::wireDefaults()
. So for example if you have the following:

---
# myApp/vendor/yapeal/config/yapeal.yaml
Yapeal:
    Database:
        database: yapeal1
        userName: IAmADummy
...


and


---
# myApp/config/yapeal.yaml
Yapeal:
    Database:
        database: yapeal2
...


then the merged settings would be:


---
# myApp/vendor/yapeal/config/yapeal.yaml
Yapeal:
    Database:
        database: yapeal2
        userName: IAmADummy
...


and all other settings would use their defaults.

Sorry I didn't have it implemented before but in the thousand other things that went into re-writing Yapeal it just fell through the cracks Sad

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.

Battle On
GoonWaffe
Goonswarm Federation
#182 - 2014-11-20 09:40:40 UTC  |  Edited by: Battle On
Dragonaire wrote:
https://github.com/Dragonrun1/yapeal/issues/83
Pushed an update that should work now if you can give it a try Battle On and let me know if it seems to work for you.


Sweet, i'll test it out asap!

Dragonaire wrote:

Sorry I didn't have it implemented before but in the thousand other things that went into re-writing Yapeal it just fell through the cracks Sad


No worries man, its just a convenience for me because in my production envirenment im doing a composer update each time i push to the server, therefor yapeal.yaml in the vendor directory is being reset, so i had to manually override it.



Oke so i've tested it and it doesnt work, and i think i know why. In your previous post you wrote this:


---
# myApp/vendor/yapeal/config/yapeal.yaml
Yapeal:
    Database:
        database: yapeal1
        userName: IAmADummy
...


however, if i fetch the package with composer, the path is "myApp/vendor/yapeal/yapeal/config/yapeal.yaml".

After further testing i found out that in Wiring.php the $dic['Yapeal.vendorParentDir'] is always not defined. This is most likely the reason why it doesnt work. However, i couldnt find the place where this cariable is actually set?

I hope this helps,
Battle On

Jita Exiles Productions is RECRUITING

Join -JXT- Public for more information

Dragonaire
Here there be Dragons
#183 - 2014-11-20 18:21:22 UTC
Battle On - it's set in Yapeal::wire() if it notices any 'vendor/' directory in the path.

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.

Battle On
GoonWaffe
Goonswarm Federation
#184 - 2014-11-20 18:48:20 UTC
Dragonaire wrote:
Battle On - it's set in Yapeal::wire() if it notices any 'vendor/' directory in the path.


thats probably the reason why its not set then. Yapeal::wire() is not called when i (for example) call "php bin/yc D:I". i tried echoing or var_dumping it, but it doesnt get into printed. However, like i said in my previous post, when i try dumping it in the Wiring class at the wireConfiguration() (where you set the path to the overriding config/yapeal.yaml file), i get the output that its undefined.

Jita Exiles Productions is RECRUITING

Join -JXT- Public for more information

Dragonaire
Here there be Dragons
#185 - 2014-11-20 21:52:00 UTC
Added setting it in the 'yc' stuff as well. Note you will only see
$dic['Yapeal.vendorParentDir']
if Yapeal has a parent vendor/ directory in its path. It should set it if you are doing something like

$yap = new Yapeal($dic);
if (isset($dic['Yapeal.vendorParentDir'])){
    print 'Parent = ' . $dic['Yapeal.vendorParentDir'] . PHP_EOL;
} else {
    print 'Parent not set!' . PHP_EOL;
}

You would need to use isset etc as $dic is not just a normal array but an instance of Pimple which in many ways can work as an array but doesn't always work well with things like var_dump().

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.

Norian Lonark
Center for Advanced Studies
Gallente Federation
#186 - 2014-11-26 21:50:07 UTC
Hi all,

just a quick question Big smile

I think I know the answer but just want to be sure. When yapeal runs and makes entries in the database tables are these entries permanent. e.g. With the wallet journal only lasting for a month if yapeal runs and makes entries in the database tables will these just keep being added too - so I can in effect go back in time longer than a month, or are they over written / deleted when the time expires?

Many thanks,

Nori.

Start wide, expand further, and never look back

Dragonaire
Here there be Dragons
#187 - 2014-11-26 22:13:21 UTC
Things like WalletJornal do build history but others like AssetList, or AccountBalance do NOT they are deleted and replaced on a per char/corp basis.

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.

Norian Lonark
Center for Advanced Studies
Gallente Federation
#188 - 2014-11-26 23:59:48 UTC
Thanks you Smile

Start wide, expand further, and never look back

Dragonaire
Here there be Dragons
#189 - 2014-11-27 00:58:14 UTC
Hi all I know it's been a while really since I added any new APIs so I decided to look at one that has been outstanding for a good long while and see about adding it. I decided MailBodies shouldn't be to hard to do so I did it. It did turn out a little more complex than I had expected because I forgot i was one of the irregular APIs that use rows with attributes and the element value but since it was only one of each I took a short cut there on how to deal with it.

A heads up for anyone using NOTICE level logging in production (btw WARNING is strongly recommended) you will start seeing some log lines about 'CachedUntil is excessively long was given ...' There isn't anything really wrong here just that Yapeal is noticing the 10 years cachedUntil time the API returns. I'll look into it some more and see if I can quiet it down some way if I get a chance.

In other news do to some RL issues I'm going to be having to move sometime in the next week to a month and not sure what kind of Internet connection I'll have or time to do anything so hopefully people that are already using Yapeal can step up and answer any questions on the forums here and I'll see if I can get someone to help with any bug fixing that might come up. One person does have access to Github and has some knowledge and experience with the code so he can probably deal with most things that might come up. He was even asking about if I had anything I want him to work on the other day in Yapeal since he has some time off work so maybe he'll get a couple of things done while I'm dealing with RL for a while Blink.

As always thanks for using Yapeal, Dragonaire

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.

Dragonaire
Here there be Dragons
#190 - 2014-12-02 19:37:50 UTC
Thanks to Melikoth for spotting this: Issue 84

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.

Jahn Ntago
Ntago Family Enterprises
#191 - 2014-12-07 00:58:44 UTC
Running 2.0.0 I am finding that all log levels are present in the log file despite having configured Error.threshold = 500 and Log.threshold = 300. The log file seems to top out just north of 100M and the yapeal process stops (after 3.5 hours).

It would appear that your monolog FingersCrossedHandler is not active and all levels from 100+ in your 2x new StreamHandlers calls are being recorded.

I have temporarily changed Wiring.php to set the desired levels.

Is my diagnosis correct and can we have a fix please?
Dragonaire
Here there be Dragons
#192 - 2014-12-07 03:28:47 UTC
Jahn Ntago - Think you are confused about how it works which I was at first too Blink The threshold keeps it from logging anything until the log message meets the threshold then it will log the prior 'bufferSize' worth of messages. It then continues to log all message regardless if log level until the end of the run. So with your settings on the first WARNING (300) level log message it'll continue to log from there plus a few before so you know what lead up to it. The question I would have is what is happening to cause Yapeal to log something with that high a level message?

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.

Jahn Ntago
Ntago Family Enterprises
#193 - 2014-12-09 09:19:06 UTC  |  Edited by: Jahn Ntago
In that case I may have to continue with my customisation. I have a 5+ hr run of data collection. In the first few minutes I get one of:

yapeal.WARNING: Failed to upsert data from Eve API account/APIKeyInfo for due to a "Syntax error or access violation: 1064"

yapeal.WARNING: Failed to upsert data from Eve API corp/CorporationSheet due to the schema not supporting REALLY big corporations (fixed by me) : "Numeric value out of range: 1264"

yapeal.WARNING: Failed to upsert data from Eve API char/ContactList due to weird formatting of names: "Invalid datetime format: 1366"

That FingersCrossed then logs EVERYTHING after one of these is not useful to me - 100MB of logs and eventual failure of the run - I guess due to a log size issue.

Let me say: your code is brilliant and does amazing things for me. Thank you.
Dragonaire
Here there be Dragons
#194 - 2014-12-09 20:47:13 UTC
Jahn Ntago - First glad you find Yapeal useful but just reading through your post I'm a bit confused by some of the partial log messages you gave.

Quote:
yapeal.WARNING: Failed to upsert data from Eve API char/ContactList due to weird formatting of names: "Invalid datetime format: 1366"
Considering there are no date or time columns in that API table I really don't see how you could get that error.

Quote:
yapeal.WARNING: Failed to upsert data from Eve API account/APIKeyInfo for due to a "Syntax error or access violation: 1064"
That would suggest some kind of SQL syntax error which though possible is very unlikely except if it's doing it for all keys but I got the impression from your post that wasn't the case.

Quote:
yapeal.WARNING: Failed to upsert data from Eve API corp/CorporationSheet due to the schema not supporting REALLY big corporations (fixed by me) : "Numeric value out of range: 1264"
Because of the mention of REALLY big corps I'm assuming you mean ones with lots of members. I do notice some possible issues with the 'memberCount' and 'memberLimit' that are probably do to some copying and pasting and not fully editing them afterwards which can probably be fixed but like the others since I don't have the complete log messages I can only guess that the changes I might make would fix it.

May I suggest you either give more complete info and log messages here or even better create each of these as an issue on GitHub so someone might have a chance to fix them for everyone instead of you having to 'customizing' Yapeal and in effect be maintaining your own private branch you would need to be constantly merge the main project into all the time as other issues are fixed there? If for some reason you feel some of the data within the log messages is to sensitive to post here or in a issue you can of course either mask with dummy data or directly E-mail it to me through my GitHub listed account and I'll take a look at it there.

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.

Legedric Striker
State War Academy
Caldari State
#195 - 2014-12-10 11:13:55 UTC  |  Edited by: Legedric Striker
Well, after Rhea Yapeal stopped working so I updated the library again (last update was made in November) and I get this:

Quote:
Fatal error: Trait 'FilePathNormalizer\FilePathNormalizerTrait' not found in /www/htdocs/w0123f51/_libext/vendor/yapeal/yapeal/lib/Yapeal.php on line 57


I searched your repository on github for "FilePathNormalizerTrait" and could only find "use" of it but no declaration at all?

/Edit: According to the master branch on your github, the folder "lib/Filesystem" is gone?
/Edit 2: Great, I just noticed you move the FilePathNormalizer into a seperate repository and as I am not able to use composer to update my project it went just nowhere...

/Edit 3: So I managed to get Yapeal up and running again (more or less a fresh reinstall using composer on my dev machine) and after I truncated the "cachedUntil" times it seems to be pulling fresh data again *crossing fingers*

EVE-Skillplan.net - Plan your pilot skill training online on PC, Mac, tablet or smartphone!

Dragonaire
Here there be Dragons
#196 - 2014-12-10 23:34:38 UTC
Push a couple fixes for some of the issues Jahn Ntago noticed.
ContactList does NOT allow UTF-8 contactNames
corpCorporationSheet does NOT allow for large membership.

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.

Legedric Striker
State War Academy
Caldari State
#197 - 2014-12-11 06:43:39 UTC
Today I came across a strange issue.

I noticed that for one of my own chars my site wasn't updating the skill queue so I checked the utilCachedUntil table to find out what's going on.

Both times, for SkillInTraining and SkillQueue showed expiry dates roughly 36 hours into the future. So I checked the dates at 06:30 and they listed times for the next day at 12:46 or so. For short or m-short cache style timers this seemed way off to me.

So I deleted the datasets for my character to see what comes in with a fresh pull:
SkillInTraining 2014-12-12 06:34:33 xxxxxxxx Char
SkillQueue 2014-12-11 06:46:10 xxxxxxxx Char

The timing for SkillQueue seems to be right, for SkillInTraining however, it seems to be way off as the xml I am getting is telling me "cachedUntil" is "2014-12-11 06:42:41" for SkillInTraining.
The time for SkillQueue after the fresh API pull was also off, but only by some minutes (<10 minutes) but it was still not matching the time I got by manually accessing my characters XML using just the URL https://api.eveonline.com/char/SkillInTraining.xml.aspx

Am I missing something here or may this be an issue?

EVE-Skillplan.net - Plan your pilot skill training online on PC, Mac, tablet or smartphone!

Tiberius Zol
Moira.
#198 - 2014-12-11 10:42:31 UTC
any solution for the "FilePathNormalizerTrait" problem without fresh reinstall?

Mr. Tibbers on twitter: @Mr_Tibbers

Mr. Tibbers Blog: www.eve-versum.de

Legedric Striker
State War Academy
Caldari State
#199 - 2014-12-11 11:05:04 UTC
Well you can still try to use "composer update -o" to update yapeal and grab the new project that way if you use composer at all.

EVE-Skillplan.net - Plan your pilot skill training online on PC, Mac, tablet or smartphone!

Tiberius Zol
Moira.
#200 - 2014-12-11 16:09:39 UTC  |  Edited by: Tiberius Zol
after workin with composer again, this is fixed for me. thanks man.

Mr. Tibbers on twitter: @Mr_Tibbers

Mr. Tibbers Blog: www.eve-versum.de