These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

 
  • Topic is locked indefinitely.
 

Yapeal PHP API library (version 1.0.5)

First post
Author
Dragonaire
Here there be Dragons
#101 - 2011-11-02 04:34:56 UTC
Are there ways to get Yapeal to violate the cachedUntil time? Yes, simple delete the cachedUntil time from utilCachedUntil table and Yapeal will at least try to refresh the data from the cache. If you also clear the XML from the file and/or database cache it will even try to get it from the API server again and depending on the API you either get an API error with a new cachedUntil time that will be even farther into the future than if you had left it alone or the same data you already had once again with a new cachedUntil time that will also be farther into the future than if you'd left it alone.

Does it make sense to do either of these things? Not really.

Now if you are just wanting to have Yapeal grab the data for say a new char when they are signing up for an account you can add the key info to Yapeal and directly call it and it will get the data but it will do it along with trying to get anything else with an expired cachedUntil time. It shouldn't even cause any problems with your regular crontab job as Yapeal does use a very simple form of API locking so multiple copies don't try to both work on the same thing at the same time but since the order they will try to get the APIs is random there's no way to predict which of them actually ends up doing the work. I am looking at adding the ability in the future to be able to call APIs directly outside of the normal crontab way Yapeal does things but there are some other things that need to change before that is possible.

The best way to make sure everything is as up to date as possible is to add the accounts to it and run the crontab every minute. I've had several developer trying to run it less frequently and than complain it's overloading their systems. By design Yapeal tries to do as much work as it can every time it runs so the less often it is ran the more it has to do each time but if you let it work as often as possible it actually works to spread out globs of API calls by delaying some of them a couple minutes so only a few cachedUntil times expire each minute. None of them will be delayed more than 5 minutes because of hard limiting plus its heavily weighted to do them on time so the upper limit is rarely seen.

Hopefully that answered you question(s) and let's you understand how things work a little better.

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.

Sable Blitzmann
24th Imperial Crusade
Amarr Empire
#102 - 2011-11-02 04:56:37 UTC
Thanks, yeah, that cleared it up a bit.

I am having a bit of trouble with this wiki page tho:
http://code.google.com/p/yapeal/wiki/UsingClassUtilClasses

There is no common_backend.php that I can see. There's a common_paths.php, but that doesn't include an autoloader and manually including the file doesn't work because it can't autoload ALimitedObject. =/
Dragonaire
Here there be Dragons
#103 - 2011-11-02 07:26:03 UTC
Wiki is outdated. What use to be in common_backend.php is now either moved to the classes the code should have been in or was moved to yapeal.php. Read through the CHANGELOG to see why etc. Lines 78, 79 are the ones for the autoloader in Yapeal.php but you'll want to look at some of the code before them to setup some of the environment you'll need.

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.

Sable Blitzmann
24th Imperial Crusade
Amarr Empire
#104 - 2011-11-02 20:24:08 UTC  |  Edited by: Sable Blitzmann
Still can't get it to work.

I included the common paths and autoloader:

require_once 'yapeal/inc/common_paths.php';
require_once YAPEAL_CLASS . 'YapealAutoLoad.php';
YapealAutoLoad::activateAutoLoad();

And it returns as:

Notice: Use of undefined constant YAPEAL_TABLE_PREFIX - assumed 'YAPEAL_TABLE_PREFIX' in /path/yapeal/class/util/RegisteredCharacter.php on line 96
Notice: Use of undefined constant YAPEAL_DSN - assumed 'YAPEAL_DSN' in /path/yapeal/class/util/RegisteredCharacter.php on line 99
Missing file: /path/yapeal/ext/ADOdb/drivers/adodb-yapeal_dsn.inc.php

Fatal error: Uncaught exception 'RuntimeException' with message 'Failed to get database connection in RegisteredCharacter' in /path/yapeal/class/util/RegisteredCharacter.php:103
Stack trace:
#0 /path/index.php(17): RegisteredCharacter->__construct('Sable Blitzmann')
#1 {main} thrown in /path/yapeal/class/util/RegisteredCharacter.php on line 103

Edit: I had to go a bit further and add this from yapeal.php:
require_once YAPEAL_INC . 'getSettingsFromIniFile.php';
if (!empty($options['config'])) {
$iniVars = getSettingsFromIniFile($options['config']);
} else {
$iniVars = getSettingsFromIniFile();
};
YapealDBConnection::setDatabaseSectionConstants($iniVars['Database']);

Still trying it out tho. =)
Dragonaire
Here there be Dragons
#105 - 2011-11-02 21:41:23 UTC
Yeah it has to read the config file or stuff doesn't work Blink

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.

Norian Lonark
Center for Advanced Studies
Gallente Federation
#106 - 2011-11-03 13:20:13 UTC
Hi,

is it possible to add additional fields to the yapeal tables.

For example with industry jobs it would be nice if I could add a couple of fields such as booked or client .

Can you do this or is there any configuration changes you would need to do, in my test I tried adding a column and when I ran yapeal.php errors came up and the data was not updated.

I was expecting this but wondered if there is any workaround?

Many thanks again :)

Nori.

Start wide, expand further, and never look back

Dragonaire
Here there be Dragons
#107 - 2011-11-03 14:14:21 UTC
@Norian Lonark - You could do what you are talking about but you would have to 'fix' the tables every time you updated Yapeal and it's database. The normal way to do what you want would be to add another table with the primary key from Yapeal's table as foreign key to your table with the other columns you need so they are linked. So for IndustryJobs you would have ownerID and jobID as the foreign key or even as you're primary key depending on what else you are adding then use a simple join in your queries.

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.

Dragonaire
Here there be Dragons
#108 - 2011-11-04 04:44:23 UTC
New version 11.307.2138 out. Made some changes to allow Yapeal to better work within MySQL server's max_packet_size limit. If you continue to have problems with 'Packet to big' try turning off xml caching in Yapeal.

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.

Dragonaire
Here there be Dragons
#109 - 2011-11-04 16:30:02 UTC
New version 11.308.0915 Since I don't have enough data to test some changes I'm trying with walking in the Wallet* API I'm pushing it out for anyone else to try. If you've been having problems with walking working give it a try and let me know if it's better.

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.

PsyKzz
Republic Military School
Minmatar Republic
#110 - 2011-11-04 22:56:08 UTC
Dragonaire wrote:
Yeah it has to read the config file or stuff doesn't work Blink


Why is it now so complicated when before we could just implement common_backend.php ?

Meh.

Dragonaire
Here there be Dragons
#111 - 2011-11-05 03:49:50 UTC
I understand it seems like a bit more work now but it has actually made it more useful because if you don't need some of the parts now you just don't call the function/methods or if you want to override something there's something with a name on it that helps you find what you need. In time it will be simplified once again but Yapeal needed to go through a transition here to get from the procedural code that was used a lot in this area to something more OOP. It has also let me reuse a lot of the same code for the scripts in install/ used to config Yapeal during installation too. Just think you only have to put your settings into config/yapeal.ini once and then can do all the setup and day to day run of it without reentering any of it again Blink

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.

Dragonaire
Here there be Dragons
#112 - 2011-11-07 04:15:40 UTC  |  Edited by: Dragonaire
Hi all got some more changes for everyone to try out this time I've overhauled all of the error and exception handling code. We can thank (blame Big smile) Zaepho for the idea. Back in post 64 he talking about using log4php and I decided to check it out and liked what I saw. Then the fun part started, converting Yapeal to using it. As anyone that's done it knows making changes to your error and exception handling is often difficult since when things go wrong it's hard to figure it out since often you get nothing to look at. The changes actually ended up going smoother than I thought they might except for some problems during the final merging. I've done some testing but since Yapeal generally runs nearly error free for me I did most of the testing by introducing temp fake errors so there still might be a few problems hiding in the changes. It would probably be a good idea to do some testing of your own before going live but it should work without problems.

Log4php has a lot of options for where, how, and what is logged. Yapeal uses only a few of the addition features available but it does mean there have been more than a few changes to what you'll see so here a quick list of some things you may notice.


  • Instead of just yapeal_error.log, and yapeal_warning.logs in log/ you will also find yapeal_info.log (replaces yapeal_notice.log), yapeal_debug.log, yapeal_fatal.log, and yapeal_trace.log. Not all of them are used yet in Yapeal but probably will be in the future.
  • The logs are limited to 100KB in size and will keep one addition old log backup for each of them. As anyone that has seen Yapeal when the API server is having problems knows the logs can fill up fast so this should at least keep them from growing totally out of control. The days of 1.5GB log files per day are over Smile
  • Log entries format has some changes do to update. I tried to keep them as much like before but some things ended up having to be in a different order which shouldn't really matter.
  • Several of the settings in the [Logging] section of config/yapeal.ini have been dropped and a couple others (log_level, log_locale) are now unused and reserved in case they are needed in the future.
  • A new setting has been added to config/yapeal.ini called log_config. It can be used to point to a custom configuration file if needed just make sure you obey warnings found in config/logger.xml.
  • There is also a new command line option -l, --log-config that can be use with yapeal.php.
  • Yapeal will also use the environment variable YAPEAL_LOGGER which can be used to set the configuration file location. The priorities are from high to low the command line, yapeal's ini file, YAPEAL_LOGGER, and finally hardwired default that is set to config/logger.xml.


There are many other changes but those are the most visible ones.

Expect some additional changes and tweaks as Yapeal is adjusted to take advantage of having additional logging levels to work with. Suggestions are welcome as always.

version 11.310.2046

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.

Zaepho
Goosewarms
Tactical Hunters and Gatherers
#113 - 2011-11-07 19:44:31 UTC
Dragonaire wrote:
Hi all got some more changes for everyone to try out this time I've overhauled all of the error and exception handling code. We can thank (blame Big smile) Zaepho for the idea. Back in post 64 he talking about using log4php and I decided to check it out and liked what I saw.


I'm absolutely ecstatic about this. It'll make emails from cron useful to me again! :) Have you seen any issues when using STDOUT/STDERR for logging? I would expect not, but want to be completely clear.

Thanks for checking log4php out. I believe this will be a very valuable change that will make your and your user's lives easier in the long term.

Thanks!
Sable Blitzmann
24th Imperial Crusade
Amarr Empire
#114 - 2011-11-08 03:38:53 UTC
Testing out the latest version. Like the new logging feature, not sure if journal walking is working correctly...

It only captures 250 rows, and it doesn't even seem to be the latest 250 journal logs (last date is 7 days ago)
Dragonaire
Here there be Dragons
#115 - 2011-11-08 04:52:15 UTC  |  Edited by: Dragonaire
Yeah when I put out the changes for logging it reverted the changes for the WalletsOops I'll have that fixed tonight. I've got to remember to pull, merge, push not the other way around P

Ok after a few more tries I got the other stuff logger stuff finish for now anyway and got the Wallet stuff back in as well.

version 11.311.2210

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.

Sable Blitzmann
24th Imperial Crusade
Amarr Empire
#116 - 2011-11-08 16:20:21 UTC
Nope still not working, unless I'm misunderstanding what is supposed to be happening.

I'm expecting Yapeal to walk backwards in the journal and collect all wallet entries that it can until it can't any more. For me, this would be thousands of records. However, it only returns hundreds.
Dragonaire
Here there be Dragons
#117 - 2011-11-08 16:29:12 UTC
What it should be doing is get one group with walking forward (no fromID) then use the oldest record from that one to walk back as far as it can which should be a month if CCP hasn't got that bugged.

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.

Sable Blitzmann
24th Imperial Crusade
Amarr Empire
#118 - 2011-11-08 16:59:58 UTC
So you've got it set to go back only a month, correct? Or is that a limit CCP imposes?

Right now, it's gathered data from 2011-10-19 and onwards... is there any way to get historical entries (as far back as the API will go)?
Dragonaire
Here there be Dragons
#119 - 2011-11-08 17:04:35 UTC  |  Edited by: Dragonaire
Limit is from CCP. They only allow the last month. Only way to have complete history since you started playing would be you started collecting it since then Blink

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.

Dragonaire
Here there be Dragons
#120 - 2011-11-10 17:48:02 UTC
So development and response times on bug reports maybe a little slower for the next week or two as I'm planning on spending most of my free time playing The Elder Scrolls V:Skyrim Smile I've already get it preloaded and I'm just waiting for them to let me actually start playing tonight after work. I will try to check every day like usual before heading to work but I might forget too Blink

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.