EVE Technology Lab

Kolb - I would assume from the all default_time_zone warnings that you are using Ubuntu and they still haven't figured out like all the other distros how to set the time zone correctly in php.ini . You probably need to fix that yourself and I'll look at adding something into Yapeal to also take care of it for it's needs. I thought I already had it setting it but must not have made it in like I thought.

On the segfault looks like you're simple running out of memory with even less than 1000 row inserts so you might want to make sure your memory_limit is high enough in php.ini. 32MB should be enough but my current setting is 128MB here on my test machine. I've not done any really memory profiling on Yapeal this time but I've used best practices to keep it as small as possible with the limit of course being the size of the Eve API XML itself. One reason I limit individual upserts to under 1001 row is to try keeping memory use lower plus work around some limits with typical MySQL setups.

Just to test it out I even tried setting it to no limit -1 same problem.

Commenting out those lines did stop the seg-fault; would you like a list of my modules to help narrow down the source of the problem for future reference?

Edit: Forget it, somehow my host had still php 5.3.x activated...

CCP is adding new char and corp Blueprints APIs which can be seen on test server now. I've adding them to Yapeal but set isActive=0 in utilEveApi for now until they get deployed to Tranquility.

Legedric Striker - Great to see you get it out for people to take a look and glad to hear you found Yapeal useful. I've took a quick look but not had time to actually try it but I'll try to get around to that soon.

Is there a way to update the DB after i downloaded the new updates via git? Without dropping existing tables?

Well, I think this may be the problem for me, my hoster is running PHP 5.3 usually on this server and to activate PHP 5.5 I have to use a CGI handler in my htaccess file.

As a result, PHP is not being run as www-data but as the main FTP user so chown is set to this user for all folders recursively.

Unfortunately this is something I cannot change...

But as PHP is running using this main FTP user and all the cache XML files belong to this user it shouldn't be problem at all. the problem seems to be that the newly generated cache files have CHMOD 644 and so they cannot be change by yapeal afterwards.

It is very strange however that yapeal can create new files without any problems but it seems like it cannot update the content of those new fiules once they are created. So the "last changed date" is changing but not the content itself as you can see above.

Other parts of my project (log4net for example) have no trouble manipulating files.

Perhaps I will try to write a second cronjob that iterates through all cache files having the wrong CHMOD and set it to 775 via PHP and that will doe the trick

Legedric Striker - Could be that some of my changes to have Yapeal do a better job of reusing the FS cache aren't right yet I'll look at it again.

Edit: Make sure you are using up to date version after I fixed issue #19. I improved the handling of cached XML files. The cache files being 0644 is fine actually since that means the owner has read/write access and as long as the user Yapeal is run as is the same user it should be fine.

Just something else I picked up on when re-reading your post is your talking about CGI and Yapeal.

Just to make it clear Yapeal should be ran with the CLI version of PHP not the one made to work with the web server. Not that anything in Yapeal really requires this but because of how CGI is setup on most shared hosts you can run into problems. Not sure that is what's going on here but I'd find the path to use the CLI version of PHP and see if it doesn't help. In most cases its something like /usr/bin/php but could be different if they are installing multiple version to allow for 5.3 and 5.4 etc.

Also it's better not to have Yapeal within you web server path as then there is always a chance that someone can try crafting a URL and access it directly as a web page. Main possible problem here is they would probably get to see the log messages that normal end up in the log file or are visible when running Yapeal directly from the command line. Shouldn't really see anything like passwords etc but still best that it's not an option.

You might also set your logging level to 250(NOTICE) and see if some more logging messages show up for you. If you want you can go all the way down to 100(Debug) here during testing and it might pick something extra up as well but generally not needed.

\lib\Xml\FileCacheRetriever::isExpired() is where it decided if the XML is expired or not. Welcome to look at it and see if you see any fault in the logic but I'm not finding any.

Ok so I just did some debugging for the isExpired function in FileCacheRetriever.php

It is ALWAYS returning false for me as my cache files are (as said) several hours old.
The reason is line 278:



$current (is taken from the cached XML file - which is way too old) is like shown above from yesterday in some cases and so it is always sure that "whatever time from yesterday +300 seconds" is lower than "now".

So I changed it to:



In addition I also changed the following lines:


into:


I am somewhat confused as the function is called isExpired I expect it to be TRUE if the cachedUntil time is in the past, so cachedUntil <= now and not like its in the original lines... same goes for the 5 minutes minimum cache which is also obviously turned around.

I somwhere saw a function called isNOTexpired which is operating the other way round (not for cache files though) so I think you may have mixed up the purpose of the isExpired function here with its actual code

After I uploaded the changes to my webspace the cached files are being updated again and the API is working as intended again!

Cheers and it's time for some sleep now!

Great, I just updated to the most recent build of your master branch