These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

 
  • Topic is locked indefinitely.
 

Yapeal PHP API library (2.0)

Author
Dragonaire
Here there be Dragons
#121 - 2014-08-31 16:05:04 UTC
They have totally changed all the Industry stuff include what some of the column values mean so chances are you can't import them but the good news is the new IndustryHistory API will let you get the older jobs with the data updated by CCP so if you really need it as far back as they allow you can get it. You are really going to be limited in this case by what CCP has done.

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.

Dragonaire
Here there be Dragons
#122 - 2014-09-04 02:13:28 UTC
Okay got some good news that should make many people happy, the char/AssetList and corp/AssetList APIs are now done and active.Big smile Has been tested here on both char and Crop side and seems to be working right but only got it work about an hour ago so might still have a minor bug or two but figured everybody had been waiting so long for it I'd put it out as soon as I could. No DB changes so should be able to grab update and let Yapeal do it's thing.

Enjoy, Dragonaire

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.

AmsTaFF Kreon
Pandemic Horde Inc.
Pandemic Horde
#123 - 2014-09-04 09:51:11 UTC
hi, how i can understand, yapeal give interface to gather data from api, but not to use it from database, yes? If i'm wrong, can you show me example? Thank you.

_______
Sorry for my English
Dragonaire
Here there be Dragons
#124 - 2014-09-04 17:29:12 UTC
AmsTaFF Kreon - I think I'm understanding your question is does Yapeal just work with the Eve API servers to pull the data and put it into a database. Yes that's what it does. It does not have anything to do with the static data dump or SDD there are other tools for that. Most applications will need both to do the, as I put it, 'numbers to names part'.

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.

Dragonaire
Here there be Dragons
#125 - 2014-09-04 19:56:35 UTC
Okay some updates have been pushed to fixes some issues I came across. First like I figured there was a couple things incomplete on AssetList. It wasn't deleting the old orphaned records. Couple of the other newer APIs had to same issue and fixed them as well. I'll try to go through them all and make sure none of the other are doing the same thing.

For those of you that haven't paid attention Yapeal for a little while now is adding into the cached XML file what is called a 'processing instruction' generated by Yapeal. Think of it basically as a comment for an application in XML. Yapeal doesn't really use it in any way at this time but it can be a useful tool for me during troubleshooting. Current it's writing the API parameters being sent to the server with the vCode cut off so it's safe to just cut and paste to places if I should ask for it. Anyway to make a long story short it let me trace down a long standing thing I've been noticing that even with just one active corp key during testing I'd often end up with more than a single IndustryJobs cache file for example after running Yapeal manually a couple times. This probably wasn't noticeable if you have several keys but is with single key testing. Before I couldn't figure out what was causing it but now I know it's because left over parameters like accountKey from accountBalance or one of the other APIs that need extra parameters was not being reset between the different API calls. It came down to one miss placed line in one of the XML network classes in the end.

Anyway I've pushed the update to fix it but I highly recommend that everyone clear out your cache/*/*.xml files to get rid of the bad files and let Yapeal refresh the cache as needed. This could save you a lot of disk space as well since the are cases where you'd end up with as many as eight files instead of one per API/key combo after a few runs.

Next I'd like a little more input on which API everybody needs next. I'll go back through thread as well to look at the votes people already made but new people can have a chance to vote now or you can update your vote if you changed your mind.

As always thank you for using Yapeal, Dragonaire.

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.

Tiberius Zol
Moira.
#126 - 2014-09-05 08:20:29 UTC  |  Edited by: Tiberius Zol
Hey, found another Problem:

Quote:

Failed to upsert data from Eve API eve/CharacterInfo for 93133983 {"exception":"[object] (PDOException: SQLSTATE[HY000]: General error: 1366 Incorrect string value: '\\xE2\\x98\\xBC 10...' for column 'shipName' at row 1 at /htdocs/eve/yapealneu/lib/Database/ValuesDatabasePreserver.php:217)"} []


Think it is caused by the special character in the shipname (☼ クリタ)

Mr. Tibbers on twitter: @Mr_Tibbers

Mr. Tibbers Blog: www.eve-versum.de

Tiberius Zol
Moira.
#127 - 2014-09-05 09:30:30 UTC  |  Edited by: Tiberius Zol
And another problem: it seems that yapeal doesn't update my marketorders anymore. in the old yapeal table all seem to be correct (actual data from today sell orders) but in the new yapeal table the last sell order is a few days old. i have deleted all the cache stuff and run yapeal manually but nothing changed. found nothing special in the logfile, besides there seem to be only one entry for 61 orders?



edit: checked again and don't know why but seem to work now.

Mr. Tibbers on twitter: @Mr_Tibbers

Mr. Tibbers Blog: www.eve-versum.de

Dragonaire
Here there be Dragons
#128 - 2014-09-05 17:10:01 UTC  |  Edited by: Dragonaire
Looks like another column that needs to be UTF-8 I'll look at changing it.

https://github.com/Dragonrun1/yapeal/issues/46 fixed.

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.

Dragonaire
Here there be Dragons
#129 - 2014-09-07 01:01:43 UTC
Fixed some things I ran across with AccountBalance, and the Wallet stuff. Came to light while I was looking at some corp stuff. One small DB table update for the corp AccountBalance table to allow for the new mercenary stuff. Fix also should keep corp/AccountBalance from making a bunch of unneeded cache files so like before worth deleting all the cache/corp/AccountBalance*.xml files and Yapeal will recreate the ones it needs.

I haven't decided for sure which API to work on next but leaning towards corporationSheet to match up with characterSheet. Looking at making it useable with and without API key info so you can use it in your apps to grab public corp info with just the corporationID but not sure if I can make it work yet so stay tuned on that.

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.

Tiberius Zol
Moira.
#130 - 2014-09-07 15:37:41 UTC
Installed the updates. Looking good so far. Good work again. :)

Mr. Tibbers on twitter: @Mr_Tibbers

Mr. Tibbers Blog: www.eve-versum.de

Tesla Ming Chent-Shi
Anoikis Exploration and Prospecting
#131 - 2014-09-08 23:43:17 UTC
Hi there -

I was wondering if there is a "getting started" tutorial that is up to date? The one I am using now is outdated and causes "not found" errors to be thrown.
Dragonaire
Here there be Dragons
#132 - 2014-09-09 00:55:53 UTC
Tesla Ming Chent-Shi - After the README.md, INSTALL.md, and CONFIG.md there really isn't much I've updated a couple of the pages in the wiki as well but since the database tables are the Yapeal API really it just a matter of looking for the info you need in them. If you are having a problem with something in particular asking your questions here in the forums is probably the best as either I or someone else should answer them and others can find them later as well when they have the same questions.

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.

Tesla Ming Chent-Shi
Anoikis Exploration and Prospecting
#133 - 2014-09-09 01:03:53 UTC  |  Edited by: Tesla Ming Chent-Shi
Thanks for the reply.

Let me edit my response:

I've successfully installed everything according to the install and config files.

From there though I'm in the dark what should be done.

Must I:


  • Execute some file on a hourly/daily chron?
  • Include some file to give me access to methods?
  • Do something I haven't noticed...


I had installed it in the root of my project - but should this be installed in PHP itself?

So really, after installation how is it run?

Thanks for your time, I should be able to hack the rest myself after this :D
Tesla Ming Chent-Shi
Anoikis Exploration and Prospecting
#134 - 2014-09-09 02:10:08 UTC
Ok, here's what I've been able to find out:

bin/php -f yapeal.php

... runs an update. Apparently if I have character API's set up in the database, running this daily or hourly should update all the characters who's info has "expired"?

I'll continue hacking away till someone sets me straight. What?
Dragonaire
Here there be Dragons
#135 - 2014-09-09 02:22:23 UTC
Base way as an application programmer is to look at it is you have a database that just magically always has the most up to date data possible. You are right in that to make that happen you do need to have Yapeal running in a crontab which is outlined in one of the Wiki pages. I don't have direct instructions for Windows because you usually do it through the GUI which is much harder to explain without a lot of screen shots etc. Also each version of Windows is a little different and I'm not going to take the time to try figuring it out with each of them. Most people will use Yapeal on a Linux server anyway so I don't expect that to be much of a problem since they have crontab.

As to how often Yapeal should run in the background I'm always very clear about that. Yapeal should run every single minute to insure the lowest server load on both ends and the most up to date data. Yapeal internally manages which APIs are ready for refreshing from the Eve API servers and trying to second guess it on that just causes you more load and problems NOT less like everybody seems to think.

How to use Yapeal for things like site signup etc does take a little more work but not much as I made a utility class you can use to add the API key info into the right table which you can find in bin/UtilRegisterKey.php. After that or possibly before you can also use any of the API classes you'll find in lib/Database/*/ directories directly just like the main Yapeal class does or even just extend from it. You'll find it in lib/Yapeal.php. You can just ignore the AutoMagic() method or override it to do what you need. The code is the best documents you will ever find in any project on how something works so don't be afraid to poke around in it a little. Blink

Just an additional hint for things like signups etc is to use the oneShot() method in the lib/Database/Account/APIKeyInfo class. It'll put the EVE API result info into the account database tables where you can check the mask and get the character and corp info you might need.

Hope this helps you get going or at least gives you some idea where to find out some more about some of the stuff. Remember the database is the interface everything else is just setup to make it work Big smile

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.

Tesla Ming Chent-Shi
Anoikis Exploration and Prospecting
#136 - 2014-09-09 02:26:07 UTC
Thanks a lot. Your work on this is magnificent and any help for dorks like me is just a bonus Big smile

Great work making a low-maintenance system! For the rest I'll just keep poking around as needed.
Tesla Ming Chent-Shi
Anoikis Exploration and Prospecting
#137 - 2014-09-10 01:51:17 UTC
CCP - we need some code-wrapping in this forum Shocked

I'm using your predefined method to set up initial API info - namely class UtilRegisterKey

This is how I've executed it:

Quote:
include( $PLUGIN_ROOT. 'includes/yapeal/bin/UtilRegisterKey.php' );
include( $PLUGIN_ROOT. 'includes/yapeal/lib/Database/CommonSqlQueries.php' );
include( $PLUGIN_ROOT. 'includes/yapeal/lib/Database/Account/APIKeyInfo.php');

$pdo = new PDO("mysql:host=localhost;dbname=".API_DB_NAME, API_DB_USER, API_DB_PASS);

$csq = new CommonSqlQueries(eve, '');

$urk = new UtilRegisterKey($pdo, $csq);
$urk->setKeyID(123456);
$urk->setVCode('Cx8j1mG6NQP56v3fxRigQW5rhZVsx8j1mG6NQP56v3fxRigQW5rhZVs');
$urk->setIsActive(1);
$urk->setActiveAPIMask(268435455);
$urk->save();


Yet it doesn't go through, and I've found that the below function adds quotation marks in places making the query invalid.

I've posted this question:
http://stackoverflow.com/questions/25756074/php-pdo-syntax-issue#25756099

Your library is expertly created, so I'm assuming I must be doing something wrong. The error is shown in above thread.

I'd use my own query, but I'd rather learn your library

Quote:
/**
* @param string $tableName
* @param array $columnNameList
* @param string $rowCount
*
* @return string
*/
public function getUpsert($tableName, array $columnNameList, $rowCount)
{
$columns = implode('","', $columnNameList);
$rowPrototype =
'(' . implode(',', array_fill(0, count($columnNameList), '?'))
. ')';
$rows = implode(',', array_fill(0, $rowCount, $rowPrototype));
$updates = [];
foreach ($columnNameList as $column) {
$updates[] = '"' . $column . '"=VALUES("' . $column . '")';
}
$updates = implode(',', $updates);
$sql = sprintf(
'INSERT INTO "%1$s"."%2$s%3$s" ("%4$s") VALUES %5$s ON DUPLICATE KEY UPDATE %6$s',
$this->databaseName,
$this->tablePrefix,
$tableName,
$columns,
$rows,
$updates
);
echo $sql;
return $sql;
}


On a side-note - is the verification code just a typical hash? If it is always a typical hash, I could build in hash validation for it in the APP I'm making.
Tesla Ming Chent-Shi
Anoikis Exploration and Prospecting
#138 - 2014-09-10 02:08:43 UTC
I'm not sure if I'm being helpful or not, but this method *seems* to have needed a correction:

http://pastebin.com/V8638K5f

Where utilRegisterKey was changed to utilRegisteredKey

And the quotes were replaced with ` on the getUpsert method, making this:

http://pastebin.com/raw.php?i=gQ4An3B0

Yet I'm not sure if perhaps it was set that way for a reason - I do not want to ruin any interdependency - so let me know if it was wrong to do this. Otherwise it inserts fine now.
Dragonaire
Here there be Dragons
#139 - 2014-09-10 20:08:38 UTC
Actually the quote marks are correct but I use ANSI mode in MySQL for all the SQL in Yapeal. What I probably do need to add into the class is the SQL to set that mode up first in a class where it's more likely to be used by others. I'll try to push out something to do that later today. If you look in lib/Configuration/Wiring::wireDatabase you'll see the exec() commands that is used in Yapeal to set the correct mode up right after the connection is created. You can add those into your code before passing it to utilRegisteredUser.

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.

Dragonaire
Here there be Dragons
#140 - 2014-09-10 22:39:47 UTC
Okay pushed out several bug fixes for things a couple of them include database updates so remember to do a 'php bin\yc D:U' to fit up the DB as well. You can see more about the issues fixed in
https://github.com/Dragonrun1/yapeal/issues/50
https://github.com/Dragonrun1/yapeal/issues/51
https://github.com/Dragonrun1/yapeal/issues/52
https://github.com/Dragonrun1/yapeal/issues/53
https://github.com/Dragonrun1/yapeal/issues/54

Finds camping stations from the inside much easier. Designer of Yapeal for the Eve API. Check out the Yapeal PHP API Library thread.