These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

 
  • Topic is locked indefinitely.
12Next page
 

iveeCore: The PHP engine for industrial activities and CREST library

Author
Aineko Macx
#1 - 2013-11-03 06:41:58 UTC  |  Edited by: Aineko Macx
I present iveeCore, a PHP engine for calculations of EVE Online industrial activities and CREST library.

Purpose and target audience

The goal of this project is to provide its users with a simple but powerful API to get information about industrial activities in EVE Online such as bill of materials, activity cost and profit or skill requirements. By hiding the complexities of EvE's Static Data Export, iveeCore helps developers to quickly prototype scripts or develop complex (web) applications. iveeCore now includes a full-fledged CREST client which further increases its utility.

iveeCore will likely be most useful for developers with at least basic PHP knowledge wanting to create their own industry related tools.

These are a few example questions that can be answered with a few lines of code using iveeCore:

  • "What is the profit of buying this item in Jita and selling in Dodixie?"
  • "What do these items reprocess to? And whats the volume of that and it's sell value in Jita?"
  • "How much does this EFT fit cost? How much is this scanned cargo worth?"
  • "What is the job cost for building this item in this system? And when built in a POS in low sec?"
  • "What is the total bill of materials to build these Jump Freighters and the minimum skills?"
  • "Is it more profitable to build the components myself or buy off the market?"
  • "Whats the total profit for copying this BPC, inventing with it, building from the resulting T2 BPC and selling the result?"
  • "How do different Decryptors affect ISK/hour?"
  • "How do Blueprint ME levels impact capital ship manufacturing profits?"
  • "How much is a month of Unrefined Ferrofluid Reaction worth?"

FEATURES

  • An API that strives to be "good", with high power-to-weight ratio
  • Strong object oriented design and class model for inventory types
  • Classes for representing manufacturing, copying, T2 & T3 invention, research and reaction activities, with recursive component building
  • Market data gathering via CREST with realistic price estimation and profit calculation & DB persistence
  • CREST data fetcher handling system industry indices, market prices and facilities
  • Parsers for EFT-style and EvE XML ship fittings descriptions as well as cargo and ship scanning results
  • Caching support for Memcached or Redis (via PhpRedis)
  • Extensible via configurable subclassing
  • A well documented and mostly PSR compliant codebase

The CREST portion of the engine (formerly the iveeCrest library) can be used in conjunction with iveeCore or independently. Some of its features include:

  • Authenticated CREST based
  • Methods for most endpoints reachable from CREST root are available, plus a few deeper endpoints. Easily expanded to support more (pull requests welcome!)
  • Gathering of multipage responses
  • Supports parallel GET requests with high performance asynchronous processing
  • Multilayer cache design
  • The index-less collections returned by CREST are properly re-indexed by IDs
  • Includes a self-contained web-script to retrieve a refresh token
  • CLI tool for automated CREST updates


Released as open source under the LGPL.

CLASS MODEL
Have a look at iveeCore's class diagram.

USAGE
Here are a few basic examples of how to use iveeCore:

//initialize iveeCore. Adapt path as required.
require_once('/path/to/iveeCore/iveeCoreInit.php');

use iveeCore\Type, iveeCore\IndustryModifier;

//show the object for 'Damage Control I'
print_r(Type::getById(2046));

//it's also possible to instantiate type objects by name
$type = Type::getByName('Damage Control I');

//Now lets looks at industry activities.
//First we need to get an IndustryModifier object, which aggregates all the things
//like system indices, available assembly lines, character skills & implants, as an
//industry context. Lots of configuration and customization options there.
$industryContext = IndustryModifier::getBySystemIdForAllNpcStations(30000180); //Osmon

//manufacture 5 units of 'Damage Control I' with ME 10 and TE 20
$manuData = $type->getBlueprint()->manufacture($industryContext, 5, 10, 20);

//show the ManufactureProcessData object
print_r($manuData);

//Lets create another IndustryModifier object for Jita, so we can calculate costs and
//profits there
$marketContext = IndustryModifier::getBySystemIdForAllNpcStations(30000142);
//set default market station to Jita 4-4 CNAP
$marketContext->setPreferredMarketStation(60003760);

//print materials, cost and profits for this process
$manuData->printData($marketContext);

//get the data for making Damage Control I blueprint copy, inventing from it with a
//decryptor and building from the resulting T2 BPC, recursively building the necessary
//components
$processData = Type::getByName('Damage Control II Blueprint')->copyInventManufacture($industryContext, 34203, 1);

//get the raw profit for running an Unrefined Hyperflurite Reaction for 30 days,
//taking into account the refining and material feedback steps, and refinery
//efficiency and tax dependent on character skill and standing, then selling in Jita
$reaction = Type::getByName('Unrefined Hyperflurite Reaction');
$reactionProcessData = $reaction->react($industryContext, 24 * 30, true, true);
echo PHP_EOL . 'Reaction Profit: ' . $reactionProcessData->getProfit($marketContext) . PHP_EOL;


DOWNLOAD
The repository and full readme can be found here:
https://github.com/aineko-m/iveeCore

Packagist:
https://packagist.org/packages/aineko-m/ivee-core

Feedback welcome!
Aineko Macx
#2 - 2013-11-03 06:42:23 UTC
Steve Ronuken
Fuzzwork Enterprises
Vote Steve Ronuken for CSM
#3 - 2013-11-03 14:14:53 UTC
I've not had a chance to dig all the way through the code, so I'm not sure if you're taking this isn't account, but the way you're calculating the waste factor suggests to me that you've missed it.

Materials in Extra materials are affected by production efficiency waste, if they also appear in the base materials. (they're never affected by the ME waste)

If I've missed you handling that correctly, then good Smile

Woo! CSM XI!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter

Aineko Macx
#4 - 2013-11-03 14:44:38 UTC
Hi Steve, I was expecting you ;)

Extra material waste should be calculated correctly. Looking at "Improved Cloaking Device II Blueprint" for instance gives identical numbers to what the eve dev wiki states, what EveHQ calculates or the eve client shows.
Have a look at the stored procedure iveeGetRequirements() and at Blueprint.php line 279.
Steve Ronuken
Fuzzwork Enterprises
Vote Steve Ronuken for CSM
#5 - 2013-11-03 14:48:18 UTC
Improved Cloaking Device II Blueprint doesn't have anything that would be affected by extra material waste.

Take a look at a Dominix when you have less than Production Efficiency 5 (my site handles it properly, if you don't have a character with less than 5. The Eve client doesn't display it correctly, until it gets to actually producing it, and even then it's a bit messed up. Which is a pain.)

Woo! CSM XI!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter

Aineko Macx
#6 - 2013-11-03 15:17:23 UTC  |  Edited by: Aineko Macx
Point taken.
Skill support is something very stub-y at the moment (assume level 5; overridable, but no everywhere atm) and I wonder if I should ever provide more than that in the library, as this would soon lead to the need of a character tracker, API link, etc. IMO something to be handled by a different part of the user application or custom extension.
Steve Ronuken
Fuzzwork Enterprises
Vote Steve Ronuken for CSM
#7 - 2013-11-03 16:37:08 UTC
I can see reasons for leaving it out, mostly because any serious manufacturer has Production Efficiency 5. And it's not too uncommon for tools to miss it (Evemon, for example, doesn't handle it right)

It's not /too/ painful to code around, mostly requiring you to pull the data in two chunks, and process in code. I've played at getting complete BOM sql, but it doesn't work quite right when it comes to RAM. Eventually I'll fix it properly, so it doesn't round down things with damage. It's just something I need atm.


Aside from that, good work Smile There's always /something/ that can be picked at. bad habit of mine.

Woo! CSM XI!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter

Aineko Macx
#8 - 2013-11-03 18:32:01 UTC  |  Edited by: Aineko Macx
It's fixed. Thanks for pointing out the error and giving a solution Smile

Btw, I don't round damage, nor the fractions due to invention chances or (hypothetical) batches of items in non-multiples of portionSize. The library deals with float amounts just fine. The user just needs to keep in mind these values are what he'll see as average over time.
Aineko Macx
#9 - 2013-11-10 17:18:27 UTC  |  Edited by: Aineko Macx
Version 1.1 released.

- Added reprocessing calculations for Type objects
- Added volume attribute to Type classes and total volume calculation to process classes
- Moved material requirements from process class to its own MaterialSet class
- Moved skill requirements from process class to its own SkillSet class
- Added custom Exceptions
- More parameter sanity checks
- Note that the file iveeCoreConfig_template.php and the stored procedure iveeGetRequirements() were changed
Aineko Macx
#10 - 2014-02-03 16:29:30 UTC
Bump before close.
Currently working on the next release.
Dunpeal Hunter
Zervas Aeronautics
#11 - 2014-02-03 16:55:18 UTC  |  Edited by: Dunpeal Hunter
This sounds like a very useful PHP library. I am kind of new to the whole coding and programming process that is required to make an application work, but i am willing to learn!

I have some experience with Ubuntu and installing applications on a headless vps server (i already own an Ubuntu 64bit VPS server that i use with my own version of ruTorrent for seeding torrents to private trackers), but when it comes to making an actual application using custom made code and using the API to receive data from another source i get completely stuck and don't know how to proceed further.

With that in mind, can you tell me how this PHP library exactly works, or how i can use it in an application myself? Where should i as an complete noob begin? Do you know any good toturials that i can use and learn from?

I dont know if this is important or not, but i have chickens.....

Aineko Macx
#12 - 2014-02-03 18:32:53 UTC
Well you will have to learn the basics of PHP coding to use the library, there's no way around that I'm afraid.

The library itself contains the distilled knowledge about how to read and process the data from eves SDE database and present it through a simple but powerful object oriented interface.
As an example, by using the library, a two line PHP command line script is enough to give you all materials and data necessary to invent and build a T2 ship plus the expected profits. That's much quicker than opening EveHQ and clicking through the blueprint manager, so I usually just hack together whatever I need.
You could use it to make indy web apps or batch scripts that calculate loads of useful bits ;)
Aineko Macx
#13 - 2014-02-08 17:01:52 UTC
Version 1.2 released

- Added reaction calculations
- Rewrote type materials and requirements handling. This fixed reprocessing calculations and most off-by-1 errors. The
memory use of the type array was reduced by up to 30%. The stored procedure iveeGetRequirements() is obsoleted by this
change, which makes for more maintainable code.
- Bug fixes and more convenience functions
- Renamed MaterialSet to MaterialMap and SkillSet to SkillMap as it better describes the classes purpose
- Renamed CopyData to CopyProcessData, InventionData to InventionProcessData and ManufactureData to
ManufactureProcessData for consistency
- Added some PHPUnit test cases covering different parts of iveeCore
- Note that the file iveeCoreConfig_template.php changed, thus adaptation of the configuration file is required
Aineko Macx
#14 - 2014-03-29 12:02:21 UTC
Version 1.3 released

- Added parsers for EFT-style and EvE's XML fitting descriptions as well as cargo and ship scanning results.
- Reworked how configuration & defaults and utility methods are organized, for cleaner separation and better consistency. Defaults now reside in IveeCoreDefaults.php and customization is done in the subclass/file MyIveeCoreDefaults.php. As a consequence, iveeCoreConfig_template.php also changed a lot, therefore configuration files will have to be redone.
- Adapted README to Rubicon 1.3 release
- More bugfixes and unit tests
Alyssa Wyatt
Bazinga Labs
#15 - 2014-03-29 16:49:55 UTC
Don't know if you're aware, but on environments with E_STRICT on, the following errors popup

Quote:
Strict standards: Declaration of Manufacturable::getReprocessingMaterialMap() should be compatible with Type::getReprocessingMaterialMap($batchSize, $reprocessingYield = 1, $reprocessingTaxFactor = 1) in \iveeCore-master\Manufacturable.php on line 140

Quote:
Strict standards: Declaration of Blueprint::getReprocessingMaterialMap() should be compatible with Type::getReprocessingMaterialMap($batchSize, $reprocessingYield = 1, $reprocessingTaxFactor = 1) in \iveeCore-master\Blueprint.php on line 702

Quote:
Strict standards: Declaration of InventableBlueprint::getBuyPrice() should be compatible with Sellable::getBuyPrice($maxPriceDataAge = NULL) in \iveeCore-master\InventableBlueprint.php on line 151

Quote:
Strict standards: Declaration of InventableBlueprint::getSellPrice() should be compatible with Sellable::getSellPrice($maxPriceDataAge = NULL) in \iveeCore-master\InventableBlueprint.php on line 151

Quote:
Strict standards: Declaration of Decryptor::getReprocessingMaterialMap() should be compatible with Type::getReprocessingMaterialMap($batchSize, $reprocessingYield = 1, $reprocessingTaxFactor = 1) in \iveeCore-master\Decryptor.php on line 150
Aineko Macx
#16 - 2014-03-29 18:38:19 UTC
Aineko Macx
#17 - 2014-06-08 10:02:11 UTC  |  Edited by: Aineko Macx
v1.3.1 - Minor update for Kronos 1.0 released.

The upcoming, comprehensive changes to industry in EvE will require refactoring of at least some of the basic components in iveeCore. As there will be many more variables to consider to determine cost and profit of activities, some of which might or might not be queryable via API, major architectural changes might become necessary. I'm currently brainstorming and also waiting for more announcements from CCP in this regard...
Yourself4
The Scope
Gallente Federation
#18 - 2014-07-25 18:10:56 UTC  |  Edited by: Yourself4
Thanks for publishing this! It's awesome to be able to do industry calcuations and devolop without having to reinvent the wheel on the whole SDE plus blueprints. I really like how the project is organized with strong object orientation that is more typical of Java than PHP. I do however have some questions regarding this project:

Does this support the new changes to the SDE and new CREST endpoints in Crius?

The git repo seems to only be used to publish new releases with no intermediate commits. Can we submit pull requests?
Aineko Macx
#19 - 2014-07-25 18:44:43 UTC
I have been working feverishly on getting iveeCore compatible with Crius. I have been making good progress but I can give no clear ETA. As so much changed it is a major overhaul and the codesize is basically doubling compared to previous versions.

The upcoming release will be able to handle CREST data including system indices, teams & specializations as well as station specific assembly line bonuses. So basically you'll be able to say "I want to manufacture X in Station Y or POS in System Z" and everything else is automagically accounted for.

I have indeed been using the git repo only for releases, but this might change when people interested in contributing show up Smile
If you'd like to contribute I'd suggest you wait for the Crius compatible version 2.0 to avoid double work. I'll be happy to look at pull requests or discuss other changes.
Aineko Macx
#20 - 2014-08-01 19:02:21 UTC  |  Edited by: Aineko Macx
Alright, version 2.0 beta is out, with Crius support. Bugs in functionality and correctness are expected. Also the documentation might still contain obsolete bits and the unit tests have not all been converted yet.

Since the industry changes required a major rework of the library, I used the opportunity to restructure it and make it a bit more PSR-compliant, with namespaces, class autoloading and style changes.
12Next page