These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

 
  • Topic is locked indefinitely.
 

Evernus [2.2 release] - the Ultimate Market Tool

First post
Author
Previous Topic Next Topic
Alex Angelier
Supplemental Shi Logistics
#801 - 2016-03-13 17:28:18 UTC
New version! No more crashes confirmed. Awesomeness!

Breathe...Relax...

Float in the void...

Know yourself... Know your environment...

Take Action...

Breathe...
Aaron Honk
Distributed Denial of Service
#802 - 2016-03-13 18:44:24 UTC  |  Edited by: Aaron Honk
.
Sabena Sema
Sema Holdings
#803 - 2016-03-16 12:01:40 UTC
I was having some trouble with evernus being extremely slow to save orders after downloading them. I noticed evernus was using SQLite and I had similar problems in my own app.

Turns out SQLite has really terrible insert performance unless you use larger transactions or change the default settings.

I solved my immediate problem by executing "PRAGMA journal_mode = OFF" and "PRAGMA synchronous = OFF" right after opening the database. I'd be happy to submit a pull request but I wanted to know Pete's thoughts on whether it's worth it to keep the journal on and wrap things like saving orders in transactions.

Doing transactions would probably mean adding two more DB related methods, one to start the transaction and one to end it. There are some error management issues to consider as well. I /think/ that simply issuing a rollback command when an error happens should be the right way to go, but I'm not sure where this should go (which is why I wanted to talk to Pete).

So Pete: Also: unless an error is recovered from without closing the database there's no need to match the begin and end transaction calls.

Oh: Also my copy of visual C had an ICE (internal compiler error) that I fixed as follows

https://gist.github.com/barcharcraz/65c4830c6d45e276fb1b

Honestly it looks to me like value is indeed a dependent template so this change is probably not standards conforming. Visual C++ has a bunch of nonstandard behavior related to name binding in templates so who knows
Pete Butcher
The Scope
Gallente Federation
#804 - 2016-03-16 13:24:06 UTC
You are right about SQLite performance - it's very slow. On a SSD, any amount of orders gets saved in a flash, while on HDD it takes much time. Solving this issue was something I wanted to battle at some point, but never actually had the time, since it might be a bit tricky. My only optimization now is splitting bulk inserts into a very large prepared statements, up to the bound variable limit. SQLite, unfortunately, doesn't support any kind of bulk import.

Your ideas are interesting, although I'm a bit concerned about journal_mode. In case of a crash or forced termination, we might loose a lot of data, and given how CREST works, I don't know if anyone would be happy about the notion of reimporting everything. And it makes debugging easier when you can simply kill the app any time you wantSmile Have you encountered such crash in the middle of transaction? We could check how this works in practice.

On the other hand, turning off synchronous mode might speed things up considerably. In my opinion the risks are negligible. Good catchSmile If you have a pull request, I'll be happy to merge it.

As for the compiler error - if you use the latest VS 2015, everything should compile fine. The initial VS release had so many bugs, I just gave up on using it.

http://evernus.com - the ultimate multiplatform EVE trade tool + nullsec Alliance Market tool
Sabena Sema
Sema Holdings
#805 - 2016-03-16 13:35:41 UTC  |  Edited by: Sabena Sema
I've never had a crash in the middle of a transaction. Still, I'm going to try experimenting with various settings. It looks like journal_mode = TRUNCATE should help. And in my own app I ended up turning off the journal and sync only for the SDE database (which I never wrote to anyways) and wrapping my order saving in transactions, which helped considerably.

I suspect the main source of the performance problems is that SQLite deletes the rollback journal file when committing each transaction. This means you have to read the directory and then write to it. I bet this is what hoses perf so badly on spinning disks.


EDIT: It seems like just disabling synchronization and letting the OS's disk scheduler do it's job is enough. I'll submit a pull request once I install Visual Studio 2015 update 1.
Sabena Sema
Sema Holdings
#806 - 2016-03-16 13:53:22 UTC
Allright I made a pull request with just the sync change. It's only one line but it should be almost somewhere around a 750x speedup.
Pete Butcher
The Scope
Gallente Federation
#807 - 2016-03-16 14:08:53 UTC
Sabena Sema wrote:
Allright I made a pull request with just the sync change. It's only one line but it should be almost somewhere around a 750x speedup.


Thanks for the PR. I must say: WOW! Just tested it on a dying HDD and saving 15k orders took few seconds. Now that's an improvementBig smile

http://evernus.com - the ultimate multiplatform EVE trade tool + nullsec Alliance Market tool
Philip Ogtaulmolfi
We are not bad. Just unlucky
#808 - 2016-03-18 23:42:16 UTC  |  Edited by: Philip Ogtaulmolfi
Pete, thank you very much. I have been playing with evernus for a couple of hours and I find it really useful.

I am still reading the thread, so perhaps this has been asked before.

I suppose that the margin is calculated against my own buy order? If that's the case, my problem is that I buy with a character and sell with a different one, so all my orders show margin 100%. Would it be possible to combine characters?

Thank you again.

Edit: Now I see that you are not calculating profit, so what I was asking is not implemented.
Hannah Tsukaya
Doomheim
#809 - 2016-03-20 20:23:04 UTC
Mergency Warpout wrote:
Image of the situation.

http://eve-files.com/dl/276046

Angel Palladium Tag selected, current market orders listed below.

Price A is correct. My order is highlighted in Yellow. It is also correctly showing that my sell order is the lowest currently.

Where is Price B coming from? It is showing a higher price for some reason.


I have this problem also.

The solution for him was: "In my case it was an issue where I assumed one corp API key was enough. Once I added a corp API key for the character I was having a problem with all the issues cleared up."

I have only 1 character API key and that character has also a corp API key added.

What am I doing wrong?
Tehg Rhind
Atlantic Innovations
#810 - 2016-03-31 01:35:37 UTC  |  Edited by: Tehg Rhind
Been a while since I've messed with this but I'm having the similar problems I've had in the past with wild asset valuations. The main problem this time seems to be in some of the more obscure items I have, like some drone alloys in some far flung stations that are showing up as worth billions due to some off the wall sell orders. Is there any way to manually set prices or something for some of these? Or just ignore them? I would really like to get a good idea of my actual asset values. Also, it looks like your concatenation puts "ISK" right next to the value, which really hurts readability. Would be nice if there was a space there.

One last thing. When I open the asset window all of the stations are expanded. Is it at all possible to collapse these? it takes ages to do it manually.

All that said still an excellent tool.
Pete Butcher
The Scope
Gallente Federation
#811 - 2016-03-31 06:13:02 UTC
You can choose which station to take the prices from. You can also turn off the ISK text in the preferences.

http://evernus.com - the ultimate multiplatform EVE trade tool + nullsec Alliance Market tool
Tehg Rhind
Atlantic Innovations
#812 - 2016-03-31 14:39:04 UTC
Cool beans about the ISK text. The problem with asset valuations is a bit more tricky though. I like using my home station of Amarr, and even in a high volume market I can find some wonky stuff in there (especially drone alloys). The workaround I have at the moment is to go to the market explorer and clear market data for those specific items. This removes them from all valuations. I think I'll have to do it any time I update prices, so it's kind of duct tape, but it works for now.
Zao Amadues
DO YOU EVEN
#813 - 2016-04-19 15:11:41 UTC
o7 Pete,

I am having a problem with the evernus install (32, and 64). installs fine, but when launched I get two errors.

ERROR 1: An error occurred during the update process. Database backup was saved as (file path). Please read online help how to deal with this situation.

ERROR 2: duplicate column name: notes Unable to execute statement ALTER TABLE market_orders ADD COLUMN notes TEXT NULL DEFAULT NULL

I have had an older version of evernus (1.28) but i believe i have completely cleared it out prior to installing the new one. this includes %AppData%

Fixes I have tried:

reinstalling (2 times)
installing older version (1.32)
full reboot after install
googling the error messages

Any help you could provide would be wonderful, thank you for your time and thanks for the awesome tool!

Zao
Pete Butcher
The Scope
Gallente Federation
#814 - 2016-04-19 15:38:16 UTC
Open regedit, go to HKEY_CURRENT_USER\SOFTWARE\evernus.com\Evernus and delete the version key there. It should help.

http://evernus.com - the ultimate multiplatform EVE trade tool + nullsec Alliance Market tool
Zao Amadues
DO YOU EVEN
#815 - 2016-04-19 16:52:29 UTC
Pete Butcher wrote:
Open regedit, go to HKEY_CURRENT_USER\SOFTWARE\evernus.com\Evernus and delete the version key there. It should help.


Thank you, worked like a charm. always tricky to get rid of things you had in the past it seems. ill remember that for next time/ with other things.

ZAO
Truxston
Republic Military School
Minmatar Republic
#816 - 2016-04-26 01:12:09 UTC
I just discovered this tool 2 days ago and I'm really impressed so far.

This tools seems to have a great depth to it, but so far all that I have managed to use is the Character Orders and the Margin Tool, which is over-the-top awesome!

2 easy questions for you:

1. Is there any sort of tutorial anywhere? I've searched and searched, but I find nothing.

2. Is there a way to make the Margin Tool disregard buy orders that are obvious margin scams? i.e. ignore buy orders where the "min volume" is greater than 1 (sometimes the min volume is equal to the "quantity", but not always)
Pete Butcher
The Scope
Gallente Federation
#817 - 2016-04-26 05:22:23 UTC
Truxston wrote:
I just discovered this tool 2 days ago and I'm really impressed so far.

This tools seems to have a great depth to it, but so far all that I have managed to use is the Character Orders and the Margin Tool, which is over-the-top awesome!

2 easy questions for you:

1. Is there any sort of tutorial anywhere? I've searched and searched, but I find nothing.

2. Is there a way to make the Margin Tool disregard buy orders that are obvious margin scams? i.e. ignore buy orders where the "min volume" is greater than 1 (sometimes the min volume is equal to the "quantity", but not always)


1. Unfortunately, there is only the help page. I have no time to make any YT videos.
2. Not yet, but will be in the next version.

http://evernus.com - the ultimate multiplatform EVE trade tool + nullsec Alliance Market tool
Iria Ahrens
Space Perverts and Forum Pirates
#818 - 2016-04-27 07:34:55 UTC
Can't install version 1.40 had to install version 1.32 which installed without problems. When I try to run 1.40 it gives me a missing dll
"The program can't start because api-ms-win-crt-runtime-l1-1-0.dll is missing from your computer. Try reinstalling..."

Reinstalling just gives same error. The x64 version of 1.40 installs to the x86 programs instead of programs like a 64bit program should. Dunno if that is cause or a side effect.

I tried installing the VC_redistx64.exe file that comes with 1.40, but I get an unspecified error 0x80240017.

There really needs to be a way to install to the same directory and just write over the previous installation.

The database recovery just totally failed me. I had to reinstall evernus anyway after reinstalling windows. So I get 1.32 running and install my api because there is only an import eve mentat not an import evernus data option. So the only option is upload to cloud. I don't want to upload, I want to download from the cloud, but the symbol is a sync symbol with the circular arrows, so I click it hoping that it will download the previous information. But it fails.

My statistics only goes back to 3/24/16 now everything before that is lost. Is there any way to recover that statistical information so my statistics graph will plot properly?

Thank you.

My choice of pronouns is based on your avatar. Even if I know what is behind the avatar.
Pete Butcher
The Scope
Gallente Federation
#819 - 2016-04-27 08:18:16 UTC  |  Edited by: Pete Butcher
That's not good. vc redist is supposed to be installed by the installer. Given your error, I assume it silently fails. The error you describe is here: http://superuser.com/questions/979546/vc-redist-x86-exe-setup-failed-0x80240017-unspecified-error The redist just fails for some unspecified reason, but there's a workaround in that topic.

If you have a dropbox sync enabled, restoring the db shouldn't be a problem. In your Dropbox account restore the file version, which you think will be the appropriate one and remove you local copy from your %APPDATA%\evernus.com\evernus\db directory. Evernus will then download the backup from Dropbox as soon as it launches, if you have sync enabled. If not, you'll probably end up with the current situation, since Evernus will use your local copy and upload it to DB, as it thinks it's newer than your backup (which technically is true). Therefore enable sync in Evernus preferences before you try to restore the backup. I see that's a situation which should be somehow protected against.

http://evernus.com - the ultimate multiplatform EVE trade tool + nullsec Alliance Market tool
Dirac Delta Function
Garoun Investment Bank
Gallente Federation
#820 - 2016-04-27 22:52:10 UTC
I've been using this tool and it's extremely helpful.

Due to the tax changes the Margin tool is not listing correct values for broker fees or sales tax, but I just changed the fee skills down until I hit my new values.

According to the old formulas it is as if I have Accounting 3, Broker Relations 0, Corp standing -5.5 and Faction standing of -6.70.

Oh dear.

This should act as a sort of workaround for now.