EVE Technology Lab

Today (before Retribution patch) I'm constantly refreshing the data export site trying to find the data dump for tomorrow's patch. Why? Well I have a program (EVE IPH - link in sig) that a decent number of people use to build items and I really wanted to get a patch ready for tomorrow when I get home from work. This is when the servers should be back up and my users would have the latest data. Most times the patches aren't so game changing that they will have an effect on EVE IPH but ones like this have big changes. We have some new items (salvage drones, mining frigs, micro jump drives, and all the cruiser/destroyer/frig mineral changes) that EVE IPH users will want to see. But not just me, there are at least 20 sites off the top of my head that do something similar to what I have and are mainstays of the EVE industry community.

As I've continued to develop EVE IPH I've tried to add easy ways to update the data so that I can deliver an updated product as soon as I get the data dump. Awhile back, I wrote an updater that I run locally to build my database for EVE IPH and it works pretty well. Not elegant but gets the job done. Everything worked great for several months of patches but two things have changed since then.

1 - The patches seem to make more and more changes or add items that are huge in the industry world. As I said above, all the 3rd party programs we have built as a community are mainstays for industry players that work to get the newest items ready for market. These changes suggest that waiting several days after a patch is too late for the community we support.

2 - The database dump is now going to be switched to YAML but slowly over time when changes are made in house. This in itself is a pain, but I think we are fine with it overall. The problem though is that without knowing what changes will come (up until now there have been minor fields replaced that I just comment out of the update because I don't use them), we won't be able to prepare our programs for updates. For instance, if you intend on making a very large switch of data to YAML that affects say killboard data or industry data, those 3rd party programs will be down for some time until they can get the data fixed or some nice dev rebuilds the data for us and posts it for download.

These 2 changes really are making it harder to maintain my program. I'm sure I'm not the only one. Not to mention that sometimes the data dump comes out and it has missing data then takes a few days to get another update. One of the great things about EVE for me as been the ability to build a program that helps the players have an easier time doing industry, and thus add to their overall enjoyment. It's also been fun just to develop. But we are doing this in our spare time. Spending days trying to figure out why the data isn't working when it did last patch isn't really fun. It's also not fun for my users to wait a week till I get time to update my data. I know you really support the 3rd party community, but times I just wonder if this is a "get to it when you can" thing.

Two things I can think of to make this better - 1 - Support the data dump process more formally. Right now it seems like this is a secondary "oh go post this data when you get a chance" job. Outside of the YAML change post, there doesn't seem to be much discussion on it. Either it's me and my lack of familiarity with YAML or it's not as big a deal yet for most of us to care. Also not much real process to knowing when the data is posted or when/how it will be. 2 - Post the data dump 2-3 days before the patch. If the patches are on Tuesdays, any way to get it on the weekend? Or at least the Monday prior? This would go a long way for helping out getting the data out to our users on patch day. Not to mention we have some time to work the YAML changes when they come up.

I'm probably rambling a bit, but the TL;DR - Can we get more support to put up the data dump a day or two before the patch? Can this be a consistent thing that is part of every patch check list so we can count on it?

Thanks!

+1

Yes. Very interested indeed!

I did at one point have reverence compiled and working, but that was ages ago, and the database schema was different in annoying little ways (I think just denormalised a little).

From the things that I've heard, what we get isn't just a dump of tables. Some of it is munged, through a process to create the tables we get, from different underlying ones.

Makes getting patch files a trifle more complex.

Simply because it is bigger than that.
As explained in this post in a somewhat similar topic, the community is bigger than just the Tech Lab forums and Community page.



Even the Community Toolkit page itself holds an invitation to the IRC channel. It's your loss if you aren't using all means available.

And I'm saying you can't standardize (spelling? dunno) this much more.
Anyways, Fuzzysteve on IRC just released his MySQL package, uploaded on eve-files.com.

Hugs to him.

http://dl.eve-files.com/media/1212/mysql-retribution-1.0-82596.dmp.zip

I find that ridiculous. If one part of the process is left out then it's not standardised, is it? The part that is left out is made redundant and never used so it might as well not even be there in the first place. It might as well say, "Join IRC or wait until someone can be bothered to post it on the forums for the rest of you losers."

I'd be interested in this as long as it's not a heavy lift. You think it would work on the test server data if we were to log in there as well? I'm assuming it's all the same and we could potentially (assuming no last minute changes) get the data early.

This could be configurable. And yes, CSM should definitely have a saying over this. Without SDE -> no 3rd party apps -> EVE Armageddon.

What I have to point out here though is that this SDE extractor will be a Windows based app. I'll try to have it Mono compatible (I can't promise that though). I just hope that Wine will be cooperative to it.

Hmm, what was their response? I don't see anything in the last minutes or the previous one. Considering there are a few key third party apps that are integral to the community, you would've thought CCP would prioritise keeping their unpaid developers happy.

I mean this is from the meetings in May/June:



Where the hell is this blog post? Why has the dev team been moved off Eve to a different project, and why are they using their dev time reinventing third party tools? You need to give them a giant kick up the arse! Nothing has changed for 6 months! You need to start shaking the tree until PrismX falls down so you can poke him with a stick until he gives in.