These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE General Discussion

 
  • Topic is locked indefinitely.
123Next pageLast page
 

CCP Development Patterns

First post
Author
Jessica Danikov
Network Danikov
#1 - 2014-01-29 10:57:29 UTC
I don't know what's going on inside of CCP, but short of arbitrary deadlines and expectations of progress that aren't being met and bad quality control, here's a few things you should think about and do, if you don't already:

A) All releases should be flexible. If the release isn't good, don't do it, postpone it until it's ready. Player trust and respect is not an infinite resource and every broken update taxes their patience.

B) All releases should be reversible. If the release is bad, roll it back, fix it, only release it when you're sure. The cost of a bad release should be an incentive for people to pipe up and say 'no, if we release now, it's going to cost us'. If anyone thinks a release isn't ready, it shouldn't happen. None of this late night burnout hacking to fix it in time for the next downtime- you'll burn out your employees that way.

C) Test, test, test. Mirror Tranquillity regularly and deploy the entire final release there (in the software industry, we call this a release candidate). Have a mass test to get people playing around with it. Most of the bugs your players ***** about on day 0 of your new release would be caught on Singularity fairly quickly. Once caught, fix those bugs, roll a new release candidate, mirror Tranquility, mass test again. Keep doing this until it's ready.

D) In-house testers. One of the greatest false economies in the software industry is not to have in-house testers. They're relatively cheap compared to your experienced employees and free up whoever is doing your testing to actually spend their time doing whatever it is they're paid to be doing- testing isn't worth their time until a problem has definitive reproduction steps. QA leads can also keep their eyes peeled for the 1/100 talent that ends up in your tester pool.

E) Be aware of your reputation. Unpopular features, for better or for worse, that's the domain of your designers and you have to ride that one out as long as you trust in their process. However, bad software process is entirely avoidable and CCP has a bad history on that front (yes, I know, it's negative bias- we notice everything that goes wrong and not all the things going right, but that's what makes it so important to be rigorous and thorough and minimise the number and severity of what bugs do make it into releases).

F)Be honest, communicative and apologetic. We're all human, we make mistakes. Just be aware if you keep making the same mistakes, you will lose good will (especially from people who know that you should know better).

Again, I don't know how much of this stuff you do, or how well you do it, but obviously there's room for improvement. Maybe a Devblog giving an insight to your development process is long overdue?
seth Hendar
I love you miners
#2 - 2014-01-29 11:01:16 UTC  |  Edited by: seth Hendar
tbh, judjing from the past 2 years releases, the various feedback and results around some features, i'd say CCP has NONE of the above, and doesn't give a crap about it.....

non non regression test is a good example, how many time some "fixed" bug just swing back?
Kryptik Kai
Sniggerdly
Pandemic Legion
#3 - 2014-01-29 11:02:36 UTC
Its amazing how many software developers play EVE

"Shiny.  Lets be bad guys." -Jayne Cobb

Kimmi Chan
Tastes Like Purple
#4 - 2014-01-29 11:05:50 UTC
Oh look another unemployed Software Developer and/or Project Manager looking for a job at CCP.

"Grr Kimmi  Nerf Chans!" ~Jenn aSide

www.eve-radio.com  Join Eve Radio channel in game!

seth Hendar
I love you miners
#5 - 2014-01-29 11:11:23 UTC
Kryptik Kai wrote:
Its amazing how many software developers play EVE

not really in fact, eve has always gethered a crowd of computer tech ppl, and many of those ARE actually, devellopers, or do work with some.


unlike many other game, i'd say the average eve player has a better understanding of computers, and the % of ppl working in this field is way higher than any other game
Riot Girl
You'll Cowards Don't Even Smoke Crack
#6 - 2014-01-29 11:17:35 UTC
The thing that confuses me a bit is why they released a patch after 6 weeks. Aren't content patches usually every 12 weeks? Well that made me think of something I read in the news about staff members being laid off from WoD and CCP saying they were going to focus more on Eve. Does this mean CCP are going to aim to release content every 6 weeks now? If they are, then idkaf about a few bugs which can be smoothed out later. Not that it ever bothered me anyway.
seth Hendar
I love you miners
#7 - 2014-01-29 11:23:22 UTC
Riot Girl wrote:
The thing that confuses me a bit is why they released a patch after 6 weeks. Aren't content patches usually every 12 weeks? Well that made me think of something I read in the news about staff members being laid off from WoD and CCP saying they were going to focus more on Eve. Does this mean CCP are going to aim to release content every 6 weeks now? If they are, then idkaf about a few bugs which can be smoothed out later. Not that it ever bothered me anyway.

actually, if it would be the case, i'd rather see them keep the 12 weeks pace and actually improve the quality of each release, by giving it more testing, and a better integration of player feedback.

this would avoid coming up with things like the ESS or the odyssey exploration, and bugs like the standing thingy we have atm.....
Nevyn Auscent
Broke Sauce
#8 - 2014-01-29 11:29:28 UTC
The biggest bug that people are complaining about got through over a month of testing on Sisi.
Stop dumping on CCP and instead have words to all those people on Sisi to actually test stuff, not just fart around in expensive blinged out space ships that they can't be bothered affording or are too scared to ever fly on TQ.
CCP Goliath
C C P
C C P Alliance
#9 - 2014-01-29 12:00:15 UTC
Jessica Danikov wrote:


C) Test, test, test. Mirror Tranquillity regularly and deploy the entire final release there (in the software industry, we call this a release candidate). Have a mass test to get people playing around with it. Most of the bugs your players ***** about on day 0 of your new release would be caught on Singularity fairly quickly. Once caught, fix those bugs, roll a new release candidate, mirror Tranquility, mass test again. Keep doing this until it's ready.

D) In-house testers. One of the greatest false economies in the software industry is not to have in-house testers. They're relatively cheap compared to your experienced employees and free up whoever is doing your testing to actually spend their time doing whatever it is they're paid to be doing- testing isn't worth their time until a problem has definitive reproduction steps. QA leads can also keep their eyes peeled for the 1/100 talent that ends up in your tester pool.
...
Again, I don't know how much of this stuff you do, or how well you do it, but obviously there's room for improvement. Maybe a Devblog giving an insight to your development process is long overdue?


Since you took the time to write this, I will give you as frank a response as possible on the subjects that are under my control.

C - We were on Singularity from the 6th January with this release. Familiar as I am with the term release candidate, being in the software industry myself, I can tell you it was deployed on Monday morning to Sisi, as is par for the course with our deployments. This might seem crazily short to you, but our development method allows us to work right up until then because we do not hamper teams with code/feature freezes and have mature teams which practice 'Whole Team Quality' (more on this later) giving them the responsibility and accountability over the quality of the features they ship. This is a process I have seen evolve in my 2.5 years here to really empower teams, and it is one that I think works well. Further than this though, we deploy daily updates to Singularity to ensure users have the latest stable build available. Combine this with the internal QA coming from the teams, and our outsourcers running a full regression suite every 2 weeks, and you're looking at a pretty robust testing process.

D - I assume on this you are referring to a test house of entry level "testers" working for minimum wage with a 50-90% bimonthly churn. This is firstly unfeasible in Iceland, as the pool of potential employees does not cover the churn that these outfits have. Instead of throwing a large amount of un/semi skilled workers at the problem, we use highly trained and experienced outsourcers to handle our regression testing, and assist with processing bug reports, providing focused testing at team request, and performing targeted exploratory testing (the most valuable type of testing, for me). Our current outsourcing partner has been with us for 3 years now, and I travel to them each year to ensure their methods and practices are keeping to the high standard established. We also have a proven conduit of taking talent from them directly into our own workforce. CCPers who have taken this route include Legion, Sledgehammer, Lebowski, Antiquarian, and even my good self!

Devblog - As the current facilitator of CCP Reykjavik's Agile Community of Practice, I tabled that we write exactly such a devblog not a month ago! We do need to get started (I actually wrote part of it around a year ago, but it got tabled at the time) but there is a plan in place and many keen contributors.

CCP Goliath | QA Director | EVE Illuminati | @CCP_Goliath

Fix Lag
The Scope
Gallente Federation
#10 - 2014-01-29 12:08:34 UTC
Antiquarian told me my computer sucked when it didn't. He's a big fat phony. :P

CCP mostly sucks at their job, but Veritas is a pretty cool dude.

Angelica Dreamstar
Aliastra
Gallente Federation
#11 - 2014-01-29 12:10:17 UTC  |  Edited by: Angelica Dreamstar
This really doesn't help understanding how such a bug can see the light on TQ, CCP Goliath. Your post just makes if look worse...

bingo, his pig not being a goat doesn't make the pig wrong, just him an idiot for shouting at his pig "WHY ARENT YOU A GOAT!" (Source)

-- Ralph King-Griffin, about deranged people playing EVE ONLINE

CCP Goliath
C C P
C C P Alliance
#12 - 2014-01-29 12:11:18 UTC
Angelica Dreamstar wrote:
This really doesn't help understanding how such a bug can see the light on TQ. Your post just makes if look worse...


That wasn't what I was trying to address, so that isn't surprising. I have addressed that in other posts so won't do so again.

CCP Goliath | QA Director | EVE Illuminati | @CCP_Goliath

Charlie Firpol
Brutor Tribe
Minmatar Republic
#13 - 2014-01-29 12:14:08 UTC
Its not a bug, its a feature and a great troll :)

The Butcher of Black Rise - eve-radio.com

mr ed thehouseofed
Wrought iron Industries
#14 - 2014-01-29 12:24:01 UTC
one of these days ccp is going grab the ten biggest complainers in this game, stick in a room and get them to fix everything they whine about ,so that way we can flame them for a change . honestly guys if you think your that good at solving all the problems with the biggest mmo ever ,**** off and make your own bloody game X as a non it guy ( i mess up half the emails i send Oops) i think they do a good trying to balance the game to suit everybody, can't be easy either.

tinfoil hat on

i want a eve pinball machine...  confirming  CCP Cognac is best cognac

Gregor Parud
Imperial Academy
#15 - 2014-01-29 12:25:47 UTC  |  Edited by: Gregor Parud
CCP Goliath wrote:
C - We were on Singularity from the 6th January with this release. Familiar as I am with the term release candidate, being in the software industry myself, I can tell you it was deployed on Monday morning to Sisi, as is par for the course with our deployments. This might seem crazily short to you, but our development method allows us to work right up until then because we do not hamper teams with code/feature freezes and have mature teams which practice 'Whole Team Quality' (more on this later) giving them the responsibility and accountability over the quality of the features they ship. This is a process I have seen evolve in my 2.5 years here to really empower teams, and it is one that I think works well. Further than this though, we deploy daily updates to Singularity to ensure users have the latest stable build available. Combine this with the internal QA coming from the teams, and our outsourcers running a full regression suite every 2 weeks, and you're looking at a pretty robust testing process.



That's all really nice but doesn't change the fact that VERY often the most hilarious bugs or issues simply make it through testing, SISI and bug reporting, obviously something isn't working as it should. Minor issues can always happen but CCP really has a vast history of royal fuckups, facepalms and "seriously?" in this regard, ones that if QA would actually play the game a bit (and have a clue on what they're doing) simply wouldn't happen.

What's the chances you have "CCP CAPLOCK" type of clowns working in QA?
Altrue
Exploration Frontier inc
Tactical-Retreat
#16 - 2014-01-29 12:37:02 UTC  |  Edited by: Altrue
Would be cool to see more people being taking bugs as positively as they can on release date.
This is not some regular MMO server with 2k players on. This is Eve, and there are some things that just wont work on TQ where they might work on Sisi. IT magic and stuff.

To me the standing bugs seemed like a pretty fun experience. Okay I wouldn't have liked that if it has been lasting for weeks, but one day, just like that, that's just an interesting experience.

Should they delay a point release date, they'd surely have people complaining about it one way or another. People will complain whatsoever.

Plus, releases have vastly been improved compared to a few years ago. Extended downtime is minimal, deployment is nearly perfect. There are obviously still some issues on the internal-testing side, but overall patch deployment has never been better.

Signature Tanking Best Tanking

[Ex-F] CEO - Eve-guides.fr

Ultimate Citadel Guide - 2016 EVE Career Chart

Adunh Slavy
#17 - 2014-01-29 12:46:25 UTC
No boot.ini issues? Ship it.

Necessity is the plea for every infringement of human freedom. It is the argument of tyrants; it is the creed of slaves.  - William Pitt

Treborr MintingtonJr
Deep Core Mining Inc.
Caldari State
#18 - 2014-01-29 12:49:39 UTC
I think more of us need to go on SiSi to improve the scope of UAT.
Sexy Cakes
Have A Seat
#19 - 2014-01-29 12:49:46 UTC
The inability of CCP to scrap crap game designs that don't sit well with the player base has always been one of their major downfalls.

Anything less than a player riot in-game and CCP keeps their heads buried in the proverbial sand.

Not today spaghetti.

Angelica Dreamstar
Aliastra
Gallente Federation
#20 - 2014-01-29 12:51:19 UTC
CCP Goliath wrote:
Angelica Dreamstar wrote:
This really doesn't help understanding how such a bug can see the light on TQ. Your post just makes if look worse...


That wasn't what I was trying to address, so that isn't surprising. I have addressed that in other posts so won't do so again.
Yes. No. Not what I'm getting at.

It doesn't matter if you wanted to address this or not.

Your post is indirectly addressing every single bug out there, but especially all the big ones that still get to TQ. It creates the impression that you hired idiots who have no clue of what they are doing, because huge issues like the standings bug go live.

At the same time you write 'highly trained and experienced outsourcers'. What people perceive is the exact opposite! And that's really bad. People could feel lied to. Some can feel as if you'd think they are idiots. An example of a rather typical reaction is below.

You missed to link to your other post, to help people not jump to ...

'yeah. highly trained and experienced outsourcers... my ass. HOW CAN YOU NOT HAVE NOTICED THIS BUG? WHAT KIND OF IDIOTS ARE YOU HIRING???' etc etc.

Get the point?

bingo, his pig not being a goat doesn't make the pig wrong, just him an idiot for shouting at his pig "WHY ARENT YOU A GOAT!" (Source)

-- Ralph King-Griffin, about deranged people playing EVE ONLINE

123Next pageLast page