These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE General Discussion

 
  • Topic is locked indefinitely.
Previous page12
 

Quality Control
Author
Previous Topic Next Topic
RubyPorto
RubysRhymes
#21 - 2011-12-21 20:46:13 UTC
Avid Bumhumper wrote:

And your response, if you really are what you claim, is predictible.
It may not even be your fault, companies in general have reached the point where it's no longer, how can I improve my customer service/app/whatever.
It's how can I squeeze more out of what I have, and how low are my customers expectations considering what they deal with daily..
If the other guy isn't doing the job either, we have nothing to worry about....


This is the new mindset......less bad == good enough. Ugh

Not saying all companies have adopted this posture, but if you look around, you'll recgonize the attitude.....




Not quite. The attitude is "Given this fixed set of resources, what's the most important thing I can fix? Done. Given this fixed (now smaller) set of resources, what's the most important thing I can fix? Done." There are some important things to fix that are simply to hard to fix before release (and aren't worth delaying the release thus delaying the improvements the release brings), and some things that aren't that hard but are less important than other things that need fixing and thus don't make it in.

It boils down to an example of a knapsack problem, with unknown (but bounded) weights and unknown (but bounded) values. Try to figure out how much value (bugfixes) you can fit into a knapsack within a certain weight (resources: Clock Time, Dev Time, Testing time, Money, etc.).

"It's easy to speak for the silent majority. They rarely object to what you put into their mouths." -Abrazzar "the risk of having your day ruined by other people is the cornerstone with which EVE was built" -CCP Solomon
RubyPorto
RubysRhymes
#22 - 2011-12-21 20:46:51 UTC  |  Edited by: RubyPorto
FORUMS R BAD at Foruming, mmmkay?

"It's easy to speak for the silent majority. They rarely object to what you put into their mouths." -Abrazzar "the risk of having your day ruined by other people is the cornerstone with which EVE was built" -CCP Solomon
Ai Shun
#23 - 2011-12-21 20:50:34 UTC
Hung TuLo wrote:
I do understand your point.

Depending on the severity and criticalness of the problem. You don't even thing of placing something into production. And yes you do have those meetings between QA and dev teams.

In the end QA sometimes does not have the "power or control" to sto something from going into production.

It mainly is the 80/20 law or the 90/10 law.

As long as 80% - 90% of the code is working as expected then a push to production is waranted. Once again that depends on the critical nature of the problem. In your example of the UI problem. If the UI was found to be working 95% of the time and the problem was minor. Should the entire push be stopped?

Most of the time. The answer to these things are not your regular QA folks or you dev folks. Its their managers that are answering to the CEO or other bigwigs that will make the ultimate decision. By and large the QA staff is doing their job.

Remember QA just reports the facts. The test the software and report problems, possible causes and possible ramifications.

QA does not cause the coding problems they find them.


Its those in the nosebleed seats that make the decisions.


If you can, read that book. It is mainly for developers, but it changed my mind about how the process should work. I know we can't always deal with ideal scenarios, but a lot of the up-front should come from development; not QA.

Seriously. I strongly recommend that book.
Atlas Oracle
Colossus Enterprises
#24 - 2011-12-21 21:00:32 UTC
Hung TuLo wrote:
I actually am a professional software quality control analyst.

<...>

Their ability isn't the issue, your attitude is.


you're a pathetic oxygen thief
Sycho Pathic
Republic Military School
Minmatar Republic
#25 - 2011-12-21 21:58:24 UTC
Anything more complicated than a bubble-sort follows that pattern.
Frankly, I'm rather impressed with how well CCP manages EVE.
Corina Jarr
en Welle Shipping Inc.
#26 - 2011-12-21 22:24:58 UTC
I wonder how many of the people who complain about the patching actually submit complete bug reports.

That is, highly detailed down, including system information, what you did to get the bug, how often it occurs, and your EVE settings.

Including troubleshooting attempts can help too (and the results of said attempts).
Ranger 1
Ranger Corp
Vae. Victis.
#27 - 2011-12-21 22:30:04 UTC  |  Edited by: Ranger 1
Ai Shun wrote:
Hung TuLo wrote:
I do understand your point.

Depending on the severity and criticalness of the problem. You don't even thing of placing something into production. And yes you do have those meetings between QA and dev teams.

In the end QA sometimes does not have the "power or control" to sto something from going into production.

It mainly is the 80/20 law or the 90/10 law.

As long as 80% - 90% of the code is working as expected then a push to production is waranted. Once again that depends on the critical nature of the problem. In your example of the UI problem. If the UI was found to be working 95% of the time and the problem was minor. Should the entire push be stopped?

Most of the time. The answer to these things are not your regular QA folks or you dev folks. Its their managers that are answering to the CEO or other bigwigs that will make the ultimate decision. By and large the QA staff is doing their job.

Remember QA just reports the facts. The test the software and report problems, possible causes and possible ramifications.

QA does not cause the coding problems they find them.


Its those in the nosebleed seats that make the decisions.


If you can, read that book. It is mainly for developers, but it changed my mind about how the process should work. I know we can't always deal with ideal scenarios, but a lot of the up-front should come from development; not QA.

Seriously. I strongly recommend that book.


I think what it basically boils down to is that if a release was held up until ALL bugs (regardless of severity) were squshed, it would never be released... especially when discussing software as unique and intricate as EVE is.

Much of the code in EVE is very unique... completely uncharted ground even for companies that have tried for years to come up with something similar, and failed.

Given the persistent nature of the environment and the countless variations of end user interactions with the software, plus the fact that (although they have made strides in this direction) there really is no effective way to stress test their code under normal use conditions prior to release, you end up with a lot of patches after the fact. It is a reality of the situation, not a failure in processes.

You can have most other types of software completely isolated on test platforms (word processors, music players, cooking programs, most other multi player games, what have you), fully test all possible uses with the maximum number of simultaneous users possible, and still have a good chance of missing something.

You can't do that with EVE.

Many of the things that make EVE unique and special to us have this inevitable downside. I think they've done a pretty fair job of minimizing that downside and implementing needed bug fixes in a timely manner.

Perfect code is a worthy goal, just (in this case) not a very realistic expectation.

View the latest EVE Online developments and other game related news and gameplay by visiting Ranger 1 Presents: Virtual Realms.
RubyPorto
RubysRhymes
#28 - 2011-12-21 22:33:28 UTC
Corina Jarr wrote:
I wonder how many of the people who complain about the patching actually submit complete bug reports.

That is, highly detailed down, including system information, what you did to get the bug, how often it occurs, and your EVE settings.

Including troubleshooting attempts can help too (and the results of said attempts).



Probably very few, though (and this is coming from a Mac user)
  • CCP doesn't exactly make the process easy (at least for the Mac)
  • One way of thinking is that users should not be required to help fix software that they're paying for (I don't agree with this point)
    • [*] Neither of the 2 main bugs affecting the Mac client produce crashlogs without special logging set up (and one doesn't produce anything even then)

    "It's easy to speak for the silent majority. They rarely object to what you put into their mouths." -Abrazzar "the risk of having your day ruined by other people is the cornerstone with which EVE was built" -CCP Solomon
    Liam Mirren
    #29 - 2011-12-21 22:43:49 UTC
    Hung TuLo wrote:
    I actually am a professional software quality control analyst.

    How may of you in this thread can say that, how many of the players can say that. What do you really know about SQC?

    It is virtually impossible to catch every single bug that is in a game this size. Its not possible

    1. There are so many computer set-ups that need to be considered.
    2. Different graphics cards that are used.
    3. Networking issues to consider.
    4. Different programing modules that need to be worked on on a daily basis.
    5. New equipment that needs to be considered in their systems.
    6. They release new code every six months that allow you to enjoy this game. We arn't talking 10 lines of code, we are talking 10's of thousands of lines of code that probably need to be modified.
    7. Intigration processes.
    8. characters that transfer from one server to other servers.
    9. Localization issues.
    10. patching issues and when it needs to happen.
    11 Release issues.
    12. Code upgrades.
    13. Code migrations.
    14. Petition requests and responses.
    15. Server Load issues


    That is a small list.


    So, when you want to ***** and complain about the patches remember this,. There is alot more than what is going on that you can't see. They have a job just like you should have a job. They are doing their job to the best of their ability and are trying to be a professional as possible.

    Don't assume they arn't doing their jobs just because something doesn't work.

    Their ability isn't the issue, your attitude is.



    Your reaction makes sense from a personal/DEV point of view, you can't (and shouldn't) blame specific DEVs or coders for issues. But you SHOULD blame the software company as a whole (management) for creating a situation where churning out less than desirable content/expansions/patches is inevitable or, worse, expected practise.

    Excellence is not a skill, it's an attitude.
    Ai Shun
    #30 - 2011-12-21 22:51:13 UTC
    Ranger 1 wrote:
    Perfect code is a worthy goal, just (in this case) not a very realistic expectation.


    Most of my C / C++ development experience comes from an ICS solution and IDS solution. (About 7 years, roughly from 2001 to 2008) Prior to that it was medical information systems and some R&D type work. So yeah, I've worked in environments where there were complex customer environments and I know our QA team bent over backwards to try and replicate the problems encountered by, for example, a chap out in the Outback with a sattelite setup that wanted to share his connection with his household while his main system was running Windows ME ...

    You can imagine ...

    But, I've changed my perspective after reading that book. While I can acknowledge that perfection is impossible; there are several mechanisms that developers can employ to not only reduce the risk but also to ensure that if the inevitable happens the system is capable of dealing with it. Yes, it is complicated. Yes, it is difficult.

    But software developers, just like any other professional be that a doctor or a lawyer or whatever, has to have the same professionalism. What happens if a doctor comes back after a week and says: "Hey, you know. I ****** up a bit. I was meant to remove your appendix, but I took your nutsac instead. My bad. It was a bug. Let me patch that for you ..."

    Everything nowadays runs off software. We trust software with almost every aspect of our lives. And getting that level of perfection is ... well. It should be a goal. Complexity should not be used as an excuse.

    That said, software development is still a relatively new discipline. We're inventing better methodologies, structures and designs every day. EVE is going on 7 or 8 years old now in terms of release, the code base must be even older! And while I suspect the code base has been refactored in that timeframe, I suspect there is still a lot of "old" knowledge and structure in there. I'm willing to cut them a lot of slack in terms of bugs because I have not encountered ONE yet in my interaction with EVE Online.

    I still think though that developers (generally) should be the ones that takes responsibility for writing their code using pair programming, test driven development, solid unit testing and so forth to take professional accountability for what they do. To ensure that what goes to QA is ******* bullet proof. And if QA can poke a hole in it, a developer should be mortified.

    Read the book if you're in a development environment. It shouldn't take you more than an evening to work through and it's a good methodology / philosophy piece. I'm taking bits of it and pushing my teams here to start working more that way. It makes our clients a lot happier Lol
    Kolmogorow
    Freedom Resources
    #31 - 2011-12-21 23:02:28 UTC
    CCP must create new huge exciting space features and content nobody has ever seen. Don't waste money and time on excessive quality assurance. Expansion and patch deployment process and quality is - despite occasional flaws - good enough for a game. This is not a software to control airplanes, medical devices or online banking security.
    Seleia O'Sinnor
    Drop of Honey
    #32 - 2011-12-21 23:47:59 UTC  |  Edited by: Seleia O'Sinnor
    Refactoring?
    Unit Testing?
    Acceptanace Testing?
    Integration Testing?
    Reduction of Deployment Cost?
    Continuous Integration?
    Lolwut?

    Odyssey: Repacking in POS hangars for modules +1,  but please for other stuff too, especially containers. Make containers openable in POS hangars.
    Previous page12