These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Information Portal

 
  • Topic is locked indefinitely.
Previous page123
 

Dev Blog: An Update on Bug Reporting, Part 2

First post
Author
CCP Habakuk
C C P
C C P Alliance
#41 - 2013-08-20 12:14:59 UTC
The problems with uploading files and sending bug reports should be fixed now. This was apparently a server configuration issue.

CCP Habakuk | EVE Quality Assurance | Team Five 0 | (Team Gridlock)

Bug reporting | Mass Testing

Cerulean Ice
Royal Amarr Reclamation
#42 - 2013-08-20 12:49:16 UTC
CCP Habakuk wrote:
The problems with uploading files and sending bug reports should be fixed now. This was apparently a server configuration issue.

http://community.eveonline.com/support/submit-bug-report/done/ wrote:
Thank you, your bug report has been submitted


success \o/
Rutger Janssen
Chanuur
The Initiative.
#43 - 2013-08-20 20:21:49 UTC
IG bug report confirmed to be working; managed to report 2 (related) bugs

Once 2 way communication has been established, will we able to see the new defects created from the old defects? I had 14 unfiltered bug reports and dozen more attached to defect; it's not that much fun to check often if they've been fixed; even worse flagged as not able to reproduce and waiting for nothing for you to fix it :(

As for ideas for dev blog 4 out of 3, numbers. Everyone knows eve players LOVE numbers. Number of oustanding bug reports, defects, average time to filter, average time to fixed, oldest bug report to be filtered, oldest defect to be fixed, % of duplicates, % unable to reproduce, % defects later on flagged as unable to reproduce etc.

Possibly compare the results from the old system(when all defects were closed with retribution?) versus the new system.
Terrorfrodo
Interbus Universal
#44 - 2013-08-23 08:41:36 UTC
I hope the new ingame bug reporting tool will be able to attach screenshots saved on the hard drive. The old tool apparently could only attach a screenshot created by the tool itself, at the time of writing the bug report. But often I would encounter a bug while playing and make a screenshot, but not create a bug report until some later time.

.

CCP Goliath
C C P
C C P Alliance
#45 - 2013-08-23 11:41:37 UTC
Rutger Janssen wrote:
IG bug report confirmed to be working; managed to report 2 (related) bugs

Once 2 way communication has been established, will we able to see the new defects created from the old defects? I had 14 unfiltered bug reports and dozen more attached to defect; it's not that much fun to check often if they've been fixed; even worse flagged as not able to reproduce and waiting for nothing for you to fix it :(

As for ideas for dev blog 4 out of 3, numbers. Everyone knows eve players LOVE numbers. Number of oustanding bug reports, defects, average time to filter, average time to fixed, oldest bug report to be filtered, oldest defect to be fixed, % of duplicates, % unable to reproduce, % defects later on flagged as unable to reproduce etc.

Possibly compare the results from the old system(when all defects were closed with retribution?) versus the new system.



You will not be able to see the new defects as they exist in a project that the bugs service does not have access to. Effectively, it is likely that the data you will receive will be a simple answer to the question "has my defect been dealt with". Due to issues we have had in the past with people not accepting decisions such as "cannot reproduce" or "by design", the current intention is not to inform the user of the resolution (as the important part - the bug report getting addressed by a developer - has already occurred at this point). One experiment I would be reasonably interested in conducting in the future (no promises, it's an idea and nothing more) is making a subsection of our defects intended for fixing in an upcoming release publically viewable, and allow users to vote on their priority. In practice this might prove to add nothing though.

The problem with numbers is that one of the issues with our old system (and one of the reasons we moved to Jira) was that it was poor at collecting metrics. Someone would have to bust out some incredible SQL-fu to get the stats above from our old system and frankly their time would be better spent elsewhere! I agree that a numbers blog would have been cool though. I know you guys love a good graph!

CCP Goliath | QA Director | EVE Illuminati | @CCP_Goliath

CCP Goliath
C C P
C C P Alliance
#46 - 2013-08-23 11:42:19 UTC
Terrorfrodo wrote:
I hope the new ingame bug reporting tool will be able to attach screenshots saved on the hard drive. The old tool apparently could only attach a screenshot created by the tool itself, at the time of writing the bug report. But often I would encounter a bug while playing and make a screenshot, but not create a bug report until some later time.


The tool itself has not changed, it has just been modified to work with the new bugs service.

CCP Goliath | QA Director | EVE Illuminati | @CCP_Goliath

Rutger Janssen
Chanuur
The Initiative.
#47 - 2013-08-24 11:25:29 UTC
CCP Goliath wrote:
You will not be able to see the new defects as they exist in a project that the bugs service does not have access to. Effectively, it is likely that the data you will receive will be a simple answer to the question "has my defect been dealt with". Due to issues we have had in the past with people not accepting decisions such as "cannot reproduce" or "by design", the current intention is not to inform the user of the resolution (as the important part - the bug report getting addressed by a developer - has already occurred at this point). One experiment I would be reasonably interested in conducting in the future (no promises, it's an idea and nothing more) is making a subsection of our defects intended for fixing in an upcoming release publically viewable, and allow users to vote on their priority. In practice this might prove to add nothing though.

The problem with numbers is that one of the issues with our old system (and one of the reasons we moved to Jira) was that it was poor at collecting metrics. Someone would have to bust out some incredible SQL-fu to get the stats above from our old system and frankly their time would be better spent elsewhere! I agree that a numbers blog would have been cool though. I know you guys love a good graph!


Thank you very much for the reply. I didn't pay attention about the difference between bug reports and defects for which I apoligize as I know the difference and the 2-layered approach continues to exist.

Although I'm probably alone in this, I would like to raise my concern about the limited feedback. I only reopened a bug report once when it was mentioned it was by design but overall I respected it. When people do not respect it/abuse the system, (temp) ban from bug reporting.

I also plead guilty to "fighting" cannot reproduce. Usually this happened with bug reports and not the defects. But if my memory serves me right, 1 defect was internally flagged as cannot reproduce, I resubmitted and later on it got fixed. I think it was after a reinstall it said the corp assets filter defaults to current region but didn't show the actual assets. This has now been fixed. (Sadly I can't get you a bug report number because that page now redirects to the new page :( )
Then there's that one bug report where I really really really went far because A. It was never made a defect B. GMs say there should have been logs when there weren't. C Senior GM couldn't explain it. D. Materials are lost without a trace E. What happens there with R.A.M.s could probably be done more efficient.

Can't remember the numbers, but the contract window throws an exception when you try to courier contrat a single repackaged plastic wrap. It was flagged as fixed but it isn't. Or it's because I have a plastic wrap pre fix and doesn't contain destination information which new repacked plastic wraps will continue to have or something.

Assuming we do get feedback from QA/BH before it's made a defect (if we don't, the below problems might occur even more often), it we don't get feedback from the defect I see the following happen:
A. Reports claiming that it has not been fixed while it just hasn't been deployed yet.
B. Because of A, it takes longer before people resubmit the report that it has not been fixed yet.
C. Reporter doesn't know if it was by design or couldn't reproduce. So whether it should be resubmitted with more info, it was by design.

So instead, I would advocate even more feedback then in the old system. In the old system, desynchronization could occur between bug report and defect status. CCP Habakuk couldn't promise anything, but hoped that less confusion would occur. With less feedback from dev to user, I would expect only more confusion.


Public issue voting does work to a degree, issues that many people have will get upvoted and fixed sooner. However issues that not many people have, would never get fixed. As evidence, corporation journal, sell OR buy filter (not both!) all wallets only returns the oldest page, I reported that a year ago, still hasn't been fixed. As I've asked before, not only count the number of votes, but also the age of the report. Ofcourse this requires tweaking about the weight, but atleast this way old less used bug reports also get fixed too.


Too bad about the numbers but I understand. And competly agree that devs better be doing other stuff, there's a long backlog of defects and bug reports ;)

Kossaw
Body Count Inc.
Mercenary Coalition
#48 - 2013-09-04 12:01:34 UTC
I know it's frustrating when users don't accept "cannot reproduce" or "by design" but they are important answers that I use all the time. Of course I guess its easier because I generally deal with my users face to face.

"Cannot Reproduce" is important because it shows that you've tried but either need more data or its a rare edge case. Typically (helpfull) users will try to get more data or a better description of the problem.

"By Design" is an important way of opening a discussion of whether the users understanding is wrong or the design is wrong. Typically its a 50/50 case.


But overall, this bug reporting system is starting to look much better than the old one. Keep up the good work guys.

WTB : An image in my signature

Previous page123