These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Information Portal

 
  • Topic is locked indefinitely.
123Next page
 

Dev Blog: An Update on Bug Reporting, Part 2

First post
Author
Previous Topic Next Topic
CCP Falcon
#1 - 2013-08-16 14:00:49 UTC  |  Edited by: CCP Phantom
Hello Capsuleers Big smile

A little while ago, CCP Goliath released a blog that detailed some changes coming to the bug reporting system, and our bug reporting tools.

Now he's back, (from outer space), to show you more about the new reporting system, and to explain to you all how you can assist in the never ending quest to eradicate bugs from EVE.

Read all about it in his latest Dev Blog here!

(EDIT: CCP Goliath here. Here are a list of known issues with the site and IGBR)

• Popup windows appears in Firefox when deleting a file

• There is an double "x" when uploading a file

• Delete a file that has already been uploaded does "not“ work

• File name on the popup window does not appear

• (IGBR) - Submitting a bug report locks up the client for longer than expected

CCP Falcon || EVE Universe Community Manager || @CCP_Falcon

Happy Birthday To FAWLTY7! <3
Mashie Saldana
V0LTA
New Eden Alliance 99013733
#2 - 2013-08-16 14:26:52 UTC  |  Edited by: Mashie Saldana
Is this part 2 of 4?

Improved bug reporting is always handy and the ability to attach multiple files at once is welcome.

To be able to communicate with the BH's in a chat style instead of adding additional comments to the original issue field will improve things a lot.

How to win EVE
Ishtanchuk Fazmarai
#3 - 2013-08-16 14:43:28 UTC  |  Edited by: Ishtanchuk Fazmarai
It's been interesting to learn how QA is performed and the new bug report function...

...but also I've had some fun trying to figure how the new jump animation was QA tested and released in a bug free albeit nausea-inducting state P

I mean, can you figure that some pesky bug prevented it from playing...? What terrible loss to players suffering headaches, migraines or dizziness! Blink

Roses are red / Violets are blue / I am an Alpha / And so it's you
CCP Goliath
C C P
C C P Alliance
#4 - 2013-08-16 14:44:04 UTC
Mashie Saldana wrote:
Is this part 2 of 4?

Improved bug reporting is always handy and the ability to attach multiple files at once is welcome.

To be able to communicate with the BH's in a chat style instead of adding additional comments to the original issue field will improve things a lot.


Part 2 of 3, with the third being "how to write the perfect bug report". If you want me to write more stuff though, happy to do it! Would need topic suggestions.

CCP Goliath | QA Director | EVE Illuminati | @CCP_Goliath
CCP Goliath
C C P
C C P Alliance
#5 - 2013-08-16 14:48:24 UTC
Ishtanchuk Fazmarai wrote:
It's been interesting to learn how QA is performed and the new bug report function...

...but also I've had some fun trying to figure how the new jump animation was QA tested and released in a bug free albeit nausea-inducting state P

I mean, can you figure that some pesky bug prevented it from playing...? What terrible loss to players suffering headaches, migraines or dizziness! Blink


So, there's a difference between functional testing and user experience testing. When testing the jump animation, it would have been tested that jumping worked as expected - you jump, you go through the gate, the animation plays, the sound plays, you land, etc. A possible user experience test case would have been doing a 40 jump run, for instance. This was done, but only after the problem became apparent on TQ. User experience testing is something we're putting a lot of focus on getting better at, and is also why Sisi feedback is so important to us.

CCP Goliath | QA Director | EVE Illuminati | @CCP_Goliath
Darkblad
Doomheim
#6 - 2013-08-16 14:55:49 UTC  |  Edited by: Darkblad
*snip* must. not. post. before. reading. last. paragraph. of. devblog.

And I assume it's intended that bugs reported with the old bug reporting system are still available at https://bugs.eveonline.com/mybugreports.asp ?

NPE – ISD – RIP
CCP Goliath
C C P
C C P Alliance
#7 - 2013-08-16 15:00:09 UTC
Darkblad wrote:
New Bug reporting page looks fine. But where's the list of bug reports I've created there?

And I assume it's intended that bugs reported with the old bug reporting system are still available at https://bugs.eveonline.com/mybugreports.asp ?


Yes and no. The old site will soon be decommissioned so you will no longer be able to access them. Once the 2 way communication/ability to see reported issues is added to the new site, you'll have that functionality back again.

CCP Goliath | QA Director | EVE Illuminati | @CCP_Goliath
Katrina Bekers
A Blessed Bean
Pandemic Horde
#8 - 2013-08-16 15:05:13 UTC
I use Jira extensively at work (IT dept over three worldwide sites).

You have NO idea how many times, talking with colleagues, I call it "Jita"...

<< THE RABBLE BRIGADE >>
Funless Saisima
School of Applied Knowledge
Caldari State
#9 - 2013-08-16 15:40:54 UTC
Quote:
Please also note that getting bug reports via the in game tool is generally more useful to us than getting them via the site, so if you can use the tool, please do so.


So you go through all that work to get the website up to date, then just tell us to use the old in game tool. Ok.
CCP Goliath
C C P
C C P Alliance
#10 - 2013-08-16 15:42:52 UTC
Funless Saisima wrote:
Quote:
Please also note that getting bug reports via the in game tool is generally more useful to us than getting them via the site, so if you can use the tool, please do so.


So you go through all that work to get the website up to date, then just tell us to use the old in game tool. Ok.


Most of the work we've been doing is creating the bugs service that interprets the data from the client and site reporting tools and converts them into Jira issues.

CCP Goliath | QA Director | EVE Illuminati | @CCP_Goliath
Vincent Athena
Photosynth
#11 - 2013-08-16 15:44:48 UTC
CCP Goliath wrote:
Ishtanchuk Fazmarai wrote:
It's been interesting to learn how QA is performed and the new bug report function...

...but also I've had some fun trying to figure how the new jump animation was QA tested and released in a bug free albeit nausea-inducting state P

I mean, can you figure that some pesky bug prevented it from playing...? What terrible loss to players suffering headaches, migraines or dizziness! Blink


So, there's a difference between functional testing and user experience testing. When testing the jump animation, it would have been tested that jumping worked as expected - you jump, you go through the gate, the animation plays, the sound plays, you land, etc. A possible user experience test case would have been doing a 40 jump run, for instance. This was done, but only after the problem became apparent on TQ. User experience testing is something we're putting a lot of focus on getting better at, and is also why Sisi feedback is so important to us.


This last paragraph seems so odd. The motion sickness issue was reported by users of Sisi. You stated that Sisi testing is important. Yet you did not do a user experience test until the issue became apparent on TQ? If you are going to wait for it to hit TQ, why is Sisi testing important? If Sisi testing is important, why did you not do the user experience test when the issue was reported on Sisi?

Also the motion sickness issue is very user specific. I do not get it. To test it you would need a susceptible person on the testing team. If you do not have such a person you need to trust your customers are actually right. And you must trust them sufficiently to actually change what you put in the game.

Know a Frozen fan? Check this out

Frozen fanfiction
Ishtanchuk Fazmarai
#12 - 2013-08-16 16:38:45 UTC  |  Edited by: Ishtanchuk Fazmarai
CCP Goliath wrote:
Ishtanchuk Fazmarai wrote:
It's been interesting to learn how QA is performed and the new bug report function...

...but also I've had some fun trying to figure how the new jump animation was QA tested and released in a bug free albeit nausea-inducting state P

I mean, can you figure that some pesky bug prevented it from playing...? What terrible loss to players suffering headaches, migraines or dizziness! Blink


So, there's a difference between functional testing and user experience testing. When testing the jump animation, it would have been tested that jumping worked as expected - you jump, you go through the gate, the animation plays, the sound plays, you land, etc. A possible user experience test case would have been doing a 40 jump run, for instance. This was done, but only after the problem became apparent on TQ. User experience testing is something we're putting a lot of focus on getting better at, and is also why Sisi feedback is so important to us.


Well, the issue was forwarded as soon as the animation hit Sisi, and nothing happened i.e. the animation was released the same, as if at that stage, what mattered was funcionality (it works in user systems) and not usability (users actually can use it). That is between unfortunate and outrageous depending on who you ask, but also points that it may be difficult to use-test things in a way that allows to do something with user feedback.

This hasn't been the first nor the worst nor likely the last feature that's been delivered in a perfectly functional yet unacceptably unusable state, but on the other hand, it's also evident that the bug hunting process is quite more effective than it used to be. I haven't run into any bug for months (or maybe I'm not paying enough attention, heh) so I would say, keep up the good work against bugs, but also give a look at what players will experience when they use any given bug-free feature.

Because sometimes animations are too abrupt and loot scatter boxes are too small and move too fast, at least to some users/players.

Roses are red / Violets are blue / I am an Alpha / And so it's you
Logix42
Taxation Damnation
#13 - 2013-08-16 18:18:20 UTC
Good to hear about the regression testing being put in place. Automation is win!

Also, very happy that bug reporting is back, time to report ALL THE BUGS

Go beyond the edge of space... Explore
Utremi Fasolasi
La Dolce Vita
#14 - 2013-08-16 19:43:57 UTC
CCP Goliath wrote:
A possible user experience test case would have been doing a 40 jump run, for instance. This was done, but only after the problem became apparent on TQ. User experience testing is something we're putting a lot of focus on getting better at, and is also why Sisi feedback is so important to us.


But.. we gave you tons of that feedback as soon as the jump animation hit Sisi - pages and pages in the Test Server forum :-(

It was fixed relatively promptly but it couldn't have been a surprise reaction when it went to TQ in that state.
Ines Tegator
Serious Business Inc. Ltd. LLC. etc.
#15 - 2013-08-16 20:30:59 UTC  |  Edited by: Ines Tegator
Yeah, sisi feedback seems to be generally ignored. I follow ship changes fairly regularly. It's pretty much accepted that once a balance change hits sisi, it's final. It gets really obnoxious, when you read entire threads about how X ship is Y broken on sisi (anciallary shield boosters are one change that stands out to my memory) and then CCP seems to be surprised when it causes problems when released and then rushes to do a hotpatch. I kind of feel like Sisi is a "preview server" and not really for testing.

I suppose you guys use it to test bugs and not so much balance/usability/feature stuff- but those things need to be included and prioritized. Above minor bugs, imo, since these things affect player experience a lot more then something small like item stacks failing to auto-split to fit the container. It's not cool to have the battlefield meta turned on it's head literally overnight (patch night), and then turned again two weeks later. It happens with feature things too (unified inventory anone?). It just reeks of sloppiness.

To end on a positive note, most patches have been relatively bug free for the past few expansions or so. Used to be I wouldn't even log on until a week after patch day cause I knew everything would be broken :) Props to the bug hunter team.

Overhaul Dscan!
Kossaw
Body Count Inc.
Mercenary Coalition
#16 - 2013-08-16 22:56:22 UTC
I find it more than a little ironic that the example "Bug" you show in the screenshots of your new tool is in fact a particularly nasty and ******** ~feature~ that has dogged Gallente stations for an exceptionally long time. This issue makes it virtually impossible to undock any large ship from a Gallente station and do anything except warp straight ahead ( if you're quick enough ) or jump ( if you have a jump drive )

It would be nice to know that you have actually formally identified this as a bug now, and will actually be working on getting it fixed .

WTB : An image in my signature
Narcil
Viper-Squad
Pandemic Horde
#17 - 2013-08-17 01:01:57 UTC
I hope this means the gallente administrative outposts are getting fixed!
Salpun
Global Telstar Federation Offices
Masters of Flying Objects
#18 - 2013-08-17 05:19:16 UTC
Any word when the in game bug reporting tool will be turned back on?

If i dont know something about EVE. I check https://wiki.eveonline.com/en/wiki/ISK_The_Guide

See you around the universe.
TheSmokingHertog
Julia's Interstellar Trade Emperium
#19 - 2013-08-17 06:10:37 UTC
Quote:
\sarcasm mode

In before...

*Hoping all those test fases create the same result as the launch of the UNI INV*

/end sarcasm mode


Keep it up, we like a great game, that has no hickups :)

"Dogma is kind of like quantum physics, observing the dogma state will change it." ~ CCP Prism X

"Schrödinger's Missile. I dig it." ~ Makari Aeron

-= "Brain in a Box on Singularity" - April 2015 =-
CCP Goliath
C C P
C C P Alliance
#20 - 2013-08-17 10:01:08 UTC
Kossaw wrote:
I find it more than a little ironic that the example "Bug" you show in the screenshots of your new tool is in fact a particularly nasty and ******** ~feature~ that has dogged Gallente stations for an exceptionally long time. This issue makes it virtually impossible to undock any large ship from a Gallente station and do anything except warp straight ahead ( if you're quick enough ) or jump ( if you have a jump drive )

It would be nice to know that you have actually formally identified this as a bug now, and will actually be working on getting it fixed .


I think it's safe to say that if we are using it as an example of a defect, we consider it a defect. It's assigned to a developer and your very own CSM member Sala is on my case about it frequently :). Haven't spoken wih the developer personally, but I imagine it will be a winter fix,

CCP Goliath | QA Director | EVE Illuminati | @CCP_Goliath
123Next page