These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

Test Server Feedback

 
  • Topic is locked indefinitely.
Previous page123Next page
 

A/B Testing framework on Singularity

First post
Author
Previous Topic Next Topic
mkint
#21 - 2014-03-31 22:28:49 UTC  |  Edited by: mkint
Um... If I remember my history correctly, this was done once before several years ago with the neocom, except the users had control of if they wanted to take part or not. And if I recall, 80% of the feedback was ignored anyway.

This version sounds like an excuse to force unwilling players to beta test for you. I wonder if this is necessary because of how sisilauncher.exe was killed in order to add the launcher.exe in spite of universal negative feedback about ever part of the process.

edit: ah, I get it... it's to reduce the amount of feedback on a feature, so you can blame us for not giving you feedback when said feature is awful.

Maxim 6. If violence wasn’t your last resort, you failed to resort to enough of it.
CCP Explorer
C C P
C C P Alliance
#22 - 2014-04-01 09:00:23 UTC
Salpun wrote:
CCP Explorer wrote:
To explain a bit more how this all works:

Right now we are testing three things on SiSi: We are testing the new tooltips, we are testing the A/B-testing framework, and we are making sure the old tooltips haven't regressed given the above development/changes. For this testing the users on SiSi were split 50/50 into the new tooltips and the old tooltips.

On TQ we will deploy this in at least three phases: First phase will be a "dark" deployment in which everyone but selected CCP devs will have the old tooltips. Second phase will be the experiment where a certain ratio of new players will get the new tooltips. Third phase will be a general rollout to all players (based on positive results from the experiment). It's during the third phase we will do the normal round of devblogs, etc.
Any idea how long this might take?
Not at this time, no. Guessing this might roll out to everyone in the next 1-2 months, but not going to make any promises.

Erlendur S. Thorsteinsson | Senior Development Director | EVE Online // CCP Games | @CCP_Explorer
Salpun
Global Telstar Federation Offices
Masters of Flying Objects
#23 - 2014-04-01 09:57:30 UTC
CCP Explorer wrote:
Salpun wrote:
CCP Explorer wrote:
To explain a bit more how this all works:

Right now we are testing three things on SiSi: We are testing the new tooltips, we are testing the A/B-testing framework, and we are making sure the old tooltips haven't regressed given the above development/changes. For this testing the users on SiSi were split 50/50 into the new tooltips and the old tooltips.

On TQ we will deploy this in at least three phases: First phase will be a "dark" deployment in which everyone but selected CCP devs will have the old tooltips. Second phase will be the experiment where a certain ratio of new players will get the new tooltips. Third phase will be a general rollout to all players (based on positive results from the experiment). It's during the third phase we will do the normal round of devblogs, etc.
Any idea how long this might take?
Not at this time, no. Guessing this might roll out to everyone in the next 1-2 months, but not going to make any promises.

While i can see the need for a slow role out and that the sample size needs to be large for the new player retention data point. Seems like a waste of a good tool to test stuff on Sisi at a larger scale while still able to use TQ candidate builds. How many A/B cases can the new system support?

If i dont know something about EVE. I check https://wiki.eveonline.com/en/wiki/ISK_The_Guide

See you around the universe.
Kethry Avenger
Ministry of War
Amarr Empire
#24 - 2014-04-01 12:07:09 UTC
CCP Explorer wrote:
To explain a bit more how this all works:

Right now we are testing three things on SiSi: We are testing the new tooltips, we are testing the A/B-testing framework, and we are making sure the old tooltips haven't regressed given the above development/changes. For this testing the users on SiSi were split 50/50 into the new tooltips and the old tooltips.

On TQ we will deploy this in at least three phases: First phase will be a "dark" deployment in which everyone but selected CCP devs will have the old tooltips. Second phase will be the experiment where a certain ratio of new players will get the new tooltips. Third phase will be a general rollout to all players (based on positive results from the experiment). It's during the third phase we will do the normal round of devblogs, etc.


So your going to change if for some people without them knowing there was a change first?
CCP Explorer
C C P
C C P Alliance
#25 - 2014-04-01 12:50:44 UTC
mkint wrote:
Um... If I remember my history correctly, this was done once before several years ago with the neocom, except the users had control of if they wanted to take part or not. And if I recall, 80% of the feedback was ignored anyway.

This version sounds like an excuse to force unwilling players to beta test for you. I wonder if this is necessary because of how sisilauncher.exe was killed in order to add the launcher.exe in spite of universal negative feedback about ever part of the process.

edit: ah, I get it... it's to reduce the amount of feedback on a feature, so you can blame us for not giving you feedback when said feature is awful.
Kethry Avenger wrote:
So your going to change if for some people without them knowing there was a change first?

Please read http://en.wikipedia.org/wiki/A/B_testing

Erlendur S. Thorsteinsson | Senior Development Director | EVE Online // CCP Games | @CCP_Explorer
Salpun
Global Telstar Federation Offices
Masters of Flying Objects
#26 - 2014-04-01 13:07:50 UTC
CCP Explorer wrote:
mkint wrote:
Um... If I remember my history correctly, this was done once before several years ago with the neocom, except the users had control of if they wanted to take part or not. And if I recall, 80% of the feedback was ignored anyway.

This version sounds like an excuse to force unwilling players to beta test for you. I wonder if this is necessary because of how sisilauncher.exe was killed in order to add the launcher.exe in spite of universal negative feedback about ever part of the process.

edit: ah, I get it... it's to reduce the amount of feedback on a feature, so you can blame us for not giving you feedback when said feature is awful.
Kethry Avenger wrote:
So your going to change if for some people without them knowing there was a change first?

Please read http://en.wikipedia.org/wiki/A/B_testing

Great link.

This topic is another reason why devs need photo privileges on the forums so you can show us the changes for such a small thing like this while still doing your new player testing. Dust forums have it Ugh

If i dont know something about EVE. I check https://wiki.eveonline.com/en/wiki/ISK_The_Guide

See you around the universe.
Merida DunBrogh
Black Screen Of Raging Defeat
#27 - 2014-04-05 22:21:14 UTC
Feedback point:
In the new Capacitor Overview you get to see the number in a constant update, which is awesome.
However, the functionality to see the % of your cap was also gotten rid of(At least on hovering) which is a bad thing because due to the capacitor fo
CCP Habakuk
C C P
C C P Alliance
#28 - 2014-04-05 23:06:03 UTC
Merida DunBrogh wrote:
Feedback point:
In the new Capacitor Overview you get to see the number in a constant update, which is awesome.
However, the functionality to see the % of your cap was also gotten rid of(At least on hovering) which is a bad thing because due to the capacitor fo


I am pretty sure, that this was already addressed, but the current version on Singularity is missing some changes to this feature.

CCP Habakuk | EVE Quality Assurance | Team Five 0 | (Team Gridlock)

Bug reporting | Mass Testing
Merida DunBrogh
Black Screen Of Raging Defeat
#29 - 2014-04-06 00:27:23 UTC
CCP Habakuk wrote:
Merida DunBrogh wrote:
Feedback point:
In the new Capacitor Overview you get to see the number in a constant update, which is awesome.
However, the functionality to see the % of your cap was also gotten rid of(At least on hovering) which is a bad thing because due to the capacitor fo


I am pretty sure, that this was already addressed, but the current version on Singularity is missing some changes to this feature.


Good to know!
Sorry for the half-posted reply there...Somehow something went wrong somewhere... Oh well, it is 2 AM for a reason :D
elitatwo
Zansha Expansion
#30 - 2014-04-19 17:45:45 UTC
Is it possible that I was 'chosen' and looking at the new UI stuff?

Eve Minions is recruiting.

This is the law of ship progression!

Aura sound-clips: Aura forever
Salpun
Global Telstar Federation Offices
Masters of Flying Objects
#31 - 2014-04-19 17:48:44 UTC
elitatwo wrote:
Is it possible that I was 'chosen' and looking at the new UI stuff?

If you see the new stuff you where chosenBig smile

If i dont know something about EVE. I check https://wiki.eveonline.com/en/wiki/ISK_The_Guide

See you around the universe.
elitatwo
Zansha Expansion
#32 - 2014-04-19 17:52:54 UTC
Salpun wrote:
elitatwo wrote:
Is it possible that I was 'chosen' and looking at the new UI stuff?

If you see the new stuff you where chosenBig smile


Hey thanks!

It appears to be so and I like them. If I had a say in this, ship it Lol

Eve Minions is recruiting.

This is the law of ship progression!

Aura sound-clips: Aura forever
Salpun
Global Telstar Federation Offices
Masters of Flying Objects
#33 - 2014-04-19 17:56:10 UTC
elitatwo wrote:
Salpun wrote:
elitatwo wrote:
Is it possible that I was 'chosen' and looking at the new UI stuff?

If you see the new stuff you where chosenBig smile


Hey thanks!

It appears to be so and I like them. If I had a say in this, ship it Lol

It got shiped to TQ a couple weeks ago but only some new players can use them.

If i dont know something about EVE. I check https://wiki.eveonline.com/en/wiki/ISK_The_Guide

See you around the universe.
elitatwo
Zansha Expansion
#34 - 2014-04-19 17:59:42 UTC
Oh, my bad!

I have to confess that I haven't been on TQ since my last skillchange. Nevertheless they should make it official and give the changes to everyone.

Eve Minions is recruiting.

This is the law of ship progression!

Aura sound-clips: Aura forever
Salpun
Global Telstar Federation Offices
Masters of Flying Objects
#35 - 2014-04-19 18:03:27 UTC
elitatwo wrote:
Oh, my bad!

I have to confess that I haven't been on TQ since my last skillchange. Nevertheless they should make it official and give the changes to everyone.

They will but they want to see if it helps retain new players better then the old ones. That takes a longer time to get a good sampling.

If i dont know something about EVE. I check https://wiki.eveonline.com/en/wiki/ISK_The_Guide

See you around the universe.
Alvatore DiMarco
Capricious Endeavours Ltd
#36 - 2014-04-19 18:40:18 UTC
Salpun wrote:
elitatwo wrote:
Oh, my bad!

I have to confess that I haven't been on TQ since my last skillchange. Nevertheless they should make it official and give the changes to everyone.

They will but they want to see if it helps retain new players better then the old ones. That takes a longer time to get a good sampling.


If only there were a way for the rest of us who are already "retained" to "opt-in" to the new stuff.
Marketing Chairman Stalin
Space Marketing Department
#37 - 2014-04-23 12:33:53 UTC
I like this

one of my Aeons has it but my Revenant doesn't

Pls amend
gawrshmapooo
J33 Monocombine
#38 - 2014-04-23 22:30:44 UTC
CCP Explorer wrote:
To explain a bit more how this all works:

Right now we are testing three things on SiSi: We are testing the new tooltips, we are testing the A/B-testing framework, and we are making sure the old tooltips haven't regressed given the above development/changes. For this testing the users on SiSi were split 50/50 into the new tooltips and the old tooltips.

On TQ we will deploy this in at least three phases: First phase will be a "dark" deployment in which everyone but selected CCP devs will have the old tooltips. Second phase will be the experiment where a certain ratio of new players will get the new tooltips. Third phase will be a general rollout to all players (based on positive results from the experiment). It's during the third phase we will do the normal round of devblogs, etc.


Sneeaaaaaaaakyyyyyyyy.

I love it!

I sell combat boosters of every strength and type. Message me to get your edge.
James Amril-Kesh
Viziam
Amarr Empire
#39 - 2014-05-14 05:41:07 UTC
Congratulations.
This moronic testing scheme led to your current tooltip fiasco.
I hope you're happy.

Enjoying the rain today? ;)
Schmata Bastanold
In Boobiez We Trust
#40 - 2014-05-14 07:12:47 UTC
I'm sure Kil2 would love new tooltips...
Shame he doesn't play this game anymore.

Invalid signature format
Previous page123Next page