These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

Upcoming Feature and Change Feedback Center

 
  • Topic is locked indefinitely.
123Next pageLast page
 

Project Discovery - Changes to the scoring system

First post First post
Author
CCP Wonderboy
C C P
C C P Alliance
#1 - 2016-03-21 11:18:58 UTC  |  Edited by: CCP Logibro
Hello scientists!

The Project Discovery team is extremely satisfied and impressed with Capsuleer interest for this scientific endeavor, you have been doing a great job so far!

However, in order to reduce the exploitability of the feature, we, in collaboration with MMOS and the HPA, have decided to stop calculating Accuracy Rating for the community consensus. Instead, we will increase the rate of training examples you get, for which you get an immediate rating.

This is only the first step in our attempt to improve Project Discovery and it's data quality, as we might introduce further changes to our scoring procedure. This change will also significantly reduce the heavy load on the MMOS servers and should ensure a smoother experience for everyone.

Feel free to leave any feedback and suggestions here, and may the cytoplasm be with you.

- CCP Wonderboy

EDIT: This change should roll out at 15:00 UTC, keep in mind this is only a change to the MMOS API so we expect no downtime. The Project Discovery client will be updated after Easter vacations.
Ralph King-Griffin
Lords.Of.Midnight
The Devil's Warrior Alliance
#2 - 2016-03-21 11:24:54 UTC
thank you for this change,
now begone with you ye cytoplasm whoures!
Yadaryon Vondawn
Vicanthya
#3 - 2016-03-21 11:53:26 UTC
Excellent change! Could you also take a look at the presentation of example/training samples? In the in-game channel we noticed that for some reason every single control image is now Nucleus or Nucleoli. This skews accuracy heavily because they are easily recognized. They seem to be mostly in the 300-500 range.

This is also tied to another request which I am sure Blink you already have on the table, remove the sample number in the lower right corner. Only show it after one presses submit because people now know that they are control images and will thus vote in a specific way.

Also since this morning the PD servers have been stable as anything. No problems whatsoever.

Keep up the good work o7
CCP Wonderboy
C C P
C C P Alliance
#4 - 2016-03-21 11:58:23 UTC
Yadaryon Vondawn wrote:
Excellent change! Could you also take a look at the presentation of example/training samples? In the in-game channel we noticed that for some reason every single control image is now Nucleus or Nucleoli. This skews accuracy heavily because they are easily recognized. They seem to be mostly in the 300-500 range.


This is for MMOS to decide, i'll forward this to them to see if they can mix things up a little.

Yadaryon Vondawn wrote:
This is also tied to another request which I am sure Blink you already have on the table, remove the sample number in the lower right corner. Only show it after one presses submit because people now know that they are control images and will thus vote in a specific way.


This is already implemented and should roll out soon(tm).


Mynxee
Signal Cartel
EvE-Scout Enclave
#5 - 2016-03-21 13:29:41 UTC
Good changes! <3

Lost in space, looking for sigs...

Blog: Cloaky Wanderer

Lasse R Farnsworth
Blue Republic
RvB - BLUE Republic
#6 - 2016-03-21 17:26:59 UTC
+1 :) Nice to see that stuff going in the game. Actually having such a fast response to reports is a nice change of pace.
Circumstantial Evidence
#7 - 2016-03-21 19:29:04 UTC
Keep up the good works! And please keep watching the "March Update feedback thread" where numerous suggestions have been made since the feature became available to all players.
Joia Crenca
Science and Trade Institute
Caldari State
#8 - 2016-03-21 23:46:53 UTC
Thank you for this change! And thanks for letting us be a part of this project! Now... about future rewards... my favorite ship doesn't look very much like a promised pony...
Nevyn Auscent
Broke Sauce
#9 - 2016-03-22 00:46:36 UTC
Some people seem to still be getting their rating affected by community consensus.
Are old slides still going to affect rating as they get processed?
Beta Maoye
#10 - 2016-03-22 02:27:01 UTC
Good to stop calculating accuracy rating for the community consensus.

Will I still see the percentages of community consensus? I think knowing what other people's choices is still an interesting feedback to players.
Barrogh Habalu
Imperial Shipment
Amarr Empire
#11 - 2016-03-22 09:05:06 UTC
Good change. Not only because of intended effects, but also because sometimes you really need a few more training examples to check yourself and filter out misconceptions.

I also hole that some of the samples will be reviewed by HPA in the future, re-calculating accuracy in the process and enabling better sorting of out results.
Keta Min
Delicious Entropy
#12 - 2016-03-22 09:14:07 UTC
CCP Wonderboy wrote:

However, in order to reduce the exploitability of the feature, we, in collaboration with MMOS and the HPA, have decided to stop calculating Accuracy Rating for the community consensus.


This is a problematic change. Now the people who are trying to game the system have no incentive to even try to be correct on the samples that don't affect accuracy. Instead of clicking cytoplasm they can now basically click anything at all.
Vardec Crom
The Harpooner's Rest
#13 - 2016-03-22 23:58:38 UTC
Keta Min wrote:
CCP Wonderboy wrote:

However, in order to reduce the exploitability of the feature, we, in collaboration with MMOS and the HPA, have decided to stop calculating Accuracy Rating for the community consensus.


This is a problematic change. Now the people who are trying to game the system have no incentive to even try to be correct on the samples that don't affect accuracy. Instead of clicking cytoplasm they can now basically click anything at all.


How will they know which ones don't affect accuracy?
Nevyn Auscent
Broke Sauce
#14 - 2016-03-23 00:33:04 UTC
CCP Wonderboy wrote:
Hello scientists!

The Project Discovery team is extremely satisfied and impressed with Capsuleer interest for this scientific endeavor, you have been doing a great job so far!

However, in order to reduce the exploitability of the feature, we, in collaboration with MMOS and the HPA, have decided to stop calculating Accuracy Rating for the community consensus. Instead, we will increase the rate of training examples you get, for which you get an immediate rating.

This is only the first step in our attempt to improve Project Discovery and it's data quality, as we might introduce further changes to our scoring procedure. This change will also significantly reduce the heavy load on the MMOS servers and should ensure a smoother experience for everyone.

Feel free to leave any feedback and suggestions here, and may the cytoplasm be with you.

- CCP Wonderboy

Something is going funky with this Wonderboy, since I'm showing rating loss still happening despite getting every test sample right. Can you please check you haven't coded every non test sample with a small downwards rating or something, because that's exactly what I'm currently seeing, and it's rather disheartening.
Keta Min
Delicious Entropy
#15 - 2016-03-23 10:01:37 UTC
Vardec Crom wrote:
Keta Min wrote:
CCP Wonderboy wrote:

However, in order to reduce the exploitability of the feature, we, in collaboration with MMOS and the HPA, have decided to stop calculating Accuracy Rating for the community consensus.


This is a problematic change. Now the people who are trying to game the system have no incentive to even try to be correct on the samples that don't affect accuracy. Instead of clicking cytoplasm they can now basically click anything at all.


How will they know which ones don't affect accuracy?

Right now the sample number is a hint. Apart from that, the control samples tend to be really clear and obvious, as opposed to the vast majority of the normal samples.
Yadaryon Vondawn
Vicanthya
#16 - 2016-03-23 11:48:13 UTC  |  Edited by: Yadaryon Vondawn
The bug that gave only Nucleoli and Nucleus control samples seem to be fixed now :)

This bugs me out as well Sad
ACESsiggy
The Scope
#17 - 2016-03-23 13:50:20 UTC
Be cool if the images loaded without having to wait 1 minute and the transmission errors for submission. Hopefully this update addressed these problems as well.

“The open-minded see the truth in different things: the narrow-minded see only the differences.”

CCP Wonderboy
C C P
C C P Alliance
#18 - 2016-03-23 13:55:06 UTC
ACESsiggy wrote:
Be cool if the images loaded without having to wait 1 minute and the transmission errors for submission. Hopefully this update addressed these problems as well.


We should see significant performance increases with this next release. :)
ACESsiggy
The Scope
#19 - 2016-03-23 14:22:15 UTC  |  Edited by: ACESsiggy
CCP Wonderboy wrote:
ACESsiggy wrote:
Be cool if the images loaded without having to wait 1 minute and the transmission errors for submission. Hopefully this update addressed these problems as well.


We should see significant performance increases with this next release. :)


Sweetness... I can finally work toward getting that combat armor suit :)

“The open-minded see the truth in different things: the narrow-minded see only the differences.”

Murkar Omaristos
The Alabaster Albatross
Unreasonable Bastards
#20 - 2016-03-23 16:53:02 UTC
CCP Wonderboy wrote:
Hello scientists!
Feel free to leave any feedback and suggestions here, and may the cytoplasm be with you.


omg. may the cytoplasm be with youShocked
123Next pageLast page