These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

Test Server Feedback

 
  • Topic is locked indefinitely.
123Next pageLast page
 

Project Discovery is now active on Singularity

First post First post
Author
CCP Paradox
#1 - 2016-01-22 18:15:37 UTC  |  Edited by: CCP Paradox
Hello fellow Scientists,

We have now enabled Project Discovery on Singularity.

This is a very interesting mini game where the goal is to classify images of immunofluorescently stained cells for the benefit of The Human Protein Atlas (HPA), which is a Swedish research group dedicated to providing a database for the scientific community of the world. Since these images cannot yet be classified with the help of computers, human interaction is needed, or in this case, capsuleer interaction. With your help, the HPA can improve their database which goes a long way towards understanding the various diseases of the world. The Sisters of EVE will reward you for each submission with LP, ISK and XP within Project Discovery, contributing to your rank.


We are now looking for feedback on it within EVE. You will find it accessible under the Neocom -> Business section, as Project Discovery.
There is also a survey to take part in, which you can find here https://www.surveymonkey.com/r/H6Z9ZRN, and we encourage you to fill this in after using the feature on Singularity.

A mass test for Singularity will also be held next week on Thursday 28th January at 17:00, which will include using Project Discovery. More details will be created next week in a new forum thread.

You can find out more about Project Discovery at the website here.
And also from this latest Dev Blog: Project Discovery needs you!

Thank you,
Team Astro Sparkle
http://mmos.ch
http://www.proteinatlas.org/
http://en.ru.is/

CCP Paradox | EVE QA | Team Phenomenon

Space Magician

Jimy F
Aliastra
Gallente Federation
#2 - 2016-01-22 18:23:06 UTC
First ^_^
Lasse R Farnsworth
Blue Republic
RvB - BLUE Republic
#3 - 2016-01-22 18:41:23 UTC
Just as a feedback. I really like all the new features, but you guys really should make a option to custimise the contrast / colorsheme. I'm not colorblind but some of the contrast / colors would make it hard for some persons. And actually for looking at pictures you might want to calibrate the program to your monitor etc ...
Crazy Kitten
The Fourth Great and Bountiful Human Empire
SONS of BANE
#4 - 2016-01-22 19:41:48 UTC  |  Edited by: Crazy Kitten
need more examples to compare for the different options, to get a better impression of whether or not an option can actually apply. only one picture for the option doesn't suffice :/

also, regarding the history: it'd be nice to be able to actually see the approved result when it's in, so i can better learn what i did wrong.

and finally, it really needs a "no clue, next!" button. no rewards, just an option to not knowingly submit bad data :/


edit: almost forgot: needs to be resizable, and more zoom options. maybe also take the "overview" outside the zoom view, to provide more visible space
Sgt Ocker
What Corp is it
#5 - 2016-01-22 19:46:01 UTC
Love Protien Discovery. Can't wait for it to go live and collect all the triple click free LP.

2k LP in 5 mins just by random clicking (42.6% accurate - without looking at what I was clicking) - risk free LP from SOE
(Get accuracy up and it pays even better but does require some interaction)
This will get abused to the end of the universe and back again.

thanks you CCP, it will be nice to earn LP while sitting in station on TS or catching up on Reddit (no need to look at what your doing), just click 3 times and get paid.

My opinions are mine.

  If you don't like them or disagree with me that's OK.- - - - - - Just don't bother Hating - I don't care

It really is getting harder and harder to justify $23 a month for each sub.

Nitchiu
KarmaFleet
Goonswarm Federation
#6 - 2016-01-22 20:04:28 UTC
I'm finding lack of examples and lack of feed back to be making the whole experience feel like I should stop trying and just click on random stuff since it doesn't seem to affect my accuracy in any real fashion.

Would be nice if we got a much larger sample of confirmed in the earlier levels. Right now I feel as if I'm punching blind. I think I did it right but since I have no feedback about if I'm making the right decisions.
The % of other people that clicked the same as you is not a good indication in part because of small sample size but the few that confirmed samples that I've done I find that it seems to be something completely different to what the majority of people have ticked.
Murtac
Brave Newbies Inc.
Brave Collective
#7 - 2016-01-22 20:21:33 UTC
Please remove the artificial delay while "Transmitting Analysis xx%". The chosen options fit into four bytes, there's no reason why transmitting them should take so long.

Make the window re-sizable and zoom the content.

And the tutorial needs A LOT more love.

Scotsman Howard
S0utherN Comfort
Test Alliance Please Ignore
#8 - 2016-01-22 20:44:10 UTC
Can you confirm or not whether the results from Singularity will actually be used in the research project?

I ask because you come to us on the singularity server to essentially try to break things and work out the bugs. However, if we are breaking things in this case (by randomly clicking to see how close we can get with the least amount of work), that would be very bad for the intended goals.

If I am able to get a 40-50% just by randomly clicking in a short amount of time, you know it will happen on the main server.

My advise would be to set the payout structure to be something like this:

0-75% - 0 LP
75-80% - 10LP
80-85% - 50LP
85-90% - 150LP
90-95% - 250LP
96-99% - 450LP
100% - 500LP

While the actual ranges and payouts could be adjusted, I think you can see that this method rewards those who actually try and get good at this. This would help discourage people from randomly clicking and ruining SOE LP.
Sven Viko VIkolander
Allemannsrett
#9 - 2016-01-22 20:49:34 UTC  |  Edited by: Sven Viko VIkolander
I've been on the test server addicted to Project Discovery for the past hour. I love it. I am finding it tough to get an accuracy rating over 50% and am particularly confused when two (or more?) options are selected. Maybe the tutorial could explain when/why you'd want to pick multiple options?

My feedback so far is primarily that the LP rewards are WAY, WAY too high. I am getting 250 LP per submission for 50% accuracy (which is equal to a coin flip, or not at all statistically useful). I think you should ONLY get LP above a tiny amount (10 or less) if you are getting above 50% accuracy. I agree with the LP payout suggested already in this thread:

Scotsman Howard wrote:

My advise would be to set the payout structure to be something like this:

0-75% - 0 LP
75-80% - 10LP
80-85% - 50LP
85-90% - 150LP
90-95% - 250LP
96-99% - 450LP
100% - 500LP



I agree with this payout structure, except maybe that 0-50% accuracy should still get a tiny amount of LP (between 1-10), and 50-75% should get a little more generous of an amount, such as 20-100 LP.


My second piece of feedback (which has also been suggested by others) is to have a skip button for when you really don't know.
Yadaryon Vondawn
YVC Inc
#10 - 2016-01-22 20:59:46 UTC
Most of the stuff is already said but I wanted to give my feedback nonetheless point by point:


  • People could easily skew the average accuracy by just deciding that every image is a....negative for example. Take a look at this image. It is very clear to anyone that played this game for over 15 minutes that that is NOT a cytoplasm. Most likely it is a negative but 93% voted for cytoplasm anyway. This can be abused and needs to be taken a careful look at. Lower accuracy should (at the start) not be punishing but later on it needs to be, lest it be abused.
  • Tie rewards to leveling up a rank. Like an additional LP reward or a special item like an implant, this will incentive people to play.
  • Do not give SoE LP, this (if implemented correctly) will be the new favourite pasttime of everyone waiting for a fleet or something. The SoE LP market will be flooded right away. I think it is better to give CONCORD LP or heavily reduce the SoE payout and up the ISK payout
  • Make the tutorial better and longer by giving more and more clear examples. Currently there are only a few images before one is thrown in. It took me some time before I got a clear understanding of how it worked and that was not thanks to the tutorial! The tutorial is the HOOK which will leer people in to this game. It has a huge potential RL impact and creates a pasttime activity. Making sure that people understand how to play it is imperative, I feel.
  • Allow players to invite other players to the image they are working on, this creates more interactive play and it probably will form groups. People can help each other which creates bonds, friendships and more of that good stuff. For that matter: a common dedicated chatchannel might be in place.
  • Resizable windows

  • For the rest I can say it is a very nice addition, one I will use very much enjoy and use. A little tweaking, especially to the tutorial and the accuracy/payout tie and bring it on :)


Lugh Crow-Slave
#11 - 2016-01-22 21:27:20 UTC  |  Edited by: Lugh Crow-Slave
with the small number of people the acuuracy is skewed i dont think we can test this all that well untill it goes live



one thing i would like is some form of Pass like i can clearly see there is a pattern but i cant say what it is i would rather not be hurting the project by just guessing

also more examples are needed as well as examples of what something is not as many things look close but have a bit of a difference

showing examples of that will help people when deciding what is most likely what
Darkblad
Doomheim
#12 - 2016-01-22 21:58:42 UTC
Add an Option to either adjust or mute the constant background sound that plays when the window is open.

NPEISDRIP

Starbuilder Stasarik
Doomheim
#13 - 2016-01-22 22:23:00 UTC
Having other options highlighted after Submitting on the tutorial is a bit confusing.

Examples:

http://puu.sh/mFNTg/39df5f351c.jpg and http://puu.sh/mFMRG/e8f3a722ec.jpg. Is there a reason for that, or is that a bug?
Destiny Dain2
Your Destiny Corporation
#14 - 2016-01-22 22:26:30 UTC
Got to head out, but I will check this out later. This should be interesting :)
Murtac
Brave Newbies Inc.
Brave Collective
#15 - 2016-01-22 22:58:42 UTC
Starbuilder Stasarik wrote:
Having other options highlighted after Submitting on the tutorial is a bit confusing.


The other option is highlighted because you should have chosen it (you can choose more than one option).
Un Lapin
Deep Core Mining Inc.
Caldari State
#16 - 2016-01-22 23:32:09 UTC  |  Edited by: Un Lapin
Okay so I spent a while on this actually TRYING to get the correct answer. It became clear that there was a disproportionate number of players purposely random clicking in order to see how far their could get. Some ideas from my experience so far:

1) The tutorial needs to be longer:
- Variations of the same formation type and to cover ALL of the options.
- Shouldn't be able to leave the tutorial until you've passed a threshold of accuracy.
- Given the length of the tutorial, a reasonable reward should be given at the end

2) A practice option:
- Rather than revisit the tutorial, include an option to practice against a database of samples which have already been solved.
- This wouldn't affect your accuracy rating.
- Reduce reward but enough to incentivize people to perfect the skill.

3) Higher accuracy threshold for payments
- To discourage people from RNG clicking to farm.
- Encourage people to use practice mode.

4) Restrict who affects the result:
- Only higher accuracy pilots count towards determining the answer to a sample.
- Lower accuracy pilots will still get same payout if correct, but will not contribute to determining if it's correct.
- The threshold could be the mean accuracy, adjusting later depending on number of false positives.
- Alternatively have people's accuracy count as an exponential multiplier i.e. their choices have more weight.

EDIT:
5) Group Play / Spectator (not that kind of group play!):
- Allow friends to team up to find the right answer.
- Submits if everyone agrees, else returns until they can't agree.
- Reward stays the same but split evenly than adjusted based on individual accuracy.
- Will help increase individual accuracy.
- And/or a spectator mode so you can be mentored by a friend.

EDIT 2:
6) Fishing out the abusers:
- Periodically give a Foreign Tissue Sample which has already been solved with a VERY high relative consensus.
- Use the results to determine those who are clearly randomly clicking.
- Give a time out from the "real" game if they're found to get too many wrong.
- Timed out players can still use the practice feature.
- Peer-review feature; high accuracy pilots have option to review the mistakes of the lowest accuracy pilots (anonymously).
- They can determine whether the wrong answer was a reasonable mistake or completely wrong.
- If a pilot's mistakes don't pass peer-review, they are timed out from the "real game" for X days.
Bertram Renning
Swords of the righteous
#17 - 2016-01-22 23:38:28 UTC
Crazy Kitten wrote:
need more examples to compare for the different options, to get a better impression of whether or not an option can actually apply. only one picture for the option doesn't suffice :/

also, regarding the history: it'd be nice to be able to actually see the approved result when it's in, so i can better learn what i did wrong.

and finally, it really needs a "no clue, next!" button. no rewards, just an option to not knowingly submit bad data :/


edit: almost forgot: needs to be resizable, and more zoom options. maybe also take the "overview" outside the zoom view, to provide more visible space


I completly agree with that.


  • We need to better understand what we do wrong.
  • the "no clue, next!" button seems an absolute necessity to me too.
  • the overall navigation inside the picture could be improve :
  • => zoom in and out with mouse-wheel
    => be able to use the "overview/minimap" by clicking at it would be nice
    => the fact it automaticly zoom when I move my mouse over the picture get me confuse


It was say before but I like to highlight that it will need some way to prevent feeding random or bogus data. As the "right" choice would be the one with the most vote, it seems that even if the reward is only gain for a "right" answer, the result could be manipulate by players for profit and would be devastating for the data's value.

All that said, it is an amazing project and nothing should keep us from making it happen.
Cixi
#18 - 2016-01-23 00:22:19 UTC
The reward curve needs to be exponential instead of linear, otherwise it is easy to farm without caring about the results Roll
CCP Paradox
#19 - 2016-01-23 00:24:40 UTC
Starbuilder Stasarik wrote:
Having other options highlighted after Submitting on the tutorial is a bit confusing.

Examples:

http://puu.sh/mFNTg/39df5f351c.jpg and http://puu.sh/mFMRG/e8f3a722ec.jpg. Is there a reason for that, or is that a bug?


At the bottom right, do you see the Help button? Click that and it explains these tiles! Blink

CCP Paradox | EVE QA | Team Phenomenon

Space Magician

Lugh Crow-Slave
#20 - 2016-01-23 00:50:44 UTC
Starbuilder Stasarik wrote:
Having other options highlighted after Submitting on the tutorial is a bit confusing.

Examples:

http://puu.sh/mFNTg/39df5f351c.jpg and http://puu.sh/mFMRG/e8f3a722ec.jpg. Is there a reason for that, or is that a bug?

those ones that are highlited were also in the image and you missed them


red X you picked wrong green check you picked right green outline you missed this
123Next pageLast page