These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Information Portal

 
  • Topic is locked indefinitely.
 

Dev blog: Project Discovery needs You!

First post First post
Author
Previous Topic Next Topic
Michael Lincoln
Science and Trade Institute
Caldari State
#121 - 2016-03-10 15:56:52 UTC
Nevyn Auscent wrote:
It's obvious people are spamming incorrect results, and the incorrect results spam is getting real attempts marked inaccurate because of 'majority' vote.
A simple fix to this would be to move the percentage at which you get the 'unknown' images from 50% to at least 75%, so people really working on it don't have to deal with as many people spamming, since the spammers will not be pushing 75% easily, while 50% is pretty easy to bounce on.

You somehow managed to say what I was trying to without it coming out gibberish. Exactly this. Glad someone else coherently said it!
MachineOfLovingGrace
V0LTA
WE FORM V0LTA
#122 - 2016-03-10 19:35:11 UTC  |  Edited by: MachineOfLovingGrace
I'd be happy to try it on TQ, but i can't get through the tutorial - it just stops loading at some point, and I need to start from zero every time I close the window. Clearing the cache doesn't seem to help.
Iwata s
ALL BETS ARE OFF
#123 - 2016-03-10 20:18:35 UTC
Hi I have a few bug reports for project discovery.

1st is sometimes it keeps giving the same answer for new slides as the previous slide despite what box's were ticked in the current slide. this means some people will be getting the slide correct and the results will be saying that there getting them wrong which would lead to incorrect data being processed once people are submitting real slides.

secondly is on occasions after completing a slide no results information will be shown and it will not allow you to continue further I am not sure if this is just a load issue and maybe linked to the issue stated above.

I am further unsure if this is just an incident with me or if others are having the same issue and just not noticing.

but I would hate for cells to be incorrectly labeled due to people being unsure what is what due to an error with the results screen as this will not only effect the research but also peoples accuracy rating and enjoyment.

Kindest Regards

Iwata S

Delivery Agent
Buzz Orti
State War Academy
Caldari State
#124 - 2016-03-11 02:07:46 UTC
Iwata s wrote:
Hi I have a few bug reports for project discovery.
...

thanks but I think bug reports go in another forum.

Please make sure to refer to it there.

Builds ship in empty Quafe bottle.
Beta Maoye
#125 - 2016-03-11 02:47:48 UTC
The results of community consensus makes me begin to doubt about the average IQ of our community. The poor accuracy evaluation system drives off players who have contributed real efforts in analyzing the slides.

To improve the positive feedback of accuracy, I think the mini game needs more control slides with known results to filter out spammers. Players with less than 50% correct rate on control slides should not have their choices be counted in the community consensus.

People playing this mini game is not because of isk. They can make a hundred times more isk by doing other activities in the game. They are doing this because they think it has significance and they can contribute for the good of humanity. The best reward for them is they are told that other players are marking effort in the project and got the same results as they do. It is important to make sure that the accuracy evaluation system truly reflects the scientific accuracy which requires time and effort instead of spamming.
HPA Darkfield Oramara
H P A
C C P Alliance
#126 - 2016-03-11 09:34:24 UTC
Zepheros Naeonis wrote:
Kata Amentis wrote:
It's quite interesting, but some of the "correct" results don't make sense from either a description or sample image stand point.

For example
Baffled
edit: More Baffled

Could someone explain how that is the right answer? asymmetrical bloby/tubular compared to the "correct" symmetrical vainy/web. ShockedWhat?UghQuestion


Is that "correct" in the sense that an above average number of people on sisi clicked on it, so we're being penalised because people are just clicking through carelessly? a bug? naff data in the project? etc. Ugh


edit: couple of samples later, I get something almost the same as before "ok, it was this last time..." wrong.... erm UghRoll

I am getting this more times than I can count and it is beginning to **** me off. I'm going to take the suggestion someone else mentioned and close/reopen after every submission to see if that helps.

EDIT:

http://i.imgtc.com/LDtorSg8cF.png

Don't mind my answer, I should have just skipped it honestly. 100% selected Cytoplasm when it clearly states that it is "seen throughout the entire cell EXCEPT FOR THE NUCLEUS (blue part)". The Nucleus is completely riddled with the cytoplasm, so how is Cytoplasm even an option in this case? I am either completely lost in my understanding of how picking between the groups works or this is just a spam fest of amazing proportions.

http://i.imgtc.com/8NYTqicNoS.png

HOLY FACEPALM

Community consensus needs to be scrubbed asap. I'm sorry to say, but there should be someone at least reviewing a sample after a certain number of submissions to determine whether they are on the right track or not. They are going to receive so many flawed results it isn't even funny.


As one of the researchers for this project I would say that your first image looks like a plasma membrane staining, together with nucleoli (and maybe also an additional cytoplasmic and nuclear staining, hard to tell when not being able to toggle the different channels). The point is however that there's very common that the protein is located to two organelles or more, which means that a cell with both cytoplasmic and nuclear staining will have staining in both the area overlapping with the red and the blue marker. The descriptions of the different locations are only true if there would be a cytoplasmic staining only. This is a bit tricky of course, but usually the different stainings don't have the exact same intensity which might make it possible to differentiate them.

I agree that the second image would rather be labeled as vesicles, but I can't really tell what the consensus for that location is from the image.

And what we have seen from the result from the test server is that you guys actually are doing really good!
HPA Darkfield Oramara
H P A
C C P Alliance
#127 - 2016-03-11 09:40:18 UTC
Iwata s wrote:
Hi I have a few bug reports for project discovery.

1st is sometimes it keeps giving the same answer for new slides as the previous slide despite what box's were ticked in the current slide. this means some people will be getting the slide correct and the results will be saying that there getting them wrong which would lead to incorrect data being processed once people are submitting real slides.

secondly is on occasions after completing a slide no results information will be shown and it will not allow you to continue further I am not sure if this is just a load issue and maybe linked to the issue stated above.

I am further unsure if this is just an incident with me or if others are having the same issue and just not noticing.

but I would hate for cells to be incorrectly labeled due to people being unsure what is what due to an error with the results screen as this will not only effect the research but also peoples accuracy rating and enjoyment.

Kindest Regards

Iwata S

Delivery Agent


Thanks for noticing! We are aware of the first problem and we have notified CCP about it. And our servers have been under pretty high pressure since we launched the game, so the loading issues might be because of that.
HPA Darkfield Oramara
H P A
C C P Alliance
#128 - 2016-03-11 09:46:41 UTC
Beta Maoye wrote:
The results of community consensus makes me begin to doubt about the average IQ of our community. The poor accuracy evaluation system drives off players who have contributed real efforts in analyzing the slides.

To improve the positive feedback of accuracy, I think the mini game needs more control slides with known results to filter out spammers. Players with less than 50% correct rate on control slides should not have their choices be counted in the community consensus.

People playing this mini game is not because of isk. They can make a hundred times more isk by doing other activities in the game. They are doing this because they think it has significance and they can contribute for the good of humanity. The best reward for them is they are told that other players are marking effort in the project and got the same results as they do. It is important to make sure that the accuracy evaluation system truly reflects the scientific accuracy which requires time and effort instead of spamming.


We do have samples that we know the answer to, and we are able to filter away the players that either get a majority of these wrong and also those who choose the same locations over and over. But as long as you do your best we are really interested in what you see in the images, and you are probably going to spot things that we have missed!
Zepheros Naeonis
TinklePee
#129 - 2016-03-11 11:22:35 UTC
HPA Darkfield Oramara wrote:
Zepheros Naeonis wrote:
Kata Amentis wrote:
It's quite interesting, but some of the "correct" results don't make sense from either a description or sample image stand point.

For example
Baffled
edit: More Baffled

Could someone explain how that is the right answer? asymmetrical bloby/tubular compared to the "correct" symmetrical vainy/web. ShockedWhat?UghQuestion


Is that "correct" in the sense that an above average number of people on sisi clicked on it, so we're being penalised because people are just clicking through carelessly? a bug? naff data in the project? etc. Ugh


edit: couple of samples later, I get something almost the same as before "ok, it was this last time..." wrong.... erm UghRoll

I am getting this more times than I can count and it is beginning to **** me off. I'm going to take the suggestion someone else mentioned and close/reopen after every submission to see if that helps.

EDIT:

http://i.imgtc.com/LDtorSg8cF.png

Don't mind my answer, I should have just skipped it honestly. 100% selected Cytoplasm when it clearly states that it is "seen throughout the entire cell EXCEPT FOR THE NUCLEUS (blue part)". The Nucleus is completely riddled with the cytoplasm, so how is Cytoplasm even an option in this case? I am either completely lost in my understanding of how picking between the groups works or this is just a spam fest of amazing proportions.

http://i.imgtc.com/8NYTqicNoS.png

HOLY FACEPALM

Community consensus needs to be scrubbed asap. I'm sorry to say, but there should be someone at least reviewing a sample after a certain number of submissions to determine whether they are on the right track or not. They are going to receive so many flawed results it isn't even funny.


As one of the researchers for this project I would say that your first image looks like a plasma membrane staining, together with nucleoli (and maybe also an additional cytoplasmic and nuclear staining, hard to tell when not being able to toggle the different channels). The point is however that there's very common that the protein is located to two organelles or more, which means that a cell with both cytoplasmic and nuclear staining will have staining in both the area overlapping with the red and the blue marker. The descriptions of the different locations are only true if there would be a cytoplasmic staining only. This is a bit tricky of course, but usually the different stainings don't have the exact same intensity which might make it possible to differentiate them.

I agree that the second image would rather be labeled as vesicles, but I can't really tell what the consensus for that location is from the image.

And what we have seen from the result from the test server is that you guys actually are doing really good!

This clarification regarding Cytoplasm and nuclear staining helps out a lot, thank you.

Also, the consensus was mostly visible, but the main reason I pointed it out was the 60% selection for Cytoplasm when it is clearly Vesticles, if anything. (Which was my choice). I don't recall now if that was the remainding 20% for Cytoplasmic categories.

Thank you for the response. I find it unusual that the test server turned out so positive when it seems completely opposite on TQ. I hope the Project Discovery team continues to evaluate the results and make adjustments so that spammers aren't benefiting from the accuracy gains while the rest of us scratch our heads.
HPA Darkfield Oramara
H P A
C C P Alliance
#130 - 2016-03-11 12:41:43 UTC
Zepheros Naeonis wrote:
HPA Darkfield Oramara wrote:
Zepheros Naeonis wrote:
Kata Amentis wrote:
It's quite interesting, but some of the "correct" results don't make sense from either a description or sample image stand point.

For example
Baffled
edit: More Baffled

Could someone explain how that is the right answer? asymmetrical bloby/tubular compared to the "correct" symmetrical vainy/web. ShockedWhat?UghQuestion


Is that "correct" in the sense that an above average number of people on sisi clicked on it, so we're being penalised because people are just clicking through carelessly? a bug? naff data in the project? etc. Ugh


edit: couple of samples later, I get something almost the same as before "ok, it was this last time..." wrong.... erm UghRoll

I am getting this more times than I can count and it is beginning to **** me off. I'm going to take the suggestion someone else mentioned and close/reopen after every submission to see if that helps.

EDIT:

http://i.imgtc.com/LDtorSg8cF.png

Don't mind my answer, I should have just skipped it honestly. 100% selected Cytoplasm when it clearly states that it is "seen throughout the entire cell EXCEPT FOR THE NUCLEUS (blue part)". The Nucleus is completely riddled with the cytoplasm, so how is Cytoplasm even an option in this case? I am either completely lost in my understanding of how picking between the groups works or this is just a spam fest of amazing proportions.

http://i.imgtc.com/8NYTqicNoS.png

HOLY FACEPALM

Community consensus needs to be scrubbed asap. I'm sorry to say, but there should be someone at least reviewing a sample after a certain number of submissions to determine whether they are on the right track or not. They are going to receive so many flawed results it isn't even funny.


As one of the researchers for this project I would say that your first image looks like a plasma membrane staining, together with nucleoli (and maybe also an additional cytoplasmic and nuclear staining, hard to tell when not being able to toggle the different channels). The point is however that there's very common that the protein is located to two organelles or more, which means that a cell with both cytoplasmic and nuclear staining will have staining in both the area overlapping with the red and the blue marker. The descriptions of the different locations are only true if there would be a cytoplasmic staining only. This is a bit tricky of course, but usually the different stainings don't have the exact same intensity which might make it possible to differentiate them.

I agree that the second image would rather be labeled as vesicles, but I can't really tell what the consensus for that location is from the image.

And what we have seen from the result from the test server is that you guys actually are doing really good!

This clarification regarding Cytoplasm and nuclear staining helps out a lot, thank you.

Also, the consensus was mostly visible, but the main reason I pointed it out was the 60% selection for Cytoplasm when it is clearly Vesticles, if anything. (Which was my choice). I don't recall now if that was the remainding 20% for Cytoplasmic categories.

Thank you for the response. I find it unusual that the test server turned out so positive when it seems completely opposite on TQ. I hope the Project Discovery team continues to evaluate the results and make adjustments so that spammers aren't benefiting from the accuracy gains while the rest of us scratch our heads.


I'm glad it helped!

We will of course keep track of how well you are doing and also keep taking your inputs into account! This is however not an easy task we have been giving you, so I'm expecting the consensus to improve (even though it's way far from bad already, most classifications are fine) with time. It takes a while to learn what to look for and to get a feeling for all the different stainings. Trust me, I'm basing this on my own experience Blink
Azerath
BLOMI
#131 - 2016-03-11 13:06:46 UTC
Normally I dont post on the forums, but I got to take the time to say Project Discovery is absolutely horrible. Almost impossible to get them even if you spend allot of time and effort on it, 8 times out of 10 its magically wrong. Very poor waste of time and resources CCP.
Zepheros Naeonis
TinklePee
#132 - 2016-03-11 13:22:11 UTC  |  Edited by: Zepheros Naeonis
HPA Darkfield Oramara wrote:
I'm glad it helped!

We will of course keep track of how well you are doing and also keep taking your inputs into account! This is however not an easy task we have been giving you, so I'm expecting the consensus to improve (even though it's way far from bad already, most classifications are fine) with time. It takes a while to learn what to look for and to get a feeling for all the different stainings. Trust me, I'm basing this on my own experience Blink

Understandable. I am noticing a bigger improvement with just today's rounds compared to yesterday. Seems to be going a lot more smoothly for me and I can see this improving the community concensus results over the next few weeks as everyone gets the hang of it / better understands the choices.
Yadaryon Vondawn
Vicanthya
#133 - 2016-03-11 15:02:52 UTC
By request of the HPA guys over at twitter here is a recording of me playing Project Discovery. In the video I explain why I make the choices I make. I hope it gives you guys a frame of reference when doing Project Discovery.

How to classify samples
Emily Stareine
Integrated System Logistics
#134 - 2016-03-11 16:10:05 UTC
Azerath wrote:
Normally I dont post on the forums, but I got to take the time to say Project Discovery is absolutely horrible. Almost impossible to get them even if you spend allot of time and effort on it, 8 times out of 10 its magically wrong. Very poor waste of time and resources CCP.


Can't say I agree with that. I have a 65% rating now and that's pretty far from 8/10 wrong.
Jaantrag
#135 - 2016-03-11 18:42:52 UTC
so yea .. tryed it .. anyway to skip the tutorial ?? .. went to 8th slide in the tutorial but the thing kept loading .. so kinda hit X and closed it .. reopened and start from 0 .. go again .. and 4th slide keeps loading for long .. so start again hopeing it will go smoother .. nope .. kinda dissapointed .. brr ..

EVElopedia < add this to your sig to show u WANT it back
Zepheros Naeonis
TinklePee
#136 - 2016-03-12 05:22:45 UTC
Yadaryon Vondawn wrote:
By request of the HPA guys over at twitter here is a recording of me playing Project Discovery. In the video I explain why I make the choices I make. I hope it gives you guys a frame of reference when doing Project Discovery.

How to classify samples

Great video. Nice to see some uncertainty on your part even compared to the community concensus as I run into this quite a bit myself, lol.
Captain Campion
Campion Corp.
#137 - 2016-03-12 12:59:59 UTC
Submission Sample Mishap
Submission Sample Mishap
Submission Sample Mishap
Submission Sample Mishap
Submission Sample Mishap

CCPLS!
Captain Campion
Campion Corp.
#138 - 2016-03-12 13:30:47 UTC
images in the Project Discovery UI not loading about half the time
March rabbit
Aliastra
Gallente Federation
#139 - 2016-03-12 14:27:30 UTC
Well... Need to say i'm feeling the game 'guess community decision' here instead of 'analyze the image and set categories'....
After about 20 images i'm starting to lose interest because i'm bad at social predictions and most of the time community chosen categories are not closer to image than the ones i have chosen.

The Mittani: "the inappropriate drunked joke"
Zepheros Naeonis
TinklePee
#140 - 2016-03-13 19:54:02 UTC  |  Edited by: Zepheros Naeonis
March rabbit wrote:
Well... Need to say i'm feeling the game 'guess community decision' here instead of 'analyze the image and set categories'....
After about 20 images i'm starting to lose interest because i'm bad at social predictions and most of the time community chosen categories are not closer to image than the ones i have chosen.

I found this to be mostly true as well, but seeing as PD was just launched it is going to take some time for this to balance out and the community gets better at correctly picking the selection.

I still select what I believe a sample to be, even if I know it is likely to be wrong according to "community concensus", because I know there is a small chance the community actually picked the right one. It really is a hit or miss.

I am level 30 right now with 72% accuracy, so it isn't completely flawed, just taking a little longer for players to... use some common sense while determining what image is what.