These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

Assembly Hall

 
  • Topic is locked indefinitely.
 

[Proposal] Minimum time on test server for mechanics changes before going live

Author
Kirren D'marr
Caldari Provisions
Caldari State
#1 - 2013-11-20 22:33:45 UTC
I believe the recent controversy over changes to RLML and RHML systems with Rubicon highlights a need for sufficient time for players to test and vet new game mechanics and provide constructive feedback prior to these changes going live.

One of the key complaints about the changes to RMLs is that the announcement was made barely a week and a half before the change was being launched with Rubicon, and only uploaded to the test server a few days before going live. A common opinion is that the launch of this particular change should have been held for a later patch in order to give sufficient time to test it first; but by the time it was unveiled, it had already been included in the planned Rubicon 1.0 build. To many, this felt like the job was being rushed out and half-done, and that feedback was ignored largely because there was no test data available to back it up.

I would think that those who are against a change such as this would welcome the opportunity to fully test out new mechanics prior to their launch, as it provides them with the opportunity to test out their theories and opinions and share experiences that reinforce their position. Likewise, I believe that those supporting mechanincs changes would also prefer this, as if they truly believe in the effectiveness of such changes, then it would be likely that at least some of the detractors would change their opinion once they have seen the changes in action. Having a reasonable amount of time to put new game mechanics to the test benefits everyone all around.

One of the statements on this particular issue was that it would be pushed live, and then possibly improved at a later date. The problem with this is that historically, once a system has gone live, it tends to be a long and laborious process to get anything changed. Once something is launched, it seems to be forgotten or ignored, and this often leaves players dealing with faulty mechanics for an extended period of time. This is unnecessary, and can generally be avoided with proper preview testing. Isn't this the primary purpose of having a test server open to the playerbase?

My proposal therefore is this: that any significant change to game mechanics be required to be uploaded to the test server for a minimum amount of time prior to its launch on the live server. If CCP is unable to load a change to the test server with sufficient time before its planned launch, then that launch must be delayed until such time that proper testing and feedback can be completed.

I will leave the exact minimum test duration up for discussion. I'd personally recommend two weeks as an absolute bare minimum, although I would like to see more. Minor tweaks or adjustments to existing systems can probably be allowed without this requirement, but anything that changes the way ships or modules are or can be used should meet this minimum.

Please keep in mind that this is not specifically about the RML situation, but something I believe to be important and valuable to any future changes. The RML discussion has merely pointed out the need for sufficient community testing as a part of the development process.

Why a switch on/off? Because the new animation doesn't add anything to gameplay and it's graphically annoying. In other words, it's worse than bad: it's useless. Simple as that.     _ - Kina Ayami_

Niena Nuamzzar
Royal Amarr Institute
Amarr Empire
#2 - 2013-11-20 23:09:25 UTC
X X X
Thaddeus Eggeras
Urkrathos Corp
#3 - 2013-11-20 23:43:32 UTC
Agreed!
Ransu Asanari
Perkone
Caldari State
#4 - 2013-11-21 00:15:55 UTC  |  Edited by: Ransu Asanari
Signed.

I understand that there was a lot of discussion internally at CCP and with the CSM and other testing teams, but there was very little time to test out this change on SiSi and provide real feedback.

In addition to the reload/magazine size changes, there were changes to fitting requirements not detailed in the Dev Blog post and forum post. Having more practical time to test this out and discuss fittings in more detail would have been helpful.
zbaaca
Republic Military Tax Avoiders
#5 - 2013-11-21 15:59:08 UTC
+1

Bugs are opportunities to cause unprecedented amounts of destruction. --Zorgn ♡♡♡

Endovior
PFU Consortium
#6 - 2013-11-21 16:37:43 UTC
Though nice in principle, it's sometimes a tricky balance between testing time and actually getting the features out. More time to play with features would be useful, but I'm not sure if it'd be worth delaying the expansion over.
Gizznitt Malikite
Agony Unleashed
Agony Empire
#7 - 2013-11-21 16:43:02 UTC

-1

The test server isn't there for us to fully vet any features they wish to implement. Sometimes they do, sometimes they don't. Furthermore, your stance is diminished by your focus on RLML's.

Did RLML's didn't undergo a "significant mechanics change" from the standpoint of how they function. Sure, their R.O.F. and reload timers were changed, but that doesn't constitute a major change in mechanics. This isn't a fundamental change to POS code that effects 80% of the player base... this was a STATs change on some very specific weapon systems. What do you expect to find with another 2 weeks of sisi play test where 99% of the forum whiners don't even bother to explore the changes in the first place!.

Time to test is a good thing, but realize that ideas are often kicked around before they reach our ears. Between Devs, with the CSM, through internal testing, etc. Sisi, and your individual testing, is pretty much the final stage of feedback before they get implemented, and while I appreciate the opportunity to bug explore myself (especially regarding new mechanics), I don't believe CCP should be obligated to give us a fully functional Sisi server for XXX days prior to release.

TheMercenaryKing
Collapsed Out
Pandemic Legion
#8 - 2013-11-21 17:16:21 UTC
There is no need for this. It's CCP's game so they can change what they want and no one in entitled to anything in this game.

THERE IS NO ISSUE WITH RMLs! The weapon is still good in many ways it just people are expecting it to be used the exact same. If there is a problem with RLMs then its the peoples understanding of how they work. It's like comparing a space shuttle which would glide down to earth where as a space capsule would just fall down. Both acheive the desired result, but go about it differently. I have been saying this many times, RMLs are Hit and Run weapons, not sustained battle weapons, use them on roams with a small or even a large group.
Thaddeus Eggeras
Urkrathos Corp
#9 - 2013-11-21 17:41:13 UTC  |  Edited by: Thaddeus Eggeras
Wrong the test serve is there to test things out on, it was first made for CCP to test new ideas and items out on. Yes it's used by players now, but it's key that the people making the game test things coming into the game somehow before it hits the game. The test server is the place for this, it's also a good place to test modules, ships, etc for being over powered and under powered and any other issues within the game.

And again wrong, CCP might have made the game, but it's the players game. The first burn Jita proved that. Upset the players enough and they could crash the economy of EVE, CCP knows this. That is why they have the CSM, and also why they ask for players input about stuff. Without the players there is no EVE.

Things like rapids being completed worthless now and not being tested and being changed within a week or so before release is completely unacceptable. All new ships, modules and etc should be tested for at least a couple weeks, to see if they have any issues, and adjusted as needed. To just through something into a game as in depth as EVE is crazy. EVE has a fine line of balance it doesn't take much to bring that line off. And EVE also has a VERY out spoken and aggreesive player and fan base, it wouldn't be wise to upset them. If CCP wants EVE to grow, and to keep the player base it has, it needs to do well thought out testing before fixing items, before new items are brought in and it needs to remember EVE is special because it isn't like other games.

And yes there are many issues with rapids now, 40secs reload in PvP is worthless for one, and won't be hard to counter at all. Once you know your enemy is using a weapon system that takes 40secs to reaload, just find away to not take a whole lot of damage and when they reload you win. Rapids were OP before, but now they are truely broken, and need to be looked at again, a long with some oher missiles too. The other thing is 40secs reload for a weapon system is a slippery slope, if they did it with rapids what else might they do it with? I think ASBs 1 minute reload is odd, and should be looked at again. These just seem to be easy and fast fixes, instead of well thoughtout and REAL fixes like old CCP would have done. I hope they get back to that soon, but who knows. Adding reload time aren't fixes, it's an easy way out. And if you disagree with rapdis being worthless, come find me and we can play, it should be a good time haha.
Kirren D'marr
Caldari Provisions
Caldari State
#10 - 2013-11-21 18:57:16 UTC  |  Edited by: Kirren D'marr
Endovior wrote:
Though nice in principle, it's sometimes a tricky balance between testing time and actually getting the features out. More time to play with features would be useful, but I'm not sure if it'd be worth delaying the expansion over.


I wouldn't recommend delaying an entire expansion over this; just hold back on the untested feature.

Gizznitt Malikite wrote:

Furthermore, your stance is diminished by your focus on RLML's.


TheMercenaryKing wrote:
THERE IS NO ISSUE WITH RMLs!


Again, this is not about RMLs - this is an issue which has been highlighted by the recent RML controversy and ensuing debate, but it is by no means unique to this single feature, and I would hope that people can be objective enought to understand the difference.

Gizznitt Malikite wrote:

Time to test is a good thing, but realize that ideas are often kicked around before they reach our ears. Between Devs, with the CSM, through internal testing, etc. Sisi, and your individual testing, is pretty much the final stage of feedback before they get implemented, and while I appreciate the opportunity to bug explore myself (especially regarding new mechanics), I don't believe CCP should be obligated to give us a fully functional Sisi server for XXX days prior to release.


I understand that these idease are discussed internally. However, the one thing that all that internal testing cannot provide is volume data. As an engineer who works in system failure analysis, I can tell you that volume data is vital to any product release. You can do all the planning and experimenting you want, but you will never cover every use case or see all the ways in which your design can break unless you can gather data en masse. Even then, there are things that may be missed, but the larger the sample size, the better the chance you have of ensuring you are releasing quality work. This is why giving these changes adequate time on the test server should be considered a crucial part of the development process.

At the core, this proposal is not about giving us time to learn the changes (although that is a nice byproduct). The key here is to give CCP the information necessary to release their features in the best possible state. How many times have we seen featrues come out which upon release were either nerfed so hard that they faded into obscurity, or on day one were broken by the players in a way CCP never imagined and become so overpowered that they dominate the game? Requiring this level of testing would not eliminate all such occurences, but it should significantly limit them and hopefully reduce their severity.


The sad part is that we should not have to request this requirement at all; this is something CCP should already be doing for their own good and for the good of the game. Frankly, the very idea that we have to ask for proper testing of features is shameful.

Why a switch on/off? Because the new animation doesn't add anything to gameplay and it's graphically annoying. In other words, it's worse than bad: it's useless. Simple as that.     _ - Kina Ayami_

TheMercenaryKing
Collapsed Out
Pandemic Legion
#11 - 2013-11-21 19:08:43 UTC
Thaddeus Eggeras wrote:
Wrong the test serve is there to test things out on, it was first made for CCP to test new ideas and items out on. Yes it's used by players now, but it's key that the people making the game test things coming into the game somehow before it hits the game. The test server is the place for this, it's also a good place to test modules, ships, etc for being over powered and under powered and any other issues within the game.

And again wrong, CCP might have made the game, but it's the players game. The first burn Jita proved that. Upset the players enough and they could crash the economy of EVE, CCP knows this. That is why they have the CSM, and also why they ask for players input about stuff. Without the players there is no EVE.

Things like rapids being completed worthless now and not being tested and being changed within a week or so before release is completely unacceptable. All new ships, modules and etc should be tested for at least a couple weeks, to see if they have any issues, and adjusted as needed. To just through something into a game as in depth as EVE is crazy. EVE has a fine line of balance it doesn't take much to bring that line off. And EVE also has a VERY out spoken and aggreesive player and fan base, it wouldn't be wise to upset them. If CCP wants EVE to grow, and to keep the player base it has, it needs to do well thought out testing before fixing items, before new items are brought in and it needs to remember EVE is special because it isn't like other games.

And yes there are many issues with rapids now, 40secs reload in PvP is worthless for one, and won't be hard to counter at all. Once you know your enemy is using a weapon system that takes 40secs to reaload, just find away to not take a whole lot of damage and when they reload you win. Rapids were OP before, but now they are truely broken, and need to be looked at again, a long with some oher missiles too. The other thing is 40secs reload for a weapon system is a slippery slope, if they did it with rapids what else might they do it with? I think ASBs 1 minute reload is odd, and should be looked at again. These just seem to be easy and fast fixes, instead of well thoughtout and REAL fixes like old CCP would have done. I hope they get back to that soon, but who knows. Adding reload time aren't fixes, it's an easy way out. And if you disagree with rapdis being worthless, come find me and we can play, it should be a good time haha.



Let me use a real life example for you. I rent an apartment and recently the new management company said they are removing the indoor pool, adding an outdoor one in the spring, and revamping the clubhouse/office. Many people were pissed about that. Those people believe they are/were entitled to the indoor pool being that they pay rent. However, they pay rent to use the apartment and amenities, the management company owns the facility and has the right to do whatever they want to it so long as it doesn't violate the existing contact the renter agreed to.

Personally I like the changes they are making, a Virtual golf center with 500 18 hole courses (so good-bye Eve) among other things, and will stay while other people clearly don't like the changed to the pool (specifically that feature) but there is no legal obligation in the contact that says the pool cannot be changed or anything of the sorts. The Company did specify that I could, if i wanted to, end my contact early with a heavily reduced fee due to the changes (1.5 months rent as specified in my contact for early termination), many people have already opted to take this and end their lease.

If you do think you are entitled to anything in Eve or by CCP then you are horribly wrong for thinking that and you have the right to walk away at anytime. CCP Being a company obviously wants to give out a quality product for business, but it's THEIR game. They CAN listen to people but DON'T have to. If you want to start another burn Jita then go ahead over one module which needed to be changed. Since they changed it, finding the new use for it to be most effective is what you should be doing.
Gizznitt Malikite
Agony Unleashed
Agony Empire
#12 - 2013-11-21 19:38:15 UTC
Kirren D'marr wrote:

Gizznitt Malikite wrote:

Time to test is a good thing, but realize that ideas are often kicked around before they reach our ears. Between Devs, with the CSM, through internal testing, etc. Sisi, and your individual testing, is pretty much the final stage of feedback before they get implemented, and while I appreciate the opportunity to bug explore myself (especially regarding new mechanics), I don't believe CCP should be obligated to give us a fully functional Sisi server for XXX days prior to release.


I understand that these idease are discussed internally. However, the one thing that all that internal testing cannot provide is volume data. As an engineer who works in system failure analysis, I can tell you that volume data is vital to any product release. You can do all the planning and experimenting you want, but you will never cover every use case or see all the ways in which your design can break unless you can gather data en masse. Even then, there are things that may be missed, but the larger the sample size, the better the chance you have of ensuring you are releasing quality work. This is why giving these changes adequate time on the test server should be considered a crucial part of the development process.

At the core, this proposal is not about giving us time to learn the changes (although that is a nice byproduct). The key here is to give CCP the information necessary to release their features in the best possible state. How many times have we seen featrues come out which upon release were either nerfed so hard that they faded into obscurity, or on day one were broken by the players in a way CCP never imagined and become so overpowered that they dominate the game? Requiring this level of testing would not eliminate all such occurences, but it should significantly limit them and hopefully reduce their severity.

The sad part is that we should not have to request this requirement at all; this is something CCP should already be doing for their own good and for the good of the game. Frankly, the very idea that we have to ask for proper testing of features is shameful.


To be frank, I while Sisi provides the full "volume" testing you reference under special situations, things like RLML aren't "volume tested" there. Sisi provides players with a risk free environment to fully vet the mechanics. When CCP changes some problematic back-end stuff, the volume testing is important. When CCP changes simple stats, it generally isn't.

Kirren D'marr
Caldari Provisions
Caldari State
#13 - 2013-11-21 20:04:55 UTC
Gizznitt Malikite wrote:
To be frank, I while Sisi provides the full "volume" testing you reference under special situations, things like RLML aren't "volume tested" there. Sisi provides players with a risk free environment to fully vet the mechanics. When CCP changes some problematic back-end stuff, the volume testing is important. When CCP changes simple stats, it generally isn't.


I disagree. When you change a weapon system like this, it has the potential to render it either completely useless, totally overpowered, or somewhere in between. The more players who can test and try it out, the more data CCP will have available to determine where on that scale this change will fall. The test server may not provide the same volume of data as the live server, but it is definitely more data than they are getting internally.

More data before release is always a good thing. After release, a lot more data will be available, but there is a lot less motivation for follow up and improvement. Once the release deadline has been met, people move on to the next project, and few features get the post release evaluation that they deserve. Plust, there is always the potential for catasrophic results and the ensuing fallout. As such, it is far better to do all that is reasonable to improve the features prior to release. To do otherwise is irresponsible.

Why a switch on/off? Because the new animation doesn't add anything to gameplay and it's graphically annoying. In other words, it's worse than bad: it's useless. Simple as that.     _ - Kina Ayami_

Thaddeus Eggeras
Urkrathos Corp
#14 - 2013-11-21 21:47:27 UTC
A video game isn't an apartemnt compnay, people have to sleep somewhere. But games, well there are 100s of them and if you don't listen to those who play said game and mess it up enough that enough move to something else, said game is gone. In the REAL world you have to listen to the consumer and also the consumer isalways right. Not testing something and changing it 10 days or so before it gets adding to game is a good way to **** those consumers off and show you don't care what they think. Do that enough, and they will have enough of you and move on.

So if CCP wants to keep their consumers, they should probably listen to them. They didn't with the walking in officer quarters and AUR and 100s of consumers attacked everything in Jita and almost crashed the server. As EVE is only on one server and the market is 100% play based, it wouldn't be hard for the consumers (players) to crash the economy and/or server. Also if they keep adding things at the last minute, especially things that ruin an idea in the game they will have th consumer loose faith in them and look around for other games that won't F them over so much.

So should CCP listen to their consumer base yes, do they I think so to a point. Should CCP test everything new coming into EVE for a couple weeks yes. Should they also test anything that might be OP or UP before trying to change them yes. Do they have to, no. But if they want to keep EVE going and keep the consumers, it be a good idea.