These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE General Discussion

 
  • Topic is locked indefinitely.
 

CCP SERVER ENGINEERS ARE IN TEST

First post
Author
Previous Topic Next Topic
Tu'yak Marowshay
Original Sinners
Pandemic Legion
#221 - 2013-07-04 23:22:56 UTC
Don't be mad goonietoonies, we were just waiting for you to jump in the huge cap fleet you have. But alas, yet another day goes by where not a single nutsack attached to a goon was seen :(


Guess it is back to grinding structures in - bombers?
snorri vidal
Aliastra
Gallente Federation
#222 - 2013-07-04 23:23:01 UTC
CCP Phantom wrote:
Shamata wrote:
CCP you just ****** one of the largets fleet battels in eve history... THANKS!

In the attempt to improve the overall fight quality and node stability in the Z9PP system at 22:41 UTC by moving away all non-critical services from this node, our engineers unfortuntely made a mistake and actually moved the critical system from the current server node instead of moving all other systems. As result every pilot in this system lost connection to the node and therefore to Tranquility.

The system Z9PP is online and accessible again now.

It is extremely unfortunate to make such a mistake and we apologize deeply for disrupting one of the truly epic battles in the history of EVE; especially since this battle already raged on for several hours and was still heating up for more! We are very sorry about this incident and will make sure that such mistakes won't happen again.


Þetta er nú bara lélegasta afsökun sem ég hef heyrt...
Jafit McJafitson
Viziam
Amarr Empire
#223 - 2013-07-04 23:23:02 UTC
Alavaria Fera
GoonWaffe
#224 - 2013-07-04 23:23:04 UTC
Harrkonen Dune wrote:
As a professional company your pitiful explanation is woefully inadequate, and unacceptable.

A very important battle which could have turned the tide has been voided by your complete incompetence.

If you wish to come out of this debacle with any sort of face you must make it right, and it is not down to the EVE community to give you the solution to the mess you've made.

You must make it right.

Haha that will never happen.

Triggered by: Wars of Sovless Agression, Bending the Knee, Twisting the Knife, Eating Sov Wheaties, Bombless Bombers, Fizzlesov, Interceptor Fleets, Running Away, GhostTime Vuln, Renters, Bombs, Bubbles ?
Arrgthepirate
Caldari Provisions
Caldari State
#225 - 2013-07-04 23:23:20 UTC
GRIEV3R wrote:
In spite of all the idiots sperging on this thread, I have one honest question: Why weren't the tackled capital ships still there when we logged back in?

They were in warp disruption bubbles with agro timers. According to everything I know about logoff mechanics, they should not have moved anywhere, regardless of server crashes, characters log offs, or any amount of tinfoil.

I'm willing to accept that it was a simple mistake and there was no foul play, but only if you can give us a solid explanation for why the carriers disappeared from the field, in contradiction to well-established logoff mechanics.


Please explain CCP.

Yumis
Ministry of War
Amarr Empire
#226 - 2013-07-04 23:23:20 UTC
Who would have thought that engineers are people and all people make mistakes...

What is important is how CCP will learn from this and I will bet isk that they will automate this process in future.
Hesod Adee
Perkone
Caldari State
#227 - 2013-07-04 23:23:20 UTC
CCP Phantom wrote:
Shamata wrote:
CCP you just ****** one of the largets fleet battels in eve history... THANKS!

In the attempt to improve the overall fight quality and node stability in the Z9PP system at 22:41 UTC by moving away all non-critical services from this node, our engineers unfortuntely made a mistake and actually moved the critical system from the current server node instead of moving all other systems. As result every pilot in this system lost connection to the node and therefore to Tranquility.

The system Z9PP is online and accessible again now.

It is extremely unfortunate to make such a mistake and we apologize deeply for disrupting one of the truly epic battles in the history of EVE; especially since this battle already raged on for several hours and was still heating up for more! We are very sorry about this incident and will make sure that such mistakes won't happen again.


Will you be implementing any changes to prevent this from happening again ?

For example, adding some code so that the system using the most resources can not be moved from its current node.
Nethras
Brutor Tribe
Minmatar Republic
#228 - 2013-07-04 23:23:30 UTC
Downtime doesn't remove ships from space or reset aggro timers, but node remapping does? Look, EVE PvE has never been a strength, the only thing that remotely matches server mechanics during fights in importance is the economy, and you **** it up this bad?

Why does this function differently than downtime? Why is it possible to "accidentally" do this without a warning that you're doing it to an extremely full system? Why the hell are you trying to excuse saving hundreds of billions of isk, if not over a trillion, with an "oops"?

There's a reason that critical functions are normally engineered with multiple checks and failsafes, because human error always DOES exist, which is why it's completely ******* inexcusable that CCP has nothing in place to mitigate that possibility here. "We'll make sure this won't happen again" isn't sufficient, fix how it was even possible to begin with, and fix it on multiple levels.
Arkon Olacar
black.listed
#229 - 2013-07-04 23:23:43 UTC
Dheeradj Nurgle wrote:
CFC Capital fleet gets saved by a boo boo 3 or 4 times, TEST grumbles and gets on with their lives.

TEST Capital fleet gets saved by a boo boo, and the SA Furries and their pets go on a Posting CTA.
md5oogle
Canadian Bacon.
Honorable Third Party
#230 - 2013-07-04 23:23:59 UTC
Way to kill **** CCP. Not only do you totally kill a 6hr fight, you do it live in front of 5000 streamers. Great PR.

Next time, maybe just leave **** alone.. Would rather have some lag then you guy's always wrecking ****.
Ayures
Perkone
Caldari State
#231 - 2013-07-04 23:24:04 UTC
Dheeradj Nurgle wrote:
CFC Capital fleet gets saved by a boo boo 3 or 4 times, TEST grumbles and gets on with their lives.

TEST Capital fleet gets saved by a boo boo, and the SA Furries and their pets go on a Posting CTA.
Alavaria Fera
GoonWaffe
#232 - 2013-07-04 23:24:06 UTC
Onictus wrote:
Tira'tai wrote:
It's not even about that it happened. It's about that all the ******* ships should've been back in space with agression timers after it came back up.

This.

Where are those carriers because they sure as **** weren't on scan

Magic happens such that the server apparently forgets about the ships since that isn't stored when the node is remapped I suppose.

Triggered by: Wars of Sovless Agression, Bending the Knee, Twisting the Knife, Eating Sov Wheaties, Bombless Bombers, Fizzlesov, Interceptor Fleets, Running Away, GhostTime Vuln, Renters, Bombs, Bubbles ?
Needmore Longcat
Dreddit
Test Alliance Please Ignore
#233 - 2013-07-04 23:24:07 UTC
Arkon Olacar wrote:
What is this, a CFC posting CTA?


everyone make sure and check in on the participation board. have to see who's carrying their weight.

also, 100% forum karma taxes during this op.
Hawkspur Firestarter
Viziam
Amarr Empire
#234 - 2013-07-04 23:24:17 UTC
Cordata Jomar wrote:
Hawkspur Firestarter wrote:
Cordata Jomar wrote:
Just like to say I'm angry and flipping tables.
My mega was alive when the node went down thanks to armour/speed/transversal.
I log in to find it dead.

~thank you~ What?



You were the dreadnaught primary for 10 minutes. You died but were bugged. Guns were cycling and not using ammo.


Welp, either you guys killed me just as the server went down and my game shut down before yours, or the server bugged.
I didn't find myself dead until I logged back in.

Before the crash I was still flying, taking damage, and fighting, taking little bits of damage from of your guys sentries, and dodging fire from the dreads (while within about 30km of all your BS). That continued right up until the second the server kicked me.

Bit annoyed that I got killed without seeing it/being able to respond.

Cool that I survived 10 mins, but still What?


10 minutes in 10 percent tidi = 1 minute

You got two cycled and we switched targets when your armor stopped moving and our guns stopped using ammo. This was well before the node crash
Traderjohn
Macabre Votum
Northern Coalition.
#235 - 2013-07-04 23:24:20 UTC
This could have happened to anyone and gone either way. Sorry to goons but either way it was a fun fight up until that point. Also we need less tidi in future events and fights such as this.

TJ
Jasp3r
Deep Core Mining Inc.
Caldari State
#236 - 2013-07-04 23:24:20 UTC
Poor engineer.. we've all screwed up at some point in IT and disrupted services for users. If you've not screwed up you've just not been doing it for long enough. Long hours and late nights are usually the reason.

I'm interested in the change control process though and was the engineer performing the migrations alone while on-call? Do multiple people need to confirm an action before it is performed on a production node? Are there any improvements to processes that can be implemented?
Jannarious White
Deep Core Mining Inc.
Caldari State
#237 - 2013-07-04 23:24:27 UTC
CCP Phantom wrote:
It is extremely unfortunate to make such a mistake and we apologize deeply for disrupting one of the truly epic battles in the history of EVE; especially since this battle already raged on for several hours and was still heating up for more! We are very sorry about this incident and will make sure that such mistakes won't happen again.


This battle is truly epic in the history of EVE because it was the first time that a CPP actively affected the outcome of a pivotal battle in null security space. It is unfortunate that "human error" was the cause, but what is CCP going to do to try to correct this mistake and try to reconcile the actions that were taking place in space when the node was terminated?
Arrgthepirate
Caldari Provisions
Caldari State
#238 - 2013-07-04 23:24:28 UTC
Yumis wrote:
Who would have thought that engineers are people and all people make mistakes...

What is important is how CCP will learn from this and I will bet isk that they will automate this process in future.


CCP can't even figure out how they programed POS what makes you think they can figure out how to automate something as delicate and complex as server management?
Darth Kilth
KarmaFleet
Goonswarm Federation
#239 - 2013-07-04 23:24:29 UTC
Before Node crash:

TEST: **** THIS, WE'RE LOSING ALL OUR CARRIERS Cry

Node Crashes

CCP: Oops, sorry.

TEST: AWW YEAH, **** GOONS HAHAHAHAHA.
Axhind
Eternity INC.
Goonswarm Federation
#240 - 2013-07-04 23:24:29 UTC
So awesome. Waiting for escalation for 5 hours and then CCP fucks it all up. So much for epic fights and escalations. When test and co **** up CCP comes running.