These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE General Discussion

 
  • Topic is locked indefinitely.
 

Unscheduled Downtime Extension – July 15, 2015

First post First post First post
Author
Previous Topic Next Topic
Terry Gillan
The Scope
Gallente Federation
#6001 - 2015-07-15 22:53:34 UTC
Wait, its online!
Now all the fun of today ends...
Pyotr Veliky
SILK ROAD
#6002 - 2015-07-15 22:53:48 UTC
Terry Gillan wrote:
Pyotr Veliky wrote:
CCP Falcon wrote:
...This is due to an issue with a code deployment today while patching for a few fixes in EVE Online. The issue caused the cluster to fail to start up correctly, and as such, Tranquility is currently unavailable for play for both EVE and DUST...

Fair enough. I assume that you have saved the previous state of the system--the one that was working before you attempted the fixes and patches. Why not roll back the changes, restore the status quo ante, and do some more testing offline?


This is the 1735th time someone suggested this. I guess it's not possible. Could lead to even worse scenarios. They need to solve the problem right now. It may even not be related to Sov changes or new content at all. It may be something old and rotten.

My friend recently fixed an error in code, which was there for very long. He works on a very large ERP-like system. But everyone expected the wrong behaviour. They tested everything, but still after the fix was deployed, everything crashed and they had to restore the previous state... :) Sometimes it's just better to keep it running.

Never fix what's not broken!

If anyone else asked the same question, it appears that there has been no response. It is standard programming practice to maintain a stable configuration as a backup whenever you deploy new code. It is hard to imagine that they would not plan for this kind of contingency. The bugs in the new configuration are obviously difficult to troubleshoot and fix. Hence, it would seem to make sense to do a rollback to the last stable configuration.
John WarpingSlow
#6003 - 2015-07-15 22:54:08 UTC
Thank you CCP staff!

( I don't know what went wrong, but it can't have been easy to get back online. )
poonanoon
The Scope
Gallente Federation
#6004 - 2015-07-15 22:54:26 UTC
Let me be the first to ask CCp this question.....

What was broken and what did you do to fix it 😏
Phugoid
Absolute Order XL
Absolute Honor
#6005 - 2015-07-15 22:54:27 UTC
Terry Gillan wrote:
Wait, its online!
Now all the fun of today ends...


Nope, back down :/

Flugzeugführer
Armagast Sin Truth
Blackstone and Fairfield Transuniversal
Sentinels of Sukanan Alliance
#6006 - 2015-07-15 22:54:47 UTC
Thanks for all your hard work guys o7
Victor Sane
A-1 Industrial and Salvage
#6007 - 2015-07-15 22:54:53 UTC
awsome ccp good team Work. love you Guys and gals
Kesinik
The Scope
Gallente Federation
#6008 - 2015-07-15 22:55:07 UTC
Now if only I could stop reading this thread !

Am I the only person that doesn't hate CCP Fozzie?
Hildebrandt Koeppl
Hybrid Flare
#6009 - 2015-07-15 22:55:39 UTC
Cluster seems back up again. A big thumbs up to the devs and IT staff.
kraken11 jensen
ROOKS AND KRAKENS
#6010 - 2015-07-15 22:56:39 UTC
ahh, bed time. lol
Siliya
State War Academy
Caldari State
#6011 - 2015-07-15 22:56:44 UTC  |  Edited by: Siliya
and now to petition for lost SP
Darkstar Mal
Caldari Provisions
Caldari State
#6012 - 2015-07-15 22:57:38 UTC
Reactor: online
Sensors:online
Weapons: online
EVE: online
All systems nominal

Thank you CCP for all your hard work (and late night) to fix EVE and get everyone back to what we love doing best, blowing **** up in space! We appreciate all the hard work that has gone into making a game of this scale and complexity, and your dedication to making sure it gets updated and improved, even if it occasionally catastrophically breaks. **** happens, right?

See ya in space!
Lil Gangrel
Absolute Order
Absolute Honor
#6013 - 2015-07-15 22:57:46 UTC
Nope, i think foozzie rocks!
Sobaan Tali
Caldari Quick Reaction Force
#6014 - 2015-07-15 22:59:00 UTC
Ah! The Drifter's infiltration of the server has failed! CCP is too stronk, please nerf yourselves!

"Tomahawks?"

"----in' A, right?"

"Trouble is, those things cost like a million and a half each."

"----, you pay me half that and I'll hump in some c4 and blow the ---- out of it my own damn self."
Gilan Isana
Science and Trade Institute
Caldari State
#6015 - 2015-07-15 23:00:18 UTC
Cephalosa Galactica wrote:
I don't care about explanations of how when or where or what happened, all I care is about skill points and the rougly 4 bill lost by my people in those 13 (almost) hours of downtime
lets not talk the positioning of our capital ships has been delayed, upsetting the balance we had stablish I out timer/peeps online ratio. thank you for the **** up ccp...expecting a response from you .... yours truly in pew -CEPHY-


BLAH BLAH BLAH, RANT BLAH BLAH GIMMIE, GIMMIE BLAH BLAH BLAH, RANT

With responses like yours i bet they they wonder why they bother.

Fortunately, others of us appreciate the hard work and dedication these people have put in to get things working again - most of them will have done well over a 12 hour shift today, just so morons like you can post idiotic comments like this because you feel you deserve to get exactly what you want when you want.

WELL Buddy, Copernicus just called, YOU ARE NOT THE CENTER OF THE UNIVERSE.

Instead of giving **** because it took so long to fix, give thanks and praise that they put in the work to get it fixed.
Zaphod Hawke
Old Vindictive Bastards
#6016 - 2015-07-15 23:01:05 UTC
Wonderful job everyone @CCP. You got it online just in time for me to get called in to work. Thanks. Evil
Snape Dieboldmotor
Minotaur Congress
#6017 - 2015-07-15 23:01:05 UTC
Pyotr Veliky wrote:
Terry Gillan wrote:
Pyotr Veliky wrote:
CCP Falcon wrote:
...This is due to an issue with a code deployment today while patching for a few fixes in EVE Online. The issue caused the cluster to fail to start up correctly, and as such, Tranquility is currently unavailable for play for both EVE and DUST...

Fair enough. I assume that you have saved the previous state of the system--the one that was working before you attempted the fixes and patches. Why not roll back the changes, restore the status quo ante, and do some more testing offline?


This is the 1735th time someone suggested this. I guess it's not possible. Could lead to even worse scenarios. They need to solve the problem right now. It may even not be related to Sov changes or new content at all. It may be something old and rotten.

My friend recently fixed an error in code, which was there for very long. He works on a very large ERP-like system. But everyone expected the wrong behaviour. They tested everything, but still after the fix was deployed, everything crashed and they had to restore the previous state... :) Sometimes it's just better to keep it running.

Never fix what's not broken!

If anyone else asked the same question, it appears that there has been no response. It is standard programming practice to maintain a stable configuration as a backup whenever you deploy new code. It is hard to imagine that they would not plan for this kind of contingency. The bugs in the new configuration are obviously difficult to troubleshoot and fix. Hence, it would seem to make sense to do a rollback to the last stable configuration.


Rollbacks are rarely tested and thus it a is almost always better to stick with the current configuration. I support their decision not to rollback.
Thal Duif
Pandemic Horde High Sec Division
#6018 - 2015-07-15 23:01:29 UTC
Sobaan Tali wrote:
Ah! The Drifter's infiltration of the server has failed! CCP is too stronk, please nerf yourselves!



you know,...

i think the drifters invented a new kind of gun... The Switch.
Ammo type = unknown
Ammo damage = Downtime on eve server
collects/rewards = Tearz

fly safe o7
Nicholas Kirk
Federal Navy Academy
Gallente Federation
#6019 - 2015-07-15 23:01:49 UTC
CCP Falcon wrote:
[img]http://i.imgur.com/2P4UoRb.gif[/img]

Hey guys,

As you’re all aware, Tranquility is still offline after today’s scheduled daily downtime.

22:40 UTC - TRANQUILITY IS NOW BACK ONLINE AND ACCEPTING CONNECTIONS!
[/b]



lol I just had 21 windows 7 updates

coincident?
Zodeac
Garoun Investment Bank
Gallente Federation
#6020 - 2015-07-15 23:02:24 UTC
CCP Falcon wrote:
[img]http://i.imgur.com/2P4UoRb.gif[/img]

Hey guys,

As you’re all aware, Tranquility is still offline after today’s scheduled daily downtime.

This is due to an issue that happened during a code deployment today while patching for a few fixes in EVE Online. The issue caused the cluster to fail to start up correctly, and as such, Tranquility is currently unavailable for play for both EVE and DUST.

Our operations and development team are working on this issue as we speak, and we’re hoping to have Tranquility back online for you guys as soon as possible.

Right now, we don’t have an ETA for when the cluster will be back online and accepting connections, but we’ll keep you updated in this thread.

Cheers,

- F

UPDATES:

13:00 UTC - AFTER SEVERAL FAILED STARTUPS, WE ARE CURRENTLY INVESTIGATING A STARTUP ISSUE
13:30 UTC - INVESTIGATION IS STILL ONGOING - NO FURTHER ETA ON UPTIME FOR TRANQUILITY
13:37 UTC - STARTUP FAILED AGAIN - INVESTIGATION IS ONGOING
14:00 UTC - INVESTIGATION IS STILL ONGOING
14:30 UTC - WE ARE CURRENTLY IN THE PROCESS OF REBOOTING AFTER FAULTFINDING AND CHANGES
15:00 UTC - FURTHER ATTEMPTS TO START UP HAVE FAILED - INVESTIGATION IS ONGOING
15:30 UTC - INVESTIGATION STILL ONGOING - NO ETA ON SERVER UPTIME
16:00 UTC - A FURTHER FAILED STARTUP AFTER A NUMBER OF CHANGES - INVESTIGATION ONGOING
17:00 UTC - INVESTIGATIONS STILL ONGOING - NO ETA ON SERVER UPTIME
17:30 UTC - WE ARE CURRENTLY FAULTFINDING - NO ETA ON SERVER UPTIME
18:00 UTC - AFTER TESTING ON SINGULARITY WE WILL BE REBOOTING TQ TO TEST A FIX
18:30 UTC - WE ARE CURRENTLY STILL FAULTFINDING ON TQ - NO ETA FOR UPTIME AT THIS STAGE
19:00 UTC - WE ARE CURRENTLY TESTING A FIX FOR THE ISSUES ON TQ - NO ETA FOR UPTIME YET
19:30 UTC - TRANQUILITY IS STILL CURRENTLY BEING EXPERIMENTED ON - WE WILL CONTINUE TO UPDATE
20:00 UTC - PIZZA HAS BEEN ACQUIRED AND WE ARE CURRENTLY REFUELLING OUR DEVS!
20:30 UTC - DEVS REFUELLED - TESTING HAS RESUMED!
21:00 UTC - WE ARE CURRENTLY REBOOTING TQ AND WILL BE TESTING A FIX OVER THE NEXT 30 MIN
21:30 UTC - TESTING IS CURRENTLY ONGOING - NO ETA FOR UPTIME
22:00 UTC - WE ARE DEPLOYING A NEW BUILD TO TRANQUILITY FOR TESTING
22:40 UTC - TRANQUILITY IS NOW BACK ONLINE AND ACCEPTING CONNECTIONS!