These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

Issues, Workarounds & Localization

 
  • Topic is locked indefinitely.
 

Random DC's from TQ
Author
Previous Topic Next Topic
Emrys Ap'Morgravaine
Deep Core Mining Inc.
Caldari State
#1 - 2012-01-16 20:36:40 UTC  |  Edited by: Emrys Ap'Morgravaine
So last year, start of December or so, lots of people started seeing a rise in random disconnects from TQ, after some hunting and tracerouting by the community the problem was pinned down to a Telia node.

CCP did some prodding, or made some calls, whatever, and all was once again well.

Until last night/this morning.

Like myself, a number of others within my alliance have started having random DC's once again, and after a chat with folks elsewhere and a bit of trace running once again we've pinned the issue down to one of the eurorings nodes.

Specifically: ldn-s2-rou-1041.uk.eurorings.net [195.66.224.54]

Can I ask everyone else who is having random DC problems to do a check, get yourself a copy of something like visualtrace and run it along side your eve client, and get some logs on the go.

@ CCP - please guys, this isn't the first time you've had problems with your routing providers, its not even the second time, I recall a lengthy statemate between your players (us) and yourselves with the LL3 provider a couple of years ago.

For the love of happy bunnies everywhere, look into it and give us some feedback will you?Attention

Em.
Emrys Ap'Morgravaine
Deep Core Mining Inc.
Caldari State
#2 - 2012-01-17 15:16:16 UTC  |  Edited by: Emrys Ap'Morgravaine
Another day, and another 2 DC's since downtime.

Please CCP - give us some sort of acknowledgement to this?

I'm still tracing the problems to the same node, with between 40 > 75% packet loss and response spikes up into the 300ms range.


Addendum: The problems also affect website/gate/forum access from time to time. Took 3 attempts to post the above message.
Othran
Route One
#3 - 2012-01-17 15:46:34 UTC  |  Edited by: Othran
Assuming you have Windows open a command window and type :

pathping www.eveonline.com

then post the output.

Or use pingplotter - unless you show the problem and the time then nobody can do much about it.
Emrys Ap'Morgravaine
Deep Core Mining Inc.
Caldari State
#4 - 2012-01-17 15:51:39 UTC  |  Edited by: Emrys Ap'Morgravaine
1 15 ms 7 ms 11 ms cpc1-sgyl27-2-0-gw.sgyl.cable.virginmedia.com [94.173.0.1]
2 7 ms 10 ms 9 ms sgyl-core-1a-ge128.network.virginmedia.net [81.97.48.209]
3 7 ms 9 ms 8 ms sgyl-core-2a-ae2-0.network.virginmedia.net [195.182.178.90]
4 14 ms 12 ms 17 ms manc-bb-1c-ae6-0.network.virginmedia.net [81.97.48.77]
5 17 ms 15 ms 28 ms manc-bb-1a-ae11-0.network.virginmedia.net [213.105.159.85]
6 28 ms 22 ms 21 ms glfd-bb-1b-ae1-0.network.virginmedia.net [213.105.175.146]
7 20 ms 19 ms 22 ms glfd-tmr-1-ae5-0.network.virginmedia.net [213.105.159.46]
8 26 ms 21 ms 26 ms tcl5-ic-1-as0-0.network.virginmedia.net [62.253.185.78]
9 * 26 ms * ldn-s2-rou-1041.UK.eurorings.net [195.66.224.54]
10 20 ms * * 134.222.109.115
11 * 24 ms 23 ms srv248-e.ccp.cc [87.237.36.248]
12 26 ms 23 ms 24 ms srv200-g.ccp.cc [87.237.38.200]

thats just one example, the timeouts and responses for that eurorings node are all over the place.

I've ran some specifics just for that node, and get equally hideous response times, so its not just throughout thats the answer.

I'm currently on a call with Virgin to see if they can map around it at their end, but if you've ever dealt with virgin support, you'll know how much of an uphill battle that is...

http://imageshack.us/photo/my-images/853/pingplotter.jpg/
Othran
Route One
#5 - 2012-01-17 16:41:16 UTC  |  Edited by: Othran
Oh you don't need to tell me mate I'm on Be and we had all this crap via L3 last month. Be's support mean well but since Telefonica took over its been lurching from one screwup to another. Still at least my jitter is probably marginally better than yours Twisted

The other thing to do is use something like http://lg.eurorings.net/ - that will show its nothing to do with you, if you pick the right node it should also show some latency spikes, just find the ones which use the same routing as you.

Edit - I remain unconvinced that these periodic screwups aren't due to CCP. They bought some company or something a couple of years ago which specialised in routing. I can't remember the details but I remember thinking "glad I'm not the poor sod who has to make this work". Decent public/private peering (not packetexchange) and bandwidth work for most folks. /me shrugs
Emrys Ap'Morgravaine
Deep Core Mining Inc.
Caldari State
#6 - 2012-01-17 16:57:13 UTC
I just had a nice chat with one of the guys at KPN (Eurorings) turns out they're loosely aware of the problem, I've sent him about a meg's worth of logs and snapshots.

I'll update as and when I have/hear more.

Em.
Scar'Face
Caldari Provisions
Caldari State
#7 - 2012-01-17 18:03:58 UTC  |  Edited by: Scar'Face
I am having issues with random dc's and then server status showing as unknown, below is ping path



Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation. All rights reserved.

C:\Users\>pathping www.eveonline.com

Tracing route to www.eveonline.com [87.237.39.197]
over a maximum of 30 hops:
0 PC [192.168.0.2]
1 10.159.44.1
2 winn-core-1b-ge-300-2134.network.virginmedia.net [62.253.121.221]
3 winn-bb-1b-so-130-0.network.virginmedia.net [213.105.175.33]
4 popl-bb-1a-ae1-0.network.virginmedia.net [62.253.185.201]
5 popl-tmr-1-ae4-0.network.virginmedia.net [213.105.159.2]
6 amst-ic-1-as0-0.network.virginmedia.net [213.105.175.6]
7 rt-dc2-ias-csg01.nl.kpn.net [195.69.144.89]
8 195.190.227.223
9 ldn-s2-rou-1021.UK.eurorings.net [134.222.229.222]
10 ldn-s2-rou-1041.UK.eurorings.net [134.222.228.94]
11 * * *
Computing statistics for 250 seconds...
Source to Here This Node/Link
Hop RTT Lost/Sent = Pct Lost/Sent = Pct Address
0 PC [192.168.0.2] 0/ 100 = 0% |
1 --- 100/ 100 =100% 100/ 100 =100% 10.159.44.1 0/ 100 = 0% |
2 23ms 0/ 100 = 0% 0/ 100 = 0% winn-core-1b-ge-300-2134.network.virginmedia.net [62.253.121.221] 0/ 100 = 0%
3 21ms 0/ 100 = 0% 0/ 100 = 0% winn-bb-1b-so-130-0.network.virginmedia.net [213.105.175.33] 0/ 100 = 0% |
4 24ms 0/ 100 = 0% 0/ 100 = 0% popl-bb-1a-ae1-0.network.virginmed.net [62.253.185.201] 0/ 100 = 0% |
5 24ms 0/ 100 = 0% 0/ 100 = 0% popl-tmr-1-ae4-0.network.virginmedia.net [213.105.159.2] 0/ 100 = 0% |
6 30ms 0/ 100 = 0% 0/ 100 = 0% amst-ic-1-as0-0.network.virginmedia.net [213.105.175.6] 31/ 100 = 31% |
7 43ms 57/ 100 = 57% 26/ 100 = 26% rt-dc2-ias-csg01.nl.kpn.net [195.69.144.89] 0/ 100 = 0% |
8 --- 100/ 100 =100% 69/ 100 = 69% rt-rou-1022.nl.eurorings.net [195.190.227.223] 0/ 100 = 0% |
9 30ms 31/ 100 = 31% 0/ 100 = 0% ldn-s2-rou-1021.UK.eurorings.net [134.222.229.222] 69/ 100 = 69% |
10 --- 100/ 100 =100% 0/ 100 = 0% ldn-s2-rou-1041.UK.eurorings.net [134.222.228.94]

Trace complete.
Ibeau Renoir
The Scope
Gallente Federation
#8 - 2012-01-17 18:14:53 UTC  |  Edited by: Ibeau Renoir
Connection trouble here too, also on Virgin Media, and CCP is not the only problem. Can't connect to lg.eurorings.net or imageshack.us...

Quote:
Tracing route to forums.eveonline.com [87.237.39.162]
over a maximum of 30 hops:
0 rhuidean [192.168.1.138]
1 DD-WRT [192.168.1.1]
2 * cpc9-horn3-2-0-gw.6-1.cable.virginmedia.com [81.107.204.1]
3 cosh-core-1a-ae1-1861.network.virginmedia.net [80.3.161.113]
4 cosh-core-1b-ge-100-0.network.virginmedia.net [80.3.161.2]
5 glfd-bb-1b-ae4-0.network.virginmedia.net [212.43.163.213]
6 glfd-tmr-1-ae5-0.network.virginmedia.net [213.105.159.46]
7 tcl5-ic-1-as0-0.network.virginmedia.net [62.253.185.78]
8 ldn-s2-rou-1041.UK.eurorings.net [195.66.224.54]
9 * 134.222.109.115
10 srv248-e.ccp.cc [87.237.36.248]
11 srv162-h.ccp.cc [87.237.39.162]

Computing statistics for 275 seconds...
Source to Here This Node/Link
Hop RTT Lost/Sent = Pct Lost/Sent = Pct Address
0 rhuidean [192.168.1.138]
0/ 100 = 0% |
1 0ms 0/ 100 = 0% 0/ 100 = 0% DD-WRT [192.168.1.1]
0/ 100 = 0% |
2 14ms 0/ 100 = 0% 0/ 100 = 0% cpc9-horn3-2-0-gw.6-1.cable.virginmedia.com [81.107.204.1]
0/ 100 = 0% |
3 12ms 0/ 100 = 0% 0/ 100 = 0% cosh-core-1a-ae1-1861.network.virginmedia.net [80.3.161.113]
0/ 100 = 0% |
4 13ms 0/ 100 = 0% 0/ 100 = 0% cosh-core-1b-ge-100-0.network.virginmedia.net [80.3.161.2]
0/ 100 = 0% |
5 17ms 0/ 100 = 0% 0/ 100 = 0% glfd-bb-1b-ae4-0.network.virginmedia.net [212.43.163.213]
0/ 100 = 0% |
6 16ms 0/ 100 = 0% 0/ 100 = 0% glfd-tmr-1-ae5-0.network.virginmedia.net [213.105.159.46]
0/ 100 = 0% |
7 15ms 0/ 100 = 0% 0/ 100 = 0% tcl5-ic-1-as0-0.network.virginmedia.net [62.253.185.78]
28/ 100 = 28% |
8 --- 100/ 100 =100% 72/ 100 = 72% ldn-s2-rou-1041.UK.eurorings.net [195.66.224.54]
0/ 100 = 0% |
9 43ms 40/ 100 = 40% 12/ 100 = 12% 134.222.109.115
0/ 100 = 0% |
10 --- 100/ 100 =100% 72/ 100 = 72% srv248-e.ccp.cc [87.237.36.248]
0/ 100 = 0% |
11 22ms 28/ 100 = 28% 0/ 100 = 0% srv162-h.ccp.cc [87.237.39.162]

Trace complete.

Interesting that I get as far as CCP's servers...

edit: Problems seem to have gone away now.

Ceci n'est pas un sig.
Emrys Ap'Morgravaine
Deep Core Mining Inc.
Caldari State
#9 - 2012-01-17 19:22:38 UTC
Anything that's on the other side of the eurorings routing I'm now having problems with, which for me includes hotmail/MSN.

Thankfully steam and skype are just fine, so at least I can play "other" games :)
Emrys Ap'Morgravaine
Deep Core Mining Inc.
Caldari State
#10 - 2012-01-17 20:28:07 UTC
Note back from my contact at KPN.

They still have some circuits down for some of their enterprise clients, mainly Virgin. According to KPN, Virgin are still having problems with some of their backbone devices in London, and that's causing knock on's to other parts of their network (including the routing from virgin to KPN (Euroring).

So yeah, its not actually CCP's fault.

My guy will update me as and when they've got an eta on repair completion.

Em.
Othran
Route One
#11 - 2012-01-18 09:53:05 UTC  |  Edited by: Othran
For people in the UK especially Internet Report UK is quite useful. Edit - its still quite useful for people outside the UK if you look at the linx router graphs on that link. If you see a lot of latency/loss on those and its happening at the same time across multiple ISPs then its likely a LINX issue.

For example it shows up Virgin problems nicely - here, here, here and so on. On those graphs you can see that Leicester, Renfrew, Bradford had identical issues at the same time, suggesting its a Virgin problem, and it affects both primary networks (core-1a and core-1b) This suggests that someone has probably screwed up the edge routers - you'd have to be deranged to work on both core primary nets at the same time but its Virgin of course so who knows Blink
Emrys Ap'Morgravaine
Deep Core Mining Inc.
Caldari State
#12 - 2012-01-18 12:05:57 UTC
Just got off the phone with Virgin, and a delightful hour long shouting match. I swear their support gets worse every time I call.

Regardless of which however, I notice that the routing is now avoiding the eurorings nodes, and is back on Telia again. I'm still seeing some large spikes and packet loss @ 213.248.83.198 (eveonline-ic-138015-ldn-b3.c.telia.net) so there's still no chance of actually getting in game as it stands, or at least not for more than 30 seconds at a time.

Various nodes along the route on virgin's own network are still a bit lossy, but not to the same extent. Not by a long stretch.
Othran
Route One
#13 - 2012-01-18 12:22:31 UTC  |  Edited by: Othran
Ah I recognise that IP address which probably isn't a good sign Roll

On Be we now go straight in via Level3 (Global Crossing as was). Over the years I've learned that Level3 appearing anywhere on a traceroute is not optimal to say the least. Edit - at least we're not getting routed via the ******* Canaries (Tenerife to be precise) and back now - I kid you not, Telefonica are utter utter fuckwits.

Hope it gets sorted for you.
Sabr Sheppard
Brotherhood of Redemption
#14 - 2012-01-19 00:29:07 UTC
I'm getting near constant DCs from eve as well. Using ping plotter I see that I'm getting near constant 30% packet loss from level3 london (4.69.139.106). When packet loss gets to 50% I DC from eve.
Emrys Ap'Morgravaine
Deep Core Mining Inc.
Caldari State
#15 - 2012-01-19 00:36:30 UTC
Yeah, I aint undocking for **** till the connection stabilises fully again.

I suspect it'll take a while though, given the last few days.