It appears that something happened in the middle of the night (on the East coast) to affect Turbine’s services. Not only is LOTRO down but all of Turbine’s web sites and other games seem to be affected.
No official news on the outage has been posted on the alternate sources of Twitter and Facebook as of yet. Given my experiences with server outages, they probably want to be sure the cause is discovered and a recovery plan is in place before addressing the public with an ETA on when services will return. Plus – It’s crapping early for us on the East coast!
Not trying to make excuses for Turbine. I know folks are upset about not being able to play and I would be too if it were my prime playing time. However, these things do happen and they’re typically not the fault of a single person tripping over a cord or hamsters dying, etc. It’s usually something along the lines of hardware failure or power outage that is the culprit.
Turbine wants you to play their games and wants their websites to be active and I’m sure they are working hard to get things back up so you can get back into Middle-earth.
I will update this post when we have some official communication from Turbine about the outage.
Update 7:00am EDT: All sites and game servers are once again available.
Update 11:11am EDT: Sapience has posted in the forums an update on the situation but the primary cause is still unknown at this point.
We’re still investigating the root cause for the outage, but we do know that this was a sort of ‘pefect storm’ that impacted both primary and secondary sytems.
There are a lot of assumptions being made in this thread and honestly most of them are wrong. The nature of last night’s issues are entirely based in hardware and connectivity failures. The way they unfolded caused a cascade of issues that lead to communication issues both internal and external (with our International GM and CS teams).
For the record, the entire community team is on call 24/7. Community Managers like myself and Tolero (DDO) are on call 24/7/365 including vacations and Holidays with very rare exceptions. No one ever hesitates to call us at 2am if need be. It’s the job and we’re happy to do it.
That said, the nature of last nights issues was such, as noted above, that communications were impacted.
We’ll continue to investigate the root cause. We’ll address the issues and failures we discover, and will adjust policies where we discover they are needed as a result of those investigations.
We’ll share what information we can when we can and if we can. It’s probably worth noting that investigations like this usually are not done in a day, but over several days.
Source: Forum Post
July 26, 2012 at 6:31 am
I was wondering why I wasn’t able to access the LoTRO site this morning. Figured it was Chrome rather than their end. Oh well, everything seems to be up and working now, I just logged into the game with no issues.
July 26, 2012 at 6:34 am
Wow… That Farmer really doesn’t want to go to the Faire!!!!!!!
Perhaps he needs to go back to bed and just let the rest of us play.
July 26, 2012 at 6:37 am
I noticed that the servers were down through http://lux-hdro.de/serverstatus.php . And it would seem they’re up and running again.
July 26, 2012 at 6:38 am
Everything just went back online.
July 26, 2012 at 6:41 am
It was very unusual how it happened especially since people were able to stay online after the lag spike at around 1 AM EST. The lack of information though has to be changed. Even a skeleton crew could have taken a minute to type “we are investigating” and leave it at that. It makes me believe they no longer have an overnight crew on staff anymore.
July 26, 2012 at 7:07 am
Techie Folk usually don’t do any Customer Care stuff, that is left to the folk like Sapience and Celestrata and usually the public face of Team Turbine goes to sleep at night and the night crew isn’t allowed to talk to us… something about them being too sleep deprived to be left alone with all of us raging gamers needing a fix. They wouldn’t stand a chance. *giggle*
July 26, 2012 at 7:40 am
Usually when I have an issue in game late night and send in a ticket, I usually get a very fast response. I haven’t had any need to lately (not playing as much or as late), but I would figure that someone working CS would have noticed the servers being down and could have posted on FB or Twitter that they “are aware of the issue” at least, or even call someone who has access to post on those sites. That way players at least know that they know about the problem. That’s my problem with issues like this, they don’t always post anywhere that they know something is wrong.
July 26, 2012 at 8:21 am
yeah..i cudnt access the website on my phone so i thoght it was my phone..came bk and it wasnt working on my pc either..
July 26, 2012 at 8:30 am
This explains me not being able to log for such a time this morning.
Thank for the info !
July 26, 2012 at 8:36 am
From Sapience:
“Good morning all, first let me apologize for the lack of communication on this topic. We’re looking into this event and we’llthey you whatever information we can as soon as we can.
I’ll update this as soon as I get into the office and have a chance to meet with the correct departments and teams.”
http://forums.lotro.com/showthread.php?470927-Was-there-a-big-outage-or-was-it-just-me-who-couldn-t-connect
July 26, 2012 at 8:49 am
So was the Faire was delayed because of the outage?
July 26, 2012 at 9:38 am
I am in New Hampshire & we have been having some really rough weather. Tornato is Maine last night. The storms are coming through today & they are saying there are some tornato possibilities for us as well.
This may have something to do with it, being they in Mass.
Wyllo of the White Mountains
July 26, 2012 at 9:40 am
and a quick note .. I really dont spell as badly as that post was & my grammer is usually better then naught .. so sorry
July 26, 2012 at 1:27 pm
I found it interesting that Sapience used the same “perfect storm” and “primary and secondary systems failed” phrasing as he used last year (on my birthday) when the game went down for about 12 hours due to power problems. (Should have had the minstrels use Anthem of Composure + Coda!)
I’m sure Turbine doesn’t like to talk about its failures, but I wonder if the primary systems have every failed, and the secondary systems have saved the day. Or if the secondary systems have failed while the primaries bravely soldiered on. It seems like the only time we hear about primary and secondary is when they both fail at the same time. And I thought that only ever happened on the bridges of starships.
July 26, 2012 at 1:40 pm
I used this chance to try out rift.
Fun game!
Still <3 lotro though
July 26, 2012 at 2:21 pm
Tried to log in and play last night (PST Time), and no-go. This may affect my leveling initiative, which may receive a revision as was already behind, and this only put me further back. Business travel next week won’t help the cause. Oh well, what are you going to do. Just thankful that it’s few and far between for this games servers compared to most.
July 26, 2012 at 2:23 pm
Warning! Warning! Danger, Will Robinson!….oh, whew it’s over.
July 26, 2012 at 6:08 pm
Is LOTRO down again? I’m at work, but cannot access any Turbine websites…
July 27, 2012 at 3:17 am
I didn’t make any assumptions. I noticed the Turbine team utilizes twitter and facebook. I noticed they use them as an advertising tool rather than an option to provide any updates from a 24/7 staff. Those are live update locations for which someone on the 24/7 team could have accessed. Not having a power outage plan for a big mmo is sort of a no-no for a nice company owned by a not so nice big company..and it wasn’t a total power outage. Many of us were in the whole time. This deserves a more reasonable apology or explanation. I am all ears.
It’s not the Server failure but the response time on the Social media for all turbine’s followers that concerned me about the quality of service. Again.