Looks like the general anger with the whole SixApart – TypePad outage problems has finally gotten through to senior management at SixApart with CEO Barak Berkowitz emailing the following to customers of the service, but as one TypePad user pointed out to me, with no offer of compensation for the lost time and busniess caused by the outages:
As you might know, some of our users have been experiencing slow performance with the TypePad service over the past few weeks. Ben and Mena, our founders, posted about this earlier this week. I want to provide more details and give you additional insight into how we got here, and what we’re doing to fix the current problems.
How did we get here?
For over a year we have been growing the TypePad service and adding capacity as needed. This growth has been extraordinary, and we are now serving ten times the traffic we were serving a year ago. About three months ago we learned that the data center we were in had no more available power, with no option to obtain more. After some evaluation of options we decided that we needed to go through the process of physically moving out of that data center. And while we would need to grow capacity, we had to stay within the limits of the power available in our current facility.
At this point a race began to procure a new data center, buy more equipment and move data, and then servers, to the new facility.
We did a search for a facility that had the capacity for us to avoid getting into this situation again. We then leased space, numerous new servers and other equipment, added ten times the bandwidth of the old facility, got a dedicated data connection between the old and new facility and began to build out new cabinets to house the servers. Two weeks ago we began the move. Due to even more rapid growth than expected in the use of TypePad, we needed the capacity a week earlier than that. So basically we lost the race to move, and the excess load on the servers in the old data center started to cause failures in systems that have created poor service for you and, unfortunately, further slowed the move.
The engineering and operations team have been working around the clock to fix problems as they arise, while continuing the move to the new data center and get the new equipment in place and your data over to the new machines.
So where are we now?
Over the last two days we have made significant progress in relieving the stress on the servers in the old data center and completing the move. Below is a chart presenting specific improvements scheduled over the next few days and over the next few weeks. Many of these items were already mentioned in the post yesterday but I want to be sure you have all the data you need to understand the process.
(chart was here in original explaining the new data processing facilities)
As you can see a great deal of activity will be happening in the next few days. This Saturday night between the hours of 9PM ‘€“ midnight PDT, we have scheduled downtime to complete some of these activities. Specific details are posted on our status page.
By the end of next week, we hope to have all this behind us and to be back to providing you the superior service you have come to expect from TypePad. We ask for a little bit more patience until that time. We all can’€™t wait to get back to the situation where our customers are delighted with our service and happy to recommend it to their friends. Until that time please accept our sincere apologies and thanks for your patience.
Chairman and CEO
Six Apart, Ltd.