Plusnet Service

A power issue has resulted in one of our core database servers being offline. We have restored temporary power, and are now rebuilding the array.

This server runs the DR site reporting database, this problem does not have any direct customer impact.

If you were logged in to the portal, you could associate your username with this problem.

This has been passed to an engineer who has visited the site and restored power.

There is further remedial work that needs to be undertaken, however this cannot happen while the server is rebuilding its RAID array.

The array has rebuilt, a cut of the database is now being copied onto the server, ETA is approx 19:00 hours this evening.

The database cut is taking longer than expected, the revised ETA for the copy to be completed is approximately 11pm

The database cut has now completed and the server is back in the replication chain.

The server has 12 hours of replication logs to process, this will complete within a few hours.

This problem is no longer a P1, however there is further work that needs to be undertaken.

The server has redundant power supplies, one of these supplies failed yesterday, a case will be raised with Sun to replace the faulty power supply.

The server cabinet also has redundant PDUs (Power Distribution Units), unfortunately one of these also failed, this was the root cause of the problem. The reason that this was the cause is that it was the PDU that supplied power to the remaining PSU in the server that failed, this will also be raised to Sun to replace the failed PDU.

This work will be scheduled in on Tuesday.

A Sun Engineer has now attended site to replace the faulty power supply in the server,

The server now has resilient power.

Resilient power has now been restored in the rack, previously resilient power was provided from a neighbouring rack in our data centre.

This problem has now been closed,