|Uptime / Load||Sep 12||Sep 13||Sep 14||Sep 15||Sep 16||Sep 17||Sep 18|
cPanel Servers - UK
cPanel Servers - USA
DirectAdmin Servers - UK
DirectAdmin Servers - USA
Plesk Servers - UK
|EC2 (Elastic Cloud)||99.974%|
|Phantom Master HTTP Check||100%|
|Phantom Slave HTTP Check||100%|
|Brixly - Main Website||99.999%|
|Brixly - Client Area||100%|
|R1Soft - DC||100%|
|R1Soft - 01||99.986%|
|R1Soft - 02||99.998%|
|R1Soft - 03||100%|
Hypervisors - UK
Hypervisors - USA
| No/minor issues (>99.9% uptime) Short outage (>99% uptime) |
Issues reported Major outage (<99%) High Load Average
The server is now back online.
We are aware of an outage on Alfa, and we are investigating this currently.
Please note that the senior team will be working on this until resolved / back online.
There is a final batch of restores which are in progress, however, in most part the accounts should be restored.
If you do have any issues, of course, get in touch so we can check for you.
Please note that the restore process would have retained the same PHP settings and variables, however for those sites using Bolt-Cache or CloudNS, this would need to be reconfigured / enabled.
Again, I cannot appreciate more the patience from you all.
As a courtesy for the issues caused, I will extend all of the services on the echo server to provide 3 months free of charge.
To add also, we are also implementing changes to ensure this simply cannot happen moving forward - we have already purchased and deployed new 'hot spares' in terms of full server deployments at a significant expense, and have also created some internal plans to improve both communications throughout such issues, as well as the implementation of a number of additional systems on our monitoring platform.
Thank you to all of you for your continued support.
Hi there, Just a quick update - We have found the issue to relate to multiple disks failing within the server, which is unfortunately irrecoverable as the RAID data is no longer intact.
Its an incredibly rare set of circumstances (the first in 6 years), where two drives have completely failed at the same time, failing parity set entirely.
To resolve this as best we can, we have already dispatched a replacement machine which we will need to restore from backups.
This, unfortunately, is an extensive process which takes time to complete. I do appreciate the frustration completely, which is why we will work throughout the night to attempt restoration of the accounts and the data. Please do note, however, that the backups will be up to 24 hours old, so there will be potential for some loss of data since the previous backup took place. Due to the explicitly rare nature of the case, and the significant inconvenience caused we will, of course, provide compensatory credit to your account as per our SLA terms and conditions. The amount of this will be calculated on restoration, depending on the amount of time taken to resolve.
Meanwhile, we appreciate your patience sincerely.
Kind regards, Dennis
The echo server is currently offline due to a hardware related issue.
We are investigating the best route for getting this back online immediately.
Apologies for any inconvenience caused.