The master and slave heads are not responding.
An intervention is underway.
Update(s):
Date: 2014-12-10 15:34:57 UTC The logs were reassembled and show a transfer from master-> slave at 4:37 following a fault on the master side.
The slave has froze when importing pools and the cluster found itself in situation where master and slave were unreachable and started but not completed failover.
It was necessary to analyze the situation by each datastore and also check the status of each SAS link, all this took time and this the price of data integrity.
Datastores were unavailable from 4:37 am to 6:32 am.
There was no impact on the integrity of data.
We will strengthen the checks that are performed when switching processes. Also we will strengthen the import simulation tests on slave heads to prevent cases where the slave is not fit 100% to take over in case of fault on the master side.
Date: 2014-12-10 08:49:29 UTC We are investigating the cause of the issue.
Date: 2014-12-10 08:48:47 UTC Service has been reestablished.
Date: 2014-12-10 08:48:03 UTC The following data stores have been impacted.
pcc-000594
pcc-000595
pcc-000596
pcc-000597
pcc-000598
pcc-000599
pcc-000600
pcc-000604
pcc-000605
pcc-000670
pcc-000677
Date: 2014-12-10 08:47:07 UTC The data stores pcc-000596 and pcc-000604 have been impacted.