OVHcloud Web Hosting Status

Current status
Legend
  • Operational
  • Degraded performance
  • Partial Outage
  • Major Outage
  • Under maintenance
FS#8341 — Load balancing a10
Scheduled Maintenance Report for Web Cloud
Completed
We received the new pictures that fix the two problems that we have reported:
1. Memory Leak
2. Slowloris / Torhammer attack

We will update the boxes and we will restore traffic of shared hsoting and ovh. * above as usual.

Update(s):

Date: 2013-03-29 23:49:04 UTC
Obviously the new fireware does not work.
We will stop there.

Date: 2013-03-29 23:48:27 UTC
The box has strained 350K of simultaneous sessions and crashed under the attack.

p19-77-a10#sh session brief
Traffic Type Total
--------------------------------------------
TCP Established 318882
TCP Half Open 64456

p19-77-a10#sh log
Log Buffer: 30000
Mar 29 2013 11:21:29 Warning [AX]:conn proxy queue depth exceeds limit (2289001)
Mar 29 2013 11:21:29 Warning [AX]:conn proxy queue depth exceeds limit (2288001)
Mar 29 2013 11:21:29 Warning [AX]:conn proxy queue depth exceeds limit (2287001)
Mar 29 2013 11:21:29 Warning [AX]:conn proxy queue depth exceeds limit (2286001)
Mar 29 2013 11:21:28 Warning [AX]:conn proxy queue depth exceeds limit (2285001)
Mar 29 2013 11:21:28 Warning [AX]:conn proxy queue depth exceeds limit (2284001)

Date: 2013-03-29 23:46:23 UTC
crash after 12 minutes of the attack that we had on shared hosting. attack is managed properly by the ACE, but not by AX.

Date: 2013-03-29 23:44:37 UTC
We will reset traffic on one of the boxes which is protected against this kind of attack.

p19-77-a10#sh log
Log Buffer: 30000
Mar 29 2013 11:15:31 Warning [AX]:conn proxy queue depth exceeds limit (465001)
Mar 29 2013 11:15:30 Warning [AX]:conn proxy queue depth exceeds limit (464001)
Mar 29 2013 11:15:30 Warning [AX]:conn proxy queue depth exceeds limit (463001)
Mar 29 2013 11:15:30 Warning [AX]:conn proxy queue depth exceeds limit (462001)


Date: 2013-03-29 23:43:14 UTC
One of the boxes has crashed. We will remove everything.

Date: 2013-03-29 23:42:37 UTC
We found the bug in the configuration that was included in redundancy with the new code.
We removed it. It works.

We pushed kimsufi.com above. It works.
OVH. * It works.
All shared hosting : it works.

Date: 2013-03-29 23:40:32 UTC
We established a parallel configuration with a new IP in order to make tests without imapcting production.
Posted Mar 29, 2013 - 23:39 UTC