OVHcloud Network Status
Current status
Legend
- Operational
- Degraded performance
- Partial Outage
- Major Outage
- Under maintenance
FS#7874 — GSW < > RBX
Incident
Report for Network & Infrastructure
Resolved
One of the links Paris GSW < > Roubaix is down.
We are investigating.
Update(s):
Date: 2013-01-10 05:58:57 UTC
*** CASCADED EXTERNAL NOTES 09-Jan-2013 23:40:07 GMT kepplerm From CASE: 6221887 - Event
The Transport NOC informed that the splicing team has been granted access to the fenced off civil works area and are now assessing the damage and how to proceed with repairs. The Transport NOC advised that the local contractor working in the civil works area is responsible for damaging the fiber and ducts.
*** CASCADED EXTERNAL NOTES 10-Jan-2013 01:06:36 GMT kepplerm From CASE: 6221887 - Event
The Transport NOC reported that the fiber cable repair is in progress at this time and
an estimated time for splicing to begin on the first end of the fiber cable is 02:00 GMT.
The second end of the fiber cable is expected to be prepared for splicing shortly. No
estimated restoral time is available currently.
*** CASCADED EXTERNAL NOTES 10-Jan-2013 02:38:20 GMT kepplerm From CASE: 6221887 - Event
The Transport NOC advised that the splicing team is nearly complete with the fiber cable
repair. It has been informed that alarms are beginning to clear but no estimated restoral
time could be provided at this time.
*** CASCADED EXTERNAL NOTES 10-Jan-2013 03:12:13 GMT caddeler From CASE: 6221887 - Event
The Transport NOC advised the splicing has been completed and all services have been restored.
Date: 2013-01-10 05:57:54 UTC
The link is performing.
RP/0/RSP0/CPU0:rbx-g2-a9#sh inter description | i gsw
Thu Jan 10 02:30:06.896 UTC
BE30 up up 1|gsw-g1-a9
BE32 up up 2|gsw-g1-a9
Te0/0/0/8 up up 1|gsw-g1-a9
Te0/0/0/9 up up 1|gsw-g1-a9
Te0/1/0/8 up up 2|gsw-g1-a9
Te0/1/0/9 up up 2|gsw-g1-a9
Te0/2/0/1 up up 1|gsw-g1-a9
Te0/3/0/1 up up 2|gsw-g1-a9
Te0/4/0/6 up up 2|gsw-g1-a9
Te0/4/0/7 up up 2|gsw-g1-a9
Te0/5/0/6 up up 2|gsw-g1-a9
Te0/5/0/7 up up 2|gsw-g1-a9
Te0/6/0/5 up up 1|gsw-g1-a9
Te0/6/0/6 up up 1|gsw-g1-a9
Te0/6/0/7 up up 2|gsw-g1-a9
Te0/7/0/5 up up 1|gsw-g1-a9
Te0/7/0/6 up up 1|gsw-g1-a9
Te0/7/0/7 up up 1|gsw-g1-a9
Date: 2013-01-09 23:35:58 UTC
*** CASCADED EXTERNAL NOTES 09-Jan-2013 20:46:58 GMT kepplerm From CASE: 6221887 - Event
The Transport NOC reported that the splicing team is on site and the replacement fiber
is expected to arrive at approximately 21:15 so that restoration efforts can commence.
*** CASCADED EXTERNAL NOTES 09-Jan-2013 21:45:09 GMT kepplerm From CASE: 6221887 - Event
The Transport NOC is reaching out to the splicing team to confirm that the replacement
fiber has arrived on site.
*** CASCADED EXTERNAL NOTES 09-Jan-2013 22:17:59 GMT kepplerm From CASE: 6221887 - Event
The Transport NOC informed that the splicing team is still working to isolate the exact
location of the fiber cut and it has been advised that the replacement fiber is on site
at this time. No estimated restoral time is currently available.
*** CASCADED EXTERNAL NOTES 09-Jan-2013 22:40:18 GMT kepplerm From CASE: 6221887 - Event
The Transport NOC advised that the fiber damage has been isolated to a fenced off civil
works area and the splicing team is currently awaiting access to be granted so that
restoration efforts can begin.
Date: 2013-01-09 23:34:15 UTC
Apparently, it is a very high cutoff between Trembley and Longueil-Sainte-Marie.
A reinforcement team were on site to accelerate the replacement of
hundreds of meters of fiber torn by works in the corner.
Date: 2013-01-09 23:27:22 UTC
PS: the long distance network of OVH is being upgraded to the new technology
100G coherent. This allows us to have protected links towards 2 optical networks.
And not to get impacted while cutting off the fiber,
even if it happens quite rarely (once within 2 years in Paris/Roubaix)
Date: 2013-01-09 23:24:14 UTC
We have tried to add 60G on the link always performing in Paris and Roubaix.
http://status.ovh.net/?do=details&id=3937
It didn't go well: One of the router cards has rebooted and crashed other cards.
Thus, we had also a disruption on the 2nd link along with the 1st which has last about 10 minutes.
The dead card has been declared with Cisco and we are awaiting the reception of the new card
within about 1H. As soon as we receive it, we will insert it in the chassis, wishing that it won't crash the chassis again.
Date: 2013-01-09 17:46:37 UTC
The cut is located at 64km north from Tremblay.
Teams are on their way to the incident location.
Date: 2013-01-09 17:45:52 UTC
The cable maintainer confirmed a problem with the cable.
A team is on its way to confim the fiber and find the location of the cut.
Date: 2013-01-09 17:44:54 UTC
We have contacted the fiber provider who confirmed that there is a cut.
Date: 2013-01-09 17:44:01 UTC
One of our fiber links Paris <> Roubaix is down, which greatly impact traffic to Paris.
We reroute a maximum of traffic to London / Amsterdam / Frankfurt in order to relieve our second link.
The problem lies between the amplification site of Marquillies and our pop in Globalswitch Clichy.
We do not yet know whether it is a fiber cut or an incident on our equipment in Globalswitch.
We are currently investigating a maintainer cable.
A team is also on their way to Globalswitch.
Date: 2013-01-09 17:40:25 UTC
We are going to cut the transit links in Paris in order the traffic to go to other routers.
We are investigating.
Update(s):
Date: 2013-01-10 05:58:57 UTC
*** CASCADED EXTERNAL NOTES 09-Jan-2013 23:40:07 GMT kepplerm From CASE: 6221887 - Event
The Transport NOC informed that the splicing team has been granted access to the fenced off civil works area and are now assessing the damage and how to proceed with repairs. The Transport NOC advised that the local contractor working in the civil works area is responsible for damaging the fiber and ducts.
*** CASCADED EXTERNAL NOTES 10-Jan-2013 01:06:36 GMT kepplerm From CASE: 6221887 - Event
The Transport NOC reported that the fiber cable repair is in progress at this time and
an estimated time for splicing to begin on the first end of the fiber cable is 02:00 GMT.
The second end of the fiber cable is expected to be prepared for splicing shortly. No
estimated restoral time is available currently.
*** CASCADED EXTERNAL NOTES 10-Jan-2013 02:38:20 GMT kepplerm From CASE: 6221887 - Event
The Transport NOC advised that the splicing team is nearly complete with the fiber cable
repair. It has been informed that alarms are beginning to clear but no estimated restoral
time could be provided at this time.
*** CASCADED EXTERNAL NOTES 10-Jan-2013 03:12:13 GMT caddeler From CASE: 6221887 - Event
The Transport NOC advised the splicing has been completed and all services have been restored.
Date: 2013-01-10 05:57:54 UTC
The link is performing.
RP/0/RSP0/CPU0:rbx-g2-a9#sh inter description | i gsw
Thu Jan 10 02:30:06.896 UTC
BE30 up up 1|gsw-g1-a9
BE32 up up 2|gsw-g1-a9
Te0/0/0/8 up up 1|gsw-g1-a9
Te0/0/0/9 up up 1|gsw-g1-a9
Te0/1/0/8 up up 2|gsw-g1-a9
Te0/1/0/9 up up 2|gsw-g1-a9
Te0/2/0/1 up up 1|gsw-g1-a9
Te0/3/0/1 up up 2|gsw-g1-a9
Te0/4/0/6 up up 2|gsw-g1-a9
Te0/4/0/7 up up 2|gsw-g1-a9
Te0/5/0/6 up up 2|gsw-g1-a9
Te0/5/0/7 up up 2|gsw-g1-a9
Te0/6/0/5 up up 1|gsw-g1-a9
Te0/6/0/6 up up 1|gsw-g1-a9
Te0/6/0/7 up up 2|gsw-g1-a9
Te0/7/0/5 up up 1|gsw-g1-a9
Te0/7/0/6 up up 1|gsw-g1-a9
Te0/7/0/7 up up 1|gsw-g1-a9
Date: 2013-01-09 23:35:58 UTC
*** CASCADED EXTERNAL NOTES 09-Jan-2013 20:46:58 GMT kepplerm From CASE: 6221887 - Event
The Transport NOC reported that the splicing team is on site and the replacement fiber
is expected to arrive at approximately 21:15 so that restoration efforts can commence.
*** CASCADED EXTERNAL NOTES 09-Jan-2013 21:45:09 GMT kepplerm From CASE: 6221887 - Event
The Transport NOC is reaching out to the splicing team to confirm that the replacement
fiber has arrived on site.
*** CASCADED EXTERNAL NOTES 09-Jan-2013 22:17:59 GMT kepplerm From CASE: 6221887 - Event
The Transport NOC informed that the splicing team is still working to isolate the exact
location of the fiber cut and it has been advised that the replacement fiber is on site
at this time. No estimated restoral time is currently available.
*** CASCADED EXTERNAL NOTES 09-Jan-2013 22:40:18 GMT kepplerm From CASE: 6221887 - Event
The Transport NOC advised that the fiber damage has been isolated to a fenced off civil
works area and the splicing team is currently awaiting access to be granted so that
restoration efforts can begin.
Date: 2013-01-09 23:34:15 UTC
Apparently, it is a very high cutoff between Trembley and Longueil-Sainte-Marie.
A reinforcement team were on site to accelerate the replacement of
hundreds of meters of fiber torn by works in the corner.
Date: 2013-01-09 23:27:22 UTC
PS: the long distance network of OVH is being upgraded to the new technology
100G coherent. This allows us to have protected links towards 2 optical networks.
And not to get impacted while cutting off the fiber,
even if it happens quite rarely (once within 2 years in Paris/Roubaix)
Date: 2013-01-09 23:24:14 UTC
We have tried to add 60G on the link always performing in Paris and Roubaix.
http://status.ovh.net/?do=details&id=3937
It didn't go well: One of the router cards has rebooted and crashed other cards.
Thus, we had also a disruption on the 2nd link along with the 1st which has last about 10 minutes.
The dead card has been declared with Cisco and we are awaiting the reception of the new card
within about 1H. As soon as we receive it, we will insert it in the chassis, wishing that it won't crash the chassis again.
Date: 2013-01-09 17:46:37 UTC
The cut is located at 64km north from Tremblay.
Teams are on their way to the incident location.
Date: 2013-01-09 17:45:52 UTC
The cable maintainer confirmed a problem with the cable.
A team is on its way to confim the fiber and find the location of the cut.
Date: 2013-01-09 17:44:54 UTC
We have contacted the fiber provider who confirmed that there is a cut.
Date: 2013-01-09 17:44:01 UTC
One of our fiber links Paris <> Roubaix is down, which greatly impact traffic to Paris.
We reroute a maximum of traffic to London / Amsterdam / Frankfurt in order to relieve our second link.
The problem lies between the amplification site of Marquillies and our pop in Globalswitch Clichy.
We do not yet know whether it is a fiber cut or an incident on our equipment in Globalswitch.
We are currently investigating a maintainer cable.
A team is also on their way to Globalswitch.
Date: 2013-01-09 17:40:25 UTC
We are going to cut the transit links in Paris in order the traffic to go to other routers.