Morano questions monitoring failure and hours-long detection gap

Thank you, Chairs.

Thank you, Commissioner, for your testimony.

I want to approach this a little differently because we could spend a lot of time today talking about software changes, routing environments, testing protocols, and...

Technical architecture.

But if you're the person having a heart attack, if someone's breaking into your house or your child isn't breathing, you don't care what piece of software malfunctioned, obviously.

You dial three numbers and you expect someone to answer.

And after hearing the testimony today, what concerns me most isn't simply that there was this technical failure.

It's how long the city appears to have gone without realizing the full extent of that failure.

So we've heard that the software change was completed, and based on what you told my colleague, Councilman Wong, we've heard

incredibly that the testing protocol was followed and that the problem manifested afterwards.

So my question is, if the system passed the city's testing protocol, how did 911 calls begin failing at 3.13 in the morning without the city knowing it was happening?

According to your testimony, the problem began at 3.13 in the morning.

The first indication from a call taker wasn't until shortly before 6 o'clock.

A support ticket wasn't opened until about 6.40.

The issue wasn't escalated to Motorola and Verizon until about 7.30.

So for hours, New Yorkers were encountering problems calling 911 while the city apparently didn't know it had a system-wide issue.

What automated monitoring should have caught that?

Was there an alarm, a dashboard?

or failed or incomplete calls.

And if those systems existed, why didn't they detect it?

Because if the first effective alarm that something is wrong with 911 is a call taker saying, I can't hear somebody, then we have a serious monitoring problem.

Thank you for the question.

No, I think those are all the right questions to be asking.

I want to address specifically, and I don't know how much technical detail you'd like me to go into, but the testing that took place, part of the reason it didn't catch the error is because

The error didn't surface until there was a session timeout.

So sessions were live.

It wasn't until the TTAL, time to live, expired, which is when some of those issues started surfacing.

To be clear, we should make sure that our testing does capture all of those scenarios going forward.

In terms of automated monitoring, because of the way the issue presented, there was no automated monitoring that caught that issue.

It is, we do have a lot of monitoring in place.

It is challenging to be able to predict every anomaly or every misconfiguration that may happen that surfaces.

That is also not to say that we are not going to take a much closer look at what systems we can put in place.

into place so that this would have been caught earlier going forward.