The public wasn’t notified about a 911 outage until about three hours after problems began because the Honolulu Police Department was struggling to come up with an alternate means of communication, officials said Monday.

Police are investigating what caused the 911 outage and failure of the regular backup system.

The department’s dispatch center started experiencing problems at 10:38 a.m. Sunday that gradually worsened over the course of a few hours, police Capt. Matt Kurihara said during a press conference at police headquarters.

He said it took communications officials hours to figure out the extent of the issue and come up with a backup solution that could be communicated to the public. 

Members of the public were finally notified about the outage at 1:50 p.m., when the city’s Department of Emergency Management pushed out an alert over the Integrated Public Alert and Warning System, department director Jennifer Walter said in a telephone interview.

Kurihara told reporters police still don’t know what caused the outage nor whether that was linked to the backup system failure. 

‘Something That We Didn’t Expect’

While there have been 911 outages before, including a brief issue reported in Honolulu and Kauai counties in April, this was the first time all of the backup systems failed at once. No other counties were affected.

“This is something that we didn’t expect. Usually we expect our backups will work,” he said. “This was the first time we had the experience where all of a sudden that doesn’t work either and now what do we do.”

He said the major cellphone carriers and Hawaiian Telcom, which provides phone, internet, wireless and television services across the state, are investigating what caused the issue. 

“Unfortunately, at this time, any technical questions like what happened, what exactly broke or what was fixed, I’m not able to answer those questions,” Kurihara said. 

Hawaiian Telcom spokeswoman Ann Nishida Fry said in a statement that questions should be directed to the wireless carriers. 

Lauren Peterson, a spokeswoman with Verizon, said in a statement Sunday’s service problem was a “third party issue that impacted all carriers.” She did not respond to follow-up questions about who the third party was. 

Sarah Rodriguez, a spokesperson with AT&T, said in a statement that the company has been in contact with Hawaiian Telcom as they investigate the root cause of the problem. 

Alert Timeline

Police dispatchers at the Joint Traffic Management Center, which houses dispatch for multiple agencies, including the Honolulu police and fire departments and the Honolulu Emergency Services Department, started to notice the problem Sunday morning when they weren’t able to transfer 911 calls to the fire department, Kurihara said.

Normally, police dispatchers take all 911 calls, then route them to EMS or the fire department depending on the type of emergency. Because they couldn’t re-route the calls, police dispatchers started taking down all of the information and bringing it over physically to the correct department within the Joint Traffic Management Center, he said. 

But the problem worsened as time went on. 

“It seemed like problems just gathered,” he said. “We eventually learned we were not getting 911 calls from the major cell carriers, T-Mobile, AT&T, Verizon.” 

Officials in the communications division started looking at the backup systems, including its “trunk line,” which is an additional phone line dedicated to receiving incoming 911 calls when the primary system is down. But that wasn’t working either. Calls were also not going through to most substations across the island’s eight police districts, he said. 

Only those calling 911 from a landline were able to get through, he said. The vast majority of 911 calls come from cellphones, he said.

Alternate Numbers

Regular calls between cellphones were still working, though, he said, so the communications division asked police officers with department-issued cellphones in each of the districts to start handling emergency calls for their area. 

The emergency alert that went out just before 2 p.m. instructed anyone experiencing an emergency to either call 911 from a landline phone or call one of a series of alternate numbers depending on what district they were calling from.

Walter said her department was notified about the 911 outage at around 1 p.m. but couldn’t push out the alert because the police department did not immediately provide the alternate numbers for callers. The emergency management department drafted the alert, and police provided the alternate numbers about 50 minutes later, she said. 

“As soon as we had the information we published them within a matter of a few minutes,” she said. 

Most of the alternate numbers provided were officer cellphones, except for a few that were numbers for district stations that weren’t affected by the outage and were still receiving calls, Kurihara said.

One number was provided for each police district, but District 4 on the Windward side had three numbers — one for Kailua, one for Kaneohe and one for Kahuku. 

When pressed on why it took the department so long to notify the public about the outage, Kurihara said it took the communications division time to figure out the extent of the problem and find a solution. 

“It’s not like on initial onset we knew everything was broken,” he said. “It seemed to happen gradually.” 

By 3:05 p.m., the outage had been fixed and the Department of Emergency Management issued a second alert notifying the public that 911 was working again. 

Kurihara said he didn’t know if any emergency calls were missed during the outage, but he hadn’t heard of anyone who experienced a major emergency and did not receive help. 

“We were able to, as far as I know, avoid any major catastrophes or incidents,” he said. 

Shayne Enright, spokeswoman for the Honolulu Emergency Services Department, said her department has not been notified that anyone received delayed medical help because of the outage.

‘It’s Falling Apart’

Kurihara said the department will look at what can be done to strengthen its backup systems but didn’t provide specifics. 

“We need to make sure we’re better prepared for any such contingencies,” he said, “but I would like to say that I think we did quite well considering this was on the fly.” 

Walter said she would like to see all the agencies involved do an after-action assessment to see what can be learned from the incident. 

The Department of Emergency Management has already started looking at what it can do better next time, she said.

For example, the department experienced an issue with the audio portion of alerts that went out via radio and television broadcast on Sunday. While the push alerts that went out to cellphones and rolling messages that appeared on people’s TV screens displayed the correct information, the audio portion of the alerts that people could hear on their TVs and radios were dictating a placeholder message that hadn’t been updated with the pertinent information, she said.

Department staff realized immediately and sent out a second alert with the correct information, she said. The department has already updated its protocols to make sure that in a future emergency situation, a person is designated to monitor the audio alert that goes out in addition to the written message. 

She said this is the first time the Department of Emergency Management has had to use its Integrated Public Alert and Warning System to notify the public about a 911 outage.

The dispatch center has also been struggling with major staffing shortages that has caused burnout and low morale.

Kurihara said 13 dispatchers were on duty Sunday. The department usually aims to have a minimum of 18, he said. 

Lakea Tjomsland, an emergency dispatcher and radio operator in Honolulu, said she wasn’t on duty Sunday but she has worked during previous 911 outages and it is “beyond stressful.” 

She said the staffing shortage is the biggest issue affecting the dispatch center and wants to see the department, which has prioritized officer recruitment, put more focus on recruiting and retaining communications professionals.

“The system and integrity of the 911 process, it’s falling apart,” she said, speaking as an individual and not as a representative of the police department. “It’s literally busting at the seams.”