Machetes aren’t usually tools a chief information officer would rely upon. But U.S. Virgin Islands CIO Tony Riddick and his team needed some long blades to hack through downed trees blocking the path to cell towers knocked offline by Hurricane Irma and Hurricane Maria last year.
Riddick and his counterpart in Puerto Rico, Luis Arocho, detailed their experiences bringing their island territories back online after two of the most devastating storms in U.S. history at the National Association of State Chief Information Officers Midyear Conference in Baltimore. Both CIOs described chaotic days following the storms, with much of the recovery work still to be completed in the months and years ahead.
“I’ve never seen so much destruction,” Riddick said. The Federal Emergency Management Agency and other U.S. agencies have already spent $800 million on the Virgin Islands’ recovery since Irma struck last Sept. 6. Nearly eight months after the storm, hospitals are still heavily damaged, and many of the territory’s 103,000 residents’ homes are still covered by blue tarps.
Meanwhile, 2017’s violent hurricane season has forced the Virgin Islands to make significant and lengthy upgrades to the infrastructure Riddick manages. After Irma and Maria knocked down at least 850 utility poles, Riddick said, the Virgin Islands is now putting its power, telephone, and fiberoptic lines underground. Riddick said that process will cost about $330 million.
Despite his talk of machete-wielding and his military experience, though, Riddick described a very troubled response to the hurricanes.
“A plan goes to hell when the first shot is fired,” the retired Army colonel said.
He called the training he and his 18-person staff had received “suspect,” leading to some early stumbles after the storms. The first emergency cell tower Riddick’s department erected after the territory lost all mobile coverage lasted all of eight hours after the generator powering it was stolen, he said.
Riddick said one reason why the Virgin Islands was so ill-prepared was that the last major hurricane to bring major damage was in 1989, before most of the technology affected by Irma and Maria existed. Among the losses the Virgin Islands sustained in last year’s storms was the territorial government’s data, which was backed up on devices stored in the same facility as the main servers. That building survived both Irma and Maria, which hit Sept. 19, but the electricity powering it did not. Riddick still hasn’t recovered the lost files.
“Have a backup of your backup,” Riddick told the audience. Specifically, he told his fellow CIOs to advise the governments they serve to put one backup in a separate facility and another in a cloud provider.
Puerto Rico’s situation was worse. Arocho rode out Maria in his data recovery center, which unlike most of the island, didn’t lose power, but its phones — land-based and mobile — were knocked out.
“I had power. I had internet connectivity,” he said. “But I couldn’t call anyone. Not my sons, not my dad, not the governor.”
Arocho said that after the storm passed, he got back in contact with Gov. Ricardo Rossello by walking through San Juan to the governor’s mansion. Puerto Rico lost 95 percent of its telecom services after Maria hit it on Sept. 20, and Arocho said most wireless carriers were unprepared for the devastation. He was hit with requests for generators, diesel and in one case, a request for the National Guard to serve as private security for one phone company. About 80 percent of the Puerto Rico National Guard’s personnel were unaccounted for in the days after the hurricane.
On top of the telecom issues, Arocho said other hurricane effects were tossed on his office, including scheduling when planes could land at a powerless airport and managing fuel and cash shortages after residents started hoarding gasoline and emptying their bank accounts.
“The CIO’s office became the [chief technology officer] for the whole island,” he told the NASCIO crowd.
One of the first bright spots in Puerto Rico’s recovery came in the form of NetHope, a humanitarian organization backed by tech companies, including Microsoft, Cisco, and Facebook, that restores IT services to disaster-stricken areas, Arocho said. More help arrived when Google brought Project Loon — a flotilla of balloons equipped with LTE signal generators — to the island. But implementation was delayed six days by the Federal Communications Commission’s approval process.
Puerto Rico is developing a new resiliency plan as it continues a long and costly recovery. Like the Virgin Islands, Puerto Rico’s plans include putting utility lines underground. Arocho said he’s also asking the companies that responded to Hurricane Maria’s destruction to leave some of their equipment behind so it can be deployed more expediently after the next storm.
“Make sure you learn from our experience,” Arocho said. “Make sure you have the things you need on-site. That was one of our key issues: getting things on the island.”