San Francisco discloses security flaw in public warning system

The San Francisco Department of Technology announced Thursday that it has patched a vulnerability to the city-county’s public siren system that would have allowed a malicious actor to take control of the system and potentially cause citywide panic.

The city first learned of the vulnerability in February after being notified by researchers from San Francisco-based security firm Bastille. A subsequent disagreement between the local government and the firm regarding how to handle the vulnerability’s remediation resulted in the two parting ways to release separate public notices of what was uncovered.

The city reports that a firmware upgrade to its system, provided by vendor ATI Systems, has “minimized the threat,” but new information to be released by Bastille on April 10 is expected to add new wrinkles to the story.

San Francisco’s public warning system, which includes sirens, voice and visual components, has never been used in an emergency since being installed in 1942 in preparation of air raids during World War II. The system — known locally as the Tuesday noon siren, because it is tested every Tuesday at noon — was repaired, upgraded and expanded in 2005 with federal funding.

San Francisco’s executive director in the Department of Technology, Linda Gerull, says her department and the Department of Emergency Management worked with ATI Systems to patch its system and are continuing testing.

Other ATI customers include large urban and rural local governments, universities, military installations, industrial sites including oil and nuclear power plants.

According to a draft public advisory obtained by StateScoop, the vulnerability found has never been seen before.

DoT Spokesperson Lauren Jones told StateScoop that a firmware update provided by ATI Systems adds a higher level of encryption to the system’s messaging, adding that before the patch, the system had weak encryption.

“Could it stop [a sophisticated actor]? I don’t know, but I think it would stop most hobbyists, pirate radio fanatics, ham radio enthusiasts,” Jones said about the new encryption.

The city’s notice states that “this encryption is designed to make each transmission unique and avoid logically detectable patterns in the messages. This communication structure will disguise the command format.”

That San Francisco’s public warning system might be used as a way to incite public panic is not mere supposition, but a reflection of how a national news audience reacted to an incident last year in Dallas, Texas.

Admitted failure by Dallas city government to configure its public siren system properly resulted in a frightening and confusing hours-long incident in April 2017 that entailed all of the city’s 156 sirens blaring through the night at regular intervals that prevented the system from overheating so the alerts would continue without a technical failure. The intrusion by an unknown actor to the radio-based system prompted more than 4,400 emergency calls to the city’s 911 center within a three-and-a-half hour period.

Dallas’ vendor is Federal Signal, the other major player in the public warning systems market.

Activated by the attacker through dual-tone multifrequency radio signaling, the city’s sirens were eventually silenced by the Dallas Office of Emergency Management when it turned off its entire radio system. The mishmash of unsecured new and old technologies that comprised the system precluded the option of simply turning off the sound or limiting outside access to stop the attack.

San Francisco did not have a prior relationship with Bastille. City officials say they do not have explicit plans for a future relationship, particularly in light of how the vendor approached with information about the vulnerability. Bastille reportedly began its communication with the city by requesting access to its warning system so it could reverse engineer the code. The city denied that request, officials said, preferring instead to resolve the issue through its vendor.

Bastille then requested access to ATI Systems’ code, a request the company declined, citing that its code was proprietary. An internal memo indicates that executives at ATI Systems did not appreciate the tone of Bastille’s request.

A Bastille representative declined to comment for this story, saying the company prefers instead to wait for the release — on April 10 — of its own account of what its researchers found.

The only other public incident involving the city’s public warning system was when sirens went off in several neighborhoods late one Saturday/early Sunday morning in Nov. 2014. After several residents called 911 and tweeted the San Francisco Department of Emergency Management, it responded by saying that there was no emergency, called it “a tech glitch” and announced that repairs were in progress.

Editor’s Note: This story was updated to clarify the level of encryption present in the patch applied by ATI Systems.