County Officials Offer Lessons Learned from Cyberattack
Officials at one of the state's larger counties by land area discussed a cyberattack their government suffered about four months ago, and how the agency will strengthen its security.
This story is limited to Techwire Insider members.
This story is limited to Techwire Insider members. Login below to read this story or learn about membership.
Imperial County, which borders Arizona and Mexico, was targeted twice on Feb. 13 by two sets of so-called Banking Trojan malware, although device event logs were overwritten, making it impossible to determine the connecting devices, Information and Technical Services Manager Henry Felix told the Imperial County Board of Supervisors at its Aug. 13 meeting. On April 10, employees received phishing emails with an invoice link, and on April 13, around 60 servers were encrypted with a “Ryuk ransomware variant,” Felix said. Officials were notified of the attack that weekend and learned that the attacker or attackers were seeking $1.2 million ransom in bitcoin. The county didn’t pay. Among the takeaways:
• The attack took down the county’s website and disrupted email and telephone systems for roughly two weeks, but the bad actors involved were not able to access any agency data, Felix confirmed. He described it as “extremely likely” the attacks in February and April involved more than one set of individuals. The county worked with the California Governor’s Office of Emergency Services (CalOES), which Felix said was “critical in helping us identify the vector and achieve containment.” The county also contacted the FBI, and is around 98 percent recovered from the breach.
Its cost to date, Board Chairman Ryan E. Kelley told Techwire, is around $1.9 million. Cyberinsurance will cover much of that, with the exception of a $50,000 deductible, Kelley said, noting that some infrastructure improvements outside the cyberattack needed to be addressed as officials rebuilt the system. The county contracted with Kivu Consulting as well as a third party to facilitate its recovery.
• The county’s backup server was also encrypted, and its recovery might have been much more difficult had the Board of Supervisors not authorized last year a tape backup to that second backup — at Felix’s recommendation.
“He brought it to the board and we approved it, not knowing what a significant role it was going to play in the events to come,” Kelley said.
Felix, who joined the agency in May 2018, said that was among his earlier recommendations.
“I had realized on arrival here, the backup strategy was exceptionally poor. I’m a big believer in the 3-2-1 backup strategy, and that’s what saved the day here,” Felix said, referring to the concept of maintaining three backup copies across at least two sets of media.
Once committing to a rebuild, officials were able to bring the county website back up over a period of about two weeks, Kelley said. The county had previously hosted its own Outlook, but subsequently brokered a new contract with Microsoft for Office 365.
• A complete service restoration will likely take the rest of the year, but the county isn’t just rebuilding. Before the attack, Imperial had contracted with Tyler Technologies to consolidate business permitting enterprisewide in the cloud, with a March delivery date. Post-breach, Kelley said officials are looking at implementing that type of architecture in areas including procurement, purchasing and accounting — and will discuss policy and procedures at a budget workshop on Tuesday. Not surprisingly, the county has beefed up its training. Officials have conducted phishing exercises on employees, the supervisor said, and will offer direct awareness training to new and existing staff.
• Asked for lessons learned, the supervisor recommended that agencies make plans for how to conduct business “without computer assistance or software” — and for later converting those paper records to digital. He also counseled governments and residents to remain patient during incidents or attacks, indicating that the county was criticized for not being more forthcoming during the attack’s early days.
“The reason we didn’t is because we didn’t want to show our hand to those who were asking for the ransom,” Kelley said, emphasizing that agencies should refuse to pay ransom.
Felix, who has a bachelor’s of science in IT from Western Governors University and was previously director of technical operations at the Imperial County Office of Education, recommended that governments require the appropriate skills when staffing technical positions.
“I think it’s important to, during the hiring process, ensure that leadership positions for information technology have a technical background," he said. "I think you’re setting up the department for failure if you don’t do that.”