Utah auditor issues recommendations to reduce bias in government software

Utah State Auditor John Dougall on Monday announced a series of recommendations for the state government’s software contracting to follow aimed at improving residents’ data privacy and reducing discrimination.

The 12 proposed guidelines were developed by a commission formed last year after it was discovered that the founder of a surveillance company hired by the state attorney general’s office had ties to violent white-extremist movements. The company, Banjo, had been building a $20.8 million “live-time” surveillance network for Utah law enforcement agencies, with access to social media, 911 feeds, police-dispatch calls, traffic cameras and other sources.

But Utah Attorney General Sean Reyes suspended the contract last April after reports surfaced that Banjo’s founder, Damien Patton, had spent part of his youth in a chapter of the Ku Klux Klan, a time during which he participated in the drive-by shooting of a Tennessee synagogue 1990 and attended meetings where speakers discussed exterminating Black and Jewish Americans.

Patton, who was 17 at the time of the synagogue shooting, pleaded guilty to juvenile delinquency; his past was uncovered only last year after the publication of grand jury testimony he gave in 1992 against a fellow Klan member. He wrote on Banjo’s website that he “extremely remorseful” for a “dark and despicable period.”

The discovery of Patton’s past prompted calls in Utah for reforms to how the state purchases technology that can be used for law enforcement and surveillance, including concerns that Banjo’s network would actually anonymize the data it collected, or that it could be used to discriminate against minority groups.

Dougall created the commission a few weeks after Reyes suspended his office’s work with Banjo. The panel included university professors, technology executives, civil rights activists and law enforcement representatives. The report the group released Monday makes several recommendations regarding data collection, including that state agencies limit their sharing of sensitive information “to the greatest extent possible to protect individual privacy” and that software used by government minimize data accumulation.

“A software application should collect no more sensitive data than it needs, and should retain that sensitive data no longer than it needs to accomplish the specified purpose,” the second recommendation reads.

The commission also recommended that vendors who offer AI-based products validate and demonstrate how their technologies collect data. It called on all vendors to mitigate discrimination, especially those offering facial-recognition products, which are frequently charged with perpetuating racial biases.

Background checks

And in what appears to be a direct response to the discoveries about Patton’s history, Dougall’s commission also called on state agencies to more thoroughly investigate the backgrounds of potential vendors’ leaders.

“Key vendor personnel may need background checks,” the report reads. “The type of checks may vary depending upon the sensitivity of data that personnel have access to. These checks need to be validated to the procuring government entity.”

Members of the commission said they hope Utah’s government agencies will follow their suggestions.

“Technology is growing increasingly powerful and it is very easy for discrimination to be perpetuated unless our public entities are careful to prevent it,” Jeanetta Williams, the president of the NAACP’s Salt Lake City branch, said in a press release. “These principles and companion questions will help our government entities to be wise in their procurement of advanced technologies.”

And while the commission was born out of an incident involving a law-enforcement contract, Dougall said he hopes all agencies across Utah will follow its recommendations.

“As the depth of expertise on the commission would not be easy to re-create for every potential contract for every agency or entity, these new documents help capture that expertise for use statewide to help protect Utahns’ privacy and prevent discrimination against them,” he said.