For computers that are fair, New York City forms algorithm task force

From water and heat to public transit, cities have laws to govern nearly every aspect of community life. New York City is now expanding this governance with legislation to monitor computer algorithms.

The New York City Council unanimously passed a piece of legislation Monday directing the creation of a task force that will advise policymakers and city staff on how computer algorithms are monitored and evaluated. The law says the new task force will study “how information on agency automated decision systems may be shared with the public and how agencies may address instances where people are harmed by agency automated decision systems.” 

While oversight bodies are common for other functions of government, this is one of the first charged with routinely monitoring government computer code for ethical violations.

As a supporter of the legislation, the ACLU lobbied for the bill and said in a blog post that algorithms demand greater oversight since they lay the groundwork for critical decisions made by government.

“Algorithms are sometimes thought of as the perfect embodiment of objectivity, but the truth is that they are highly vulnerable to human bias,” NYCLU Executive Director Donna Lieberman wrote. “As algorithms become ubiquitous, this bill provides an opportunity for the city to grapple with the critical questions they present, and develop policies that promote transparency and prevent abuse.”

In New York City, algorithms influence how first responders are dispatched, how business license applications are processed, they calculate department budgets and handle 311 requests like graffiti removal.

“A flawed algorithm can lead to someone being trapped in jail for no good reason or not receiving a public benefit,” NYCLU Legislative Counsel Rashida Richardson said in a statement. “This bill is the first in the nation to take such a broad view of the problem and recognize that, for algorithms to benefit society, they must be subject to public scrutiny and a mechanism to remedy flaws and biases.”

Even with its strong support from the city council, the task force will have the difficult job of determining how the city can monitor algorithms that manage its most sensitive data and critical services. 

The city has not yet announced when the task force will be established.