Are Biased Algorithms Even Preventable?

Technology promises a wide range of efficiencies; one of the most promising and yet elusive is the removal of human bias from decision making. A recent paper published by New York University (NYU) researchers presents a glimmer of hope for a less biased future. The study found that during Covid-19, “the majority of Black borrowers who received aid from the $800 billion relief program got their loan from a financial technology company, not a bank.” The study explains that fin-tech companies and large banks that employed algorithms and automation demonstrated higher rates of loan approvals for Black borrowers compared to small banks not using the technology. 

In the study, the research showed that this distinction was even more pronounced in areas with higher instances of racial animus and that when companies switched to an automated process midway through the funding program, “the share of loans to Black-owned firms noticeably increased after the switch.” 

This glimmer of hope comes at a time when there is widespread criticism of biased algorithms, and while the failures of biased algorithms have led to painful repercussions, these errors are hopefully part of the solution. Unlike humans, algorithms and automation are easily changed and adjusted when issues are identified. Dr. Howell, a member of the research team at NYU writes, 

You can constrain an algorithm to meet fair-lending standards, and you can ensure the data it trains on isn’t biased. That may be hard to do, but it’s a clear and objective possibility. Whereas when you have a human loan officer who is in front of someone and making a decision, you can never do that.

As Dr. Howell points out, there are ways to reduce and eliminate bias in algorithms, although the trick is that these involve human intervention which is subject to bias. 

That being said, before assessing the data or algorithm, it is likely beneficial to investigate our own biases. By recognizing our own potential biases, we can be on alert as to whether they impact our work and if they do appear, we can course correct. 

Here are a few other ways to prevent biased algorithms:

1. Check that the historical data the model or algorithm is being trained on is not biased. For example, in 2015, Amazon created a tool to assess the likely success of an applicant given their resume. Amazon used historical data made up of mainly men and then tried to use that data to predict the success of a future candidate pool which included men and women. The program was shut down when it was discovered that it was discriminating against women. 
2. Check that the outcome of the model is not biased. For example, if two people have all the same characteristics but one is Black and one is White and the algorithm predicts that the Black person has a higher risk of incarceration, this would be an instance of bias. 
3. Check that the outcome of humans and the algorithms together do not lead to biased outcomes. We sometimes assume that when a person is given all the facts, they will make the right decision. But this is not true. Especially when it comes to risk. In a paper written by Ben Green, a postdoctoral scholar at the University of Michigan and an assistant professor at the Gerald R. Ford School of Public Policy, he found that, “People respond to risk assessments in biased ways..So, even if we were to say, ‘OK, this algorithm might meet certain standards of fairness,’ the actual impacts of these algorithms might not satisfy those constraints when you think about how humans are going to respond.” 

There is reason to hope that technology will play an influential role in creating a less biased future society but in order to get there, there is still a lot of work to be done.