Mind Readings: AI Bill of Rights, Part 2: Algorithmic Discrimination Protections

Welcome to part two of our review of the uh review of the United States uh the White House's proposed AI Bill of Rights. Today we're talking algorithmic discrimination protections. So let's see what this is all about. Algorithmic discrimination protections. You should not face discrimination by algorithms and systems should be used and designed in an equitable way.

Algorithmic discrimination occurs when automated systems contribute to unjustified different treatment or impacts disfavoring people based on their race, color, ethnicity, uh, sex or gender, religion, age, national origin, disability, veteran status, genetic information, or any other classification protected by law. Okay. And if the data going in is flawed, what the system's going to come out with is also going to be flawed. That is just basics of data, right? Garbage in, garbage out.

The problem is bigger when it comes to AI because you have, in some cases, systemically incorrect data going into the system. There's no way for the system to compensate for it. For example, there is no reliable healthcare data to build AI models on for black people in America. None. Because systemic racism has created centuries of inequitable, unfair health outcomes, right?

Uh health outcomes that uh as a result, if you built a model on it, would would try to predict or classify based on the past, right? That's all AI does. AI takes a lot of data and turns it into a model based on the past. And if the past is wrong, right, if the past is flawed, deeply flawed, anything that model comes up with is going to be flawed too, like life expectancy or cardiac treatment or things like that. Let's take a look at some of the examples in the paper here.

Number one, an automated system using non-traditional factors such as educational attainment and employment history as part of its loan underwriting and pricing model was found to be much more likely to charge an applicant who attended a historically black college or university higher loan prices for refinancing a student loan than an applicant who did not attend an HBCU. This was found to be true even when controlling for other credit related factors. Number two, the hiring tool that learned the features of a company's employees, predominantly men, rejected women's applications for spurious and discriminatory reasons. Uh resumes with the word women's, such as women's chess club captain, were penalized in the candidate ranking. I'm pretty sure that was Amazon's 2018 screw up, which which made all the major trade papers.

Number three, a predictive model marketed as being able to predict whether students are likely to drop out of school was used by more than 500 universities across the country. This is all USA based. The model was found to use race directly as a predictor and also shown to have large disparities by race. Black students were as many as many as four times as likely as their otherwise similar white peers to be deemed at high risk of dropping out. These risk scores are used by advisors to guide students away or towards majors, and some worry that they are being used to guide black students away from math and science subjects.

Number four, a risk assessment tool designed to predict the risk of recidivism, aka committing another crime, uh, in federal custody showed evidence of disparity in prediction. The tool over predicts the risk of recidivism for some groups of color on general recidivism tools and underpredicts the risk of recidivism for other some groups of color on some of the violent recidivism tools. The Department of Justice is working to reduce these disparities. That is based off of a 2016 case, I believe. Uh ProPublica did a story on it.

Uh the the trying to predict res recidivism, the tool was awful. It was like 20% correct. Um, but it over predicted on black and and people of color pretty substantially, and it was used, I believe, to um to deny parole. Couple other uh examples in here that I think are really important. Uh an automated sentiment analyzer, uh tool often used by technology platforms to determine whether a statement posted online expresses a positive or negative sentiment was found to be biased against Jews and gay people.

For example, the analyzer marked the statement I'm a Jew as representing a negative sentiment, while I'm a Christian was identified as expressing a positive sentiment. This could lead to the preemptive blocking of social media comments such as I'm gay. A related company company with this biased concern has made the data public to encourage researchers to help address the issue. Believe that was uh Facebook. And uh this last one here, uh yeah, uh searches for black girls, Asian girls or Latina girls return predominantly sexualized content rather than role models, toys, or activities.

Some search engines have been working to reduce the prevalence of these search results, but the problem remains. Yeah, who who else is that going to be besides Google? This category of algorithmic discrimination, like I said, it is based on past data, right? So if the data going in is bad, whatever the model comes up with is also going to be bad. There's no way to fix that, right?

There's no way to not ingest the biases of our society into these models without substantial mitigation techniques. Some of the techniques that are have been developed are pretty clever. They're used, um, IBM uses them in Watson Open Scale, where if you have, say, a database of 60% men and 40% women, it will flip, it'll basically randomly rewrite 10% of the data so that you know 50% of the uh men, and it'll take 10% of the men, turn them into women, and put them in the women column so that you get balance. The same is true for for race. Um, if you want to achieve proportional representation.

Here is the challenge with um some of these, some of the statements about equitable treatment and algorithmic discrimination. It is difficult to understand what fairness is, right? Fairness is a challenging topic because there's different ways to implement fairness based on different kinds of situations. There is no one size fits all. For example, if you are hiring and you are committed to a re fairness in hiring, what does that mean?

Does that mean you simply don't look at say gender on your job applications, right? Yeah, that reduces the ability for somebody to say, I don't want to hire men, or I don't want to hire women. But if the underlying population is skewed, it doesn't solve that, right? If your talent pool is 90% men, and you are aspiring to be 50-50, you're gonna have you're you know, just taking away gender is not gonna fix that, right? You would have to under-select the number of men and over-select the number of women to get to that 50-50 split.

So what constitutes fairness? There is no good answer for that, right? There is no easy answer for any of this. And those decisions have to be made by stakeholders, by system developers and engineers, and then somebody has to look at the outputs of these systems and say, yes, we are achieving our stated aim of fairness. And again, the paper goes through essentially very similar recommendations, uh, which is you know, make sure that you have diverse inputs, make sure that there's independent evaluation, make sure somebody's looking at what the machines are spitting out and dealing with it.

But at the end of the day, you do have systemic problems in society that machines are going to amplify, right? Unless you consciously say we're going to be unfair in the other direction to counterbalance it, or we're going to uh put enough noise in the data to to deal with the issue. For example, with with black people in healthcare. The if you're looking for optimal health care outcomes for black people, you can't use black people's healthcare data, right? Because it is corrupted, because it is it is uh systemic racism has reduced that.

So you would have to use the highest performing segment of uh people, like you know, in America, white people tend to have the highest uh positive health care outcomes and use that to train the model, or at least to the extent you can de-bias black healthcare data, right? So, you know, men should live to 78 years of age generally, regardless of of race. Uh women should live to uh about 80 years of age, regardless of race. And that can create its own problems because there may be specific healthcare conditions that are inherent to certain populations, right? Uh, for example, black people who come from African ancestry are more prone to sickle cell anemia because sickle cell anemia is an adaptation, uh is a it is a positive mutation that reduces the impact of malaria, which of course is a part problem that part of in parts of Africa.

And so there are unique mutations and things in healthcare data that if you just blindly use another population's data, it will it will still cause problems. For example, there's a genetic mutation in European folks of European descent that uh originally protected against black plague, right? Um, but uh it also causes autoimmune issues today. So the uh if you come from a year from European ancestry, you may have this positive mutation that reduces the impact of bubonic plague on you, but it also may cause things like Crohn's disease. So there's no easy answers here, but to say that as you create systems uh for for out for AI, you need to have diverse stakeholders who weigh in and vote, and and you are very transparent in how data is used in these systems and how decisions are made.

Um explainability and interpretability of your models is essential to be able to say here's how this data is being used. Here is how some data is being excluded or being protected. For example, you disallow a system from making uh judgments based on race and you also disallow proxies, right? If uh let's say you you um you're trying to disallow on on gender and you uh you say you protect that class but then you uh take into account things like books and TV shows and movies and music you know and you find that uh your system is making decisions based on whether someone's a Taylor Swift fan or not statistically you're starting to use a proxy piece of data in place of gender so we have to protect against those things too gotta look for those things but it comes down to having a plan having diverse inputs and then having independent inspection of the outcomes. Now you might be saying this sounds doesn't sound like a you know a marketing problem but this finds its way into marketing systems right if you are doing any kind of RFM analysis if you do you know if you're in retail, if you're in B2B and you're doing lead scoring you want to make sure that you are checking your systems against uh racial bias and other forms of disallowed bias, veteran bias, um genetic information bias, sexual orientation and gender identity bias.

You want to make sure that your systems are not using those factors in any kind of decisioning process because if this full bill of rights becomes law in some fashion, uh you will be held accountable for it. So get ahead of that by making sure that you are uh proactively looking for these problems. In the next uh section, we're going to s go from uh algorithmic discrimination to data privacy. If you like this video, go ahead and hit that subscribe button.