You Ask, I Answer: Vetting Marketing AI Vendors for Bias?

In today's uh episode, Tracy asks, what are some questions you should ask vendors to better understand what data they use in their algorithms to make sure it's not biased? So it's not a question, well, it's not just questions we need to ask. Bias is like any other um any other professional vetting that you want to do in that there are things to look for and things to request from a uh a vendor, the same way that you would uh vet a vendor for equal opportunity uh employment for non-discrimination for fiduciary responsibility. Uh there are so many different aspects to auditing and doing your due diligence on a company. And checking for uh bias and artificial intelligence and machine learning models really shouldn't be any different than checking to see if a vendor is you know Title VII compliant, right?

If the vendor discriminates against uh people uh in hiring, you probably would want to do that and know that. You know, when you look at uh any of these audit forms that you're required to fill out if you've ever been through a corporate audit, they're delightfully fun. Um but there are lots of questions about you know what's your process around hiring, what's your process around uh alignment to the Equal Opportunity Employment Act, uh all these different ways to look for problems. When it comes to um bias in AI and dealing with vendors, it's important to understand what kinds of bias to look for. There's six places you want to look for it.

And we've got other videos in this show. If you want to head over to the YouTube channel, you can see um you know, better definitions and stuff. But the six areas, I'll bring this up here. The six areas where bias creeps in uh in AI and machine learning are people, strategy, data, algorithms, models, and actions. So let's talk about each one of these as it relates to a vendor.

Number one, people is easy. Um is has been hired, right? Who are the people working on the models and algorithms? Um, who are the people building the software? If you look at the development team uh or the engineering team, and you see a complete lack of diversity, there's probably gonna be a problem, right?

Even if it's not intentional, just having a uh monolithic view of the world, if it's a bunch of you know, um say 20 mid-20s Caucasian males, that's your whole development team, they have a natural mindset that does not include people who are black, because they're it's not in their experience, right? It's not saying that they're they're bad people, it's just they simply do not have the experience. If none of them are female, they have no frame of reference for things that females, people who identify as female might be interested in, right? So that's an easy one. Look at the people, look at the composition of the people, look at the diversity of the people, and if you don't see any diversity, you know there's a problem.

This, by the way, applies not just to AI and machine learning, but to every vendor. If you're hiring like a PR agency, go to that agency's leadership team. If you see a whole bunch of uh people who look exactly the same, there's a diversity problem. There is which means there's a diversity of ideas problem. Second, strategy is where bias can creep in.

What is this the the strategy that somebody's going for? Really good example of this. Facebook has a strategy of engagement, right? They care about getting eyeballs stuck to their site, which means their algorithms tend to promote things that keep people engaged, like making people angry and afraid all the time. Um and so there the outcomes from that strategy have been as we've all seen uh pretty substantially negative, right?

Um we see you know flourishing of hate groups and all these things because that's the strategy. Um did they intend to allow like Nazi groups to flourish? Probably not but it is a natural outcome of an incomplete strategy or a strategy that was not informed by uh a diverse set of objectives? Yes. Third, data.

Bias creeps in in data. Uh where did the data come from? Right? Where uh this is uh what's called data lineage or data provenance. Uh how good is the data is the data itself balanced is it representative IBM has a fantastic uh toolkit called uh the IBM Fairness 360 toolkit uh if you're fluent in Python you can download this for free run it on your data um declare any protected classes things like you know age gender uh veteran status disability sexual orientation gender identity race uh religion and in your data it will then say hey this model does not look representative or this model has a lot of drift or this model uh is or this data is likely to behave badly um so checking your data the lineage of the data is important where the where did your data come from if your data came from uh sources that themselves are biased uh that can be a big problem for example uh black American healthcare all the data is wrong right because of systemic racism, you cannot get really good large scale data on black American healthcare because there isn't good data.

Systemic discrimination has created an entire pool of corrupted data. Number four, algorithms. So the algorithms are the individual choices that you make uh for uh what your model is going to do, what strategy you're going to pursue pursue from an algorithm point of view. This is things like deciding if you're going for like gradient boosting uh for uh generalized the linear regressions, all these different choices. Bias can creep in here because if you have somebody who doesn't understand the full objectives and doesn't have a background in a diversity, they may choose a computationally efficient algorithm, but not necessarily one that is fair.

So this would be a case, for example, of using you know something like uh a straight up uh gradient boosting model versus something like Pareto uh multi-objective optimization. The algorithms are very different. Pareto optimization allows you to essentially do what's called trade-off analytics. You will get a less well-performing model, but it uh it performs against you know many, many different objectives as opposed to one objective, kind of like what Facebook versus like LinkedIn, how they function, they function very differently because of their optimization uh algorithms. Number five, the model itself.

The model can drift. Um the model when it takes in data, as it takes in new data over time, it can drift. The most famous example of this is the Microsoft Tay chat bot, uh, which was corrupted by trolls basically. Um within 24 hours, it became a porn spewing neo Nazi uh chat bot. It was it was trained properly, but it drifted and it drifted and it didn't have guardrails to keep it on the rails.

So that's a place where bias can creep in. And last is the the actions. What do you do with the model? Right? What do you where what are you going to use this model for?

Um, this is a good example of this is a Martech vendor I saw that was effectively reinvented redlining, right? They they built a uh tool to identify ideal customers and it reinvented redlining. And so bias crept in in what their model was going they're going to do with the model. So that's in a very short period of time. A all the places that bias can creep in throughout the process.

When you're auditing vendors, when you're doing a due diligence, ask them for their documentation about how they prevent bias in each of these areas, right? You would not get on a plane if you walked in the cockpit and you saw there was no quick reference handbook, and there was no pre-flight checklist, and you know the pilots are just kind of winging it, right? Do not get on that plane. Um because that is an unsafe plane. There's no documentation, there's no process, there's no validation that things are working uh as they should be.

The same is true with AI uh and bias, right? If a company has no documentation, no processes, no uh rigor, no checking for bias in each of these areas with real checklists, like real documented checklists. Here's the bullet points that we look for at each stage of our projects, then there's a good chance bias is crept in. And in turn, that means there's a good chance that the uh what they produce is also biased too. So look for those, ask for those as part of your process.

And if they can't produce it, it's probably a problem, right? That's the the easiest way to vet a vendor. Ask them for the documentation, call it part of compliance or whatever. Um and the vendors themselves should recognize that if they don't have this, they themselves are at legal risk, right? Because they can't prove that they're not biased.

So, great question. We could spend a whole lot of time on this. Uh great question. If you have follow-up questions, leave them in the comments box below. Subscribe to the YouTube channel and the newsletter.

I'll talk to you soon. Take care. Want help solving your company's data, analytics, and digital marketing problems? Visit trustinsights.ai today and let us know how we can help you.