You Ask, I Answer: Third Party Data and Trustworthiness?

In today's episode, Ashley asks if you choose to use public data sets for your uh machine learning models like from Amazon or Google, can you trust that those are free of bias? Hmm. Well, so there's a couple of different things here. Uh companies like Amazon and Google don't typically offer data sets. What they do offer are either models uh or uh APIs of some kind.

So Amazon, for example, has recognition and SageMaker and all these things, and these have APIs behind them that have pre-trained models. Google, uh many of the services in Google Cloud perform perform the same way. Uh for example, Google Speech to Text, things like that. Google also does release actual models themselves, like the T5 Transformer Library, which you can uh install into like Google Colab uh or your local Python environment and use uh their pre-trained models. And then there yes, uh for example, Google Datasets does offer uh raw data.

Now, let's talk about that. Can you trust uh that these are free of bias? In a word, no. Uh in a word, you cannot blindly trust anyone's uh uh machine learning models, data is that are to be free of bias because you don't know what's in it. So as an analogy, suppose that uh you have a a jar of jalapenos, right?

Uh a nutrition label that says uh five calories per serving, and what would I expect to see in this jar? Jalapenos, right? As an ingredient, and probably vinegar and water, right? Because that's maybe some salt. That's what's in here.

Can I trust that that if I just look at the label alone that that's what I'm getting? Well, when I look at this, I go, let's see, jalapenos, water, vinegar, salt, dehydrated onions, dehydrated garlic, calcium chloride is a firming agent, sodium benzoate is a preservative, polysorbate 80, which why is that in there? Turmeric for color. Why is polysorbate 80 in here? You don't need an emulsifier for peppers in a jar.

Anyway, um can I trust what's on the label? Can I trust that what's on the label is is accurate? For example, where did the jalapeno where are they jalapenos grown? Uh were they grown free of most you know harmful pesticides? Um in this case, this jar is not labeled organic, so probably not.

Um the other hand, if you're in the EU and you had this exact same product, could you trust that it was free of pesticides? Yes, much more so because EU regulations for foods are much more stringent than the United States. The same analogy applies to machine learning and data science. What uh the model says is important, but also what went into the model to make the model is just as important to be uh free of bias, to be free of uh both kinds of bias, both human and statistical. There are, for example, uh any number of cases of bias that was unintentional.

Uh somebody did not mean for their data set to be uh biased, or did not mean for their model to be biased, but it was because they didn't do any due diligence when putting it together. Most uh probably famous case of this is Amazon, when it attempted to uh build uh hiring uh AI to screen resumes. They trained it, they weren't looking for bias, and the model stopped hiring women. Right? Uh, because nobody did any checks.

So, what's the solution? Uh can you build from these systems and trust them? Well, there's two different ways to handle this. First, um, the first is to build your own model, which is expensive and time consuming, but it is the only guarantee that the data going into it is trustworthy because you will have vetted it and made it trustworthy and tested it. Uh, if you are uh somewhat familiar with Python, uh IBM has the Fairness 360 toolkit, which is a phenomenal toolkit, totally free, totally free, to test data sets for bias.

And if you are building your own model, you would use that to uh to validate your data before the model is constructed, and then you can be reasonably sure that your model uh uh is free of at least of data going in being biased. You do still have to monitor it. You still do have to have, for example, the protected classes that you're monitoring for declared, and you still do have to ensure that the model when it's running is not drifting out of the the uh rails that you set for it, the guardrails. For example, if you said that you know uh gender must be 50-50 split, um for 40-4010, uh 40-4020, um, then you would have to monitor and say, okay, how far outside is acceptable to you know, is a 1% drift acceptable, is a 5% drift acceptable? At what point do you say, hey, we need to either rein the model back in, retrain it, um, or balance it in some way to get it back on the rails?

So that's one aspect is uh the you have to build it yourself and train it and monitor it. Two is a risk judgment. Um trust in a model is proportional to the risk that you're incurring with the model. So if I am building a machine learning model to recognize sentiment in tweets, how vitally important is that going to be? No one's probably going to die.

If I'm using it for like social media engagement monitoring, probably nobody's going to die. I might make some bad judgment calls. I could cause some damage to my brand, but for the most part, it's not super serious. On the other hand, if I am producing, say, a new vaccine, it had better be really, really, really unbiased. It better be really representative.

It had better be really any model I build to try and assess the efficacy of something or uh identify a drug candidate had better be pristine in its freedom from bias because it could actually kill people. The risk level is substantially higher, so the the standards that we must hold that model to are much more stringent. Facial recognition for say, like at a trade show booth, relatively low risk, right? If you misidentify somebody for you know as as a gimmick to attract people to your trade show booth, not huge. Facial identification being misused by police, big deal.

A life-threatening deal. Um so you had you'd better make sure that that model is properly trained and unbiased. You know, a lot of these uh models and data sets and pre-trained models and APIs from major vendors, uh, is what is the level of risk, and what is or the consequences if it gets it wrong. Bear in mind that an awful lot of machine learning models are biased, especially in facial recognition uh and in natural language processing. Natural language processing has a lot of hidden biases.

The most obvious one of which is uh most of them are trained on the English language. And English is, uh I forget who said it, is a language of privilege. Uh it is the a language of the wealthier part of the world. You know, it is not the majority language in the world. Um, and there are many, many, many, many billions of people who speak other languages.

And our many of our machine learning models are not well suited to recognizing or processing those models. Now, if you think some of the the things that AI does with English are hilarious, you should see what they do to other languages. Um when you give that some consideration and who speaks English and what race they are, and what gender they are, and what income level they are, um, what ethnicity they are, uh, what religion they are. You can see how even something as simple as using the English language could introduce biases into your models. So keep that in mind.

It's all about trust and risk. How much trust do you need in the model? How high is the risk? Uh, and that dictates uh whether you should be training your own versus using a third party's. If you have follow up questions, leave them in the comments box below.

Subscribe to the YouTube channel and the newsletter. I'll talk to you soon. Take care. Want help solving your company's data analytics and digital marketing problems? Visit TrustInsights.ai today and let us know how we can help you.