You Ask, I Answer: Can AI Solve Word Problems?

In today's episode, Ravi asks, can AI solve word problems? This is a question from the YouTube channel. Um, it depends. It depends on how we define word problems, what kind of problems we're trying to solve using words. AI techniques in a domain called natural language processing, absolutely can take words, take text, and process them and then deliver useful outcomes, deliver some kind of uh analysis that can help us make decisions.

Uh super simple example will be something like sentiment analysis or emotions and tones based on the language people use in writing, can we ascertain using AI the tone of a piece of text? And the answer is yes. Uh, we can do it. And the accuracy rate, depending on how you're using it, uh and which library and which technology can range anywhere from 70% to uh 95% accurate. Uh again, it depends on how much uh compute power you have to throw at it and and such like that.

Can computers and and machine learning techniques understand language? Uh that is not within their reach yet. And a really good example of this is if you go to any of the tools that allow you to use the open AI uh GPT2 uh simulator, uh the the model, the language model, you can start typing a sentence, and the computer will sort of autocomplete the net the rest of that sentence, maybe the next sentence as well. Uh hugging face has one called Write with Transformer. If you want to Google that, you can try it out.

If you type in questions for which there should be a logical answer that shows understanding, the machine can't do it. The machine can't process it in such a way that shows that it understands the question you're asking it. It is only predictive based on patterns it's already been trained on. So a real good example, if you type in a few math questions, like what's five plus eight? What's twelve divided by four?

Uh questions like that. The machine will spit out text based on patterns, but not the actual mathematical answer. It's not reading the question and understanding the answer. It has no ability to do that. And therefore, we know that it's still just statistical prediction at this point, not actual understanding, not reading it and knowing, oh, this is what you mean to ask.

That's one of the reasons why with all these smart devices and things we have, um, they're still not really showing any kind of understanding, and and they mess up a lot because they are trying to process probability. The way all really all natural language processing works is underneath the hood, every word in a sentence, paragraph, a document is turned into a number uh representing the different words in that sentence. So uh my dog ate my homework would be like one, two, one, three, four, right. And come up with if you have my dog ate my one, two, one, three, probability says the next number should be four homework, right? But it could be other things, steak, bread, meal, etc.

But probabilistically, it's in that in that context, based on previous patterns, for homework would be the answer. That's what's happening underneath the hood of almost all natural language processing, and as a result, uh it shows that the machines don't understand. They can only recognize patterns and replicate them. That requires machines to have domain expertise and cross-pattern thinking that isn't computationally in the cards yet. Um it's not gonna be soon, because again, it requires much, much larger computational capabilities.

There is the possibility that in the next five or ten years, as uh quantum computing becomes more stable and more usable, that we could see that substantially change, but for right now, uh it's not within the cards. So, can an AI solve word problems? Can an AI process natural language? Absolutely. In terms of what you do with this information, if you have large bodies of text that you need to process, social media mentions, emails, uh web pages, etc., and you're trying to understand them, there are a number of excellent libraries out there that do this uh in the R programming language or in the Python programming language, but all of them are f are well, the the major ones are all open source, they're free of charge, and if you have the technology uh and the uh technical aptitude, uh you can build and use some of the top language models in the world for free.

Um there are a lot of vendors that are charging uh surprisingly expensive amounts of money to do the same net level of natural language processing, but it is something that is if you have the technical aptitude or you have someone on staff who does, um, you can get access to those same resources that the company's charging a lot of money uh charge and build your own applications. It takes a long time. It is not something you do overnight unless your programmers are really, really good. Um and there's a lot of trial and error and getting ramped up, but it is within your reach. So if you're thinking about using some of this stuff, take a look at what's out there, and that you you'll probably take one of three approaches.

Either build it entirely yourself with existing models, build a hybrid version with APIs from a major tech vendor like Google or IBM, or buy something off the shelf for an awful lot of money. Um those are probably the the three major approaches that you'll take. So give that a look. Um if you want to get into natural language processing, uh as always, please uh leave your follow up comments below in the comments box, subscribe to the YouTube channel and the newsletter. I'll talk to you soon.

Take care. Want help solving your company's data analytics and digital marketing problems, visit TrustInsights.ai today and let us know how we can help you.