You Ask, I Answer: Accounting and Tax Large Language Model Strategy?

In today's uh question, Allison asks, are you aware of any large language models with tax and accounting data or any being developed? Or crazy question how to develop one. When it comes to accounting and tax, the regulations change all the time, so it would be important for the model to be updated as needed, which adds to the complexity of our needs. Okay, so there are two approaches towards the use of large language models. One is the perfect memory approach, where you try to train a model and fine-tune it to have perfect memory.

The model runs, it doesn't need any other data sources. It knows what to do in any given situation. Perfect memory models are good, they're fast, they have a very large upfront cost to train them, and they go out of date really quickly because the moment something changes, the model doesn't know it because you have to retrain it on a regular frequent basis. The second architecture, and the one that we see a lot more companies uh taking, is the language model is an interpreter. It's an interpreter that connects to other systems, and those other systems can have the data.

So, for example, when you use Microsoft Bing's chat, Bing is not asking GPT4 for the answers. Bing is asking GBT4 to take the conversational thread that a user asks and convert it into queries that are compatible with Bing's search engine. It goes through, it returns the search data to GPT4 and says, hey, summarize the data that I've given you and output it as language to the user. So the large language model in that case is not being leveraged for its ability to know things, it is being used for its ability to convert other formats of data into natural language, into and out of natural language. Of these two approaches, I mean they're they're both good approaches.

You know, uh perfect memory means big upfront training costs, goes out of st out out of date really fast, but very, very fast, very, very capable. The interpretation version is lower upfront cost because you're just using a language model for its language purposes, bigger infrastructure cost, um, and bigger operating costs because there's more machinery being used to uh to do the work, right? Because the model does not know everything. The model's only there to interpret. However, in this situation where you're talking about tax data, accounting data, financial data, and the changes in tax regulations, you would probably want the interpreted model where you have an underlying database of some kind.

Typically, when we're talking about large language models, we're talking about vector databases. You would want a vector database that was constantly being primed and fed the accounting and tax data that you want, and then your language model takes in individual queries, looks first at the vector database and says, Hey, what do you know about escrow taxes? And then if it comes up with less good answers there, then we'll ask, you know, it'll default to asking the it's uh itself as a language model. But most of the time the answers are going to come from the vector database for a given query. And that's the approach I would take if I was being asked to build something like this, rather than try to fine-tune a model.

Now you might want to fine-tune the model in the beginning to give it a good sense of all the language that's really important. Um, you know, there's gonna be terms and accounting that no one else uses, and you would want to make sure the model knew of them, understood them from a statistical perspective, and could generate them, and then you would uh feed the model data to and from the database that contains all the current information. So that's the approach. Um, it's not crazy to build one. It's not crazy to build a system like this.

It is expensive, it is laborious because you have to gather up all the data you want to train the model on. You can't just give it, you know, five pages of stuff. You need to give it a good amount of information. But it's not crazy to do it. And lots of people and lots of companies are building custom models or custom integrations, hybrid models where you have a language model that does the interpretation, and then you have a data source that is kept up to date and clean and structured well.

But it's a really good question. Um shameless plug. Consulting on this stuff is what my company, Trust Insights does. Uh so if you have questions about wanting to implement this kind of system and the strategy and maybe even the blueprint for building the system itself, hit reply, leave a comment, do something. That's because again, we're happy to help with this.

It's literally one of the things that we do. Uh, good question, Allison. Thanks for asking. Talk to you soon. If you like this video, go ahead and hit that subscribe button.