Mind Readings: AI Content Detectors Deep Dive Part 4

Welcome back. This is part four in our AI content detector series. Today we're going to talk about building the term paper of the future using generative AI. So we're going to walk through a process of how you might want to teach a student how to use generative AI to write a term paper of the future. And we're going to talk about this from the perspective of how would you go about building a great term paper?

A machine will write it, but a student has to do all the thinking to make it happen. For the topic, let's pick a topic. Uh let's see what we got here. Uh I'm looking at my kids' old stuff. Let's do so.

My kid years ago, a couple of years ago, did an article or a paper on the Supreme Court's impact on the economy? Uh, and whether or not the Supreme Court has had a net positive or net negative on the economy, which is an interesting paper. First, how would you go about instructing a student to write this paper? Probably suggest in the era of AI to use various tools to find credible sources of data. So you could use a traditional tool like this.

Let's do uh Supreme Court cases and economic impacts and see what we've got here. The non-impact sequence regulatory taking cases. Does the Supreme Court really matter? And you might want to teach the student, hey, like look for citation numbers and then look for PDFs. So let's take a look here.

We've got cases. Let's limit this. Well, you know, let's limit this to the 21st century. So looking at a few of these different papers here. Let's see, penalizing punitive damages, selecting on the economy.

Okay, so we got a few of these papers. Let's go ahead and grab the URLs because we will want to use them for APA citation. So we want to remember where we got our stuff from. And let's get the papers themselves. So here is the non-impact of regulatory uh rulings.

Grab this and download it. Grab this. Next, let's use a true AI tool. We use perplexity AI, and we'll ask the same uh question. Find peer reviewed studies, research, and cases that examine the economic impact of supreme court rulings.

Restrict your search to papers and content published in the 21st century. Let's go ahead and turn on pro mode. You get five free pro modes a day, and it's going to go through and try and answer this question. Now let's see what we have here. We have a study by Donald Sanger.

Looked at established methodology for establishing the court's cases. Let's take a look at the source on this. Scholar Commons. Let's take a look at the citation. You know what?

That might be worth including. Let's save that. Uh let's see, contribute contribution to economic quality. This is the Gilman article. I think we might have yeah, we have this one already.

Uh impact on specific economic areas, property. That's from Kiplinger. Those are all Kiplinger articles, which I would not say is a peer-reviewed source. Let's see, that's NF. Who is this?

That's a lobbying group, Project Syndicate. Okay. So we've got a decent selection of data. Our next stop should be Google's free notebook LM. So this is a research tool that uses Google's Gemini.

But what it does is it will only answer questions that you give it about a specific topic. So you have you have to give the questions and you have to give it the data itself. It cannot pull data from something that you did not give it. So I'm going to take all of those papers that we just looked at. I'm going to drag them in here.

And what it's going to do is it's going to start digesting these things down. A lot of people, by the way, love the audio preview, can make a little podcast episode out of this. But I think for us, we're going to do the briefing document, the table of contents, the study guide, and the frequently asked questions. Let's do timeline too while we're at it. And what this does is it makes cards.

This makes little flash cards that uh examine the data we uploaded. And let's see here, frequently asked questions about these things exploring judicial behavior. Should we create one more summary? This tool gives us the ability to ask questions of these data sources specifically, which is super helpful. So let's start by asking it based on the information provided.

Summarize the economic impact of Supreme Court case rulings on the US economy. And what is really nice that it does is it will go through and it will try to answer this question only on information that we've given. Let's go away. Says, based on the sources provided, have impacted the economy in the following ways. The court generally favors business interests over those of consumers and employees.

Supreme Court's rulings on economic policy influence lower courts. The court's decisions on issues like punitive damages have been criticized for lacking economic grounding. The court's decisions can have significant and immediate impacts on specific aspects of the economy. So let's copy this and let's put this in our scratch file because that's a useful summary. Let's ask it next.

Based on the information available, have Supreme Court case rulings been a net positive or net negative to the overall US economy. Let's see what it comes up with. Determine the overall impact. The sources don't offer a conclusive answer, they do provide insights into different facets of the court's influence. Pro-business tendencies impact on lower courts of policy.

So again, the same stuff, role of macroeconomic conditions. We're going to take that and we're going to save that information as well. Uh and we're basically creating a little scrapbook here. Next, we want to move in out of notebook LM into the standard uh generative model. So I'm going to use Google's AI Studio because I want to use the most advanced model, which is Gemini Pro 002.

I'm going to just make a few minor settings changes here to make it sure it can spit out lots of long output. And we're going to first ask it. Um, well, this time around, we don't have to prime the model because we just did that in notebook LM. So we're gonna say, first today, we're going to be writing a position paper on the impact of the US Supreme Court on the US economy. Here is some background information.

And let's go ahead and put in our notes from earlier, and then here are some source documents. We're gonna add in the same papers we started with. One of the things that's true about generative AI is that the more data you provide, the less it makes things up. So if we can provide more information up front, it will do a better job. So let's go ahead and just feed it this information and have it start to review and think about it.

Okay, after about three minutes of thinking, it came up with uh a very brief uh uh sort of regurgitation. This is unhelpful, right? This is this is no different than a term paper where the the student just spits back the information. What we might want to say is, well, you need to have a point of view. So suppose I were to prompt it by saying I want the focus of this paper that the Supreme Court is ultimately a net negative on the US economy in its current form, and there's a bunch of reasons why.

My position paper needs to come up with some recommendations because the paper just complaining does little to advance democracy based on everything we've discussed so far. What might be some potential suggestions? This is part and parcel of how education would change. Not from, hey, just use this tool to write a paper faster, but based on the summaries and the evidence and the information we've gathered, seeing in Notebook LM that it came up with, like, yeah, there's the way it the Supreme Court has has ruled on things in the past, has ultimately been very pro-business and favors business interests over those of consumers and employees. Depending on your perspective of how the world works, that might be a good thing or a bad thing.

And so if I was teaching students about this, I would say reviewing the conclusions from Notebook LM, what position would you take? What point of view would you have on this? And then go back and prompt the AI model to help you think through the that position, right? So in this case, we've asked it. Um, I want some recommendations because I think this is what we're doing for this paper.

Um, I think it's a bad thing. So, what would you suggest? So, we want the the student to think critically and go, well, what are some options? What are some ways you could fix this? Okay, about three minutes later, after the model has chewed through uh our suggestions, it has revisited its first draft, and it has come up with some suggestions.

It says some of these uh negative influences encountered by promoting economic expertise on the court by including the selection of justices, enhancing transparency and accountability. Um the court should embrace uh greater transparency decision making processes, uh campaign finance reform itself, expanding access to justice, and promoting civic engagement and voter participation. Now, again, this is this is an improvement. What would you, if you were an instructor, what would you say to the student? What critical what would you say?

How would you critique this if we think about this as the Socratic method that is the the the machine is the student and the student is the debater the student is the one who should be asking tough questions of the machine so we can do this by teaching the student hey I want you to to consider things that you might not have so I'm going to say based on these suggestions I want to identify a path forward that has the fewest negative unintended consequences especially to marginalized populations that are already disadvantaged in Americ in American society additionally I want to reduce the likelihood that unethical politicians could further take advantage of or willfully misinterpret the law and the rulings of the Supreme Court to their advantage I want you to evaluate the paper so far based on these criteria and score your suggestions on a one to 10 scale where 10 is the most positive impact let's do a minus 10 to plus 10 scale so minus 10 to plus 10 scale where plus 10 is the most positive impact especially on marginalized populations and minus 10 is the most negative impact especially on marginalized populations. Review your suggestions, score them, and then wait for my input. So what we're doing here, we are doing critical thinking. We're saying, well, these suggestions sounds great. What could go wrong?

What how could these these suggestions be misinterpreted? Because again, if we're talking about things like civics, and in this particular paper that we're writing, um, we might want to consider these points. We want to consider not just advocating for position, but asking what could go wrong. That is that is a critical thing that very, very few people in general are doing right now. So let's go ahead and see what it comes up with.

Okay, so what it has evaluated is evaluated um promoting economic expertise on the court as a plus eight, enhancing transparency and accountability is a plus seven, campaign finance reform is a plus nine, expanding access to justice is a plus ten. Um, and it talks about each of the reasons for uh why these things would be impactful, um, but also acknowledging the difficulty of them. So we might want to further say, ask it of these suggestions, keeping in mind the hyperpolarized political environment today. Which one of these paths forward would be most practical to achieve? Okay, after about a minute and a half of thinking, it says enhancing transparency and accountability likely presents the most practical path forward in today's hyperpolarized political environment.

While all the suggestions have merit, campaign finance reform and expanding access to justice face significant political obstacles. Umoting economic expertise while beneficial depends heavily on the individuals. Civic and voter participation addresses a symptom rather than a root cause. Transparency and accountability offers a less politically charged entry point. So that's a a pretty interesting conclusion.

It's not the one I thought it was going to pick. Again, if you're teaching this to a student, you want to have a discussion and say, which one would you have picked and why? And how does the tool's reasoning sound to you? Now I think this actually sounds you know interesting in reasoning. Um, we're gonna wrap this up by saying ever based on everything we've talked about so far, let's propose a new outline for the paper that has these rough categories, and they are introduction, current situation, likely outcomes, suggestions for future changes, impact, feasibility, selecting a path forward, conclusion.

And we're gonna have the the software build a new outline for a paper. So our outline is now beefed up based on what we've provided. Briefly introducing the Supreme Court's role, the current situation, the likely outcomes of inaction, proposals for a more just future, anticipating the impacts of reform, assessing feasibility, strategic path forward, conclusion. And we can now say using this new outline in all the conversations so far, write the paper section by section. Now, what happens after this?

Machine has created a paper, and instead of having 10, 20, 30, 40, 50 students generating papers, maybe you have teams of three or four or five people, and one person on that team generates a paper. And then what's generated, the rest of the team debates. The rest of the team corrects, the rest of the team argues with. Because even though the machine is creating this thing, treat it like any other academic source, like any other academic reading and say, okay, well, what is it, what's good about this paper? What's bad?

What are the blind spots? What got missed. And so what we're doing here is we're trying to teach students to do a lot of critical thinking, to ask probing questions, to have a point of view based on the research that they've done and the tools that have helped them summarize it, and ultimately then criticize the output of the machines. That is probably the most important takeaway from all this getting people to not implicitly trust what the machine spits out. It's not about writing a paper.

It's about thinking. It's about helping people to think, helping people to think critically, helping people to develop the skills to think critically, which in this case requires a lot of research, a lot of gathering of information and having a point of view. And so it's not the output isn't the paper, the output is the thinking. And so if we can adapt our educational processes to embrace the use of AI, we will do much better. Okay, so that's part four of this series.

Let's move on now to part five, the final part. Uh, where do people go from here? So stay tuned for that. If you enjoyed this video, please hit the like button. Subscribe to my channel if you haven't already, and if you want to know when new videos are available, hit the bell button to be notified as soon as new content is live