Episode 196: Can educators leverage AI's mistakes as teaching moments?

So welcome back to another School of School podcast. I am here with one of my favourite podcasters of all time, Adam.

It's like when you say this is one of my favourite children. It's like, you know, like one of my favourite podcasters. There's two of us that are regularly podcasting. I'm one of the two favourites. Yeah. Yeah. I know. Well, this is the thing. This is all I say to my children when they ask me. I've seen this before. know, my daughter asked me, who's your favourite child? You.

Well, because...

That's why I had to say one of, you know, he's not here today, but you know Andy would get his nose out of joint if I said my very favourite podcaster.

My son asked me, who's your favourite child? You, you're done, good. You're both my favourite child. You know, here we go, okay? Anyway, yes, here we are digressing.

That's right. But don't let them have that conversation. Okay, but we digress. So, it's just Adam and I here today and that's okay because we always have lots to talk about and maybe we'll get a word in without Andy here. But we were just talking, before we started recording, we were just talking about everybody's favourite topic, AI.

technology and how it's impacting schools and or at least how it's being integrated into our lives everywhere and schools would be one of those small pieces of where they're invading our lives. So Adam, did you have any comments?

Yeah, I wanted to sort of flip it on its head because often what we hear in the press a lot, and there's no doubt that this is going to be part of it, that AI can do this and AI can do that, and AI will take the jobs and everything that it does very well. And it does a lot of things very well. But on the flip side of that, it does some things really badly. And one area that AI, although I know they're working on sort of remedying this,

But one area that has been notoriously poorly done is reasoning and mathematics. So a lot of the large language models, a lot of the AI models that are being used just don't seem to be able to cope with even some quite low level reasoning and mathematics. And I was thinking about this and thinking, I was listening to someone talk about this who was involved in the development of

the new models for chatGPT, OpenAI. And there was another mathematician talking about it and he was saying that how he uses AI, he's a university lecturer. I thought it was brilliant. And he said, advantage of the fact that it's wrong. And so I think that what he was doing is he would pose AI or one of the models, a question that he knew it was going to get it wrong

and then ask the children to critique it. Why is it wrong? Is it wrong? And what would you do to put it right? Where do you think this is going wrong? And so really taking advantage of that because that in itself is quite an advanced skill. Often we wouldn't use a mistake as a start, know, like, so-and-so's got this wrong because it's actually very difficult. But taking advantage of the fact that

these models are getting things wrong and then critiquing it was one of the ways that I thought it was a really nice use of it to take advantage of that.

I'm curious though, Adam. Adam, how did he know it would get it wrong? Has he just tried it enough that...

You know, we could put any of the problems, like in our book, into one of the, you know, you could put it into Chat or put it into whatever model you're using. And very often the reasoning part of it will fall down or it just be an incorrect answer. So I think, you know, it won't get everything wrong all the time, but I think it reminds us that at this stage, the reasoning aspect of it, having to be able to reason your way through a problem.

at the moment is a very human skill. Of course it's being worked on. So I think you've got to know in the first instance, but I think it also highlights the children. Here you go. Here's this, because we just assume that AI is right all the time, right? We just assume it's got all the answers and it doesn't really. And the other thing that he does with his students that I thought was fascinating and brilliant, these models learn, right? That they learn.

And so he was saying that what he did with the students is asked his students to teach the model to pass an exam that included reasoning. And I thought, this is brilliant. Like this is really clever stuff. that, you know, how do you teach them? And of course, in order to teach someone or something to pass some of these areas that they couldn't previously, you've got to have a really good understanding of it. so I thought that these, like turning it on its head and identifying and saying these are the limitations currently with some of these models, how can we either help it or exploit the fact that these models still have a long way to go when it particularly in reasoning and mathematics.

It's a really interesting perspective and good for him to figure that out. Like how can I use this in the classroom to our advantage, you know, to take AI's mistakes and turn it into a learning moment, you know, for the students. I think then it also forces the students to use their reasoning.

to figure out how are we going to teach this AI this moment.

Absolutely. Yeah. And it reminds us that these things get things wrong, you know, and that we've got to be mindful of that. And sometimes they get them wrong spectacularly, you know, sometimes there's some questions you just go, this is like, you really like this is so this is so poor. I think I think is that and of course, way that this like just listening to a talk, the way that this began, of course, is, you know, are people going to lose their number sense? they going to, is it, they're going to be doing all those things?

And he sort of made the point, said, well, calculators were introduced some 50, 60 years ago, right? And what happened, and what happened over time was, is that the constant long use of really long tedious calculations reduced. So we focused on a different area. So we didn't have to focus so hard on that bit there.

But was that at the detriment of other areas? No, not at all.

It meant that there were other aspects of mathematics that could then be focused on more because the heavy lifting of these big long calculations could now be done using calculator. So we didn't have to attend to them. So it's kind of like that fear of here comes some technology, which certainly happened. He was talking about the introduction of the calculator. And I think that's the thing. think that what we have to be mindful of is that with everything that comes along and helps us.

because there will be some aspects of it that helps us. Perhaps that just, we reframe it and that it frees us up to then focus on some other areas that allow us to put more focus into, which may make things better in the long run. Because I don't think anyone would argue that the fact that we've now got just calculators for simple or simple calculations or actually long calculations and complex calculations are being done by machine.

Is that a bad thing? No. So I think it just got me thinking about, and I know we've discussed this before, that sometimes in these discussions it becomes so...

just so sort of basic and dismissive. You know, we're going to lose this or we're not going to be able to do this or whatever it is.

Well, yeah, I think the fear with AI is, you know, that we stop learning. We stop having to think and...

And in this example that you've just given, to think that we could do that instead, where we turn it back onto the AI and say, actually, this is incorrect. Let me show you how it's done, is brilliant. And I mean, I have these discussions with my own kids all of the time about the use of AI in classrooms. And it's becoming the norm.

and very difficult to stop. So you have to embrace it. So I think the question is, how can we embrace it in the most positive way possible? And this is one example of that, you know, where finding those mistakes, AI is not perfect, number one, finding those mistakes and...

and taking advantage of it in the sense that it now becomes a learning opportunity for your students, to me, makes so much sense. There's hope.

Yeah, totally. And I think those areas where it does get it right. So in some areas, know, like heavy liftings, like you ask it, can you spot a pattern? I think this is happening. Can you run this a thousand times and see if you can spot a pattern? So I think those sorts of things as well where it does work with, you know, like spotting patterns and big calculations and long, long.

computation, all those sorts of things that I think that again, it takes part of it out where, you know, this is what mathematicians do is they put forward a conjecture, they say, I think this is happening. But then you've got to prove it and you've got to, you know, and sometimes the only way to do it is to replicate it over and over and over again, or use increasingly different numbers or situations. And I think again, that's got to be beneficial.

because you need to understand and have the reasoning in the first place that you need to look for a pattern. And if it does spot a pattern, what does that mean? You've still got to analyze it. You've still got to decide what that means. It's just saying, yeah, I can see a pattern and this is what the pattern is. You've still got to interpret that and do something. And I think it's those types of things that, you know, 50, 60 years ago, here's the calculator, and that's kind of like the first step. And I've got no doubt that at that time,

there probably would have been many people thinking, if we give the children this, we're going to have a nation that can't do maths. And I think it's the same thing. And I think it's that thing around, right, well, if we can focus in on certain parts that AI does really well, perhaps it leaves us space to do these other parts that are really important that it can't do. We've got to decide what does this mean. So it's telling us this, OK, but what does it mean?

And I think it's those types of things that I just think that, I don't know, I think it's having that balanced approach, isn't it? It's looking at it and saying, well, AI is going to be, you know, and I just think our generation is incredible. We've had the introduction of the personal computer. We've the introduction. I sort of talk about our people of our age. We've had the introduction of the internet.

And now, you know, we've got AI and I think all of those things, well, so yeah, of course there's some ills that come with it. I think that some of the advances that can happen, you know, I think that they could be really exciting and I think that, you know, it could really support it. Of course, Yep.

And they should be exciting. Yeah. So listeners, there's your challenge. Use AI to your advantage. Don't rely on it. Share the ideas. Absolutely.

Yeah, yeah, and share the ideas. think share those ideas because they might not be immediately obvious. And I wish at times we had like a, we probably have actually. You know the old school saying, send ideas in the back of a postcard to P.O. Box, blah, blah, blah, blah. I'd love to hear them, you know, like post them, copy, you know, on socials, just, you know, tag Maths — No Problem! You know, I'd love to know how people are using it. Yeah, for sure, for sure. Because I think that

it is going to be a challenge and it's going to accelerate real quick. And if we can maximize the learning out of it, then it's got to be a good thing.