Episode 165: Comparing education systems - Should we take international results with a pinch of salt?

Andy Psarianos

Hi, I'm Andy Psarianos.

Robin Potter

Hi, I'm Robin Potter.

Adam Gifford

Hi, I'm Adam Gifford.

Andy Psarianos

This is the School of School Podcast.

Welcome to the School of School Podcast.

Adam Gifford

Welcome back. It's another episode of the School of School Podcast. The regulars, they're here-

Robin Potter

Hey, hey.

Adam Gifford

This afternoon. How are you doing, Andy? How you doing, Robin? Hope you're both well.

Robin Potter

Good, yes. Very well.

Andy Psarianos

Yeah, great.

Adam Gifford

Good, good. Well, what I wanted to talk about today, last year, the end of last year, PISA results came out. In education, we're often comparing international systems. I think that maybe it's time to have a look and think what do we need to consider when these things come out? Do we look at the results? Are they the be and end all, or is there more that we need to think about? And maybe look at them with a bit more, not necessarily sophistication, but just questions about should we be just looking simply at those results, of big international studies and adopting whatever we see is at the top or what sorts of things might we consider when we're comparing international education systems. Do either of you have any thoughts on this?

Andy Psarianos

Any thoughts?

Adam Gifford

I'll take any thoughts, yeah.

Andy Psarianos

I've got a lot of thoughts on it. It's charged. These studies are done for good reason and I think it's important that they're doing because it should be informing people. They're a longterm view. It's not a short term thing. It's not like, "Yeah, we won the competition!" That's not what it's about.

There are two big studies for mathematics. PISA obviously, which most people are familiar with. And Trends in International Mathematics and Science. I think the name implies how it should be interpreted, is trends in international mathematics and science. A trend is what we're looking at. Which countries are going this way? Up. Which ones are going down? That's what you got to look at.

Look, when I say Finland to most people in education, what do they think? A lot of people still think Finland is the best country in the world in education. They're still being referred to ... Yeah, that was the case in 1993 or something. It was a headline because the first studies came out at that time. Everybody was, "Oh my God, Finland's the best country in education." Yeah, no. They were the best country in education in 1993. We all looked at what they were doing in 1993. Not all of us, but a lot of people said, "Well, we got to do what Finland's doing." So they went and looked at what Finland was doing. They said, "We've got to be more like the Finnish system."

Well, guess what? Since 1993, or whenever it was, I don't know if it was '93. Whenever it was that they were at the top of PISA, they've been dropping like a stone ever since. What you got to ask yourself when you look at a country and their education system is not what are they doing today, what have they been doing for the past 15 years? What they're doing today could be completely wrong. It's what they did in the last 15 years because it took 15 years for them to get their 15-year-olds to the stage that they're at right now.

That's the thing, that you got to understand how to interpret these things. They're dangerous because the press likes to take headlines and says, "The UK is doing better than most other European countries." Yeah, they are today but that's not based on what they're doing today necessarily. It could be based on what's happened in the last 15 years. It's charged. They're dangerous and they're helpful.

Robin Potter

Thinking about it, this is a worldview. How can you look at one system and say, "Oh, Finland, okay we should all be doing that. We should all encourage the kids not to read until they're eight years old." What's good for one is not necessarily good for another. That needs to be kept in mind as well. Culturally, we're very different, from a global perspective. Not only is it that we need to look at the past 15 years, maybe we need to look at the culture of each country and figure out what's the right fit. I don't know. Would it be the right fit in Canada to apply the same elements of the curriculum from Japan? I don't know, maybe it would. Maybe some of it would, maybe it wouldn't.

Andy Psarianos

Yeah. I think like all things, all research, the devil lies in the detail. You have to be able to discern which things are relatable to your circumstances and which things are not. You talk about Japan and Finland. Culturally, I don't know a tremendous amount about either culture, I wouldn't call myself an expert. But just even from the outside, I think you could tell that culturally, they're pretty different.

Adam Gifford

Yeah. Japan is a good one. Just in the recent PISA results that were just released at the end of last year, the Japanese did really well in their mathematics. We can all, perhaps we jump on board the Japanese model. But then you look, there's another statistic that's given in the PISA report, which is around maths anxiety. It's really high in Japan, it's really high. Then the question becomes, well hang on a second. Do we do it at the expense of anxiety? Is there an issue around that?

We start to look at these different models, and like Andy's saying, I think the devil is in the detail. But another piece of research that I read that I thought was really interesting was they asked policymakers from the countries that took part in PISA how influential the PISA result was to their decisions in making policies for their countries. For the vast majority, it was really influential. But interestingly, countries like Singapore was one, where PISA almost was just a confirmation of what they already knew and what they were doing, as opposed to, "We need to look at this and find out which country we're going to follow or what the influence is going to be."

Like you're saying, Andy, we've got to remember that the education system that these children grew up in is different. It's happened. It's already happened. If we jump onto a system or we jump onto a country, and we think about what they're doing there, New Zealand was victim to it. New Zealand used to be hitting heady heights and doing very well in PISA. You could say, "Well, we'll follow the New Zealand system," but then it fell off a cliff.

I think it's really interesting. I think that it is massively influential more so in the media, more so in politics because it's an international comparison. But I do think we need to be really careful to listen to those countries that are doing well. I'm sure if you talk to someone in Japan, I'm just assuming this, they'll want to say, "Well, we're going to look at other models which decrease anxiety in mathematics. How can we get the same results but without having our children feel that towards the subject?"

Andy Psarianos

If you look at countries that do really well ... Again, we have to be careful what we mean by are doing really well. You mentioned Japan, you mentioned Singapore. What's interesting about Japan and Singapore? Japan and Singapore, they were always at the top. Singapore hasn't always been at the top. But since the '90s, Singapore has been probably the top country more times than any other country and they've always been in the top three. Japan's always up there. There's a couple of other places that do remarkably well like that, South Korea being another one. They're all ... Economists talk about the Asian factor. There's something happening in Asia.

This is where I think these studies are important. Now look at South America. South America is the opposite story. South America, by global standards, is really underperforming. I'm not even going to talk about Africa. South America is really underperforming. They're really underperforming, I don't know what else to say. They're the opposite of Asia. You've got Asia and South America.

If you look at those two, you got to say well, what's happening in those places? Well, what you find in Asia is that they're very different education systems in Asia. What's happening in Japan is not all that similar to what's happening in Singapore or South Korea. The big similarities is that they seem to have the system worked out. It's a systemic thing. It's like the politicians know what role they're supposed to play, the teachers know what role they're supposed to play. Policies and teachers are not working against each other, they're working together. They have developed a system. There's not just one system, but they've developed a system that works.

If you look at places that are suffering, it's perpetual systematic chaos. There is no real system and the systems seem to be broken. There's all these false incentives. The labour unions are constantly fighting the government, and the teachers all have one idea about what should be happening, the government has a completely different ... The system just doesn't work. The schools, the education of teachers is a mess, and blah, blah, blah. That's where the problems are.

Very rarely is it about what's in the curriculum. Oh, we should be teaching fractions in year three or year one, or wherever we start. That's usually not the problem. That's the lesson to learn. But that's what you got to look at. Then there's other big lessons as well. You might say, as a policymaker, "Okay, the problem is is that people don't study a topic long enough or they don't spend enough time on a topic. If we want our kids to be better at maths, we need to make sure that they study maths until the end of secondary school. Every year, they need to take mathematics." You think that the answer lies in there.

But the facts don't support that because if you look at countries in South America, many of them are forced to spend a very long time studying topics but they still do quite poorly at them. It's not the amount of time in the classroom that makes a difference, it's how effective the time that they have in the classroom actually is. That's where these research studies become important because the answers are in there. You can spend 20 years studying mathematics and know absolutely nothing if you're not taught. Well, not absolutely nothing, but you're not going to be great at it if you're poorly taught. It doesn't matter how long you spend. But you could spend a shorter period of time, get the right mindset for the subject and come out being a lot stronger.

That's the kind of details that we should be looking at in these things.

Adam Gifford

Yeah, I agree.

Andy Psarianos

And not looking at the scorecard.

Robin Potter

But then it goes back to again, you look at Japan, and wow, they're doing incredible at maths, even if they're mentally stressed. But now we look at Venezuela. If they implement what Japan is doing with their maths, is it going to make a difference? Or again, socially or culturally, is it-

Andy Psarianos

It will.

Robin Potter

Is it too different to try and instil the Japanese?

Andy Psarianos

I think it can but you need to understand which components are the ones that are going to make a difference. If you look at, let's say okay, talk about Japan. Japan uses lesson study for professional development a lot. Their teachers are incredibly knowledgeable and they use lesson study. One could derive from that that we need to do lesson study like the Japanese do lesson study.

Well, there's a lot of cultural stuff in the Japanese lesson study that wouldn't necessarily work in a different environment. Because Japanese really like a lot of, what's the word I'm thinking of? Call it discipline, call it structure, respect, all this kind of stuff. It's a big part of their culture. But those aren't the things in lesson study that make lesson study effective. Don't fixate on those things because it's not about discipline. It's about the process, and the process and experience of the people that partake in lesson study. You might have to take that idea, but make it adapted to your culture. In Finland, it's unlikely the Japanese lesson study in a Japanese way would work because no one respects someone just because they're older in Finland, I would expect. While in Japan, that's a big deal. There's just cultural elements that you need to ... They're not the ones that are important. It's figuring out which bits are important.

Adam Gifford

Then it's the time it takes to make it systematic.

Andy Psarianos

Yeah.

Adam Gifford

You got to remember that 15-year-old ... If you think, okay that 15-year-old, say they were trained by a teacher, that teacher needed to train for X number of years. Now you're going back 18 years in the past. In order to make that system work, politicians have to say, "We're just going to keep hands off and not make this a political argument." This, I guess, is one of the problem that can exist with those figures, is that that can be the stick that you beat up the opposition with. "This has failed them. We need to do this and we need to do it tomorrow." But for that systemic change, going all the way back, which includes the training of the teachers, the funding, all sorts. There's so many different ingredients that feed into it, as Andy said. Yeah, there are cultural differences.

I think of the Japanese model where you've got the expert observer that might give a half-hour lecture at the end of a lesson study. Would that sit well with a group of English teachers tomorrow? Possibly not. Possibly not. We'd need to do something different or work towards that, or whatever it might be. But there's certain elements, like Andy was saying, that you take the parts that you think, "Yeah, this is proven," but you've got to give them time to become part of the system. You can't expect that to happen overnight.

If you look at the top performing systems, they haven't happened because someone introduced a policy a year ago. They've been working at them. We'll just talk about Singapore and Japan. Both of those, both those countries evaluate things over years, decades. I went to a course, oh about six months ago, a Japanese professor who lives in the States showed us this problem. He was talking about this problem. That problem has been discussed and evaluated for over 100 years. As of the last time they discussed and looked at that problem, they still think it's fine as it is. But next year, they'll have another look.

I think that sums stuff up for me, is that the evaluation and change needs to take time, and be gradual and thorough. But if policy changes just sweeping, and there's the potential in the UK for that to happen every four or five years potentially, then that's a worry. That's a worry. Like you say, Andy, I think it's easy just to look at some parts of what these studies tell us and ignore the fact that these systems have been set up over decades. Decades and decades.

Andy Psarianos

Thank you for joining us on the School of School Podcast.