PISA incredulity

Rather than join in with the accusations and recriminations about the PISA results announced today, I would instead like to look at the sort of questions that were asked.

Apparently there are six levels in the mathematics section, and a sample question from each level is here. The first four levels are so easy that they're not worth bothering with, so I'll concentrate on the sample questions for Levels 5 and 6.

Level 5 Question

The Gotemba walking trail up Mount Fuji is about 9 kilometres (km) long. Walkers need to return from the 18 km walk by 8 pm.

Toshi estimates that he can walk up the mountain at 1.5 kilometres per hour on average, and down at twice that speed. These speeds take into account meal breaks and rest times.

Using Toshi's estimated speeds, what is the latest time he can begin his walk so that he can return by 8 pm?

Level 6 Question

Helen rode her bike from home to the river, which is 4 km away. It took her 9 minutes. She rode home using a shorter route of 3 km. This only took her 6 minutes.

What was Helen's average speed, in km/h, for the trip to the river and back?

Assuming that these sample questions reflect the actual degree of difficulty of the test itself, my problem is that I can't see how people could fail to answer them correctly. Yet the OCED average getting the Level 5 question right was only 13%, and the UK average 12%. For the Level 6 question the average for both the OECD and UK was only 3%.

So I would like to ask a genuine question. Are there people reading this who weren't able to work out the answers? Please don't be reticent, because according to PISA you'll be in good company. 7 out of 8 got the first one wrong and 32 out of 33 got the second one wrong; and I'd imagine the maths ability of the average 15 year old isn't too far removed from that of the average adult.

If these figures are even remotely accurate, it points to the whole of the so-called developed world being in a mess. And therefore it seems rather pointless to argue the toss about whether Wales would be doing OK if our figures suddenly rose to 1 in 6 and 1 in 25.

Update - 17:20, 10 December 2013

The full set of PISA maths questions for 2012 is here.

Unfortunately, it isn't clear from the paper what the time limit is, or whether calculators are allowed. However, as some square root calculations are required, I would guess they are.

Have fun.

Bookmark and Share


Anonymous said...

You can take a sample of the questions here, OECD's website: http://www.oecd.org/pisa/test They really are very easy.

Anonymous said...

So what are the answers ........ or are you just keeping them to yourself like all the kids in Wales do?

MH said...

Thanks, 19:58. However I had already provided that link. That's where the questions came from.


Are you really saying that you need me to provide the answers, 20:11?

Anonymous said...

MH 20:22

Well, if not you, who else?

You see, this is half the problem with education here in Wales. We think just turning up is enough, turn up at school, turn up at exams. Heaven forbid ever having to work something out, heaven forbid having to proffer an answer and heaven forbid anyone daring to suggest that the answer is wrong, wrong even by a tiny amount!

God help Wales!

Anonymous said...

MH -I was able to answer all the questions fairly easily, and I'm confident that I would have answered them correctly more quickly still when I was 15. However, although my mathematical ability is above average, I don't think I have ever been in the top 3%. I am amazed by the indicated success rates in relation to these questions. Furthermore, I don't understand why level 6 is any more difficult than level 5 or level 4. I would have thought that if you could answer one of those, you would have been able to answer the others.

Anonymous said...

Anon 23:56, You may have answered the questions but are your answers correct?

Anonymous said...

Anon 00:45 - Er....yes. If you follow the link, the correct answers are supplied once you have submitted your own answers. I assume you haven't followed the link; or perhaps you did and it was all a bit beyond you.

Owen said...

Those are, I would judge, fairly typical Intermediate-level GCSE maths questions, though I'm not sure if there still is an Intermediate-level. Here's a sample paper for Higher Level for want of comparison. We need to remember that the A*-C pass rate for GCSE maths is 52.8%.

I don't think it's the nature of the questions themselves that's the problem. It's being able to complete lots of them in a set time (2 hours I think, and PISA is a long test). Plus, sample questions tend to be easier than the real thing. Here's a test paper from 2006 (pdf).

Oh, and to "ruin" it for everyone - 11am and 28 km/h.

Anonymous said...

Correct answers, Owen, well done.

Mind, can you help me out with something I don't understand. In the second question Helen is riding her bike for a total of 15 minutes and covers 7 km. As such, she'd cover 28 km in one hour, making her average speed 28 km per hour.

But if I work out her average speed during her first session (4km's in 9 minutes) and her average speed in her second session (3km's in 6 minutes) and then average the two readings together I get a slightly different answer:

(4/9)*60 = 26.67 km/hr
(3/6)*60 = 30 km/hr

Average speed = 28.335 km/hour

Why is this answer incorrect?

Pads said...

Anon 20:51 - that won't work because you dividing by 2 and the two journeys were not equal.

I'm please to say I did both answers in my head - that's O Levels for you. *smug*

Tarian said...

Thanks for highlighting this. It is simply staggering. The worst part about this debacle is the utter complacency and 'It's not me guv!' attitude of the government, unions and teaching colleges. As a former teacher myself I am convinced that no reform is possible because every single pillar of the education system is rotten and blinded by self interest. Education is really very simple - get well trained and intelligent people to educate small groups of children, start with the basics and move on to more complex matters only when they have grasped the fundamentals. The endless obsession with an ever growing list initiatives/schemes/interventions and throwing cash at buildings and IT schemes has failed abysmally. You can succesfully educate kids (and adults) on a shoestring - the history of our own country proves it. All we need is to focus on the fundamentals of teaching (not the fantasy land methods of training colleges and inspectorates) and engender an attitude of valuing education and self improvement among our people. At present no political party has any idea of what to do.

MH said...

Sorry to come back so late in response to the more recent comments.

I suppose it is obvious that some people. like 20:11, couldn't do it. But there's no blame attached, for a lot of kids are in the same position.

Looking at some reactions to PISA, particularly this page of head teachers' reactions, there probably is mileage in the idea that the questions were posed in an unfamiliar way. In other words that they required a degree of "verbal unravelling" that is different from the way a GCSE question would be asked.

20:51 presents this perfectly. When stripped of the inessentials, Question 6 breaks down into five very easy calculations:

What was the total distance travelled? ... 4 + 3 = 7 km
What was the total time? ... 9 + 6 = 15 mins
How many minutes are there in an hour? ... 60
What is 60 divided by 15? ... 4
What is 7 multiplied by 4? ... 28

But if you go at it like a bull at a gate, you end up wondering what on earth to do with figures like 26.67 km/hr and 30 km/hr ... and someone like Pads needs to help you out. The rule of thumb I was given when I was fifteen is that the answers to questions like this are nearly always simple, round numbers. If you end up with a complicated fraction, you've probably got the answer wrong.

I was going to ask what was so strange about Question 6 in the light of Question 5. But 20:51 has all but answered it. Quite rightly, Question 5 points out that going uphill is slower than going downhill. Therefore it's odd that Helen's speed on the journey down to the river is slower than her speed on the journey back.


I didn't intend this post to be about the politics of the way we educate our children, but I will say this. I don't think that we should look for political solutions to educational problems. If you put a politician on the spot about it, the only thing they can "do" about it is come up with political solutions ... more and more and more of them. But the fault is ours, as a society, for expecting politicians to be able to solve the problem. By and large, educational problems are better solved by those who dedicate their lives to education. It would generally be better to leave it to them to identify what external factors are problematic, and for politicians to propose solutions to those specific problems.

It is clear to me that relative poverty is the biggest factor in poor educational achievement. Therefore the relative decline in educational achievement in Wales is to some extent explained by the relative decline in our economic performance. I think Professor David Egan made that point well when the last PISA results were published, as I mentioned in this post almost exactly three years ago.

This time round, the same David Egan has pointed to Victoria Winkler's post on the Bevan Foundation blog as the most sensible response to these new PISA results.

I would say that although mitigating the effects of poverty, as emphasized by Huw Lewis, is a perfectly laudable thing to do, it would be much better to deal with poverty itself than to simply attempt to mitigate its effects. If Labour are serious about improving educational achievement, it is hard to understand why they don't want control over the economic levers (primarily tax powers) that would enable them to reduce relative poverty in Wales.

MH said...

Owen gave a link to the 2006 questions. The full set of questions for 2012 is here.

Unfortunately, it isn't clear from the paper what the time limit is, or whether calculators are allowed. However, as some square root calculations are required, I would guess they are.

Have fun.

Post a Comment