Monday, 23 March 2015

The Big Book of Data Nonsense: entry #789

remember a comedy sketch from many years ago in which two gibbering old men thought up collective nouns and then entered them into a thick, dusty ledger. Finally one says something along the lines of "I think that will do for today", whereupon the ledger is closed to reveal the title on the cover:

The Big Book of B******s

Just a couple of old guys with nothing better to do than stand around making stuff up but at least they weren't hurting anyone.

I recalled this sketch yesterday when I received the following tweet from someone regarding the foundation stage and progress measures:

"talking to HT today who has been told by Ofsted and LA his progress judged from National EYFS not actual. Bonkers."

At first I had no idea what the author was going on about. I assumed that they must be referring to the measuring of progress from end of EYFS to end of KS1, and that this related to the Ofsted guidance stating that 'children who reached a good level of development at the end of reception ought to be reaching at least the expected standard (i.e. 2b) by the end of key stage 1'. Flawed but at least sort of understandable.

That was not what the tweet was referring to. It was actually referring to the measuring of progress across the reception year. Sorry? What? Read that tweet again:

"talking to HT today who has been told by Ofsted and LA his progress judged from National EYFS not actual. Bonkers."

Hang on! There IS no 'national EYFS' for the start of the foundation stage - there is no on entry data collection - and even if there was why would that be used as a baseline for measuring progress. This isn't what is done across key stages 1, 2, 3, 4 or 5, so why would anyone think it was right and proper or made any degree of sense at EYFS?

All I can come up with is that those involved have made an assumption of the national average on entry level for pupils at the start of EYFS. Given that pupils will range from just 4 years old (48 months) to nearly 5 (59 months) at the start of the reception year, this could be anywhere from 30-50 high to 40-60 high, in terms of development matters bands that many schools use to track progress. 

But having ascertained your national average level on entry, it still doesn't explain why anyone would measure progress from that point rather than the actual on entry assessment. There are numerous obvious problems here:

1) As stated above, there is no such thing as a national average on entry level for EYFS. You'd have to make one up.

2) Any progress made by lower prior attainment pupils - those in a lower age band on entry (there are many!) - will be ignored up to the point of the assumed baseline.

3) Depending on what you decide to use as your start point, it will benefit the more able, who may be assigned a lower baseline.

4) This is not how progress is measured at any other key stage.

5) It contradicts Ofsted's own guidance in paragraph 198:

The handbook quite clearly states that inspectors should 'take account of the proportions of children who have made typical progress from their starting points'.

THEIR starting points; not a made up one based on non-existent data.

The tweeter informs me that the school's own data shows that pupils have made good progress across the foundation stage from their actual starting points but that data and protestations were falling on deaf ears. I even begin to wonder if those concerned are confusing attainment and achievement and morphing it into some kind of Frankenstein measure. It doesn't bear thinking about - an horrendous situation to be in, especially when such utter nonsense can have such dire consequences.

I am left with a mental image of our data-experimenting friends, having dispatched the headteacher with a wave of a hand, writing a new entry into the ledger - measure EYFS progress from a made-up baseline. Looking pleased with their day's work, they acknowledge each other's efforts with a satisfied nod, before closing the book to reveal the title on the cover:

The Big Book of Data Nonsense

There are still plenty of blank pages to fill in that weighty tome.