Wednesday, 28 June 2017

Primary assessment for non-primary people: a very quick guide

The DfE collects a bewildering array of data from primary schools. This quick guide to statutory assessment is here to help.

Early Years Foundation Stage Profile (EYFSP)
This assessment takes place at the end of the reception year when pupils are 5 years old. It 
comprises 17 early learning goals (ELGs) against which pupils are assessed as emerging, expected, or exceeding. If pupils meet the expected level of development in the 12 prime and specific ELGs then they are deemed to have reached a ‘good level of development’ (GLD). The percentage reaching GLD is the key performance measure, which is shown in the Ofsted Inspection dashboard but is not in the publicly available performance tables.

Phonics Screening Check (PSC)
Carried out at the end of Year 1, pupils attempt to decode 40 words, half of which are real and the other half made-up. Pupils managing to decode 32 or more out of 40 have achieved the expected standard, and the percentage doing so is another key measure. As for EYFSP, school results are presented in the Inspection dashboard but are not available in the public domain.

Key Stage 1 (KS1)
At the end of year 2 pupils receive a teacher assessment in reading, writing, maths, and science. In science pupils are simply deemed to have met or not met expected standards (EXS or HNM). In other subjects the majority of pupils are assessed as either working towards (WTS), working at the expected standard (EXS) or working at greater depth (GDS). Pre-key stage assessment frameworks are available for those pupils that are working below the standard of the curriculum. Pupils take tests to inform the overall teacher assessment in reading and maths; there are no tests for writing and science. A grammar, punctuation and spelling test is provided but it is non-statutory and no data is collected. The DfE collect pupils’ overall teacher assessment in each subject – they do not collect the test scores – and the percentage achieving expected standards and greater depth in reading, writing and maths are the key measures. Pupils’ KS1 results also act as the baseline for primary school progress measures but this may change in future if the DfE implement a baseline at the start of reception. A school’s KS1 results are shown in the inspection dashboard but are not available in the public domain. 

Key Stage 2 (KS2)
At the end of Year 6, pupils sit tests in reading, maths, and grammar, punctuation and spelling. 
Achieving a score of 100 or more indicates that the pupil has met the expected standard and a score of 110 is deemed to be a high score. There are no tests for writing and science so pupils receive a teacher assessment based on the KS2 teacher assessment frameworks*, and these mirror the format of KS1 with a binary result for science (HNM or EXS), and more differentiated outcomes in writing (WTS, EXS, GDS). Pre-key stage frameworks are used to assess pupils working below the curriculum standard, with more pre-key stage categories than at KS1. The DfE collect test scores and teacher assessments but headline measures are mainly based on scaled scores in reading and maths, and teacher assessment in writing.  Headline measures include:
  • % achieving expected standards in reading, writing and maths combined (1 measure)
  • % achieving high standards in reading and maths, and greater depth in writing (1 measure)
  • Average scores in reading and maths (2 separate measures)
  • Average progress in reading, writing and maths (3 separate measures)
There are floor standards linked to these results with an attainment floor set at 65% achieving expected standards in reading, writing and maths combined, and progress floor thresholds that change each year. Schools' results are available in the public domain as well as in the inspection dashboard.

Note that only reading and maths scores are used for Progress 8 baselines.

*Teachers also make an assessment of reading and maths. This data is collected but is not used in headline measures.

Links:
Performance tables: https://www.compare-school-performance.service.gov.uk/
DfE Statistics: https://www.gov.uk/government/organisations/department-for-
#education/about/statistics
Key guidance: https://www.gov.uk/education/school-curriculum

Tuesday, 6 June 2017

The Data Burden

In implementing a new approach to assessment and tracking we must first weigh up our desire for data against any impact on workload. Sometimes those seemingly minor tweaks can produce a butterfly effect, placing huge demands on teachers' time that are most likely disproportionate to any benefit gained from the data. Before embarking on a new assessment journey always start by asking: who is this data for? what impact will it have on learning?

Many schools have implemented systems with a small number of key learning objectives in each subject, encouraging teachers to have the confidence to make judgements on pupils' competence within a broad framework. Some schools however have broken these key objectives down into many smaller steps - perhaps to enable progress to be measured over shorter periods - and this is having an adverse impact both on teacher workload and possibly on learning itself as we seek to assess and record everything a pupil does. This may be done to improve accuracy but such micromanagement of assessment is undermining professional judgement and is unnecessary, demoralising and counterproductive. There is a great irony in procedures put in place supposedly for the purposes of school improvement that actually take teachers' time away from teaching. It is hardly surprising that some headteachers are experiencing a backlash. An anti-tracking rebellion is underway in many schools.

I recently visited a school to help them set up tracking for the reception year. Initially the school wanted to track against development matters statements in each of the areas of the early years foundation stage profile, making an assessment every half term. After discussion, they decided that they would just focus on reading, writing and maths. I took a look at the list of statements - there were around 60 for each subject across the various month bands. I got out my calculator:

60 statements

3 subjects

6 terms

45 pupils

That comes to 48,600 assessments that the Early Years teacher would have to make across the course of year.

Let's say each assessment takes 5 seconds to enter onto the system. This works out as 9 days per year for one teacher.

Perhaps the teacher wouldn't have to update each assessment each term but you take the point. It's a potentially huge burden. Thankfully the headteacher and deputy went a bit pale and decided to go back to the drawing board. Their tracking system now requires the teacher to record an assessment against each of the 17 areas of the foundation stage three times per year. Somewhat more manageable and the early years teacher was certainly happier. They can spend more time doing more important stuff, like teaching.

But yesterday I had a lengthy twitter chat with awesome @misterunwin who challenged my thinking further. In his school they do no objective level tracking at all in their system. No box ticking, no rag rating statements, no assessment by numbers. Teachers make an overall judgement of where each pupil is at each half term and this produces all the data the school needs for analysis and reporting purposes. Teachers therefore spend very little time on data entry, freeing them up to do the stuff that has real impact: planning lessons and teaching children. Detailed pupil progress reviews are used to reveal gaps, inform next steps and identify where further support is needed. Staff are very happy with this arrangement, and improved staff morale will obviously benefit the school in many ways.

I have always tried to encourage schools to keep lists of objectives to an absolute minimum; to strike that balance between impact on learning and impact on workload. However, I completely understand why schools will want to reject any approach to tracking that takes teachers' time away from teaching, which is why some schools are choosing not to track in this way at all. I doubt that we are about to see schools ditching tracking systems en masse - I'm certainly not advocating that, by the way - but we do need to be mindful of the how these approaches can eat into teachers' precious time and the adverse impact this can have.

Start by getting your calculator out.

The beginning of the end

The DfE's primary assessment consultation, which closes on the 22nd June, focuses on a number of aspects of statutory assessment but it really centres on the baseline, the point from which pupils' progress to the end of KS2 will be measured. The DfE's preferred option is to set the baseline at the beginning of the reception year, and this has been supported by the likes of the NAHT (with caveats), the Headteachers Roundtable, as well as numerous headteachers and senior leaders that I've spoken to over the past year or so. I do, however, recognise that this is a contentious and emotive subject with many in the profession being understandably concerned and opposed to the idea of assessing children at such a young age. Is it possible to assess 4 year olds with any degree of accuracy? What about the pupils' month of birth, which can make such a big difference at that age? And can assessment be disruptive and stressful for children if they are not fully settled in to school life?

These are valid concerns but most primary schools already assess children on entry into reception so this is not new. No doubt the primary reason for carrying out such assessments is to support children in their learning but all too often schools are seeking to establish a baseline from which to measure progress. Pupils are therefore shoehorned into various bands with associated point scores in order to count steps of learning, or plotted onto RAISE-style progress matrices, which show them moving from one band to another. Those that progress, for example, from the 30-50 low band on entry to meet expected standards at KS1 are deemed to have made 'above expected progress', and are colour coded green or purple for good measure. Simple stuff that usually satisfies the demands of governors, the LA advisor and even an Ofsted Inspector.

The problem is that this data is fairly meaningless. On-entry assessments, carried out for the purpose of teaching and learning, are being commandeered for progress measures in order to respond to the increasing pressures of accountability. Such conflicting aims result in perverse incentives, which inevitably skew the data.  It is therefore no surprise that most pupils, according to schools' own data, are below average on entry and appear to make good progress across key stage 1. The reality, if the data remained true to its intended purpose, may look somewhat different and would probably be more informative, too.

The other issue is that many of these assessment practices are immensely time consuming, and it is often in the reception year that tracking is at its most excessive. Teachers dutifully tick off numerous development matters statements, in order to 'level' a child and supposedly measure their progress, despite this being contrary to the purpose of the assessment. It is, in short, a colossal waste of time. Would it not, therefore, be preferable to have a dedicated and universal, standardised baseline assessment, which would afford more robust comparisons of pupils, cohorts and schools, instil greater confidence in progress measures, and free other forms of assessment from the damaging influence of perverse incentives?

However, this is clearly an unpopular opinion. Someone recently suggested I was confusing assessment with accountability, of losing sight of, or perhaps never really understanding, the true purpose of assessment. As a data analyst I admit I am more focussed on accountability and performance measures - that's my job - but I do understand that the main purpose of assessment is to help children learn, to identify gaps and barriers, and inform next steps; and that these principles are put at risk by accountability. However, doesn't accountability in education require some form of assessment? Aren't they inextricably linked? Or am I being naïve or narrow minded? Perhaps this is more about confusion over the purpose of assessment: formative or summative, low or high stakes, for teaching and learning or monitoring school standards. Can assessment be all these things without getting wrenched apart in a tug-of-war between opposing forces? It would appear not. We only have to look at how the Foundation Stage Profile is being put at risk as pupils' development in specific early learning goals is being used to establish prior attainment groups for key stage 1 measures in the Inspection dashboard. And concerns about the validity of key stage 1 assessment, used as a baseline for key stage 2 progress measures, are nothing new.

A baseline therefore, whether taken in the reception year or at the end of key stage 1, is most robust if it has a single purpose: to act as a start point for future progress measures. There may be some formative by-product but that's not the main reason for carrying it out. Perhaps the reason why one particular assessment - one rooted in the principles of the foundation stage profile - became so dominant the first time round, was because we lost site of the main purpose of the assessment, or never truly understood it in the first place. Either that or it was a protest vote from a profession concerned about yet another accountability measure. But let's face it, the purpose was never very well explained; that the baseline was required to produce a standardised score, which would be used to construct prior attainment groups for a future VA measure. Pupils scores at key stage 2 would then be compared against the average score of pupils nationally with the same baseline score. That's pretty much it.

Accountability measures are not going anywhere soon, so we have to consider whether we want an accountability system based on attainment or progress. Most would probably go for the latter and so we need to work out how this is best achieved. It makes sense to measure progress from the earliest point possible but this doesn't necessarily have to be the beginning of reception year. It could be from the end of reception year, which would mean modifying the EYFSP, or from a separate assessment at the beginning of year 1. Whatever happens, it is obviously fairer to judge school performance on the basis of the progress pupils make and we need to recognise that the current process of measuring progress from key stage 1 to 2 is flawed, inaccurate and not fit for purpose. Future measures need to be far more robust, more standardised, and take account of as much of pupils' journey through school as possible.


Either that or scrap the entire system and start again.