Category Archives: Uncategorized

If data were the ocean, I’d send you a float

I’ve been out and about a lot recently, visiting lots of schools to talk about both Observe and Analytics. It’s probably just one of those weeks but recently I feel that data is taking a backwards step and I’ve found myself having to bite my tongue on more than one occasion.

‘We do things differently here’

There are two sides to doing things differently. Of course, it could be that a school is being completely innovative and fresh, but what if  no one else is doing it because it’s actually not very good? If this is the case, and there is no room for discussion they may never know.

This could be coincidence and is of course based on my own experiences but it strikes me that the schools who ‘do things differently’ are more like to also be suffering with data overload. Not feeling quite ready to abandon the complex and intricate systems they have been developing in some cases, for years, they are increasing workload and spending money without a return on their investment.  I visited a school recently who were producing reams of paper after each assessment point (of which there were 6 per year group in a big school), printing them all out, and then diligently delivering to pigeon holes for middle leaders, year group leaders and class teachers. Not only is this resource and paper heavy but is it altogether the wrong approach. I would go so far as to say that it is actually detrimental to the data cause, one I feel committed to defending. How will this exercise improve or impact on teaching and learning? I couldn’t see how in this instance, and voiced diplomatically only to be stopped in my tracks with….(drum roll please).

‘We do things differently here’

Wood for trees?

As this post is intended to be helpful as opposed to musing, the below steps outline what I believe to be an effective, simplistic approach.

  1. Pick relevant data. Class teachers do not need to know what last year’s exam residual was for a subject they don’t teach, relevance is key here. Class teachers need specific information about the pupils in their classes, where they’ve come from and where they need to go.
  2. Find it. The very act of finding the data is one of the first natural steps in understanding and interrogating, without which I predict any mass produced document will be lost under a pile of marking or worse, go in the bin. I understand why schools would want to provide a certain amount to avoid anything bordering on administration, but I don’t think disseminating the finished product works. Most schools now have SISRA, 4matrix or they’re doing stuff in their MIS but they can’t be expected to use it without quality training (see previous post on training staff here).
  3. Choose a focus. Asking middle leaders or class teachers to ‘analyse their data’ is a big and wide ranging task which increases fear and confusion. Middle leaders could choose to focus on one or two groups of pupils at a time, or ask faculty members to look at one each in detail. Alternatively, how about looking at levels of progress OR attainment vs targets OR class residuals.
  4. Build case studies. Instead of trying to tackle everything all at once, case studies of individual pupils can be powerful when shared within and across faculties. They also form a strong evidence base for an Ofsted visit and are thoroughly worthwhile on several other levels.
  5. Review and review. Data isn’t a one off task, and you’ve never really ‘done your data’. Ideally the cycle should be undertaken often and be planned in to faculty meetings or other directed time. If not, then it a little and often approach is best which then gradually (believe me, it will be gradual) becomes part of regular planning and monitoring.

I haven’t met a teacher yet, who would find a thick, stapled data document of much practical use. In conclusion, if it’s not practical is it just red tape?

Advertisements

On 2014 so far…

As I write this post from my living room come home office, we are within touching distance of being able to use a full and finished version of SISRA Observe, our new lesson observation tool. BETT is upon us, the SISRA Analytics pilot is 3 days in and our first ever annual conference, Empowering Improvement is just 8 weeks away. SISRA has grown exponentially over the last couple of years with no signs of letting up, having taken on a new school every day since the autumn term began in September! The office has a buzz I haven’t felt before but it feels wonderful and I hope I speak for everyone when I say I’m ready for whatever 2014 may bring!

Life can be cruel sometimes and losing my Dad unexpectedly three weeks ago made Christmas a sad time in my household, sprinkled with the odd moment where I was left feeling that any future success would be bittersweet without an important supporter there to see it unfold. I’m told time is a healer, and given little other option I have chosen to believe it. One thing I do know, is that I couldn’t have come this far without the professional support of the SISRA directors Terry and Jon who have shown complete faith in me, and without the love and understanding from my wonderful husband, Jason.

On SISRA Observe…

It’s been an exciting and challenging first project, and seeing the ideas which have consumed my thoughts being created (as if by magic) on a screen has sent me from the extremes of happiness and fear and trepidation at various times over the last 10 months. It’s just not normal to go to sleep thinking obsessively about what colour pie charts would work best!

But I am here nonetheless, and SISRA Observe is looking and feeling really great. I don’t have children, but I can only imagine the relationship between myself and Observe as one akin to a mother and daughter (yes, Observe is female). Like a proud mother I have talked to anyone and everyone about what Observe does, and why I hope it will make a difference to the lives of teachers, helping them to monitor and improve the quality of teaching while importantly saving them time and administration.

Where it all started…

My career in education started at The Dean Trust in Trafford. The trust is made up of Ashton on Mersey Academy, which has recently received its fifth Outstanding Ofsted report, Broadoak School, which received its first Outstanding while I was there, and most recently Forest Gate Academy. Having dealt with every type of school data in this role, I already had a solid understanding of what lesson observations involve and had created something of my own in Excel to allow easy analysis and evaluation for the SLT, something I now realise is done by one or more senior leaders in the vast majority of schools.

While at the Dean Trust, I would spend many an hour with Kevin Green who was an Assistant Head at the time but who has since gone on to achieve a Deputy Headship at Manchester Health Academy. We would sit in his office with a cup of tea and a few biscuits and talk over the data for KS4, testing out hypotheses and playing about with the latest assessment data trying to spot trends or patterns in the large Y11 cohort. Shortly after bringing SISRA Online in to the academy, I remember summoning him and the Head of Year 10 to said office, where I sat them down and performed a mental arithmetic style SISRA quiz, which I then marked and returned back to them! I bet they just loved me!

It was also Kevin who first asked me to look at the teaching and learning data, sparking my interest and giving me the opportunity to create something Excel based to help him identify potential coaching and mentoring pairs. In the sometimes relentless school environment, I was grateful for the relationship between Kevin and I, which was one of mutual respect and trust and having seen each other recently at an ASCL day, I was able to fill him in on the project.  I also took the opportunity to express my thanks to him for treating me with kindness and for playing an important part in my own development personally and professionally. As the famous Maya Angelou quote goes…

“People will forget what you said, people will forget what you did, but people will never forget how you made them feel.”

Never one to be content with helping just one school, the time came for me to move on from the Trust and embark on a new challenge with SISRA.  It wasn’t long before I began thinking more and more about the teaching and learning data, along with other things like behaviour and attendance which I used to be involved with. I knew that I could help schools to use their lesson observation data more effectively, using the themes from the work Kevin and I had done and so I decided to put pen to paper. Sure I could have created a lovely spreadsheet for schools to use and then give it to anyone who was interested, but the thought of thousands of senior leaders spending Sunday nights inputting data to a spreadsheet just didn’t sit well with me, and the realisation that I could change that was too big a chance to miss. I wrote down a brief plan and spoke to Terry who asked me to flesh it out and run the figures to see if it was viable. I dutifully went away and came back with a detailed document, mock up screen shots and lots of graphs, tables and charts showing the analysis I wanted to be able to provide.

The rest, as they say, is history.

So back to my living room come home office where I sit with the very highest hopes for the success of SISRA Observe. We all know hoping won’t get you very far so on that note, I have work to do. 

Thanks for reading.

‘Twas the last day of term….

‘Twas the last day of term and all through the school, most had gone home except one lonely mule.

The autumn analysis had all gone to pot, “I’ll have to redo it, the whole bloody lot.”

The children were giddy and hard to control, their ears waited eagerly for the last bell to toll. Staff room conversations turned to talk of mince pies and beer, as the last bell of the year rang out loud and clear.

But the data manager alone would have to stay behind, to complete the analysis before the Christmas unwind.
“There must be something more, something better than this, a spreadsheet so large, mistakes were easy to miss!”

Eventually he conceded and went home to his wife, Tired and generally fed up with his data life.
He prayed for a solution to cure his analysis woes, without even realising, the answer was right under his nose.

The following day refreshed and restored, He took to the net for his worries to be cured.
He clicked on a link for SISRA Online, and then sweetly exclaimed “This must be a sign!”

Right-click, save to favourites,Right-click, ctrl + C,
Compose new email message,Recipients, SLT.

He quickly pressed send, his heart thumping fast,The answer to his prayers had come to him at last.
SISRA Online could do so much more,Than the spreadsheet which had made his job such a bore.

The first question back came straight from the Head,Saying why can’t we stick with our spread sheet instead?
“Our teachers need something that’s easy to use,This will help them use data, and not just confuse!”

The head came quickly back with questions about cost,For admissions were down and funding had been lost.
Formalities were discussed, and the Head agreed,That it would save him time and allow him to lead.

A quick call to SISRA, and the demo was booked,SLT gathered round and within minutes they were hooked.
Why have we been drowning in the data abyss,While 1200 schools were accessing all of this!

The data manager waited patiently to hear,And read review after review on the Twittersphere.
Praise was abundant and case studies noted,But none of that mattered until SLT had voted.

As the new message tone chimed out from the laptop,An email displayed from the man at the top.
He waited for a moment before taking a seat,If they had decided in favour, his life would be complete.

“Merry Christmas to You” the email subject read,The recipient pinched himself as he saw what it said,
Our gift to you, is way over time, But here is your login for SISRA Online.

Target Setting doesn’t have to be hard work….

Those of you who follow my tweets regularly will already know that I exist mainly on a diet of data, more data and the occasional large glass of Brancott Estate (if you haven’t yet had the pleasure of this delicious Marlborough, stop reading and get to Tesco).

As part of my job at SISRA, I am lucky to be able to gain first-hand experience in many school environments which makes it much easier for me to judge objectively, take snippets of the best and pass that information on to those who could benefit. Along with the wonderful work being done up and down the country, I also see some practice which frankly, keeps me awake at night. I often wonder how and why this variety exists within schools, and more importantly how, as one person I can contribute to helping drive the standard of data use in the right direction. If you’ve come this far then I don’t need to tell you that if the use of data isn’t contributing to teaching and learning, then we shouldn’t be doing it.

I recently visited an outstanding school to do some work with their middle leaders. I was greeted with trepidation as the staff filtered in to the session, but I didn’t take it personally and as we settled in to the session it emerged that the staff hadn’t been briefed beforehand which explained their suspicious faces. We began looking at the most recent exam results, discussing attainment, progress and comparisons with the previous year which all seemed to go well, and most staff were keen to follow the session which I measured by the proportion simultaneously checking their emails when they thought I wasn’t looking.

All was well until I mentioned the T word… I was surprised and saddened to find that not one of these talented, passionate middle leaders could tell me how the targets for the pupils in their subjects had been set, let alone engage in open discussion about them and as a result there was a complete disconnect between the staff and their data as well as several other harmful consequences.

So, what did I recommend for this school?

1.       Training staff to use estimates

Even as an expert, I sometimes feel overwhelmed by the amount and complexity of the data available to schools so it’s vital that differentiated, relevant training is provided. At the very minimum, ALL staff should know the difference between an estimate, prediction and target (in that order) so in case you didn’t, or need refreshing I have summarised below:

What is it? Source
Estimate “Data says that pupils with your profile are most likely to achieve a B grade.”“40% of pupils with your profile achieve a B grade and 12% achieve an A.” Key Stage 2 results, FFT Estimates, CATs indicator, MidYIS, internal baseline testing
Prediction “If your current poor efforts and attitude are maintained, I predict you will achieve a C grade.”“As you’ve been working so hard, you’re currently on course for that A!” Estimate + professional judgement (attitude, personal circumstances, personality)
Target “It might be a challenge, but I really think you could aim for an A grade.” Prediction + teacher and pupil aspiration

As a summary of the information above, I really like this which is taken from www.fft.org.uk

FFT

So, if we assume that a target requires a prediction, and to get that we require an estimate, we can start to look at the different types of estimates schools might want to consider.

  • Key Stage 2 results – personal opinions on the validity of these results aside, schools will be judged on the progress each pupil makes from their respective result, so it’s always a good idea to keep them in mind.
  • FFT Estimates – clue’s in the name folks, these are NOT targets! FFT Estimates in their various flavours are probabilities based on key stage 2 data, national performance and historical school results.
  • CATs Indicators – gives numerical outcomes in verbal, non-verbal and quantitative tests, with 100 being the national average. Provide a +/- profile where there is a significant difference between performance in any of the three strands. Interesting comparisons can be made between these scores and KS2 results.
  • MiDYIS – as with CAT, tests pupils in a variety of areas then produces probability and chance style data.
  • Internal baselines – when done rigoursly, internal baselines can provide valuable input for target setting, especially in subjects which are not taught in primary.

2.       Making a prediction

To make a prediction, teachers should be able to use estimates, then have the freedom to add in their professional judgement and anything else not quantifiable (attitude, personality, etc.).

I have no objection to a skilled data manager or senior leader using the data in weird and wonderful ways to provide another layer of estimation to the target setting process, in fact I think that’s really great when done properly, but estimates are not useful if staff cannot understand what they are and where they have come from.

Some schools may opt to measure staff performance against these predictions instead of the more aspirational targets set in conjunction with pupils.

3.       Deciding on a target

Assuming that both an estimate and prediction have been made, a target can now be set. I strongly believe that pupils should be involved in this process in order for them to take ownership and to encourage accountability should their progress or attitude slip during the year.

One popular, and powerful way to do this is to share chance probabilities with pupils then discuss where they see themselves. Represented on a simple graph or as a table,  looking at this data with subject teachers will enable targets to be truly aspirational and personal. Involving the pupils in this process also secures a verbal commitment from them, encouraging accountability.

Graph

Progress Leaders/Heads of Year /House can then monitor progress towards these targets, identifying underperformance and putting interventions where necessary.

There are many good ways to set meaningful, challenging targets for pupils. No one method will suit every school, subject or pupil so it’s important to draw from as many places as possible before deciding on what’s right for your school. On the flip side, there are also many ways to disengage teachers, dissociate pupils from their own data, and create a culture of mistrust between teachers and senior leaders.

Targets are an essential ingredient in improving attainment and progress and I’d like to see more schools devoting the time and energy needed to get it right!

Removing barriers to effective data use

The importance of data in education is now well accepted and schools have become adept at gathering and storing lots of data about pupils and their achievements. Many teachers have become expert at using achievement data to plan lessons, design seating plans, identify underachievement and to analyse performance of their classes at various points in the year.

As a data consultant with SISRA, I know only too well the challenges faced by school staff at all levels and work hard in my job (and often my free time) to help teachers overcome their data phobias. There are perceived barriers in almost every school I visit, but the good news is that they are (nearly all) surmountable!

Helping staff to overcome barriers to using data really depends on whether they can’t or won’t ‘do data’. In addition to those who can’t or won’t, there are also those who ‘can’ and ‘will’ – the people I like to refer to as the ‘champions’. These are the people who will help make data training a success in your school and can help drive data use from the staffroom. Encouraging these champions to get involved with the training sessions can have a positive impact on the attitudes of other staff, which will hopefully permeate all levels of the school hierarchy.

Quick Look. Can’t Won’t Do Data

Those who can’t do data…

‘I’m rubbish with computers’

Let’s face it, everyone sometimes feels the familiar pang of frustration that technology can sometimes cause, but for those who really struggle using office applications and management systems, the defences are up before they have even accessed the data they need. Having a knowledgeable and patient data manager can help, but helpful guides, handouts and reminders will also make staff feel genuinely supported. Weekly drop in sessions are a great way to support any staff wanting extra help, and running refreshers at busy times might just reduce the stress levels, especially during report writing periods for example. Creating a dedicated area on your school VLE is a great way to store handouts and guidance so they can be accessed easily by those who need them.

‘I don’t have time!’

It is important for staff to be able to access key data at times and places convenient to them which is why web-based applications and management systems are becoming common place. Many of us work from home on a regular basis and so being able to access data via the web can allows teachers the required flexibility. Many schools now have data on the agenda for every faculty meeting and set aside time, either at the start of a lesson, or during form time where pupils can access their own data and discuss with a member of staff. How about protected time around data collection windows? For some schools, that’s a step too far :).

Won’t do data

‘Let me get back to my actual job’

Teaching staff must feel that using data is worthwhile and will contribute to teaching and learning overall in order for them to use precious time interrogating and analysing it. In other words, if teachers cannot see how the data can impact on teaching and learning, you have little chance of persuading them to use it. Demonstrating the relevance of the data to teachers can be achieved with high quality training; either provided by a willing person in school or an external consultant. This type of training can be particularly effective in small groups of teachers from different faculties. Grouping staff and asking them to examine pupils in a given teaching group based on progress from KS2-4, progress towards target and subject residual, can highlight where pupils are responding to a particular member of staff or topic. One of the best parts of my job is having detailed discussions about specific pupils when I am on-site at a school. Looking at the performance of a pupil across all the subjects they take can reveal some surprises, and further discussion can allow sharing of practice between faculties. Comparisons with National Residuals allows staff to see how their classes compare with other pupils across the country.

‘What are you going to do with it?’

Performance management and performance-related pay are pertinent topics at the moment, which is vilifying data even further and there is unfortunately much mistrust around the use of exam results and lesson observation data to inform performance management. Senior leaders can promote trust by allowing class teachers to observe them, and use the same data to evaluate performance in the same way that they will be judged. During training, senior leaders could use their own group to demonstrate where a pupils is underachieving, or perhaps look at their own exam results from the previous year. Having an open and transparent system for data analysis can also promote honest discussion without the secrecy that has historically surrounded these types of data.

I believe that most people can learn to use data well, and some will even grow to love it! Senior leaders with responsibility for data can often find themselves alone in their enthusiasm; however a ‘whole school approach’ to data is the only way it will ever be truly embedded and used to maximum effect.