(1 of 3) Measuring the Impact of Technology on Learning

How do we measure the impact of technology used to support learning?

In the first of three posts I reflect on technology’s role in education and how we can measure the impact as we utilise it in our classrooms.

Learning is a complex system, particularly in schools where we try to turbo charge it from 9am to 3pm. The measure of that learning can often be less complex, such as a letter or number to quantify the learning or benchmark the progress since the last check.

Technology has always been part of learning…

Chalk, slate, paper, pens, books and calculators… but we are specifically referring to new technology that has become so ingrained in our lives, therefore schools are grappling with whether and how to embed this technology in learning. If a school gives every child an iPad, will this improve learning? Of course it won’t, any tool is simply a small part of the system but we have become used to measuring impact on learning so much in schools we need to be able to measure the impact of technology on learning but it is crude and dis-ingenuous to measure the impact simply in results. However, schools can spend heavily on technology when budgets are shrinking and the investment needs to be justified.

Measuring Impact in Schools

The measures we are used to are

  • Results
  • Lesson Observations
  • Progress indicators

If I were to visit your school and say

  1. “Implementing technology to support learning will improve results.”
  2. “Lessons with technology are better than those without.”
  3. “Learning with technology will improve student progress”

I sincerely hope you show me the door as I almost definitely don’t have evidence to back the statements up. Furthermore, if those silver bullets existed you would already be using them.

However, I do strongly believe that there are some learning experiences that technology can enhance, improve and provide that are not possible without it.

 

wpid-wp-1391683768119.jpg

There are times we have reflected on lessons and note that mini-whiteboards, post-it notes, laminated cards or highlighters could have made the lesson more successful. This is not because it saves paper, looks pretty but because it improves progress, understanding, explanations, discovery, feedback and so on. Hence, I aspire that technology sits less as an entity on its own and becomes ingrained in learning and teaching. There are fantastic tools freely available to teachers that can enhance feedback (The Education Endowment Foundation and Hattie have research to suggest this has a significantly positive impact on learning. For balance you may wish to also read @LearningSpy’s blog post summarising some objections to the use of effect size and this post questioning the statistics in Hattie’s work) to students or facilitate more effective group work or peer review.

If I were to visit your school and say

  1. “Implementing technology saves teachers time.”
  2. “Lessons with technology allow for more personalised learning.”
  3. “Learning with technology will improve student engagement”

I hypothesise that I would be much less likely to be shown the door. Technology can be a distraction, irrelevant or a positive impact on learning and I suggest the measures we use to assess the impact of technology need to be discussed and a clear set of agreed measures brought in to general use. The team at Google for Education tried this with a vote on twitter:

Technology used to enhance personalised learning requires some cultural change in schools as it challenges the teaching styles of many successful teachers, the learning preferences of good children and the digital literacy of both groups. Alongside any implementation of a change to embed technology in learning is a need for training, discussion and a clear vision for learning. The training deficit that I have become aware of as I try to embed technology in learning was highlighted more publicly in late 2015.

On 15 September 2015, The Organisation for Economic Co-operation and Development (OECD) published a report called “Students, Computers and Learning” on technology in the classroom and snippets from it formed rather sensationalist headlines, including this one on the BBC website.

Computers ‘do not improve’ pupil results, says OECD

The report looks at computer usage across a number of countries and compares outcomes. Using OECD’s Programme for International Assessment (PISA) results from 2012 the report identifies “no appreciable improvements in student achievement in reading, mathematics or science in the countries that had invested heavily in ICT for education”.

Do we ‘invest’ in ICT for Education?

Is investment in ICT for education or technology for learning simply monetary? My school has an investors in people plaque in reception and I am pretty sure that doesn’t mean they pay more than other schools. However, the reference to investment in ICT in the OECD report is a quantitative measure. Despite the media headlines the report is detailed and aware of the data it has and has not used when summarising the findings.

Technology can amplify great teaching, but great technology cannot replace poor teaching.”

OECD: Students, Computers and Learning: Making the Connections, 2015

Schools will have to choose the level and focus of investment in technology for learning. Whether that takes the form of devices, training, open discussion, design thinking projects, bring your own device (BYOD) policies or even new learning spaces is a big and difficult decision for school leaders to make. Especially as the impact of these measures are unclear, hence this blog post.

The discussion around technology in schools is often confused,  school leader. There is a distinct difference between school provided technology to support learning and a personal device used by a student. This distinction can be overlooked in the discussion that surfaces via bloggers, newspapers and those with vested interests. An Ofsted spokesperson was quoted in the Times Educational Supplement (TES):

“Pupils bringing personal devices such as laptops or tablets into school can be extremely disruptive and make it difficult for teachers to teach,” an Ofsted spokesperson told TES.

This quote is then translated into this headline:

Ofsted warns against ‘extremely disruptive’ tablets in school

Richard Vaughan – TES December 2015

Here is a response to the TES article, (Ensure you also read a follow up post including Ofsted’s response as the reference to disruptive technology was specifically to do with technology in schools not for the use in learning) including a search through Ofsted reports for references to technology’s impact on learning.

The decision six months ago to equip all students with tablet computers has not been universally welcomed by parents and carers, but the positive impact on students’ learning is obvious. The computers help students to work independently, they give all students equal access to online resources and they provide an excellent communication tool between teachers and students.

Ofsted Report June 2012 – Secondary (School overall judged as Outstanding)

The impact of technology on learning to this inspection was ‘obvious’ so we know it when we see it. The impact measures suggested by this inspector are

  • Independence
  • Equal access
  • Effective communication

Did this school implement tablet computers because they identified weaknesses in these areas? When I work with schools advising them on their use of technology to support learning the first document I request is the school development plan as there is no point developing technology or the use of it in anything that does not support the key focuses of that school. Therefore the measures have to be the same measures schools put in place every day, week term and year.

The OECD report summarises the implications of the findings and suggests that schools are not yet able or ready to embed technology in learning and leverage the potential. I agree, that we are yet to develop a clarity on how we want technology to blend into pedagogy and schools. I feel this provides more support for a drive to agree tangible measures for the impact of technology. What is the problem we are solving? The implications section concluding the OECD report implies the following potential benefits of technology in education:

  • Equality
  • Digital Literacy
  • Teacher and Student Collaboration

I suspect a search through school development plans will find many that want equality, more officially known and closing the gap. A number of approaches can support this and technology, planned and implemented well, can help. However, a poorly executed implementation of technology can bring into sharp focus the inequality between students. We often hear educators present the now familiar concept that “we are preparing our students for jobs that don’t exist yet” (if I had a penny…) and therefore the use of technology in schools is necessary to prepare them for the world they are going into. However, I didn’t have any such preparation and I think I am doing fine. Technology is already allowing teachers to share and collaborate and we have dumped millions of mediocre to poor resources onto our blogs, social media and resource sites.

What would your measures of impact be?

In part 2 of this 3 parts blog series I want to focus on the tool versus the teacher.

 

How might we… Evolve #Edtech

Without talking about tech?

After my enthusiastic and bold message to my boss during Ewan McIntosh’s keynote (thanks to lisibo for the reflections and great sketchnotes!) at Practical Pedagogies in Toulouse I have been preparing to meet with him to start making that dent.

My role is to embed technology in learning across the academy chain, however I am coming to the conclusion this is best achieved by not talking about the technology (or at least as little as possible). Conversations about learning, independent learning, sharing and collaboration have been more effective in moving the use of technology forward in an effective way.

Hence, if my colleagues are creating innovative ideas and looking to make them happen, technology that supports learning will follow.

“How might we raise the aspirations of our children?”

“How might we shared best practice across our schools?”

“How might we assess without levels?”

“How might we connect with other cultures?”

“How might we support our local community?”

These are learning based problems that can have a technology aspect in their solutions but technology is not the solution alone. I often refer to a former colleague who was/is a self-confessed “techno-phobe” and nervous of my #edtech role. However, one lunch in the canteen she idly mentioned whatsapp, which she was merrily using on her new iphone. The technology gave her no worries or concerns…. Why? Because it allowed her to view endless pictures of her beautiful granddaughter. It solved a problem and enhanced her experience of being a new grandmother.

The design thinking process seems to be an ideal tool for identifying problems and developing ideas so I want to try and begin to introduce it to my organisation.

IMG_20151029_224921Part of this process has been re-reading Ewan’s book “How to Come Up with Great Ideas… and actually make them happen”. It is a fascinating combination of Ewan’s experience and a journey through the design thinking process. The aspects I have picked up in particular are ideas about trying to move excellence out of individual classrooms, how meetings do not support innovation and looking to define problems, great problems that can lead to great ideas. I have used a lot of post-its, though this seems to be a big part of design thinking!

The key ideas I want to get across are:

  • Help us immerse in our schools and look at the details (avoiding doing what we have always done)
  • Valuing people and their ideas (design thinking gives us a way to get the best ideas from our colleagues)
  • Make our organisation innovative and focussed on learning (Make the big thing, The big thing)
  • Schools can remain individual and autonomous while having a common language for creating innovation across the academies (A key vision of our Multi-academy Trust, MAT)
  • Subtle was of getting everyone utilising technology effectively (Evolving Edtech maybe?)

It feels fuzzy but I am assured this is how all great ideas should feel at the beginning. However, fuzzy doesn’t always allow people to buy-in.

The practical things I am asking for are

  • A notebook for a team/teams to write a bug list and ideas wallet
  • A room to display the immersion of our core team in our schools
  • Lots of post-its
  • Permission to form a project team that represents out entire organisation

My next post may be my last if I cannot convey the impact I think design thinking could have on our organisation.

Watch this space

Ben

Does UK have “edtech integrators”?

I started writing a post some time ago:

“I don’t think we do and if not why not?

What is a edtech integrator?

Chris Betcher wrote an interesting post about his role as a tech integrator. This is a term I have heard from US based educators using tech and Chris is based in Australia. I have no knowledge of such a role in the UK beyond enthusiastic teachers like myself developing a role for ourselves to support the use of technology for learning within our own schools.

A true tech integrator seems to straddle a number of schools in alliances, districts or chains and oversee the training, implementation and support of technology to support and enhance learning.

Why does US have edtech integrators?

With a system like our Local Authorities still in place the US districts have a clear need for tech integrators as they will implement district wide implementations of Google Apps, Office365, ipads or chormebooks and they need people in place to make sure the investment has impact and there is consistent use of the new tools.

Where is edtech on the UK schools agenda?”

Since beginning that draft… er I think I have become one?

From September I will be working as a mathematics teacher for three days a week and supporting technology for learning across my school’s academy chain for two. The role is aimed at working with the school leaders in the academy chain to get the most out of their existing technology to support learning and develop longer term strategies to enhance their provision to support learning and teaching with technology in effective and efficient ways.

I initially met with heads of the schools and the response has been fairly positive in that they seem to be very open to the idea of someone supporting the school’s strategy in an area they do not always feel is one they possess excessive expertise in.

So, I am hoping that I will be able to share the role of an “edtech integrator” in UK with you.

Exciting times ahead

#giesummit New Hampshire

I have been fortunate to have been invited to present at a few Google in Education summits run my Appsevents across Europe, delivering sessions on Google Classroom, Flubaroo for Google Forms, Super Quiz for Google Forms and e-portfolios. I am delighted to say I have been invited to New Hampshire to deliver sessions and a keynote presentation on the second day.

New-Hampshire-summit-ad
Find out more https://goo.gl/0LDysj

I wouldn’t profess to be any expert but simply an enthusiast for the possibilities technology can bring to the classroom and student learning. Sharing my insight is enjoyable and rewarding but I have always taken away ideas from fellow presenters and delegates alike, who are always amazing me with their initiative for ways in which they are using simple free tools to enhance learning in their classes.

I am particularly excited to have the chance to share with Holly Doe in New Hampshire as I was fortunate enough to attend the same Google Teacher Academy as Holly in London in December 2013. During our two day event she shared an inspiring use of Google’s tour builder with us. Her students had created an immersive account of a journey and developed some amazing story-telling skills along the way. I reflected on how delighted I would be as a father if my son had that learning opportunity.

So, of I go to New Hampshire confident of coming back to UK with some new gems to try in my own classroom or share with the schools I am going to be supporting from September. If you haven’t considered such an event yet I hope you might consider one near you. Check our the appsevents website for details of summits near you.

Enjoy

Ben Rouse

Pedagogy with #GAfE – Marking and Feedback in Google Docs

Since introducing Google Apps for Education (GAfE) at my school in September 2013 I have been struck by the way that individuals will begin using the tools in their own way to improve their pedagogy. It shows how great teachers pick and choose the tools that are right for them and their classes.

As with all good edtech, the tool just refines the great pedagogy.

One great example of this is a language teacher who I work with who has developed her marking and feedback using Google documents. She has made critical choices about how to use the tools and has worked hard at training her class to submit their work via Google Drive and respond to her feedback. This demonstrates some key features of how to develop pedagogy with technology.

  • Embedding technology takes the same time and effort as any pedagogical tool
  • Your class need to be trained in how to produce the work your expect
  • Be persistent
  • Don’t expect it to go smoothly (just like any lesson or change in your classroom)

In preparation for their language qualifications the students have to develop a text covering some key topics. This teacher identified the sharing and collaboration of Google docs as a perfect tool to help with drafting, re-drafting and providing feedback. The benefits have been:

  1. saved time
  2. improved the impact of teacher feedback
  3. changes can be reviewed and tracked
  4. better student outcomes!

Not my interpretation but the feedback of the teacher. It has not been a smooth ride, managing your Google Drive and that of the students is a necessity, something that can be dealt with by using Google Classroom. This teacher understands how Google Classroom works but having spent time developing her class to use a system of sharing their work with her and then she would organise it in her folders, she did not want to change. We have discussed trialling Google Classroom for the same task in the summer term when there is a little more time to develop new strategies.

I have taken a piece of work completed by one of her students and used draftback.com to show the evolution of the document to try and demonstrate the power of feedback via a Google document over the traditional piece of paper that can be lost by the student or ignored as they write their second draft without the teacher’s invaluable feedback. I hope the video gives a good idea of the development of the piece of work thanks to great pedagogy from a great teacher.

As with all good edtech, the tool just refines the great pedagogy.

Enjoy

Ben Rouse

Evolving edtech indeed…

I was clicking around one of my class blogs and found a link to a website I used to use. I found it interesting to see the difference between my old wikispace, where I shared resources and the new site I am developing to house resources for maths students.

Before:

Wikispace site created after I started making instructional videos and uploading them to youtube.

After:

Google Site that I am developing to host the large range of instructional videos from my youtube channel. Mr Hegarty has a better site…

I am finding that change is the only constant so I am beginning to question the time and effort I put into these sites as I know I will change my mind on the best way to share resources and start a new project soon. In fact I sketched the design of a new site the other day…

Enjoy

Ben

Google Educator Group UK 2014-15

What will this academic year hold for teachers using Google tools and how can the Google Educator Group help?

GEG UK Banner 2560 x 1440

Google Educator Groups have sprung up around the world since they were launched and I have been involved with the UK one. We have had some hangouts and there is a Google plus community of around 350, however some other groups such as GEG Melbourne and many across the US have had events and meetups. The GEGs in countries like India and Philippines have huge numbers and they appear to be thriving.

Some regional GEG groups have started in the UK.

GEG UK London

GEG UK North East

GEG UK South East

The UK has pockets of educators using Google Apps for Education (GAfE) very effectively but the number of schools with GAfE accounts suggests there are possibly thousands of teachers with access to use the tools.

  • Do those teachers know how to use the tools and what they can do?
  • Do they want to more about how they can be used?
  • Would they attend training on a weekend?
  • Would they be interested in meeting teachers who are using the tools for learning?
  • Can the current members of GEG UK provide any guidance for school leaders in schools with GAfE?
  • Do any UK schools give a budget to their teachers to choose their PD for the year?

If you read this I would imagine you are not the target for the GEG UK but the people you work with are. Can you ask them if they are interested in learning more about the potential for Google Apps for Education and if so what support, resources and events can GEG UK provide.

Thanks

 

Ben Rouse