Will robots soon be conducting pupil-assessments?

By May 19, 2017
robots at school

A recent research paper by Professor Rose Luckin of University College, London points to the potential advantages of Artificial Intelligence systems for pupil assessment.

Might such things as school examinations, tests, marked classwork and progress checks soon all be a thing of the past? They will be if Rose Luckin, Professor of Learner-Centred Design at the Knowledge Lab at the University College, London (UCL) Institute of Education, has her way. She argues that the way school pupils are assessed today is unsatisfactory. “Decades of research have shown that knowledge and understanding cannot be rigorously evaluated through a series of 90-minute exams. The prevailing exam paradigm is stressful, unpleasant, can turn students away from education, and requires that both students and teachers take time away from learning. And yet we persist in relying on these blunt instruments, sending students off to universities and the workplace ill-equipped for their futures,” writes Professor Luckin in a research paper.

She says that the main reason why we continue to use the periodic ‘stop and test’ examination approach is that there is no satisfactory alternative. Or rather this was the case until quite recently because, according to the UCL professor, recent advances in Artificial Intelligence mean that today we can certainly consider a different approach. She writes: “The application of AI to education has been the subject of academic research for more than 30 years (…). The evidence from existing AI systems that assess learning as well as provide tutoring is positive with respect to their assessment accuracy. AI is a powerful tool to open up the ‘black box of learning’, by providing a deep, fine-grained understanding of when and how learning actually happens.”


Drawing up a profile for each pupil

So what might an alternative education assessment system, based on Artificial Intelligence, actually look like? Rose Luckin argues that nowadays AI is capable of “forming an evaluation of each student's progress (…) over a period of time.” The way it would work is that information on the curriculum, subject area and learning activities pursued by each student, “plus what counts as success” at each stage of each activity, would first be collected. AI techniques, such as computer modelling and machine learning, would then be applied to this information and the AI assessment system would form an evaluation of the student's knowledge of the subject area being studied.

The Education professor further points out that AI assessment systems can also be used to assess students’ skills, “such as collaboration and persistence, and students’ characteristics, such as confidence and motivation.” The information collection and processing carried out by an AI assessment system would take place over a period of time. This period of time may be a whole school semester, a year, several years or more, unlike a standard 90-minute exam, which focuses on only part of the pupil’s skills and characteristics, with particular emphasis on memory, and at a brief instant in time, underlines Rose Luckin.  

The output from AI assessment software can be synthesised and interpreted to produce visualisations of a student's knowledge, skills or resource requirements, which she calls Open Learner Models (OLMs), that will help teachers understand their students’ approach to learning and so adjust their teaching to the perceived needs, while also enabling students to track their own progress and encourage them to think about their learning processes.

Rose Luckin gives as an example the AIAssess generic assessment system developed by researchers at UCL Knowledge Lab. Well-suited to ‘hard’ subjects such as mathematics and science, the system sets pupils exercises of increasing difficulty, adapting the questions to the individual’s knowledge level and gaps, plus what the professor calls his/her ‘metacognitive awareness’ – i.e. understanding of one’s own learning abilities, reactions and needs. Based on this data, the software system guides each student towards the solution to each problem, providing hints and tips geared to the student’s individual profile. Every answer given provides the computer with yet more data that enables the system to refine the individual profile. The system thus provides the teacher with a picture of the basic knowledge level of each pupil, gaps and weaknesses, areas for improvement, and tips for optimising pupil progress. For instance, some students need to be guided by a series of precise intermediate questions before being able to arrive at the solution to a maths problem, while others can manage the task in fewer steps; some pupils need more encouragement, and so on.


Helping to create a fairer education system 

Rose Luckin argues that the use of AI systems of this kind would help to promote social equality in the education system by enabling the provision of more personalised education over the long term, an approach which currently remains restricted to the children of those wealthier families able to pay for private coaching and tutoring. She insists that “AI would provide a fairer, richer assessment system that would evaluate students across a longer period of time and from an evidence-based, value-added perspective” that takes account of more than a simple ‘right’ or ‘wrong’ answer approach to testing.  AI assessment systems “would be able to demonstrate how a student deals with challenging subject matter, how they persevere and how quickly they learn when given appropriate support.” This important information on pupils’ abilities is not gathered by the current assessment systems, thus wasting opportunities for useful feedback.

Luckin nevertheless sounds a note of caution regarding the use of AI systems in education. For instance, measures will need to be taken to ensure the security and confidentiality of pupil data. The approach used must also be able to “make transparent the AI system's reasoning (…) and it will be essential to be able to explain the assessment decisions made by any AI assessment system and constantly provide informative feedback,” she underlines.

AI promoting personalisation and efficiency

The use of Artificial Intelligence is nowadays being considered in numerous fields and domains, and the education sphere is no exception. Andrew Ng, who was one of the founders of online education platform Coursera, recently left his post with Chinese firm Baidu to concentrate on applying AI to education. In April, in an interview with US online media organ The Verge, Microsoft founder Bill Gates discussed the potential benefits of AI in creating teaching materials.

As long ago as 2014, Google launched Classroom, a tool designed to promote online communication between teachers and students, enabling such interactions as teacher-student information, submission of homework assignments by students and the return of marked work by the teacher. Meanwhile Facebook is partnering with Summit Public Schools, a local schools network in the San Francisco Bay area, which is also financed by the Gates Foundation. The purpose of the partnership is to design an individualised teaching software package and make it accessible free-of-charge to all.

All these various initiatives are based on the same idea: to switch from an ‘off-the-peg’, ‘one-size-fits-all’ mentality in education to a ‘tailor-made’ approach, helping teachers to provide more individual, more personal assistance to every student. The intention is not that AI will replace teachers, but help and support them. Says Joel Mokyr, an Economic History professor at Northwestern University: “I doubt we’ll ever be able to do without human teachers. But a teacher assisted by intelligent machines that can obtain information on each student and compare him/her with millions of others so as to select the best approach for his/her profile – that would be even better.  Artificial Intelligence is not in competition with us, it complements us, improves us, makes us smarter, better informed.  AI doesn’t replace people, it makes them more efficient. And when you think just how much education needs to change, that’s all the better!”

Legal mentions © L’Atelier BNP Paribas