Automatic Diagnostics with Intermediate Steps in Mathematics Education Erasmus Project

Project Title

Automatic Diagnostics with Intermediate Steps in Mathematics Education

Project Key Action

This project related with these key action: Cooperation for innovation and the exchange of good practices

Project Action Type

This project related with this action type : Strategic Partnerships for school education

Project Call Year

This project’s Call Year is 2016

Project Topics

This project is related with these Project Topics: Key Competences (incl. mathematics and literacy) – basic skills; ICT – new technologies – digital competences

Project Summary

Competence in mathematics has been identified at EU level as one of the key competences for personal fulfilment, active citizenship, social inclusion, and employability in the knowledge society of the 21st century. Concerns about low student performance, as revealed by international surveys, led to the adoption in 2009 of an EU-wide benchmark in basic skills, which states that by 2020 the share of 15-year-olds with insufficient abilities in reading, mathematics and science should be less than 15%. Low achievement in mathematics is a common concern for all European countries. It is an issue associated not only with the effectiveness of teaching and learning, but also with providing an equitable system of education.

An extensive review of research evidence on what works for children with mathematical difficulties has concluded that interventions should ideally be targeted towards an individual child’s particular difficulties (Dowker, 2004). Individual support has proven to have significant impact on children’s performance (Wright et al., 2000, 2002), and reporting progress on achievement is important for improving mathematics skills. ICT can be of crucial help here. Digital learning environments used for stepwise solving homework are just as effective as individual tutors (VanLehn, 2011), and have the added advantage of giving teachers information about the progress of individual students.

The objective of the Advise-Me project was to develop flexible support for detailed diagnostics of mathematical competences of pupils, and to use this in existing digital testing and practicing environments in mathematics education. The diagnostics summarise the knowledge, skills, and competences of a pupil, and can be used by teachers to identify underachievers. With this information, teachers can direct their attention to help pupils overcome particular difficulties. Pupils can use diagnostic reports to get an accurate understanding of their own performance and progress.

Most digital testing and practicing tools in mathematics education review only a final answer, and do not look into the steps taken to reach the final answer. However, intermediate steps provide essential information for a detailed assessment of student skills. Assessing intermediate steps also corresponds to the pen and paper practice, and is a wish from the mathematics education sector.

Five organisations from the Netherlands, Germany and France, participated in the Strategic Partnership of the Advise-Me project: The Open University of the Netherlands (project coordinator), Utrecht University, Cito, Saarland University, and Paris-Est-Creteil University. Together, these groups have a long tradition in research on software technology for learning and teaching, mathematics education, educational technology, and automated assessment.

The project studied the automatic assessment of intermediate steps in mathematics education for the ‘Numbers’ and ‘Relationships’ domains. We focused on high-level mathematical competencies related to setting-up an algebraic expression or equation, and then solving it. We developed general feedback and assessment services that can be used by existing digital learning environments. The services recognise steps and solution approaches in free-form answers and use this information to update a user model of mathematical competencies. The developed technology was tested in classes.

The results of the project are threefold. (1) We developed innovative technology for calculating detailed diagnostics in mathematics education, offered as an open, reusable set of feedback and assessment services. The diagnostic information is calculated automatically based on the analysis of intermediate steps. (2) The detailed diagnostics have been integrated into the Digital Mathematics Environment (DME) and Pepite, which are advanced and widely-used digital environments for practicing mathematics. We have investigated interoperability standards to simplify communication between systems. (3) We have designed and executed pilot and evaluation studies for assessing the quality of the diagnostics. Almost 500 pupils participated in the studies. Data collected from the studies was made publicly available.

The project improved the state of the art in math education and automated digital testing environments, and accelerated the introduction of reusable (feedback and assessment) services. The collaborative design and development of such a shared set of high-quality services, and the innovation that such a collaboration brings, is a step forward towards tackling persistent problems in math education across Europe.

EU Grant (Eur)

Funding of the project from EU: 357025 Eur

Project Coordinator

OPEN UNIVERSITEIT NEDERLAND & Country: NL

Project Partners

• CITO BV
• UNIVERSITE PARIS XII VAL DE MARNE
• UNIVERSITEIT UTRECHT
• UNIVERSITAT DES SAARLANDES