Print
News

In or out?

10. Nov. 2017

Developing and experimentally testing an intervention against early school leaving.

At the end of August, the project Jump@School, which was funded under the European Commission’s LifeLong Learning successfully, came to an end. Among others, in just over three years, the project managed to develop an intervention to combat early school leaving and experimentally test it. All the processes and experiences gained within this project in its lifetime are compressed in the final project publication available in 5 languages (EN, DE, IT, ES und PL). ZSI was responsible for the evaluation of the pilot. The results of the impact evaluation and the formative evaluation as well as the lessons learnt from an evaluation point of view are available in the following three main reports:

  1. Report on impact evaluation
  2. Report on formative evaluation
  3. Aftermath lessons learnt report

What was Jump@School really about? Below is a summary of the implementation of the evaluation design and the analysis of the pilot.

The Jump@School pilot study implemented a ‘two-group control group design’ (see report on evaluation design for more details). The target group for the pilot were high school students aged between 14 and 17 at risk of early school leaving. The agegroup was chosen based on the definition of early school leavers from the European Commission which states that early school leavers are: ‘those young people (18-24 years old) who leave education and training with only lower secondary education or less, and who are no longer in education and training’  (Thematic Working Group on 'Early School Leaving', 2013, p. 8). Jump@School aimed to provide a preventative measure of early school leaving. Therefore, students who were not yet in this situation – that means younger than 18, had to be considered. However, some exceptions were included in the sample as the selection of students was based on grade levels rather than individual students; therefore grade levels with the most pupils between 14 and 17 were selected.

Two schools each, in Spain and Italy were selected to take part in the Jump@School intervention mainly based on their size and willingness to participate. As the sample size had been determined at the project application stage n=110 per school, it was of vital importance that the schools were large enough so as to identify at least 110 at risk of early school leaving. Due to fears of attrition, the sample size was eventually increased to 120. Initially it had been thought to determine whether a student was at risk of early school leaving based on three main indicators: School performance (grade point averages (GPA)), attendance and disruptive behaviour. As the different schools collect different kinds of data, only the GPA was uniform among all four schools and as a result, the GPA was the only indicator used to determine whether a student was at risk of early school leaving. The Performance benchmark was based on research by Allensworth/Easton 2007, who stated that students with a GPA of 2.0 or less at the end of their first year of high school should be considered at risk of dropping out. As at the time of selection only the GPAs at the end of the school year 2014/2015 was available, for the purpose of this study, this threshold was translated to a GPA of 2.0 or less at the end of this school year (2015/2016). Based on this indicator as well as absences by mid-January 2016, students who fell below the set threshold were considered at risk of early school leaving.

Proportionate random sampling was then carried out to select 120 students at risk of early school leaving according to grade level. These participants in each school were then randomly assigned to either the intervention or control group based on the grade level (randomised block design). After the randomisation process, ex-post assessments on the randomisation procedure were carried out to determine whether the two groups were balanced on the following indicators: GPA, gender, age and absences (Italy only).

The participants allocated to the intervention group then received the Jump@School intervention (see the operational model) while the control group did not. However, both groups filled in the same questionnaire at the beginning and at the end (after 5 months) of the intervention. The GPAs of all students selected were collected at four points in time regardless of their participation in the Jump@School intervention: June/July 2015, December 2015/January 2016 , June/July 2016 and December 2015/January 2016. Where possible, also data regarding absences and disruptive behaviour was also collected.

Due to post-assignment attrition in Spain, caused by many students selected to take part in the study not receiving parental consent, it was decided to carry out two rounds of the experiment in Spain. Here, in September 2016, after checking which students had returned to the next school year, the remaining eligible participants in each school were randomly allocated to the intervention and control group just as it was done for the first cohort and the students allocated to the intervention group, who received parental consent, received the intervention from October to December 2016. Both groups like in the first cohort completed the same questionnaires at two points in time and their GPA was collected at three points in time: June/July 2015, June/July 2016 and December 2015/January 2016. The grades from December 2015/January 2016 were not collected for this cohort.

Find out more about the project as whole in the final publication!

All the products of the project are available under: http://jumpatschool.eu/documents/

Downloads:

Related Articles: