Evidence for Social Policy in Europe
To date, researchers in our network have conducted over 90 ongoing or completed randomised evaluations of social programmes and policies across 21 European countries, with a particular focus on the education and labour market sectors.
Here we highlight a selection of insights generated by these studies over the past decade.
Research conducted by J-PAL affiliates in Education in Europe has focused on narrowing the achievement gap for students from poor or marginalised backgrounds, including through promoting access to high-quality education for all, helping children develop skills associated with improved learning, increasing access to information about educational opportunities for both children and parents and providing targeted remedial education to help students catch up as needed.
Improving educational opportunities for children from disadvantaged backgrounds
Children from economically disadvantaged backgrounds or regions often have access to fewer resources—in and out of school—for academic success, and those from immigrant or minority backgrounds may face additional barriers. These factors can become obstacles to obtaining quality employment opportunities later in life.
A growing body of randomised evaluations has sought to understand what forms of targeted interventions can help better connect these students to school resources in an effort to improve learning outcomes and promote social mobility. A recent review of experimental evidence on tutoring programmes from North America showed that tutoring generally led to large improvements in student learning, particularly for students in earlier grades and when conducted during school by teacher or paraprofessional tutors.
- In Italy, researchers partnered with the education ministry to evaluate whether a programme offering tutoring and career counselling for high-performing immigrant students could narrow attainment gaps between native-born and immigrant students at the end of middle school. Grades 7 and 8 are an important inflection point in the Italian system, when students choose between academic, technical, or vocational tracks for secondary education.
- The Equality of Opportunity programme provided career counselling and tutoring for immigrant children with high academic potential, in an attempt to better align their goals and aspirations with their ability.
- The programme improved immigrant boys’ academic performance and led to higher enrollment rates in academic and technical high schools, eliminating the gap between immigrant and Italian-born students. No impact was found on girls’ achievement, as there was no education gap between them and native girls to begin with.
- Lower-performing immigrant students in schools where the programme was run (who were ineligible for the program) also benefited from the program, while there was no impact on native students. These results are consistent with previous evidence suggesting that peer effects are stronger within groups with a similar background.1
- Another recent study in Italy examined whether teachers hold implicit biases that might lead them to award lower grades to immigrant students. Teachers participating in the study showed above-average implicit bias against immigrants on the basis of an implicit association test as compared to teachers in other European countries. But informing teachers of their results led them to increase the grades they assigned to immigrant students and, for maths teachers, to lower the likelihood of failing immigrant students.
- Similarly, in Turkey, researchers found that Syrian refugee students assigned to teachers with stronger ethnic biases (as measured by implicit bias tests) were more likely to be socially excluded, leading to socially segregated classrooms.
- Marginalised groups may have limited access to information about basic services. In Bulgaria, providing information to parents about the benefits of kindergarten was found to increase kindergarten attendance among Roma children.
Researchers measured the effects of offering free access to kindergarten, providing financial incentives to parents, and informing parents about the benefits of kindergarten.
While all three interventions led to increases in kindergarten participation, parental aspirations for their children, and household income, offering free access to kindergarten was the most cost-effective way to increase participation.
Initially the intervention led to a decrease in minority children’s learning, which was associated with a decrease in time children spent at home benefiting from activities with their parents and to a kindergarten curriculum not adapted to their needs. However, in a follow-up study three years after the intervention, the impact was shown to have reversed: kindergarten participation was associated with an improvement in minority children’s learning.
- Refugee children are particularly vulnerable to social exclusion and violence in schools. In Turkey, where ethnic composition in schools has changed abruptly in recent years, researchers measured the impact of a programme implemented by the Ministry of Education that aims to build cohesion between different groups.
An extracurricular programme on perspective-taking—the ability to consider another person’s viewpoint—was delivered in classrooms to Syrian refugee students and their Turkish classmates. Researchers then tested whether participation in the programme could reduce anti-social behaviours, such as bullying, violence, and social exclusion. While aiming to foster social cohesion in schools serving both Turkish and Syrian refugee students, the programme never explicitly mentioned ethnicity.
Both refugee and host students who were part of the programme became less violent and showed higher levels of trust, reciprocity, and altruism towards their peers, mainly due to improvements in their perspective-taking and ability to control their impulsiveness.
The programme also helped refugee children improve their host language skills.
In Germany, researchers conducted several randomised evaluations2 of a low-intensity programme aimed at fostering informal learning and psycho-social development by providing primary school children with a high-skilled volunteer mentor for one year.
Providing a mentor increased children’s prosocial behaviour (i.e., behaviour intended to benefit others), helping close the gap between students from different socioeconomic backgrounds.
The programme also improved educational outcomes among children from low socioeconomic backgrounds, who received better teacher assessments and were more likely to enrol in more advanced secondary school tracks.
Four years after the intervention, children who had a mentor were more honest, as measured through a game involving predicting and reporting the outcome of rolling a dice, particularly for children whose parents have a less warm parenting style or are less trusting.
In Italy, in response to potential learning losses following the closure of schools at the outbreak of the Covid-19 pandemic, researchers evaluated the impact of an online tutoring programe on the academic outcomes of middle school students (grades 6 to 8) from disadvantaged backgrounds. The programme was delivered by volunteer university students for free and consisted of help with homework in three subjects: Math, Italian, English.
The researchers found that the programme increased academic performance, especially for students from lower socioeconomic backgrounds, and improved psychological well-being, particularly for immigrant children.
1Sacerdote, Bruce. 2011. “Peer effects in education: How might they work, how big are they and how much do we know thus far?” In Handbook of the Economics of Education, ed. Eric Hanushek, Stephen Machin, and Ludger Woessmannno, 3:249–277. Elsevier. doi: https://EconPapers.repec.org/RePEc:eee:educhp:3-04.
2Kosse, Fabian, Thomas Deckers, Pia Pinger, Hannah Schildberg-Hörisch, and Armin Falk. 2020. "The formation of prosociality: causal evidence on the role of social environment." Journal of Political Economy 128, no. 2: 434-467.
Falk, Armin; Kosse, Fabian; Pinger, Pia. 2020. "Mentoring and Schooling Decisions: Causal Evidence." IZA Discussion Papers, No. 13387, Institute of Labor Economics (IZA), Bonn.
Falk, Armin, Fabian Kosse, Hannah Schildberg-Hörisch, and Florian Zimmermann. 2020. "Self-assessment: The role of the social environment." CESifo Working Paper Series 8308, CESifo.
Abeler, Johannes, Armin Falk, and Fabian Kosse. 2021. "Malleability of preferences for honesty." CEPR Discussion Paper No. DP16164.
Shaping minds to promote learning
Researchers are increasingly exploring the role of non-cognitive skills—such as patience, self-control, and perseverance—in driving successful performance at school. Recent work has shown that these skills are indeed malleable in children and adolescents, and new evaluations are helping us understand how development of non-cognitive skills may help children perform better in certain learning tasks, set higher aspirations and exert more effort in learning. These interventions tend to have generally small, yet positive impacts, often at low costs.
- A broad review of experimental evidence in education technology, including studies in the Netherlands and in Romania, found that interventions focused on promoting a “growth mindset”—the belief that intelligence and ability can be improved with hard work—tend to work only for specific groups of students and have small, although meaningful, effects.
- Research has shown that students with a “fixed mindset”, believing that intelligence and talents are fixed and unchangeable traits, tend to avoid academic challenges and lack resilience. In Norway, a low-cost information campaign about the brain’s potential to change and grow was found to improve high-school students’ perseverance and, in some cases, their academic performance.
Researchers conducted a computer-based intervention to measure the impact of a programme designed to encourage a “growth mindset” among students, as well as their subsequent effort in school.
After being exposed to the programme, students were more likely to believe intellectual ability is malleable, to seek out challenges, and to display improved academic performance three weeks later.
The intervention led to short-term improvements in academic performance among lower-performing students and among those who began the session with a fixed mindset and thus had limited confidence in their ability to learn. These findings suggest that students’ perseverance can be shaped by their beliefs in their ability to learn.
- In France, a light-touch intervention to encourage a growth mindset among middle school students from disadvantaged backgrounds helped girls improve their academic performance and reduced truancy, improved students’ attitudes in class, and helped students be more optimistic.
Researchers evaluated the impact of a low-intensity series of classroom discussions (three one-hour sessions per year for four years) on students’ beliefs and perceptions of their abilities to succeed in life.
The program’s effect was concentrated among girls; impacts on boys’ performance were insignificant.
- In Turkey, researchers have partnered with the Ministry of Education to develop a series of randomised evaluations examining the malleability of non-cognitive skills.
- They found that an educational programme focused on teaching patience to students in primary school led them to make more patient choices and receive better behaviour grades.
- A separate study evaluated the impact of introducing fourth-grade students to a curriculum designed to foster increased effort and more ambitious goal-setting (or “grit”) among students. Teachers highlighted the role of effort and the plasticity of the human brain, while underscoring the importance of setting goals and responding constructively to failure.
Students exposed to the curriculum had higher scores in maths and were more likely to engage in challenging tasks, persevere after negative feedback, set goals, and engage in skill-building activities.
The programme also helped close the gender gap in student competitiveness by increasing girls’ willingness to compete.
Based on these results, researchers are now working with the Ministry to scale up the programme.
Increasing access to information on schooling to improve student learning
Providing information about schooling and encouraging parents to be more involved in their children’s education can be a highly effective, low-cost way to help students and their parents set more realistic expectations. However, the content of the information and student characteristics such as academic standing may be an important determinant of effectiveness for these programmes.
- In France, a series of evaluations of structured group meetings between parents and school directors increased parents’ involvement in their children’s learning, improved student behaviour, reduced absenteeism and dropout rates, and helped students make choices that reflected more realistic educational expectations.
- Two interventions for parents of sixth-grade students helped them become more involved in their children’s learning both in school and at home, in turn reducing student absenteeism and disciplinary sanctions—even for their children’s classmates whose parents did not attend the meetings.2,3
- In a follow-up study, a similar programme for parents of ninth-grade students reduced high school dropout and grade repetition among low-achieving students mainly by helping parents and their children make more realistic educational choices and apply to tracks that matched their academic performance.
- Based on the results of these evaluations, in 2010 the education ministry made the programme available to all interested public schools in the country. (See more on the scaling up of this programme.)
- A lack of information regarding more selective high-school tracks (or pathways to higher-ranked universities) may curtail students’ educational ambitions.
- In France, high school students were assigned to a programme of academic tutoring and mentoring delivered by students of the prestigious École Normale Supérieure to evaluate whether exposure to successful peers paired with intensive support could change their university choices.
- The impacts of the programme differed among higher- and lower-performing students. The former saw improved academic performance and elected more selective university education tracks. Lower-performing students performed worse and were less likely to enter a selective track. Due to the program’s intensity, participation may have come at the expense of time spent on homework, which may have hurt lower-performing students more.
- An academic tutoring and career counseling programme for immigrants in Italy also had differential impacts depending on students’ previous academic records. For immigrant students whose career choices were not aligned with their abilities, the programme helped them bring their aspirations in line with their academic achievements.
- However, providing students with performance feedback may negatively affect their academic performance, particularly when students underestimate their standing relative to peers. In Spain, informing university students about their academic performance relative to that of their peers led to an increase in student satisfaction but also to a short-term decrease in academic performance which was concentrated among students who had perceived their ranking lower than it was in fact. This decrease faded over time—likely due to the fact that, with time, the amount of information provided decreased and comparison group students tended to gain more accurate information about their performance.
- These results are consistent with previous research suggesting that differences in prior beliefs regarding their own and others’ ability play a key role in students’ response to feedback.4
- Based on this evaluation, the university (University Carlos III) decided to discontinue the programme due to its negative impact on student performance.
- In Romania, researchers examined whether providing parents with information on the differing educational outcomes of students attending local schools affected their high-school choices for their children. Parents submit a list of top school and track choices, and the government allocates students to schools using an algorithm that uses track preference rankings in the order of students’ scores. This information improved the accuracy of households’ beliefs regarding school quality, leading low-achieving students who failed to gain admission to their top choices to enrol in higher-value added schools.
2Abdul Latif Jameel Poverty Action Lab (J-PAL). 2011. “Getting Parents Involved.” J-PAL Policy Briefcase. Last modified February 2011. https://www.povertyactionlab.org/publication/getting-parents-involved.
3Abdul Latif Jameel Poverty Action Lab (J-PAL). 2011. “School Communication Strategies and School Outcomes in France.” J-PAL Evaluation Summary. https://www.povertyactionlab.org/evaluation/school-communication-strategies-and-school-outcomes-france.
4Kuhnen, Camelia N. and Tymuła Agnieszka. 2012. “Feedback, Self-Esteem, and Performance in Organizations.” Management Science 58, no. 1 (August): 94–113. doi: https://doi.org/10.1287/mnsc.1110.1379.
Targeted remedial education to help students catch up
Extensive research from other regions, including the United States and India, has shown how tailoring instruction to students’ learning levels—regardless of grade level—can be a powerful tool for helping narrow the attainment gap for students who have fallen behind. New research is evaluating different models for offering intensive learning support at different schooling levels.
- In France, one recent evaluation explored whether offering access to a high-quality boarding school to secondary-level students from disadvantaged backgrounds could shift their aspirations and lead to increased attendance and academic performance.
- Enrolment at the Sourdun boarding school improved students’ maths skills, but only for initially high-performing students, and had no impact on students’ language skills in French. This is consistent with other findings that students’ maths aptitude continues to develop during adolescence, while language abilities might be set during childhood.
Students experienced reductions in an overall measure of their well-being in their first year, as they faced separation from friends and family and a more demanding academic environment. However, they adjusted to their new environment by the program’s second year.
The boarding school also shifted students’ schooling choices over the longer term, leading them to opt for more selective baccalaureate tracks at the end of high school.
- Another study in France focused on early primary years, as children who face difficulties in reading and writing from the start of their education may fall further behind their peers in subsequent years. A recreational after-school tutoring programme consisting of reading stories and playing word games helped low-achieving first grade students develop an increased taste for reading and academic subjects but had no impact on their reading or maths skills, compared to the standard remedial education programmes offered in other public schools.
- A programme of intensive after-school academic tutoring was detrimental for weaker students. Participating in the time-intensive programme appears to have taken away from time available for homework, which may have harmed weaker students.
In Denmark, researchers evaluated the impact of a two-week, school-based intensive learning camp for low-performing students who were deemed ‘not ready’ for further education at the end of compulsory schooling (age 15-16). Researchers measured two versions of the camp: one focused on maths and Danish and one where 30 percent of time was dedicated to developing students’ non-cognitive skills.
The programme led to small improvements in academic outcomes 5 weeks later but these did not persist after one year.
The programme had no impact on participants’ non-cognitive skills. Researchers suggest that the reason why this programme may have failed to generate such impacts where others have succeeded (see “Shaping Minds”) could be linked to the older age of the students, and that non-cognitive skills might be less malleable among this age group.
Education technology programmes to improve student learning
- In Russia, in an evaluation involving approximately 6,000 third-grade students, researchers measured the impact of a computer-assisted learning (CAL) programme as a substitute for traditional learning.
- They found that when offered for 45 minutes, the programme had a positive effect on students’ maths and Russian language test scores. Doubling the amount of time students spent using the programme did not double test scores and reduced student motivation. The effect might have diminished as the time spent on CAL increased beyond a level because of waning student interest, increased fatigue, or distractions from using the computer for entertainment.
Environment, Energy, and Climate Change
A body of evidence from North America points to the potential power of nudges for shifting individual consumption habits.
- In France, researchers partnered with the local authorities to measure the impact of a programme that provided households with sensors to monitor indoor air pollution and information on the negative effects of wood heating on health and the environment.
- Both types of information led to an increase in participants’ awareness of the negative effects of wood burning and in their intention to reduce the use of wood burning as a source of heating.
- While providing general information on the effects of wood heating had no impact on wood heating, and thus on indoor air pollution, participants who received personalised information about their household’s levels of indoor air pollution saw a 20 percent reduction in PM2.5 levels.
In Europe, we look forward to generating more policy-relevant evidence on programmes and policies related to climate change, environment, and energy, with a particular focus on climate action. As part of this work, we published a blog post on the potential of rigorous evaluations to help European policymakers tackle climate change.
Research by J-PAL affiliates has evaluated the impact of financial services and products in helping households smooth consumption, make investments, and manage risk. Many of these studies have drawn on interventions evaluated in low- and middle-income countries elsewhere and sought to test their applicability in Europe. A separate stream of research has explored how changes to the ways that charitable organisations raise money can lead to increased donations.
Household finance and consumption
Cash transfers, microcredit, and targeted remittances are financial management tools that can be useful in helping households face unexpected expenses and take advantage of opportunities.
Despite its potential to expand access to credit for low-income borrowers, traditional microcredit has not had a transformative impact on borrowers’ income and long-term consumption on average.
- A body of evidence from low and middle-income countries has shown that microcredit improved financial management for some households but did not have a transformative impact on household income or consumption on average. In an evaluation in Bosnia and Herzegovina, researchers partnered with a large microfinance institution to investigate the effect of providing access to microcredit to applicants who were slightly under-qualified for traditional loans. Microloans helped increase business ownership and self-employment but did not improve profits or incomes in the short run.
Researchers found that the amounts loaned were insufficient for households to finance their investments, leading households to reduce either their savings, when available, or their food consumption, to free up additional resources.
The loans also led to a decrease in school participation, as young adult children of microloan recipients spent more time working for the family business.
Remittances are one of the largest types of international financial flows to low- and middle-income countries and have been shown to improve educational outcomes of migrant workers’ relatives, as migrants tend to spend a large part of their income on the education of relatives back home.
- In a study in Italy, researchers found that offering migrant workers the option to label their remittances for educational purposes when sending them increased the amount of money transferred to their home countries by 15 percent. Migrants were also overall more likely to send money home.
- These results are consistent with other evidence5 6 on the way saving and spending behaviour can change when money is labelled for certain purposes.
Targeted cash transfers are another tool for reducing poverty and influencing household spending that have been evaluated in many contexts around the world. This global evidence suggests that the impact of these payments may depend in part on who in the household receives these payments.
- In one study in North Macedonia, targeting conditional cash transfers to mothers, rather than male household heads, enhanced women’s decision-making power and increased household spending on food.
5Karlan, Dean and Leigh Linden. “Loose Knots: Strong versus Weak Commitments to Save for Education in Uganda.” Yale University Economic Growth Center Discussion Paper No. 1037, Yale Economics Department Working Paper No. 12, April 2016.
6Benhassine, Najy, Florencia Devoto, Esther Duflo, Pascaline Dupas, and Victor Pouliquen. 2015. “Turning a Shove into a Nudge? A ‘Labeled Cash Transfer’ for Education.” American Economic Journal: Economic Policy 7, no. 3 (August): 86–125. doi: http://dx.doi.org/10.1257/pol.20130225.
Consumer protection remains central to financial inclusion efforts to ensure consumers can select and use the most appropriate financial products. However open questions remain on how consumers make financial decisions and how best to protect consumers.
- In Turkey, researchers evaluated the impact of informational reminders and price discounts on bank clients’ overdraft usage. Messages on the availability of overdraft protection services increased usage. However, offering a discount on overdraft fees actually reduced overdraft usage, possibly by reminding clients of the costs associated with overdrafts.
- In the UK, researchers studied the effects of increasing the credit card interest rate among low-income, financially risky borrowers. The rate increase reduced demand among borrowers who appeared more financially stable but on average borrowers continued using the credit card and incurred higher interest rate charges.
- Also in the UK, researchers evaluated the effectiveness of various designs for disclosing information on the interest rates of comparable savings products to support consumers’ financial decisions in order to understand their impact on consumer behaviour. Although most disclosure designs increased the number of consumers that switched to higher-yielding savings accounts, the effects were relatively modest.
- Follow-up surveys with a subset of consumers revealed that many often did not remember the disclosure information or remembered it incorrectly, thus suggesting that a lack of attention might be driving the low impact.
Encouraging charitable donations
Charitable organisations often rely on private donors to support activities that may advance the public good. How can choices charities make about how they ask for donations impact the amount of funds they are able to raise?
- In a series of evaluations in Germany, researchers tested the potential of different communication strategies to increase charitable donations. They found that lowering the effort required for making donations, by sending a pre-filled bank transfer form in letters to potential donors, increased the response rate. When the letters offered a suggested donation amount, this yielded fewer donations but increased the average amount given. When the letter included information on the existence of a lead donor, this almost doubled the size of the average donation.
J-PAL affiliated researchers in Europe have evaluated a range of interventions designed to contribute to women’s empowerment, including through efforts to improve soft skills to reduce gender gaps among school children, provide cash transfers to increase women’s decision-making power, and promote gender equality in hiring.
- In Turkey, researchers found that an innovative education intervention focused on improving elementary students’ grit helped reduce the gender gap in willingness to compete by encouraging girls who were likely to succeed to compete, likely because the grit curriculum made girls more optimistic about their own performance. (For information on other effects of the programme, please read the education section above.)
- In a similar study in Germany, an intensive, year-long mentoring programme exposed girls from lower socioeconomic backgrounds to female role models, building on previous evidence showing that role models are influential on young women’s education choices. The programme helped narrow the gender gap in both competitiveness and in earnings expectations by encouraging girls to compete.
- These results suggest that beliefs about earnings are malleable at a young age and that exposing children to female role models can reduce the gender gap in children’s earnings expectations.
- In France, an intensive tutoring programme in high schools had no impact on students on average. However, among students with better academic records, tutoring led more girls to enrol in scientific preparatory courses and more boys to enrol in literary preparatory courses, suggesting that the programme helped reduce gender-based stereotypes that influence schooling choices.
- In North Macedonia, researchers found that providing conditional cash transfers to women rather than male household heads, changed household spending on food by increasing women’s decision-making power. Researchers used a novel approach to measure empowerment by asking women how much money women were willing to give up in order to have control over it (the more women were willing to forego, the lower their decision-making power in the household).
- These results are in line with other evidence, mostly from Latin America, suggesting that targeting conditional cash transfers to women can increase women’s decision-making power in the household. For more lessons on supporting women’s agency—inside and outside the household—please see J-PAL’s new review.
- In Spain, the government passed a law in 2007 to ensure greater female participation in top positions and achieve gender parity in selection committees in state administration, public organisations, and firms. Researchers took advantage of the random allocation of candidates to selection committees to measure the impact of committees’ gender composition on women’s employment outcomes. They found that female candidates were less likely to succeed in the public examinations when they were evaluated by a mixed committee, compared to all-male committees. Male candidates, on the contrary, were more likely to succeed when evaluated by mixed committees.
- Researchers find suggestive evidence that the presence of female evaluators may have influenced the voting behaviour of their male counterparts. Researchers believe the results are driven by evaluators’ own perceptions rather than by a difference in candidates’ performance. In a similar study in Germany, an intensive, year-long mentoring programme exposed girls from lower socioeconomic backgrounds to female role models, building on previous evidence showing that role models are influential on young women’s education choices.
European governments face an array of challenges related to labour markets, from a rising unemployment gap between persons with lower and higher levels of education, to large discrepancies in long-term unemployment rates across different European countries, to the rapidly changing nature of jobs.7 Many of these challenges are likely to grow in the wake of the Covid-19 crisis. To identify potential lessons to some of these challenges, J-PAL affiliates have conducted rigorous evaluations of programmes aiming to help jobseekers (re)enter the labour force, foster skills in entrepreneurship among youth, increase demand for labour, and reduce discrimination in hiring.
7Eurostat. 2020. “Unemployment statistics and beyond.” Eurostat Statistics Explained. Last modified June 25, 2020. https://ec.europa.eu/eurostat/statistics-explained/index.php/Unemployment_statistics_and_beyond#Unemployment_by_level_of_education.
Job search assistance: What do jobseekers need?
Job search assistance programmes have generally been shown to be effective in improving employment outcomes for jobseekers in high-income countries, but questions remain about how to best structure this assistance. By partnering with public employment agencies across Europe, researchers have sought to answer what forms of assistance—and how much—jobseekers need.
- When designing counselling programmes, particular attention should be given to programme intensity. Increasing programme intensity, by assigning fewer jobseekers to individual case workers or by increasing the frequency of counselling meetings, was shown to increase employment in several studies in France and Denmark. More research is necessary to determine the optimal intensity of such programmes.
- In France, in one study in partnership with the public employment services, an intensive counseling programme (assigning three times fewer jobseekers to a case worker and holding meetings on a weekly, as opposed to monthly, basis) for jobseekers at risk of long-term unemployment helped them find work sooner than the standard low-intensity counseling programme.
- In two other studies in France, intensive job counselling programmes were found to also reduce recipients’ dependence on welfare, and when targeted at youth who had dropped out of university, intensive counselling improved youth’s perceptions of their career prospects and increased the number of job interviews they received.
- In Denmark, findings from three studies showed that frequent individual meetings between people who have recently become unemployed and caseworkers significantly improved employment rates, with results persisting in the medium and long run.
- These findings are consistent with other evidence on the importance of establishing close interactions between jobseekers and their caseworkers and suggest that it may be profitable to establish these early in the unemployment spell.
- However, meetings with case workers may be less effective for participants who have been unemployed for longer periods of time. In Denmark, an evaluation of an intensified programme for such youth found that simply increasing the frequency of meetings with case workers did not increase employment. The programme actually reduced employment for youth with a lower level of educational attainment (as time in the programme may have reduced time in work or searching for a job).
- Lessons from a pair of evaluations (report in French) in France showed that high-performing youth offered personalised job search counseling were more likely to obtain an apprenticeship than non-participating youth. Researchers suggest the programme did not make participating youth more employable but rather increased the efficiency of their job search efforts.
- In Germany, researchers evaluated the effectiveness of informational brochures designed to give jobseekers information about job search strategies and the consequences of unemployment, while motivating them to actively look for a job. While the intervention had no measurable effects on the general population, it did improve the employment and earnings of jobseekers at risk of long-term unemployment, and production and distribution costs were less than €1 per brochure.
- In France, researchers evaluated the impact of an apprenticeship support programme in increasing the number of apprenticeship contracts obtained by job seekers and reducing the likelihood of interrupting such contracts ahead of their end date.
- The first stage of the intervention, which consisted of personalised job counselling, was effective in helping participants obtain more apprenticeships. However, a second phase failed to reduce the number of broken apprenticeship contracts.
Job search assistance: How to deliver most efficiently?
Delivery method is another important aspect to consider when designing job search assistance programmes. Are individual programmes more effective than group ones? Can private providers deliver these programmes more effectively than public actors?
- While some European governments have begun outsourcing employment programmes to private providers, findings from two evaluations suggested that private contractors were not more effective at delivering these services than public ones.
- In Denmark, researchers measured the impact of public and private employment service providers and found that both had similar impacts on labour market outcomes. However, private providers were less cost-effective and had lower client satisfaction compared to public providers.
- In France, an evaluation that compared intensive job-search counselling programmes provided by the public employment services and by private companies found that the public programme was more than twice as effective at helping jobseekers find a job within six months than the private programme. Poorer performance by private providers may have been driven in part by how payments to the firms were structured.
- Several European countries have introduced statistical targeting software to help target job assistance programmes to jobseekers who need them most.8 The impact of new technologies may be limited, however, when caseworkers are unwilling to adopt them.
- In Switzerland, a statistical targeting software that provided caseworkers with recommendations on which labour market programmes were more likely to help jobseekers find employment had no effect on caseworkers’ choices, as most did not access the prediction software or ignored its recommendations.
- Collective counselling programmes may offer a cost-effective alternative to individualised programmes, and provide the added benefit of peer support and information-sharing opportunities.
- In France, a collective job-search programme targeting youth from socio-economically disadvantaged backgrounds was found to be more effective in helping youth find a stable, full-time job than individual job counseling.
- Making job search assistance programmes available may not lead to much uptake by jobseekers; information barriers or a lack of awareness of potential benefits may act as obstacles to enrollment. Could providing incentives play an important role?
- In France, researchers measured the impact of a conditional monthly stipend to encourage youth participation in a job placement programme. The stipend increased programme participation but had no impact on participants’ job search efforts or employment rates, and in the short run it acted as a disincentive for work, reducing youth employment. The results were potentially due to the nature of the programme: (1) participants received the stipends automatically, without having to commit to finding better work, and (2) the stipend size declined as employment income increased.
- More research is needed to understand the impact of increasing stipend conditionality by tying its allocation to employment achievements and of allowing participants to accumulate stipends alongside their income.
8Countries include Germany, Sweden, Finland, Denmark, and the United Kingdom. For more information, please see Behncke, Stefanie, Markus Frölich and Michael Lechner. 2009. “Targeting Labour Market Programmes – Results from a Randomized Experiment.” Swiss Society of Economics and Statistics 2009, no. 145(3): 221–268. doi: http://www.sjes.ch/papers/2009-III-1.pdf.
Job search assistance: Risk of displacing other jobseekers
Evidence from two studies suggests that where available jobs are scarce, job placement programmes may have unintended displacement effects: The recipients of these services may obtain jobs at the expense of non-participants, leaving the latter potentially worse off than before the programme.
- In a large randomised evaluation of a counseling and placement programme with nearly 30,000 jobseekers in France, researchers found that, while counselling helped participants find work sooner, it led to the displacement of jobseekers who did not receive the programme. Rather than increasing the number of jobs available, the programme merely re-allocated jobs within the large number of jobseekers.
Targeting job creation through stimulating labour demand
Another policy strategy to increase employment may be to stimulate firms’ demand for labour through interventions that target firms’ recruitment costs. Efforts to trigger increased demand in markets where it is low may be an alternative strategy that is potentially more cost-effective and less likely to create negative displacement effects as some job search assistance programmes mentioned above.
- In an evaluation in France, researchers partnered with the state employment agency to measure the impact of subsidising recruitment costs on the number of vacancy postings and hiring across the job market. They randomly offered free recruitment services to over 8,000 small and medium-sized firms and found that access to these services helped firms increase the number of vacancies they posted and hire more permanent employees.
Evaluations of programmes designed to foster entrepreneurship in Europe have shown mixed results and have not generally led to an increase in business creation. In France, two seemingly similar entrepreneurship interventions targeting young people from lower socioeconomic backgrounds showed different impacts on youth employment:
- For one study, researchers partnered with a microcredit NGO to evaluate a package of coaching, guidance, and financial support to help young people create and sustain independent businesses. The program’s selection criteria meant that only the most promising—and thus the most likely to succeed, regardless of the support received—candidates participated. The programme did not increase entrepreneurship and led to a delay in business creation, and participating youth were more likely to be unemployed and earn lower revenues from their business 28 months after the start of the programme.
- In another evaluation, researchers measured an entrepreneurship programme run through the local services of the French state employment agency that consisted of self-directed vocational training aimed at developing the autonomy and decision-making skills of young people interested in starting their own company. Participants were ultimately less likely to start their own business, but benefited from a more stable employment situation and greater financial autonomy for participating youth, likely because of the way the programme was built to focus on improving youth’s intrinsic motivation rather than on creating a business.
- These results suggest that providing information and business training may be insufficient to encourage business creation, and that tackling other barriers, such as low self-confidence or a lack of entrepreneurial spirit, may be a more effective way to support unemployed youth.
Reducing hiring discrimination
Research suggests there are many instances of hiring discrimination against minority and underrepresented groups on the basis of factors such as race, ethnicity, immigration status, gender, religion, and sexual orientation. Randomised correspondence studies, in which researchers send out fictitious resumes to job openings to test responses, have found cases of discrimination in a number of European countries, including Czechia, France, Germany, Ireland, Italy, Sweden and the Netherlands.
One proposed policy response to address this kind of hiring discrimination has included removing identifying information from job applications, but the impact of such measures appears to depend on the characteristics of hiring firms, namely whether they were discriminating against minority groups in the first place.
- In France, an evaluation with the public employment agency found that making resumes anonymous reduced the likelihood that firms interview and hire minority candidates. Researchers found that firms that had volunteered to participate in the study were more likely to interview and hire minority candidates than those who declined. In this case, removing information on minority status prevented the firms from favourably considering minority applicants.
- Further research is necessary to analyse the impact of mandatory anonymisation to capture those firms that would not voluntarily anonymise their screening process.
Political Economy and Governance
Research by J-PAL affiliated researchers in Europe in the Political Economy and Governance sector has focused on evaluating measures to increase electoral participation across Europe as well as how simple tweaks to how governments collect taxes can increase compliance.
Increasing electoral participation
To identify potential mechanisms to improve voter participation in Europe, a number of studies have measured the impact of various partisan and nonpartisan information campaigns on citizens’ electoral participation, including voter registration, turnout, and election results.
In France, researchers evaluated the impact of organising home visits to facilitate voter registration by providing unregistered voters with information or assistance regarding the registration process. These visits increased both registration and engagement, suggesting that lowering registration costs can encourage participation. This was particularly true for groups who may be less engaged in elections (including immigrants and those who spoke a language other than French at home).
- In another study in France, providing information about upcoming elections encouraging people to vote during home visits increased voter turnout among immigrants, although these impacts faded one year later. Immigrants may have been more receptive to the campaign, as they had significantly less political information than non-immigrants. Evidence on the role of door-to-door information campaigns to increase voter turnout is more mixed.
- In Italy, a campaign that targeted inactive voters increased voter turnout and persuaded undecided voters, but only when canvassed by (impartial) paid volunteers and not by political candidates. In France, a voter consultation campaign to encourage people to vote for a specific candidate did not impact turnout but had a long-term impact on voters’ opinions and voting decisions.
- Current evidence from low- and middle-income country contexts also suggests that providing voters with information about candidates can impact who they vote for, generally increasing vote share for less corrupt, more qualified, and better performing candidates.
- In Turkey, researchers evaluated the impact of a programme delivering information via two randomised door-to-door campaigns to approximately a quarter million voters before a constitutional referendum intended to weaken constraints on the executive on voter behaviour and ideology. The campaigns were organised by members of the largest party opposing the referendum and carried out by party volunteers.
- The results showed that exposing voters to information from non-state-owned sources drove voters to adopt “extreme” positions with regards to the referendum, either against or in favour, and was associated with greater partisan polarisation in presidential, general, and local elections over the next two years.
- These results support the thesis that, in a setting where voters disagree on whether strengthening the state is a good policy (the topic of the referendum), reducing censorship can backfire and cause some voters to switch their party affiliation and support the incumbent.
The salience and strength of national identity (including as compared to individual regional identities) can play an important role in shaping the nature of individuals’ political participation.
In Spain, researchers estimated the impact of the temporary exposure of men performing compulsory military service to individuals from different regions. Before compulsory service was abolished in 2001, young Spanish men were assigned to different regions for military service at random.
They found that increased exposure to individuals from different regions led to improved sentiments and attitudes towards people from the region of service.
This effect was stronger among individuals who originated from regions with strong secessionist movements, for whom sustained interactions with people from different regions also led to an increase in national identity sentiments.
Increasing tax compliance
Despite being the main source of national revenue and of funding for public goods, tax is often imperfectly collected. To help countries increase tax compliance, a series of studies have tested the effect of shifting how their communications are framed and simplifying steps for compliance.
- In Latvia, researchers partnered with the Ministry of Finance to measure the impact of sending emails containing several types of behavioral messages on individual tax compliance and found that framing non-compliant behavior as a deliberate choice was the most effective way to increase tax declaration submission by the deadline.
- In a similar study in Poland, researchers tested the impact of sending tax enforcement letters with behavioural messages, such as highlighting social incentives such as public goods and social norms, emphasising sanctions for noncompliance or framing nonpayment as an intentional choice, to delinquent taxpayers. They found that behavioural messages significantly improved tax compliance relative to the status quo letter, with hard-tone messages being more effective than soft-tone ones.
- In Belgium, sending simplified letters and making the consequences of non-compliance—such as fines and tax increases and/or follow-up enforcement—explicitly led to higher rates of tax compliance, while morale messages promoting social norms had no impact and sometimes decreased compliance.
- A study in Germany also revealed that simplifying tax notification letters and highlighting the possibility of tax enforcement may increase taxpayers' compliance. At the same time, offering rewards and recognition for timely payment increased the probability of evading.
- Taken together, these results suggest that behavioral insights can be a low-cost and easy way of improving the effectiveness of tax collection efforts and act as a complement to tax reform, particularly in contexts where such reforms may be difficult to implement.
- In the UK, researchers varied the type of language used in 40,000 letters alerting people to an overpayment of benefits, framing a failure to pay back as either an active choice (an act of commission) or a lack of action (an omission). They found that reframing failure of repayment as an active choice generated over $1.3 million in additional revenues.
Cross-sectoral Evidence on Social Inclusion
Efforts to promote social inclusion can take many forms: from programmes to reduce obstacles to educational achievements for children, to labour market policies focused on improving the employment prospects of all, and to measures to better meet the health and housing needs of disadvantaged communities.
In launching the European Social Inclusion Initiative in 2019, J-PAL Europe reviewed a broad range of evidence generated from evaluations of different programmes and policies designed to help break the intergenerational transmission of poverty. A short four-page summary of the review's key insights and priority research questions can be found here.
To learn more about our work on social inclusion, see our European Social Inclusion Initiative page.