Evidence in Europe
To date, researchers in our network have conducted over 80 ongoing or completed randomized evaluations of social programs and policies across 20 European countries, with a particular focus on the education and labor market sectors.
Here we highlight a selection of insights generated by these studies over the past decade.
Research conducted by J-PAL affiliates in Education in Europe has focused on narrowing the achievement gap for students from poor or marginalized backgrounds, including through promoting access to high-quality education for all, helping children develop skills associated with improved learning, increasing access to information about educational opportunities for both children and parents, and providing targeted remedial education to help students catch up as needed.
Improving educational opportunities for children from disadvantaged backgrounds
Children from economically disadvantaged backgrounds or regions often have access to fewer resources—in and out of school—for academic success, and those from immigrant or minority backgrounds may face additional barriers. These factors can become obstacles to obtaining quality employment opportunities later in life.
A growing body of randomized evaluations has sought to understand what forms of targeted interventions can help better connect these students to school resources in an effort to improve learning outcomes and promote social mobility. A recent review of experimental evidence on tutoring programs from North America showed that tutoring generally led to large improvements in student learning, particularly for students in earlier grades and when conducted during school by teacher or paraprofessional tutors.
- In Italy, researchers partnered with the education ministry to evaluate whether a program offering tutoring and career counseling for high-performing immigrant students could narrow attainment gaps between native-born and immigrant students at the end of middle school. Grades 7 and 8 are an important inflection point in the Italian system, when students choose between academic, technical, or vocational tracks for secondary education.
- The Equality of Opportunity program provided career counseling and tutoring for immigrant children with high academic potential, in an attempt to better align their goals and aspirations with their ability.
- The program improved immigrant boys’ academic performance and led to higher enrollment rates in academic and technical high schools, eliminating the gap between immigrant and Italian-born students. No impact was found on girls’ achievement, as there was no education gap between them and native girls to begin with.
- Lower-performing immigrant students in schools where the program was run (who were ineligible for the program) also benefited from the program, while there was no impact on native students. These results are consistent with previous evidence suggesting that peer effects are stronger within groups with a similar background.1
- Another recent study in Italy examined whether teachers hold implicit biases that might lead them to award lower grades to immigrant students. Teachers participating in the study showed above-average implicit bias against immigrants on the basis of an implicit association test as compared to teachers in other European countries. But informing teachers of their results led them to increase the grades they assigned to immigrant students and, for math teachers, to lower the likelihood of failing immigrant students.
- Marginalized groups may have limited access to information about basic services. In Bulgaria, providing information to parents about the benefits of kindergarten was found to increase kindergarten attendance among Roma children.
Researchers measured the effects of offering free access to kindergarten, providing financial incentives to parents, and informing parents about the benefits of kindergarten.
While all three interventions led to increases in kindergarten participation, parental aspirations for their children, and household income, offering free access to kindergarten was the most cost-effective way to increase participation.
Initially the intervention led to a decrease in minority children’s learning, which was associated with a decrease in time children spent at home benefiting from activities with their parents and to a kindergarten curriculum not adapted to their needs. However, in a follow-up study three years after the intervention, the impact was shown to have reversed: kindergarten participation was associated with an improvement in minority children’s learning.
- Refugee children are particularly vulnerable to social exclusion and violence in schools. In Turkey, where ethnic composition in schools has changed abruptly in recent years, researchers measured the impact of a program implemented by the Ministry of Education that aims to build cohesion between different groups.
An extracurricular program on perspective-taking—the ability to consider another person’s viewpoint—was delivered in classrooms to Syrian refugee students and their Turkish classmates. Researchers then tested whether participation in the program could reduce anti-social behaviors, such as bullying, violence, and social exclusion. While aiming to foster social cohesion in schools serving both Turkish and Syrian refugee students, the program never explicitly mentioned ethnicity.
Both refugee and host students who were part of the program became less violent and showed higher levels of trust, reciprocity, and altruism towards their peers, mainly due to improvements in their perspective-taking and ability to control their impulsiveness.
The program also helped refugee children improve their host language skills.
1Sacerdote, Bruce. 2011. “Peer effects in education: How might they work, how big are they and how much do we know thus far?” In Handbook of the Economics of Education, ed. Eric Hanushek, Stephen Machin, and Ludger Woessmannno, 3:249–277. Elsevier. doi: https://EconPapers.repec.org/RePEc:eee:educhp:3-04.
Shaping minds to promote learning
Researchers are increasingly exploring the role of non-cognitive skills—such as patience, self-control, and perseverance—in driving successful performance at school. Recent work has shown that these skills are indeed malleable in children and adolescents, and new evaluations are helping us understand how development of non-cognitive skills may help children perform better in certain learning tasks, set higher aspirations, and exert more effort in learning. These interventions tend to have generally small, yet positive impacts, often at low costs.
- A broad review of experimental evidence in education technology, including studies in the Netherlands and in Romania, found that interventions focused on promoting a “growth mindset”—the belief that intelligence and ability can be improved with hard work—tend to work only for specific groups of students and have small, although meaningful, effects.
- Research has shown that students with a “fixed mindset”, believing that intelligence and talents are fixed and unchangeable traits, tend to avoid academic challenges and lack resilience. In Norway, a low-cost information campaign about the brain’s potential to change and grow was found to improve high-school students’ perseverance and, in some cases, their academic performance.
Researchers conducted a computer-based intervention to measure the impact of a program designed to encourage a “growth mindset” among students, as well as their subsequent effort in school.
After being exposed to the program, students were more likely to believe intellectual ability is malleable, to seek out challenges, and to display improved academic performance three weeks later.
The intervention led to short-term improvements in academic performance among lower-performing students and among those who began the session with a fixed mindset and thus had limited confidence in their ability to learn. These findings suggest that students’ perseverance can be shaped by their beliefs in their ability to learn.
- In France, a light-touch intervention to encourage a growth mindset among middle school students from disadvantaged backgrounds helped girls improve their academic performance and reduced truancy, improved students’ attitudes in class, and helped students be more optimistic.
Researchers evaluated the impact of a low-intensity series of classroom discussions (three one-hour sessions per year for four years) on students’ beliefs and perceptions of their abilities to succeed in life.
The program’s effect was concentrated among girls; impacts on boys’ performance were insignificant.
- In Turkey, researchers have partnered with the Ministry of Education to develop a series of randomized evaluations examining the malleability of non-cognitive skills.
- They found that an educational program focused on teaching patience to students in primary school led them to make more patient choices and receive better behavior grades.
- A separate study evaluated the impact of introducing fourth-grade students to a curriculum designed to foster increased effort and more ambitious goal-setting (or “grit”) among students. Teachers highlighted the role of effort and the plasticity of the human brain, while underscoring the importance of setting goals and responding constructively to failure.
Students exposed to the curriculum had higher scores in math and were more likely to engage in challenging tasks, persevere after negative feedback, set goals, and engage in skill-building activities.
The program also helped close the gender gap in student competitiveness by increasing girls’ willingness to compete.
Based on these results, researchers are now working with the Ministry to scale up the program.
Increasing access to information on schooling to improve student learning
Providing information about schooling and encouraging parents to be more involved in their children’s education can be a highly effective, low-cost way to help students and their parents set more realistic expectations. However, the content of the information and student characteristics such as academic standing may be an important determinant of effectiveness for these programs.
- In France, a series of evaluations of structured group meetings between parents and school directors increased parents’ involvement in their children’s learning, improved student behavior, reduced absenteeism and dropout rates, and helped students make choices that reflected more realistic educational expectations.
- Two interventions for parents of sixth-grade students helped them become more involved in their children’s learning both in school and at home, in turn reducing student absenteeism and disciplinary sanctions—even for their children’s classmates whose parents did not attend the meetings.2,3
- In a follow-up study, a similar program for parents of ninth-grade students reduced high school dropout and grade repetition among low-achieving students mainly by helping parents and their children make more realistic educational choices and apply to tracks that matched their academic performance.
- Based on the results of these evaluations, in 2010 the education ministry made the program available to all interested public schools in the country. (See more on the scaling up of this program.)
- A lack of information regarding more selective high-school tracks (or pathways to higher-ranked universities) may curtail students’ educational ambitions.
- In France, high school students were assigned to a program of academic tutoring and mentoring delivered by students of the prestigious École Normale Supérieure to evaluate whether exposure to successful peers paired with intensive support could change their university choices.
- The impacts of the program differed among higher- and lower-performing students. The former saw improved academic performance and elected more selective university education tracks. Lower-performing students performed worse and were less likely to enter a selective track. Due to the program’s intensity, participation may have come at the expense of time spent on homework, which may have hurt lower-performing students more.
- An academic tutoring and career counseling program for immigrants in Italy also had differential impacts depending on students’ previous academic records. For immigrant students whose career choices were not aligned with their abilities, the program helped them bring their aspirations in line with their academic achievements.
- However, providing students with performance feedback may negatively affect their academic performance, particularly when students underestimate their standing relative to peers. In Spain, informing university students about their academic performance relative to that of their peers led to an increase in student satisfaction but also to a short-term decrease in academic performance which was concentrated among students who had perceived their ranking lower than it was in fact. This decrease faded over time—likely due to the fact that, with time, the amount of information provided decreased and comparison group students tended to gain more accurate information about their performance.
- These results are consistent with previous research suggesting that differences in prior beliefs regarding their own and others’ ability play a key role in students’ response to feedback.4
- Based on this evaluation, the university (University Carlos III) decided to discontinue the program due to its negative impact on student performance.
2Abdul Latif Jameel Poverty Action Lab (J-PAL). 2011. “Getting Parents Involved.” J-PAL Policy Briefcase. Last modified February 2011. https://www.povertyactionlab.org/publication/getting-parents-involved.
3Abdul Latif Jameel Poverty Action Lab (J-PAL). 2011. “School Communication Strategies and School Outcomes in France.” J-PAL Evaluation Summary. https://www.povertyactionlab.org/evaluation/school-communication-strategies-and-school-outcomes-france.
4Kuhnen, Camelia N. and Tymuła Agnieszka. 2012. “Feedback, Self-Esteem, and Performance in Organizations.” Management Science 58, no. 1 (August): 94–113. doi: https://doi.org/10.1287/mnsc.1110.1379.
Targeted remedial education to help students catch up
Extensive research from other regions, including the United States and India, has shown how tailoring instruction to students’ learning levels—regardless of grade level—can be a powerful tool for helping narrow the attainment gap for students who have fallen behind. New research in France is evaluating different models for offering intensive learning support at different schooling levels.
- One recent evaluation explored whether offering access to a high-quality boarding school to secondary-level students from disadvantaged backgrounds could shift their aspirations and lead to increased attendance and academic performance.
- Enrollment at the Sourdun boarding school improved students’ math skills, but only for initially high-performing students, and had no impact on students’ language skills in French. This is consistent with other findings that students’ math aptitude continues to develop during adolescence, while language abilities might be set during childhood.
Students experienced reductions in an overall measure of their well-being in their first year, as they faced separation from friends and family and a more demanding academic environment. However, they adjusted to their new environment by the program’s second year.
The boarding school also shifted students’ schooling choices over the longer term, leading them to opt for more selective baccalaureate tracks at the end of high school.
- Another study focused on early primary years, as children who face difficulties in reading and writing from the start of their education may fall further behind their peers in subsequent years. A recreational after-school tutoring program in France consisting of personalized tutoring helped low-achieving first grade students develop an increased taste for reading and academic subjects but had no impact on their reading or math skills, compared to the standard remedial education programs offered in other public schools.
- A program of intensive after-school academic tutoring was detrimental for weaker students. Participating in the time-intensive program appears to have taken away from time available for homework, which may have harmed weaker students.
In Denmark, researchers evaluated the impact of a two-week, school-based intensive learning camp for low-performing students who were deemed ‘not ready’ for further education at the end of compulsory schooling (age 15-16). Researchers measured two versions of the camp: one focused on math and Danish and one where 30 percent of time was dedicated to developing students’ non-cognitive skills.
The program led to small improvements in academic outcomes 5 weeks later but these did not persist after one year.
The program had no impact on participants’ non-cognitive skills. Researchers suggest that the reason why this program may have failed to generate such impacts where others have succeeded (see “Shaping Minds”) could be linked to the older age of the students, and that non-cognitive skills might be less malleable among this age group.
Environment, Energy, and Climate Change
A body of evidence from North America points to the potential power of nudges for shifting individual consumption habits.
- In France, researchers partnered with the local authorities to measure the impact of a program that provided households with sensors to monitor indoor air pollution and information on the negative effects of wood heating on health and the environment.
- Both types of information led to an increase in participants’ awareness of the negative effects of wood burning and in their intention to reduce the use of wood burning as a source of heating.
- While providing general information on the effects of wood heating had no impact on wood heating, and thus on indoor air pollution, participants who received personalized information about their household’s levels of indoor air pollution saw a 20 percent reduction in PM2.5 levels.
In Europe, we look forward to generating more policy-relevant evidence on programs and policies related to climate change, environment, and energy, with a particular focus on climate action. As part of this work, we published a blog post on the potential of rigorous evaluations to help European policymakers tackle climate change.
Research by J-PAL affiliates has evaluated the impact of financial services and products in helping households smooth consumption, make investments, and manage risk. Many of these studies have drawn on interventions evaluated in low- and middle-income countries elsewhere and sought to test their applicability in Europe. A separate stream of research has explored how changes to the ways that charitable organizations raise money can lead to increased donations.
Household finance and consumption
Cash transfers, microcredit, and targeted remittances are financial management tools that can be useful in helping households face unexpected expenses and take advantage of opportunities.
Despite its potential to expand access to credit for low-income borrowers, traditional microcredit has not had a transformative impact on borrowers’ income and long-term consumption on average.
- A body of evidence from low and middle-income countries has shown that microcredit improved financial management for some households but did not have a transformative impact on household income or consumption on average. In an evaluation in Bosnia and Herzegovina, researchers partnered with a large microfinance institution to investigate the effect of providing access to microcredit to applicants who were slightly under-qualified for traditional loans. Microloans helped increase business ownership and self-employment but did not improve profits or incomes in the short run.
Researchers found that the amounts loaned were insufficient for households to finance their investments, leading households to reduce either their savings, when available, or their food consumption, to free up additional resources.
The loans also led to a decrease in school participation, as young adult children of microloan recipients spent more time working for the family business.
Remittances are one of the largest types of international financial flows to low- and middle-income countries and have been shown to improve educational outcomes of migrant workers’ relatives, as migrants tend to spend a large part of their income on the education of relatives back home.
- In a study in Italy, researchers found that offering migrant workers the option to label their remittances for educational purposes when sending them increased the amount of money transferred to their home countries by 15 percent. Migrants were also overall more likely to send money home.
- These results are consistent with other evidence5 6 on the way saving and spending behavior can change when money is labeled for certain purposes.
Targeted cash transfers are another tool for reducing poverty and influencing household spending that have been evaluated in many contexts around the world. This global evidence suggests that the impact of these payments may depend in part on who in the household receives these payments.
- In one study in North Macedonia, targeting conditional cash transfers to mothers, rather than male household heads, enhanced women’s decision-making power and increased household spending on food.
5Karlan, Dean and Leigh Linden. “Loose Knots: Strong versus Weak Commitments to Save for Education in Uganda.” Yale University Economic Growth Center Discussion Paper No. 1037, Yale Economics Department Working Paper No. 12, April 2016.
6Benhassine, Najy, Florencia Devoto, Esther Duflo, Pascaline Dupas, and Victor Pouliquen. 2015. “Turning a Shove into a Nudge? A ‘Labeled Cash Transfer’ for Education.” American Economic Journal: Economic Policy 7, no. 3 (August): 86–125. doi: http://dx.doi.org/10.1257/pol.20130225.
Consumer protection remains central to financial inclusion efforts to ensure consumers can select and use the most appropriate financial products. However open questions remain on how consumers make financial decisions and how best to protect consumers.
- In Turkey, researchers evaluated the impact of informational reminders and price discounts on bank clients’ overdraft usage. Messages on the availability of overdraft protection services increased usage. However, offering a discount on overdraft fees actually reduced overdraft usage, possibly by reminding clients of the costs associated with overdrafts.
- In the United Kingdom, researchers studied the effects of increasing the credit card interest rate among low-income, financially risky borrowers. The rate increase reduced demand among borrowers who appeared more financially stable but on average borrowers continued using the credit card and incurred higher interest rate charges.
Encouraging charitable donations
Charitable organizations often rely on private donors to support activities that may advance the public good. How can choices charities make about how they ask for donations impact the amount of funds they are able to raise?
- In a series of evaluations in Germany, researchers tested the potential of different communication strategies to increase charitable donations. They found that lowering the effort required for making donations, by sending a pre-filled bank transfer form in letters to potential donors, increased the response rate. When the letters offered a suggested donation amount, this yielded fewer donations but increased the average amount given. When the letter included information on the existence of a lead donor, this almost doubled the size of the average donation.
J-PAL affiliated researchers in Europe have evaluated a range of interventions designed to contribute to women’s empowerment, including through efforts to improve soft skills to reduce gender gaps among school children, provide cash transfers to increase women’s decision-making power, and promote gender equality in hiring.
- In Turkey, researchers found that an innovative education intervention focused on improving elementary students’ grit helped reduce the gender gap in willingness to compete by encouraging girls who were likely to succeed to compete, likely because the grit curriculum made girls more optimistic about their own performance. (For information on other effects of the program, please read the education section above.)
- In France, an intensive tutoring program in high schools had no impact on students on average. However, among students with better academic records, tutoring led more girls to enroll in scientific preparatory courses and more boys to enroll in literary preparatory courses, suggesting that the program helped reduce gender-based stereotypes that influence schooling choices.
- In North Macedonia, researchers found that providing conditional cash transfers to women rather than male household heads, changed household spending on food by increasing women’s decision-making power. Researchers used a novel approach to measure empowerment by asking women how much money women were willing to give up in order to have control over it (the more women were willing to forego, the lower their decision-making power in the household).
- These results are in line with other evidence, mostly from Latin America, suggesting that targeting conditional cash transfers to women can increase women’s decision-making power in the household. For more lessons on supporting women’s agency—inside and outside the household—please see J-PAL’s new review.
- In Spain, the government passed a law in 2007 to ensure greater female participation in top positions and achieve gender parity in selection committees in state administration, public organizations, and firms. Researchers took advantage of the random allocation of candidates to selection committees to measure the impact of committees’ gender composition on women’s employment outcomes. They found that female candidates were less likely to succeed in the public examinations when they were evaluated by a mixed committee, compared to all-male committees. Male candidates, on the contrary, were more likely to succeed when evaluated by mixed committees.
- Researchers find suggestive evidence that the presence of female evaluators may have influenced the voting behavior of their male counterparts. Researchers believe the results are driven by evaluators’ own perceptions rather than by a difference in candidates’ performance.
European governments face an array of challenges related to labor markets, from a rising unemployment gap between persons with lower and higher levels of education, to large discrepancies in long-term unemployment rates across different European countries, to the rapidly changing nature of jobs.7 Many of these challenges are likely to grow in the wake of the Covid-19 crisis. To identify potential lessons to some of these challenges, J-PAL affiliates have conducted rigorous evaluations of programs aiming to help jobseekers (re)enter the labor force, foster skills in entrepreneurship among youth, increase demand for labor, and reduce discrimination in hiring.
7Eurostat. 2020. “Unemployment statistics and beyond.” Eurostat Statistics Explained. Last modified June 25, 2020. https://ec.europa.eu/eurostat/statistics-explained/index.php/Unemployment_statistics_and_beyond#Unemployment_by_level_of_education.
Job search assistance: What do jobseekers need?
Job search assistance programs have generally been shown to be effective in improving employment outcomes for jobseekers in high-income countries, but questions remain about how to best structure this assistance. By partnering with public employment agencies across Europe, researchers have sought to answer what forms of assistance—and how much—jobseekers need.
- When designing counseling programs, particular attention should be given to program intensity. Increasing program intensity, by assigning fewer jobseekers to individual case workers or by increasing the frequency of counseling meetings, was shown to increase employment in several studies in France and Denmark. More research is necessary to determine the optimal intensity of such programs.
- In France, in one study in partnership with the public employment services, an intensive counseling program (assigning three times fewer jobseekers to a case worker and holding meetings on a weekly, as opposed to monthly, basis) for jobseekers at risk of long-term unemployment helped them find work sooner than the standard low-intensity counseling program.
- In two other studies in France, intensive job counseling programs were found to also reduce recipients’ dependence on welfare, and when targeted at youth who had dropped out of university, intensive counseling improved youth’s perceptions of their career prospects and increased the number of job interviews they received.
- In Denmark, findings from three studies showed that frequent individual meetings between people who have recently become unemployed and caseworkers significantly improved employment rates, with results persisting in the medium and long run.
- These findings are consistent with other evidence on the importance of establishing close interactions between jobseekers and their caseworkers and suggest that it may be profitable to establish these early in the unemployment spell.
- However, meetings with case workers may be less effective for participants who have been unemployed for longer periods of time. In Denmark, an evaluation of an intensified program for such youth found that simply increasing the frequency of meetings with case workers did not increase employment. The program actually reduced employment for youth with a lower level of educational attainment (as time in the program may have reduced time in work or searching for a job).
- Lessons from a pair of evaluations (report in French) in France showed that high-performing youth offered personalized job search counseling were more likely to obtain an apprenticeship than non-participating youth. Researchers suggest the program did not make participating youth more employable but rather increased the efficiency of their job search efforts.
- In Germany, researchers evaluated the effectiveness of informational brochures designed to give jobseekers information about job search strategies and the consequences of unemployment, while motivating them to actively look for a job. While the intervention had no measurable effects on the general population, it did improve the employment and earnings of jobseekers at risk of long-term unemployment, and production and distribution costs were less than €1 per brochure.
Job search assistance: How to deliver most efficiently?
Delivery method is another important aspect to consider when designing job search assistance programs. Are individual programs more effective than group ones? Can private providers deliver these programs more effectively than public actors?
- While some European governments have begun outsourcing employment programs to private providers, findings from two evaluations suggested that private contractors were not more effective at delivering these services than public ones.
- In Denmark, researchers measured the impact of public and private employment service providers and found that both had similar impacts on labor market outcomes. However, private providers were less cost-effective and had lower client satisfaction compared to public providers.
- In France, an evaluation that compared intensive job-search counseling programs provided by the public employment services and by private companies found that the public program was more than twice as effective at helping jobseekers find a job within six months than the private program. Poorer performance by private providers may have been driven in part by how payments to the firms were structured.
- Several European countries have introduced statistical targeting software to help target job assistance programs to jobseekers who need them most.8 The impact of new technologies may be limited, however, when caseworkers are unwilling to adopt them.
- In Switzerland, a statistical targeting software that provided caseworkers with recommendations on which labor market programs were more likely to help jobseekers find employment had no effect on caseworkers’ choices, as most did not access the prediction software or ignored its recommendations.
- Collective counseling programs may offer a cost-effective alternative to individualized programs, and provide the added benefit of peer support and information-sharing opportunities.
- In France, a collective job-search program targeting youth from socio-economically disadvantaged backgrounds was found to be more effective in helping youth find a stable, full-time job than individual job counseling.
- Making job search assistance programs available may not lead to much uptake by jobseekers; information barriers or a lack of awareness of potential benefits may act as obstacles to enrollment. Could providing incentives play an important role?
- In France, researchers measured the impact of a conditional monthly stipend to encourage youth participation in a job placement program. The stipend increased program participation but had no impact on participants’ job search efforts or employment rates, and in the short run it acted as a disincentive for work, reducing youth employment. The results were potentially due to the nature of the program: (1) participants received the stipends automatically, without having to commit to finding better work, and (2) the stipend size declined as employment income increased.
- More research is needed to understand the impact of increasing stipend conditionality by tying its allocation to employment achievements and of allowing participants to accumulate stipends alongside their income.
8Countries include Germany, Sweden, Finland, Denmark, and the United Kingdom. For more information, please see Behncke, Stefanie, Markus Frölich and Michael Lechner. 2009. “Targeting Labour Market Programmes – Results from a Randomized Experiment.” Swiss Society of Economics and Statistics 2009, no. 145(3): 221–268. doi: http://www.sjes.ch/papers/2009-III-1.pdf.
Job search assistance: Risk of displacing other jobseekers
Evidence from two studies suggests that where available jobs are scarce, job placement programs may have unintended displacement effects: The recipients of these services may obtain jobs at the expense of non-participants, leaving the latter potentially worse off than before the program.
- In a large randomized evaluation of a counseling and placement program with nearly 30,000 jobseekers in France, researchers found that, while counseling helped participants find work sooner, it led to the displacement of jobseekers who did not receive the program. Rather than increasing the number of jobs available, the program merely re-allocated jobs within the large number of jobseekers.
Targeting job creation through stimulating labor demand
Another policy strategy to increase employment may be to stimulate firms’ demand for labor through interventions that target firms’ recruitment costs. Efforts to trigger increased demand in markets where it is low may be an alternative strategy that is potentially more cost-effective and less likely to create negative displacement effects as some job search assistance programs mentioned above.
- In an evaluation in France, researchers partnered with the state employment agency to measure the impact of subsidizing recruitment costs on the number of vacancy postings and hiring across the job market. They randomly offered free recruitment services to over 8,000 small and medium-sized firms and found that access to these services helped firms increase the number of vacancies they posted and hire more permanent employees.
Evaluations of programs designed to foster entrepreneurship in Europe have shown mixed results and have not generally led to an increase in business creation. In France, two seemingly similar entrepreneurship interventions targeting young people from lower socioeconomic backgrounds showed different impacts on youth employment:
- For one study, researchers partnered with a microcredit NGO to evaluate a package of coaching, guidance, and financial support to help young people create and sustain independent businesses. The program’s selection criteria meant that only the most promising—and thus the most likely to succeed, regardless of the support received—candidates participated. The program did not increase entrepreneurship and led to a delay in business creation, and participating youth were more likely to be unemployed and earn lower revenues from their business 28 months after the start of the program.
- In another evaluation, researchers measured an entrepreneurship program run through the local services of the French state employment agency that consisted of self-directed vocational training aimed at developing the autonomy and decision-making skills of young people interested in starting their own company. Participants were ultimately less likely to start their own business, but benefited from a more stable employment situation and greater financial autonomy for participating youth, likely because of the way the program was built to focus on improving youth’s intrinsic motivation rather than on creating a business.
- These results suggest that providing information and business training may be insufficient to encourage business creation, and that tackling other barriers, such as low self-confidence or a lack of entrepreneurial spirit, may be a more effective way to support unemployed youth.
Reducing hiring discrimination
Research suggests there are many instances of hiring discrimination against minority and underrepresented groups on the basis of factors such as race, ethnicity, immigration status, gender, religion, and sexual orientation. Randomized correspondence studies, in which researchers send out fictitious resumes to job openings to test responses, have found cases of discrimination in a number of European countries, including Czechia, France, Germany, Ireland, Italy, Sweden, and the Netherlands.
One proposed policy response to address this kind of hiring discrimination has included removing identifying information from job applications, but the impact of such measures appears to depend on the characteristics of hiring firms, namely whether they were discriminating against minority groups in the first place.
- In France, an evaluation with the public employment agency found that making resumes anonymous reduced the likelihood that firms interview and hire minority candidates. Researchers found that firms that had volunteered to participate in the study were more likely to interview and hire minority candidates than those who declined. In this case, removing information on minority status prevented the firms from favorably considering minority applicants.
- Further research is necessary to analyze the impact of mandatory anonymization to capture those firms that would not voluntarily anonymize their screening process.
Political Economy and Governance
Research by J-PAL affiliated researchers in Europe in the Political Economy and Governance sector has focused on evaluating measures to increase electoral participation across Europe as well as how simple tweaks to how governments collect taxes can increase compliance.
Increasing electoral participation
To identify potential mechanisms to improve voter participation in Europe, a number of studies have measured the impact of various partisan and nonpartisan information campaigns on citizens’ electoral participation, including voter registration, turnout, and election results.
In France, researchers evaluated the impact of organizing home visits to facilitate voter registration by providing unregistered voters with information or assistance regarding the registration process. These visits increased both registration and engagement, suggesting that lowering registration costs can encourage participation. This was particularly true for groups who may be less engaged in elections (including immigrants and those who spoke a language other than French at home).
- In another study in France, providing information about upcoming elections encouraging people to vote during home visits increased voter turnout among immigrants, although these impacts faded one year later. Immigrants may have been more receptive to the campaign, as they had significantly less political information than non-immigrants. Evidence on the role of door-to-door information campaigns to increase voter turnout is more mixed.
- In Italy, a campaign that targeted inactive voters increased voter turnout and persuaded undecided voters, but only when canvassed by (impartial) paid volunteers and not by political candidates. In France, a voter consultation campaign to encourage people to vote for a specific candidate did not impact turnout but had a long-term impact on voters’ opinions and voting decisions.
- Current evidence from low- and middle-income country contexts also suggests that providing voters with information about candidates can impact who they vote for, generally increasing vote share for less corrupt, more qualified, and better performing candidates.
The salience and strength of national identity (including as compared to individual regional identities) can play an important role in shaping the nature of individuals’ political participation.
In Spain, researchers estimated the impact of the temporary exposure of men performing compulsory military service to individuals from different regions. Before compulsory service was abolished in 2001, young Spanish men were assigned to different regions for military service at random.
They found that increased exposure to individuals from different regions led to improved sentiments and attitudes towards people from the region of service.
This effect was stronger among individuals who originated from regions with strong secessionist movements, for whom sustained interactions with people from different regions also led to an increase in national identity sentiments.
Increasing tax compliance
Despite being the main source of national revenue and of funding for public goods, tax is often imperfectly collected. To help countries increase tax compliance, a series of studies have tested the effect of shifting how their communications are framed and simplifying steps for compliance.
- In Latvia, researchers partnered with the Ministry of Finance to measure the impact of sending emails containing several types of behavioral messages on individual tax compliance and found that framing non-compliant behavior as a deliberate choice was the most effective way to increase tax declaration submission by the deadline.
- In a similar study in Poland, researchers tested the impact of sending tax enforcement letters with behavioral messages, such as highlighting social incentives such as public goods and social norms, emphasizing sanctions for noncompliance or framing nonpayment as an intentional choice, to delinquent taxpayers. They found that behavioral messages significantly improved tax compliance relative to the status quo letter, with hard-tone messages being more effective than soft-tone ones.
- In Belgium, sending simplified letters and making the consequences of non-compliance—such as fines and tax increases and/or follow-up enforcement—explicit led to higher rates of tax compliance, while morale messages promoting social norms had no impact and sometimes decreased compliance.
- A study in Germany also revealed that simplifying tax notification letters and highlighting the possibility of tax enforcement may increase taxpayers' compliance. At the same time, offering rewards and recognition for timely payment increased the probability of evading.
- Taken together, these results suggest that behavioral insights can be a low-cost and easy way of improving the effectiveness of tax collection efforts and act as a complement to tax reform, particularly in contexts where such reforms may be difficult to implement.
Cross-sectoral Evidence on Social Inclusion
Efforts to promote social inclusion can take many forms: from programs to reduce obstacles to educational achievements for children, to labor market policies focused on improving the employment prospects of all, and to measures to better meet the health and housing needs of disadvantaged communities.
In launching the European Social Inclusion Initiative in 2019, J-PAL Europe reviewed a broad range of evidence generated from evaluations of different programs and policies designed to help break the intergenerational transmission of poverty. A short four-page summary of the review's key insights and priority research questions can be found here.
To learn more about our work on social inclusion, see our European Social Inclusion Initiative page.