J-PAL North America’s Education, Technology, and Opportunity Initiative funds randomized evaluations of education technologies in school districts and other organizations across the country. Read more about our previously and currently funded projects below.
Big Word Club Evaluation
Researchers: Philip Oreopoulos, Ariel Kalil, Susan Mayer
Location: United States
This project aims to evaluate the effectiveness of a classroom-based program intended to increase the vocabulary of preschool and primary school children. The program, called the Big Word Club (BWC), consists of videos, books and activities intended to help children learn one new word per day over a school year. The intent of the program is not only to teach specific new words, but also to increase children’s interest in words and literacy in general and thereby improve school success.
Teaching Cognitive Behavioral Therapy through Video Games
Researchers: Ben Castleman, Jenn Doleac
Location: Richmond, Virginia, United States
Access to technology/Computer-assisted learning
Bridging the Digital Divide: A Randomized Controlled Trial of Providing Internet Access
Researchers: Peter Bergman, Elizabeth Setren, Susha Roy
Location: United States
A “digital divide” exists in access to technology and internet connectivity. Policymakers argue that this divide impedes lower-income and minority students’ ability to complete schoolwork and succeed academically. This narrative has inspired government and private entities to direct large investments toward bridging the digital divide in the hopes of reducing achievement disparities across socio-economic groups. However, there have been no large‑scale experiments testing the idea that access to the internet improves student achievement. This study identifies the causal effect of bridging the digital divide on student achievement by randomizing the provision of 10,000 smart phones, tablets and hot spots, with 5-year data plans, to high school students who lack home internet access. This project is project is a part of a larger initiative, the Sprint 1Million Project, which is providing these devices and data plans to one million high school students across the country. This study will inform policies that aim to provide internet and digital devices to low-income students.
Technology-enabled behavioral interventions
My Student’s Team
Researchers: Todd Rogers
Locations: Wake County (Raleigh), NC; Manchester, CT; Windsor, CT
Strong relationships with non-parental adults are associated with greater student success. Despite this, many children receive little educational support from adults in their families’ social networks – even from those with whom they have relationships. This randomized experiment evaluates a scalable intervention called “My Student’s Team,” which entails proactively inviting and empowering parent-nominated adults who have pre-existing relationships with students (e.g., family members, neighbors, coaches) to support students’ educations. Students’ parents name an adult “supporter” whom they permit to receive school-related information about their children. Supporters of students assigned to the intervention group are sent regular communications, such as class schedules, progress reports, attendance updates, behavior reports, invitations to school activities, notifications of upcoming tests, and conversational prompts. An implementation pilot was conducted during the 2015-2016 school year followed by a modestly-powered roll-out in 2016-2017. We now plan for a well-powered roll-out during the 2017-2018 school year, which is currently being implemented. We predict that this intervention will improve end-of-year course grades, grade point average, test scores, and attendance. This research will also enhance our understanding of how adults within students’ social networks can influence student success.
The (Null) Impact of Tax Credit Information on College Enrollment in Texas
Researchers: Dayanand Manoli, Peter Bergman, Jeffrey Denning
Location: Texas, United States
Evidence suggests that informational and behavioral “nudges” which help individuals overcome procrastination or inattention can affect educational attainment, college enrollment, and student achievement. Researchers conducted a randomized evaluation to test the impact of a large, information-only intervention about tax credits and financial aid for college on college application and enrollment. Information about tax credits did not influence reenrollment, reapplication, or enrollment, even for students who had viewed the informational emails about tax credits for college.
PILOT: Mindful Parenting: A Pilot Study Leveraging Neureoscience and Technology to Promote Well-being and Child Development at Home
Researchers: Philip Oreopoulos, Ariel Kalil, Susan Mayer
Location: Chicago, IL
Recent evidence suggests that programs aimed at increasing “mindfulness” are successful at reducing the cognitive demands of stress and increasing focus and attention. This pilot study would be the first to apply this approach to improving parenting among disadvantaged families. In partnership with Chicago's Children's Home and Aid Society, it would explore the feasibility of developing a technology-based mindfulness training intervention (i.e., with apps and videos) combined with behavioral devices such as reminders and goal-setting with the goal of improving executive function, and in turn the quality and quantity of time parents interact with their children and other parent and child outcomes. The results of the pilot work will be used to inform the development of a JPAL Full Research Project Grant Proposal to conduct a large-scale RCT. The ultimate goal of this research is to develop a cost-effective scalable approach for fostering mindfulness in parents in order to foster children’s cognitive and non-cognitive development
PILOT: Student Coaching: How Far Can Technology Go?
Researcher: Philip Oreopoulos
Location: Toronto, ON, Canada
Nearly half of college students in United States fail to complete their bachelors’ degree within six years. One-on-one coaching and tutoring can lead to improvements in student outcomes, but are far more expensive than technology-driven alternatives. Researchers conducted a randomized evaluation to test the impact of three different coaching methods on academic outcomes: one-on-one in-person coaching, text messaging, and an online values and goals exercise. One-on-one coaching substantially improved student outcomes, while text messaging and the online exercise had no detectable impact.
PILOT: Financial Aid Information and College Applications: The Role School Counselors Play
Researcher: Phil Levine
Location: United States (210 different designedated market areas)
Many American high school students know only the sticker price of college and make application decisions without properly incorporating the availability of extensive financial aid, particularly at elite colleges and universities. School counselors often cannot overcome this barrier because they are overwhelmed with high caseloads and have difficulty explaining the complexity of financial aid to students. This project will conduct a randomized controlled trial to examine whether providing guidance counselors with simple tools can help. Counselors assigned to treatment group schools will receive promotional material regarding MyinTuition, a simplified financial aid calculator that was first used by Wellesley College in 2013; it is expanding to more than a dozen highly selective colleges and universities beginning in the spring of 2017. It offers students individualized estimates of college costs factoring in financial aid upon providing six basic financial characteristics. MyinTuition takes the average user just three minutes to complete. It may help inform school counselors of the aid available to their students and provide an efficient way to communicate that information to them. The effectiveness of this approach can be determined by comparing application rates from treatment group secondary schools relative to control group schools at colleges and universities using MyinTuition.
Leveraging Education Technology to Increase Community College Transfer Rates
Researchers: Ben Castleman, Kelli Bird
Location: Virginia, United States
Community colleges enroll approximately 40 percent of all US undergraduates and they enroll a disproportionate share of low-income and first-generation college students in higher education. For instance, low-income college students are three times more likely to attend a community college compared to high-income students. Many community college students intend to transfer, but few actually do so. Closing this gap between intention and successful transfer is a promising strategy for increasing bachelor’s degree attainment in the United States, and for reducing socioeconomic disparities in college completion that have widened over time. This study proposes to build on the foundation of related research by providing community college students in Virginia with (1) personalized information about their progress towards transfer and meeting GAAs at four-year institutions across the state, (2) personalized guidance about specific four-year institutions in the state where students appear to have a high probability of success upon transfer based on the transfer and degree attainment trajectories of similar students from prior cohorts, and (3) access to remote, one-on-one advisors who will use course mapping tools and other technology solutions to support students to plan and execute a personalized transfer preparation pathway.
Testing External Validity for Online and Electronic Messaging Efforts to Improve College Achievement and Retention
Researchers: Philip Oreopulos
Locations: United States
Over the last four years, Phil Oreopoulos been researching how online exercises, text messages, emails, and electronic calendars can be used to improve college academic achievement. The platform design works remarkably well in getting students to engage and contemplate advice within their own contexts. Instructors impose a small participation grade at the start of a course for completing an online ‘warm-up exercise’. This leads to exposing virtually all students to randomized content designed to improve mindset, study habits and motivation. Despite hints of improved study time, mental health and very enthusiastic feedback about user experience, Oreopoulos’ studies have yet to estimate markedly improved course grades or retention from my tested programs, including ones based on previous promising studies. Since these ed-tech experiments were conducted with high degrees of precision, fidelity, and internal validity, Oreopoulos proposes to better understand their external validity, to determine whether online exercises and electronic messaging hold promise or not. Oreopoulos proposes to make the platform easily available to and customizable by other researchers and colleges. In particular Oreopoulos aims to test these programs at U.S. 2-year community colleges and 4-year public colleges where the college dropout rate is much higher than the Canadian college previously examined.
Improving Educational Achievement Through Fitbit-Assisted Better Sleep Habits
Researchers: Sally Sadoff
Location: Pittsburgh, PA
The proposed study investigates how technology-assisted behavioral interventions can help individuals improve their sleep habits in order to improve educational outcomes. In prior work, we find that incentives for meeting sleep goals increase sleep and also find suggestive evidence that the incentives improve academic performance. Building on these findings, we will test the impact of the following interventions among 3,000 undergraduates: (1) Technology only and (2) Technology and Incentives. The Technology intervention aims to lower the costs of shifting and sustaining habits, including the costs of tracking sleep and remembering to go to bed on time. Through wearable technology (Fitbits) and a custom smartphone app, we will provide participants with reminders to go to bed and immediate feedback about sleep duration. The Technology and Incentives intervention aims to develop habits building on cue/reward models of habit formation. The Technology intervention will provide the cue to go to bed on time and sleep adequately; we will combine this with an associated reward (either Financial or Non-Financial) provided immediately each morning for meeting sleep goals. We will measure the impact of the interventions on sleep habits and academic performance.
Pocket Aces: Commitment and Incentives Against Smartphone Usage for Students
Researchers: Billur Aksoy, Scott Carrell, Lester Lusher
Location: United States
Nearly half of US smartphone owners make an effort to limit their use, but only 30% succeed. Usage is particularly high among teens, who spend an average of nine hours per day on their phones. Studies have linked increased smartphone usage to decreased sleep, lower student learning, and negative worker productivity. This study centers on a series of field experiments conducted with Pocket Points, an app that acts as a commitment device and provides tangible incentives to students for curtailing smartphone usage. Students open the app, lock their phone, and start accumulating “points” while the app verifies the student’s location and activity using GPS coordinates. Points can then be used to get discounts at participating businesses. A pilot study found that students randomly encouraged to use Pocket Points at Texas A&M experienced positive academic outcomes, including higher grades and improved in-class focus. In this study, we implement a series of treatment arms across a multitude universities to incentivize staying off the phone while in the classroom as well as during sleep hours. Outcomes will be collected through app usage, surveys (e.g. sleep quality), GPS coordinates (e.g. time spent on campus), and transcripts (e.g. course grades).
Evaluation of Talking Points
Researchers: Ariel Kalil and Susan Mayer
Location: United States
TalkingPoints is an interactive personalized messaging platform aimed at increasing communication between parents and teachers, increasing parents’ engagement in their children’s schooling and improving children’s executive functioning skill in grade 3. TalkingPoints has two components. TalkingPoints Multilingual Messaging facilitates communication between teachers and parents by providing an easy-to-use platform that promotes two-way translated messages in over 30 languages. TalkingPoints Coach provides parents tips for communicating with teachers and other information about their children’s schooling. Messages between parents and teachers come directly from TalkingPoints. Both local and federal law requires schools to provide programing intended to promote parental engagement in their children’s education. We assess the effectiveness of TalkingPoints in a school-based randomized field experiment over one school year in 50–65 schools using objective measures of parent communication with schools, engagement with children and children’s executive functioning.