News: Data, Research & Evaluation

Critical New Research Shows Positive Pell Impacts

Thursday, July 11, 2019  
Posted by: Bill DeBaun, Director of Data and Evaluation
Share |

You’d be hard pressed to find bigger supporters of the Pell Grant than NCAN. The program helps to make college possible for over 7 million students annually, and NCAN would like to see the maximum Pell Grant award doubled to keep up its purchasing power. Despite the significant public investment in this program, the body of evidence around its impact is scant. Given this, and at the risk of sounding hyperbolic, a recent study may be one of the most important ever published about the Pell Grant.

ProPelled: The Effects of Grants on Graduation, Earnings, and Welfare,” by Jeffrey T. Denning, Benjamin M. Marx, and Lesley J. Turner provides evidence that “within four-year institutions, eligibility for additional grant aid significantly increases first-time students’ degree completion and later earnings.” The research specifically looks at the effect of receiving a maximum Pell Grant (i.e., qualifying for an automatic zero expected family contribution (EFC)). The study also finds that for students attending community colleges, receiving a maximum Pell Grant “affects enrollment decisions of individuals on the margin” and pushes them to attend. It is important to emphasize here that the study does not examine the effect of receiving any Pell Grant versus no Pell Grant but instead the effect of receiving an automatic zero EFC that makes a student eligible for a maximum Pell Grant.

The study finds first-time students at four-year institutions who receive a maximum Pell Grant are 1.5 percentage points (p.p.) more likely to receive a bachelor’s degree within four years of matriculating and 3.3 p.p. more likely to graduate within five- and six-years of matriculating. These percentage point gains represent 10%, 11%, and 8% increases, respectively, over students who were just barely ineligible for a maximum Pell Grant.

The study also finds that for these first-time students, each additional $1,000 of grant aid received at college entry increases the probability of graduating within five years by about 5 percentage points (about 15% compared to completion rates of students not eligible for a maximum Pell Grant). Also, each additional $1,000 of grant aid received at college entry is associated with a greater than $1,000 increase to earnings starting four years after enrollment.

For community college students, first-time students eligible for a maximum Pell Grant were 7%-10% more likely to enroll than their peers just ineligible for a maximum Pell Grant; for returning students eligible for a max Pell, that boost to enrollment was 3%-8%. There are additional results this blog will highlight later, but these are the big-ticket items.

Results discussed, let’s consider how the authors got there.

The study uses student-level data from Texas’ public colleges and universities and includes data on about 148,000 students from the 2007-08 to 2010-11 academic years, about 37,000 of which are first-time-in-college students. The authors take advantage of the automatic zero threshold in the EFC calculation to examine the effect of receiving a maximum Pell Grant. As most NCAN members know, students with a family adjusted gross income (AGI) below an annually-fluctuating threshold qualify for an automatic $0 EFC and a maximum Pell Grant. The statistical technique the authors use is called a “regression discontinuity design (RDD),” and before your eyes glaze over, this strategy involves looking at students just above and just below a given cutoff to see how their outcomes differ.

Readers, let’s have a little sidebar to talk about why RDD is important. First, recall that correlation and causation are different. X and Y having a relationship (either positive or negative) is not the same as X causing Y to happen, which is a much stronger bit of evidence. Next, recall that randomized controlled trials (RCTs) are the gold standard of evidence in experimental research; they involve taking a population, randomly splitting it into two groups (a treatment group, which receives the intervention being tested, and a control group, which receives no intervention or a placebo), and examining the results with the ultimate goal of demonstrating causality between the treatment and those results.

RCTs can be difficult to perform in social sector research. For example, in the case of researching the effectiveness of the Pell Grant, an RCT would involve taking a group of Pell-eligible students, randomly giving some of them Pell Grants (the treatment) and some of them nothing (the control) and then seeing what their outcomes, postsecondary and otherwise, were. Astute readers can see how such an approach might be problematic for the students not receiving the grant aid.

In situations where RCTs are not feasible, researchers turn to approaches that fall under the heading of “quasi-experimental designs (QED).” RDD, such as the one used in “ProPelled,” is one of these approaches. QED allows researchers to estimate the impact of an intervention without random assignment into treatment and control groups. Instead, QED relies on “natural experiments” where things like policy changes (e.g., shifting to mandatory FAFSA completion for high school graduation) and eligibility cutoffs (e.g., the automatic zero threshold for the Pell Grant) separate subjects, in this case students, into treatment and control groups. Specifically with regression discontinuity design, researchers look at students just above and just below an eligibility cutoff and hypothesize that there is nothing different about students just on either side of that cutoff except for eligible students receiving the treatment (i.e., a maximum Pell Grant). Then researchers compare the outcomes of students who received the treatment and those who did not and estimate the causal impact of having received the treatment.

Sidebar completed and your daily dose of statistics administered, the reason that “ProPelled” is so important is that it estimates the causal impact of receiving a maximum Pell Grant and finds positive impacts on student outcomes. Not just the completion effects, at four-year institutions for first-time enrollers, and enrollment effects, for community college students, but also that students who receive a maximum Pell Grant borrow less and go on to earn more than their peers who were ineligible for a maximum Pell Grant (year 4 and beyond). In the short term for first-time students enrolled at four-year institutions, receiving a maximum Pell Grant had minimal effects on credits attempted, GPA, first-year earnings or likelihood of second-year persistence. As noted above, the study finds big gains to maximum Pell Grant recipients’ earnings just four years after first enrolling.

Notably, the study’s results are less effusive for returning students; the authors “find no statistically significant effects of eligibility on returning students’ graduation, earnings, or taxes.”

The authors conclude by discussing the policy implications of their research and making some proposals. They note that not only does providing additional grant aid to first-time students benefit them, but the cost to providing that aid is likely to be recovered by the government within 10 years. “Our results indicate that additional grant aid raises welfare for these students and likely pays for itself several times over,” the authors note. Given this, and the benefits to graduation and earnings for first-time students, the authors also suggest that “the aid not only speeds up degree receipt but increases the overall number of college graduates.”

This important research deserves to be amplified loudly and widely to education stakeholders and especially policymakers who now have good evidence of the causal impact of the Pell Grant program.

(Photo by princess on Unsplash)