Evaluation Shows Bottom Line Impacts Student Enrollment

July 25, 2014
Bill DeBaun, Program Analyst

An evaluation of Bottom Line (summary), an NCAN member, shows significant impacts on the postsecondary enrollment and persistence rates of the students served by the program. This was especially true if those students matriculated to a set of schools encouraged by Bottom Line. The evaluation was conducted by the University of Virginia’s Ben Castleman and Harvard’s Joshua Goodman who “argue that this evidence suggests that intensive college advising can have meaningful impacts on college enrollment decisions and may improve persistence and, ultimately, degree completion.” 

Among the other findings in the evaluation are that receiving Bottom Line’s services had a particularly strong effect for ESL students and that “Bottom Line advising results in students attending more affordable institutions.” Students participating in Bottom Line enrolled in institutions with average net prices that were $7,400 lower than those enrolled in by their peers who did not receive advising from Bottom Line. The authors note that, “This is not an indication of the actual costs such students are paying because IPEDS’ measure is an average across all students,” but that “It does…suggest that students are being diverted into four-year colleges with lower sticker prices, higher amounts of grant aid, or some combination thereof.”

Bottom Line was founded in 1997 and maintains both access and success programs in Boston and Worcester, MA and New York City. Students apply to Bottom Line in the second half of their junior year and admissions are based largely on family income, first generation college-going status, and cumulative GPA. Students with GPAs above 2.5 are targeted to ensure that they are prepared for collegiate coursework. Students who matriculate to one of 20 four-year institutions encouraged by Bottom Line (based on their graduation rate and tuition) are eligible to also receive success programming for up to six years following enrollment.

The selection of students around a specific GPA allowed Drs. Castleman and Goodman to use a statistical design known as “regression discontinuity” to compare students who were similar in nearly every way except for receiving Bottom Line’s services. This allowed for the isolation of the specific effect of Bottom Line.

Based on the questions that the evaluation leaves unanswered, including “whether Bottom Line’s impact on where students enroll and persist are sufficient to justify the costs of the program” and the fact that the evaluation was limited to examining Bottom Line’s impacts on students close to the 2.5 GPA cut-off, the authors and Bottom Line are collaborating on a “multi-cohort experiment across Bottom Line’s Massachusetts and New York sites” that will track students for 6-8 years after high school graduation to see how receiving Bottom Line services impacts completion as well as persistence.

This large-scale randomized controlled trial (RCT) was recently selected to be funded by the Coalition for Evidence-Based Policy. This award was given to “fund low-cost randomized controlled trials (RCTs) in areas of high policy importance, with the goal of demonstrating the feasibility and value of such studies to a wide policy, philanthropic, and research audience.”

Back to Blog

Leave a Reply:

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License