Emily K Center Embarks on New Evaluation Plan

January 20, 2015

Bill DeBaun, Program Analyst

“Empowerment is a big word here for students, staff, and parents,” says Dalia Wimberly, Director of Program Administration at the Emily Krzyzewski Center (EKC) in Durham, North Carolina. That empowerment comes for students of all ages who participate in the Center’s K to College Programs. Through the Pioneer Scholars (1st-8th grades), Scholars to College (9th-12th grades), and Scholars on Campus (postsecondary) levels, the EKC offers services tailored to students no matter their stage of life, and Center staff use data and evidence to be sure they are being as effective as possible.

“We are trying to look at what kinds of factors we should be looking at or tracking for students, not all of which we’re collecting yet, but we’re making progress on it,” adds Wimberly.

After joining the EKC in June 2013, Wimberly began examining what students were doing at which stages of the program. Lots of data was being collected from each individual level. For example, elementary and middle students were receiving the Woodcock-Johnson Test of Achievement and traditional pre-college metrics (SAT/ACT, GPA, etc.) were being collected for high school students. Where these metrics were falling short, however, was in where they were connected across the programs. “We are in a unique position because we work with students across a 14 year continuum,” says Wimberly. “There’s an opportunity to track students over the long term in some of these characteristics.”

In order to better understand the metrics they use and determine which are the most valuable for promoting student success, EKC is working with respected research firm RTI International on a more formal program evaluation. The intention is to vet some of the metrics and indicators that the Center has been hearing about in the college access and success field. The Center is seeking “vetted, evidence-based, research-backed tools and assessments to measure non-cognitive indicators but also some of the academic indicators,” says Wimberly.

The Center and RTI are working from a five domain, 14-year evaluation model which was developed by paring down EKC’s curriculum into elementary, middle, high school, and college steps. At each level, there is a slightly different goal and focus.

For elementary and middle school students, the aspiration is to be well-prepared for a college preparatory curriculum in high school. This level’s activities include academic skill development through individualized tutoring with licensed teachers and volunteers five days a week. Students are separated into small groups, each of which works with 2-3 volunteers. Students need to be able to master academic work and move ahead in areas in which they have interest.

High school students have weekly meetings that focus on their becoming competitive applicants for colleges and universities. Freshmen work specifically on time management, organizational, and study skills. They do not receive much specific subject tutoring unless students are struggling on something specific. As students continue on their pathway to college and advance through high school, they receive guidance on all of the requisite college-going activities: making a college list, visiting colleges, writing applications and essays, practicing mock interviews, and undergoing college/scholarship interviews, among others.

Once they matriculate, on-campus college scholars stay in contact with the Emily K Center in their first and second years to make sure they make a successful transition to their college campuses. Interactions at this point include monthly check-ins as well as winter workshops.

In recent years, the Emily K Center experienced a particular need to standardize their high school curriculum. The Center was committed to doubling the size of its high school program and consequently saw an increased need for efficiency and standardization across what students were being taught at each grade level. In some cases, students were not getting what they felt was important to them or they were getting more than enough, depending on the curriculum specialist associated with their group. Standardization was needed to even out the under- or over-servicing.

In moving from 60 to 120 students, EKC began to look critically at their high school students. Staff talked to other NCAN members and looked at research in the field around the skills and knowledge needed to succeed in high school and college. From this research, they developed five domain areas: college planning, personal management and leadership, academic skill development, career exploration, and parent empowerment. Once these were developed, the Center and RTI International began to further segment the individual indicators of each domain by grade level. Of course, some domains and indicators are more relevant to some grade levels than others. After segmenting the indicators, “we really began to reach out to RTI and doing research on our own and saying…’If we want to track these pieces of info, what is the best measure or assessment or scale to get that from students?’” says Wimberly. Some of that research was conducted by the Center, but RTI has also been helpful in assisting. Together they have sought research about, and scales that measure, positive youth development.

This new domain and indicator system is being piloted this school year. It is intended to give the Center a baseline of students’ intellectual aptitude and an understanding about whether students are really grasping the intellectual skills needed for college.

In order to ramp up data collection for the evaluation model, EKC collects report cards from students, and analyzes, collects, and stores teacher comments (e.g., on a student’s attention span, participation, etc). Parents sign a release for EKC to discuss their children with their teachers. All of that information for high school students is logged in the Naviance platform used by EKC. This qualitative information is also collected for elementary and middle school students, and it is stored for these students in a hard copy file. The end goal is to be able to code qualitative data (including data on a student’s non-cognitive attributes) and to set up an early warning system and communicate that information back to the students, their parents, and their schools.

Collecting all of this data is important, but it is also important to put it to use in order to evaluate student progress and make sure students are getting the supports that they need.

At the end of the year at the elementary and middle school levels, EKC tutors fill out both a survey and a student assessment form, in which they comment specifically on students’ progress and highlight areas where students are doing well and still need support. The survey includes some of these topics as well, but it also places a student’s performance on a scale. Both instruments were developed in-house but were also vetted through the relationship with RTI.

In the high school program, there are surveys at the end of the year (and bi-monthly progress check-ins, that include quantitative and qualitative elements). At this level, the qualitative aspect “is even more rich with qualitative data,” says Wimberly. Lead counselors each have three to four students, and each week a counselor has to report on that student’s progress.

Dalia Wimberly identifies two big changes that have come along with this more purposeful tracking and measurement of student data. The first “is even just giving a name to these domain goals and clearly defining specific outcomes we are hoping to see for students across each program.” This has emphasized the expectation that students will go to college and empowered students, staff, and parents to all talk about these domains.

The second change is that although EKC was already collecting data fairly well within each program, “the data relevant to each grade level operated in a silo as it related to that grade level…The individual indicators look different, but being able to see how what we’re doing with elementary school students leads into middle school students and then into high school students,” says Wimberly. “That has been a really big change…[because] the programs need to be fantastic together and need to be linked together.”

The goal of this platform and this revamped and recharged evaluation model is being able to track students across all 14 years. This way, there will not be a whole new data set in each transition across program levels. Ideally, EKC will be able to better serve students because, for example, they will be able to see an 8th grade student’s progression from the previous seven years. Ultimately, EKC wants to be able to pinpoint the specific domain needs that individual students have as they transition further along in the program on the way to college. Stay tuned for what will surely be more from the Emily K Center as they continue in their first year of this new evaluation plan.

Back to Blog

Leave a Reply:

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License