EDSOURCE. FEBRUARY 14, 2013. Students planning to attend one of the nation’s 4,500 colleges and universities have a new interactive College Scorecard touted by President Obama in his State of the Union address as a tool “to compare schools based on a simple criteria – where you can get the most bang for your educational buck.”
Community college leaders say the focus on costs and graduation rates is a flawed lens for measuring their worth. Give it a few weeks, though, and the California Community College Chancellor’s Office will be launching its own scorecard tailored to the broad mission and local scope of the state’s 112 campuses.
The federal scorecard is “very four-year centric data,” explained Patrick Perry, Vice Chancellor for Technology, Research and Information Systems for California Community Colleges. “It tracks first-time, full-time freshmen degree-seeking students. That’s a small percentage of who’s coming to us.”
In late March or early April the chancellor’s office is expected to release AARC 2.0, an updated and more accessible version of its current Accountability Reporting for the California Community Colleges, an unwieldy 856-page document mandated by a 2004 state law.
The new scorecard will shine a light on key indicators of success and provide data on how well each of the 112 campuses is measuring up. There are six specific categories, or what Perry calls “momentum points,” that research has shown are correlated with student success. These are based on progress over six years.
- Persistence Rate – the percentage of students seeking a degree or transfer to a four-year school who remain enrolled for three consecutive terms,
- 30 Unit Rate – the percentage of first-time students seeking a degree or transfer who earn at least 30 units,
- Student Progress and Attainment Rate – the percentage of degree-or-transfer seeking students – separated into cohorts of those who start in basic skills and those who begin in college-level classes – who earn a degree, earn a certificate or transfer to a four-year college or university,
- Basic Skills Progress Rate – the percentage of students who start out in remedial classes who go on to succeed in college-level courses,
- Career Technical Education – the percentage of students who complete a career technical education program and earn a degree, earn a certificate or transfer, and
- Career Development and College Preparation Rate – the completion rate for students in non-credit career development and non-credit college prep courses, such as English as a second language, which are offered at about a third of the state’s community colleges.
Paul Steenhausen with the Legislative Analyst’s Office, who worked on the scorecard, said it contains multiple layers of statistics that he likens to a pyramid. The tip contains the aggregated statewide figures; below that is more nuanced information that “allows researchers and analysts to drill down and get even more granulated data.”
Bakersfield College microbiology professor Janet Fulks, who worked on the algorithm, said there are so many levels of data that she could follow a group of students placed in basic skills math to see how many semesters it took to pass and move into college-level classes, and then whether they graduated or transferred. Colleges could track students who complete education plans to see if they do increase completion rates. And all that data can be disaggregated by race, sex and ethnicity. Where faculty see problems, they can make changes and try new strategies to help students succeed.
“I am telling you it’s really unbelievable,” said Fulks. “Without that kind of data how do we improve the way that students complete their educational pathways?”
The scorecard data has been sent to each individual college for review. They have 30 days to verify or change the information.
The economic payoff
A companion community college data file that’s due to roll out at about the same time as the scorecard lets students look into the future before choosing a major or career technical program. Under an agreement with the state Economic Development Department, the community college system will show the median wages for students based on their major two years after they graduate and five years after that.
The early analysis, said Perry, shows that career technical education graduates in nursing and other health occupations, fire technology and some information technology fields are doing the best. On the other hand, students considering fine arts may want to start seeking a patron as soon as possible.
The federal scorecard, not even a week old, is already getting mixed reviews. “We are dazzled,” exclaimed Nancy Coolidge, coordinator of federal and state financial aid programs at the University of California. She said it’s easy to use and gives students and their families a good indication of college costs. “We are generally impressed with the transparency and the intuitive way it functions,” said Coolidge.
She said the scorecard would have been helpful to have a few days ago, when she was working with a family and had to use a different database, which she finds clunky and not intuitive. “This is much more accessible and easy to look at. I think this is a step in the right direction.”
Here’s how it works. Let’s say a high school student has her heart set on attending college in San Diego. If she and her parents look up San Diego State, they’d see that the median cost after grants and loans is about $8,500 a year, the average loan is nearly $15,600 and the six-year graduation rate is 65.6 percent. Comparing that to UC San Diego, they’d see the median cost is just over $12,600 a year and those families borrow more than $17,700. But the six-year graduation rate is significantly higher at 85.3 percent.
But the federal scorecard still has some bugs to work out. Early in the day, when Coolidge typed in the University of California, up popped Bethesda University of California in Anaheim.
Other problems are potentially more serious. Debbie Cochrane, research director for The Institute for College Access and Success (TICAS), said some of the data lacks context and “could be misleading rather than enlightening,” particularly for community colleges.
On loan default rates, the institute cautions that the scorecard doesn’t report the percentage of students at each college who borrow money. In a blog posted just hours after the scorecard was released, TICAS gives the example of American River College. The community college has a 27.5 percent default rate, more than twice the national average. But what the scorecard doesn’t show is that only 8 percent of students at the college take out federal loans. “Implying that the default rate alone is indicative of student outcomes more generally does a disservice to would-be students,” they write.
The federal data also derives its median student loan amount by lumping together all loans without regard to whether a student borrowed for only one year or for four years. “You could have a school where everyone drops out after six months, but their median debt is going to be very low,” said Cochrane. “It gives you no scope of the borrowing.” Cochrane said the institute has already made the U.S. Department of Education aware of their concerns.