Presidents Are Divided on Best Ways to Measure Quality

By David Glenn. In a year when public concern about the cost and purpose of college education is rising, a new survey has revealed an undercurrent of anxiety among college presidents about the quality of teaching and learning on their campuses.
More than a quarter of the presidents in the Pew Research Center survey, done in association with The Chronicle, said they worried that their faculty members were grading too leniently. More than half said students spent less time studying than they did a decade ago. And when asked how the public should assess a college's quality, the presidents did not show much faith in the student-engagement surveys and student-learning examinations that have come to prominence in the last decade. Instead, the yardsticks that got the most support were measures whose reliability is often questioned: graduation rates and accreditation. "It's surprising to me how relatively low the numbers were for any kind of assessment measures or surveys of engagement as effective gauges of college quality," said David C. Paris, executive director of the New Leadership Alliance for Student Learning and Accountability. Only 17 percent of presidents surveyed regarded those tools as "very effective."
"Presidents clearly don't think there are surveys or tests out there that really get them to effective assessment," said Mr. Paris, whose organization has recruited 76 presidents to sign public pledges to improve learning on their campuses. "In some ways, that's a depressing finding. We still have a long way to go as an industry in getting to the point of saying, This is what quality is, and here's how we'll know it."
Barbara Couture, president of New Mexico State University, agrees with Mr. Paris that higher education needs to develop universally understood measures of quality. But she believes those measures should have more to do with labor-market outcomes than with direct tests of learning. "The measure I didn't see on the survey, which is one I think we're all being asked to respond to, was, How well are students being prepared to enter the workplace?" she said. Ms. Couture believes that as states develop more-sophisticated longitudinal-data systems, the public will demand to see reports about the average salaries and unemployment rates of people who majored in a particular field at a particular college. But she cautioned that such reports would need to be interpreted carefully, taking into account the characteristics of the students who attend each institution. "What we're proud of here is that we take in students who are low-income, first-generation college students," Ms. Couture said.
Other college leaders are not so enthusiastic about using the labor market as a bellwether of college quality. David W. Oxtoby, president of Pomona College, said the public discourse about job preparation had become toxic. "There's this idea that the purpose of college is to train you for that first job, and anything else is just sort of extra, a waste of money," he said. Narrow vocationalism, he fears, will erode colleges' traditional missions of civic preparation and scientific inquiry. But if Mr. Oxtoby dislikes Ms. Couture's vision of labor-market measures, he is equally skeptical of Mr. Paris's desire for better public yardsticks of learning. Mr. Oxtoby is not an admirer of the National Survey of Student Engagement or the Collegiate Learning Assessment, and he was pleased to see that such measures were not strongly endorsed in the survey.
"The fact that presidents are all over the board in terms of different ways to assess quality—to me, that's just a reflection of reality," Mr. Oxtoby said. "There's not a single measure. There are many different measures. "I think this whole set of answers just shows that we don't have good indicators for the public," he continued. "I don't think any of them by themselves are very valid. Nor is there going to be some magic bullet down the road if we just work harder."
The Land of the Easy A
If presidents have mixed emotions about external measures of learning, they are also uneasy about their campuses' primary internal measures: grades. Twenty-seven percent of presidents surveyed said they believed their faculty members graded too leniently. "I do fall on that side of worrying about leniency," said Douglas C. Bennett, president of Earlham College. "Every grading system we've had for the last 150 years winds up suffering from inflation, and the only way we ever fix that is to throw out the grading system and introduce a new one. You have to throw away the currency and issue entirely new currency."
So what can presidents do to accelerate that process? Mr. Oxtoby said that presidents can set a tone, but that combating grade inflation really needs to be the work of faculty leaders. Pomona's faculty recently clarified what is meant by A, B, C, and D grades, with the aim of making C's and D's seem less radioactive to students. "We want to provide some standards and encourage instructors to use a broader range of grades," Mr. Oxtoby said. "Because grading, of course, becomes much less useful if everyone is getting an A."
Presidents' concern about lenient grading did not vary by their institutions' selectivity, but it did vary by institutional type. Thirty-three percent of presidents of four-year private institutions said they worried about grading, but only 24 percent of those at four-year public colleges said they did. The poster child for that theme might be E. Gordon Gee, president of Ohio State University. Mr. Gee said he was pleased with the grading standards at Ohio State. "Our faculty are very responsible about grading," he said. "Our deans and our department chairs look pretty closely at those patterns."
But when Mr. Gee was previously a president of private institutions, he said, grade inflation was a persistent concern. "At Vanderbilt and Brown, we had to work very hard on that problem. We used to laugh about the fact that 90 percent of our students were in the top 10 percent, which meant that our grading procedures were significantly skewed." The public-private discrepancy should be no surprise, Pomona's Mr. Oxtoby said. The intimacy of the smaller classes at many private colleges, he said, makes professors wince at giving C's. "At a public university, if you've got a class of 200, it's just a name on a piece of paper, whereas here, you know the particular student. You think, 'How could I possibly give a low grade and hurt this student's future chances?"
Students Lounge?
Fifty-two percent of presidents said they believed students were studying less than their counterparts did a decade ago. (Interestingly, the presidents who were least likely to say that were those who had been in office for 11 years or more.) Mr. Bennett, of Earlham, said those numbers should be interpreted cautiously. At his own institution, and at similar liberal-arts colleges, he said, students are being asked to do more demanding and sophisticated assignments than they were a decade ago. "When you ask students, How much are you reading? How much are you writing?, there's quite a spread across institutions," he said.
Eileen B. Wilson-Oyelaran, president of Kalamazoo College, said she believed that the survey reflected a healthy level of ferment among college leaders about the improvement of college learning. "There's a lot of debate out there," she said. "Each institution needs to find a way of communicating its learning outcomes, and there are lots of complex conversations about what those things might be. It's not as if institutions are running away from the responsibility."