LONs vs Grades
From Institutional Effectiveness and Accreditation
Catherine Wehlberg recently posted to the ASSESS Listserv that she "would love to hear your thoughts about the recent Inside Higher Ed Article, Missing the (Grade) Point - http://www.insidehighered.com/news/2013/05/20/study-challenges-data-and-... or the original article in Educational Researcher (Is the Sky Falling? Grade Inflation and the Signaling Power of Grades). These articles have created something of a discussion storm among our Deans and I’d like to hear the perspective of my assessment colleagues. Thanks, in advance!"
I saw this query about grade inflation in the more general context of David Shupe's keynote talk at last year's AALHE annual conference. His point was that program definitions of learning outcomes will (or at least should) change the way higher education is organized. He focused on credits, but the same argument could be made about grades. Clear, reliably assessed learning outcomes should replace not only credits, but also grades. I will be presenting evidence at AALHE this year based on ratings by 16 faculty members using two types of scales both embedded in the same Survey Monkey form. One scale included 14 dimensions based on Likert-type rubrics or (SWELL rubrics--Sequence Which Expands Little by Little) that are very much like grades, except that instead of one grade per course, 16 grades are provided.. The other was based on 12 dimensions of developmental rubrics, where every level of every dimension had a unique definition of student behavior. The levels in the latter are roughly related to program-beginning behaviors, behaviors acquired in fundamentals courses, behaviors acquired by the end of the program (i.e., these are the program outcomes), and behavios acquired long into the careers by people who make inspiring contributions to the area of expertise.
The resulting data included more than 4,000 ratings and are shown on page 8 of my PowerPoint for The Intricate Unfolding of Assessment Systems (http://www.vwc.edu/academics/academic-affairs/institutional-effectivenes...). The differences is one of the largest I've seen in my 50 years of research. The SWELL rubrics, much like grades, did not discriminate between levels of experience (for each institution in each decade sophomore GPAs are nearly the same as senior GPAs, regardless of grade inflation or selectivity of schools). On the other hand, the developmental rubrics discriminated student experience at the 10 ^-23 level of significance (roughly the equivalent of marking an atom, tossing it into a cubic centimeter of copper and finding it by chance).
We define the third developmental level (the program outcomes) as those necessary to go out and get a job in the area of expertise--which of course will vary with the type of expertise (one can go out and get a job with a biology BS, but not as a physician). But when faculty are asked merely to rate the performance of students, however, the ratnngs change with expectations which, in turn, change vary from decades to decade and school to school. On the other hand, when they are charged with defining behaviors related to target jobs, they make absolute discriminations (either the student has shown the behavior or not). Sometimes they sense that a student has fallen between the levels, but usually can agree on whether they find the student ready for the working world in the dimension they are assessing. Furthermore, we can expect the learning outcomes to change gradually over the decades, so adjustments need to be made. But the valuable point is that when they do, new definitions are created. When these are publicized, then employers and the general public can know what types of behavior the faculty member has used to certify the student as being employable.
I would enjoy continuing the conversation at AALHE in Lexington, KY in two weeks. It would be fun to hear people's reactions to the query "Should learning outcomes replace grades?"