Comments "SEI scores, VIGRE, Grades and Independent Testing"

by Boris Mityagin, June 4, 2002

Over the last thirty years, the system of student evaluation of teachers in colleges evolved - step by step - as a result of an implicit plot by academic administrators [who are unable to fill classes by students properly prepared to attend these classes] and by unqualified students [who want to be awarded high grades without either having or getting skills and knowledge].

For great educational experience - nobody talks about great education - teachers is an obstacle or a nuisance. They should be intimidated and pushed to certify illiteracy by perfect grades. The use of student evaluation by
administrators in making personnel decisions on promotion, tenure and salary adjustments became the whip which keeps the faculty in line and keeps grade inflation (or to say simpler, cheating of the public) intact.

These abstract comments are not necessarily related to OSU or our department but they give a general framework all of us function in. All these thoughts went through my mind time and again when I've learned about results of
the Qualifying Exam in Analysis 1 1/2 months ago.

22 graduate students took the exam. They are not novices in Analysis or Mathematics; they've been selected as the best candidates who applied to our graduate program in Mathematics. Over the previous 12 months, they attended
Classes 787.03 SU-01, 651 AU-01 and 652 WI-02 or 750 AU-01, and other classes in Analysis. These classes (except 787.03) do not necessarily prepare the students for this Exam but, no doubt, students who are successful in
these classes should not have any problem with the Exam. The following numbers could be of some interest for all of us.

(a) Among 22 students, who took the Exam, 11 attended Class 787.03 SU-01. They've earned [or just got as a gift] the following grades:

A A- A- A- A- A- A- A- A- A- A- .

All of them, fresh from 787.03 SU-01 with these grades, failed the Exam in Fall 2001. Another try, and in April 2002 10 of them failed the Exam again but one has passed.

(b) Few students took a series 651-652. Their grades were:

AA AA *B+ *A [B- B] BB- A-A- B+B+ AA AA- B+A-.

(* means that the student did not attend Class 651 AU-01. [B- B] are grades from 651 AU-00/652 WI-01)

Of these 11 students one passed and 10 failed the Exam. SEI score of this instructor in 652 WI-02 was 4.8 (on the scale 1 to 5 with 5 being high). It seems students' satisfaction was almost perfect. [Two weeks later they failed an independent testing but who cares?!]

(c) Finally, let us mention Class 750. There were 11 of these
students among 22. Their grades are

B- B- [B+] [B-] B+ B+ B B B A [B] .

( [X] is a grade from 750 AU-00.)

Three of them (just those with grades of A, B+, B+) passed the Exam and 8 failed.

[The SEI score of the AU-01 instructor was 4.8]

Numbers are small (less than 30), and no statistically valid analysis is possible. This anecdotal evidence in this isolated case demonstrates nothing, but it gives a weak legitimacy to the conjecture that the Department of Mathematics has designed and now runs the machine of recycling mathematical illiteracy.

Of course, there are other numbers. In October 1, 2001 only 5 students (less than 30%) of 17 who tried passed the Exam. In Spring 2002 only 8 students (less than 37%) of 22 who tried passed the Qualifying Exam in Algebra.

We can analyze the same data as in (a)-(c) for related Classses in Algebra. Moreover, we can go back 12 years (i.e. to the outset of our current system of Qualifying Exams) and study the history of Qualifying Exams and compare their results, the grades in related classes and the SEI scores of instructors there. I am ready to do this job (and present the results of rigorous statistical analysis to the Department and OSU community by the end of AU-02 quarter) with understanding that the administration will give me an access to all these data.

Maybe, it is illusory rarity that students have great satisfaction (SEI score is close to 5!) when they do not
learn the basics? Yes, but the real rarity is the existence of this instrument (Qualifying Exam!!) which provides an independent testing of student learning or knowledge, something you cannot find anywhere in the entire structure of educational process although everybody talks and talks about accountability.

Paradoxically, this independent testing is done by the same people who are instructors in classes (a)-(c). I do not mention the names; it does not matter, it could be any of us. But in one case we are accountable to the SEI system and student satisfaction it requires, so no real testing, - only home-take assignments, tests and exams, - and high grades buy high SEI scores although students do not get skills and knowledge (they are becoming willing coparticipants of this cheating). In another case (design and grading of Qualifying Exam) it is done by us in such a way that we are responsible to our professional conscience (or even its ashes) and our colleagues in a committee
who straightforwardly judge your grading, i.e. your mathematical judgement.