University of Texas Adopts Plan to Publish Performance Data on Professors & Campuses

[amborin]

U. of Texas Adopts Plan to Publish Performance Data on Professors and Campuses

By Katherine Mangan

Austin, Tex.

The University of Texas system's Board of Regents unanimously approved a sweeping plan on Thursday that responds to calls for more transparency about the performance of its campuses and spends $243.6-million to raise four-year graduation rates, expand the use of technology, and improve efficiency throughout the system.

snip

The plan unveiled on Thursday designates money to create a "dashboard"—an interactive, online database—to give students, parents, and legislators access to detailed measures of departments' and colleges' productivity and efficiency. Data on individual professors will probably also be included, although Dr. Cigarroa stressed that each campus would be able to develop its own system of metrics and the details have not yet been worked out.

The overall plan will help ensure that taxpayers are getting their money's worth from the system's nine academic campuses and six health institutions, he said.

snip

......includes the online database, which will track productivity over time......

snip

http://chronicle.com/article/U-of-Texas-Adopts-Plan-to/128800/



this will give new meaning to the mantra, "publish or perish"

[bluestateguy]

1. The more you make it publish or perish

Professors will devote more of their time to research and even less to teaching, and the quality of teaching will suffer.

And it could encourage professors to churn out a lot of mediocre crap articles just for the sake of getting publication points rather than working toward larger projects like major books which can take many years to complete.

[amborin]

3. and potentially encourage more grade inflation...

student evaluations depend heavily...

[SteveM]

8. How would this "grade inflation" occur?

[amborin]

9. how?

http://opinionator.blogs.nytimes.com/2010/06/28/student-evaluations-part-two/?scp=1&sq=grade%20inflation%20and%20evalutaions&st=Search

http://www.nytimes.com/2001/12/09/opinion/why-grade-inflation-is-serious.html?scp=2&sq=grade%20inflation%20and%20evalutaions&st=Search

[SteveM]

16. People have been bitchin' about "grade inflation" for years...

But no one has proposed much of a solution to this "problem." The touted "bell curve" (which takes a lot of data, I understand, to show some kind of meaningful phenomenon or measure, not usually found even in a large survey lecture), strikes me as a mechanistic approach. If a professor sets certain standards (learning data and information, or changing behavior), and everyone scores 80% on some kind of test, then the grades would presumably be "B" or "A." To apply a "bell curve" to this would be a gross distortion to the standards the instructor/department establishes.

Of course, not using the "bell curve" does not remove the challenge for the professor. If all the students do well, then perhaps the professor's standards are too low. That may be a way to thwart "grade inflation," but is it a sure-shot authentic approach? Raising standards may very well beget yet another way to escape the accusation of grade inflation, but then that becomes the goal, not the more important one of meeting standards of the course; standards established by the prof or his/her department.

I'm not sure how this really relates to student evaluation: Professors who have dished out a load of Bs and As have received evaluations of "dull course," "lack of point," "lack of challenge," even "lack of knowledge." The same has been said of profs who hand out many Ds and Cs. Clothing, hat & cane pizzazz behind the lectern, team-teaching, special guests, etc., don't really hoodwink students into giving out good evaluations to a professor if the course is a stinker.

In my limited teaching experience at the college level, those students who did poorly had more prosaic ways to achieve that "lofty" goal: They didn't show up, took a cook-book test (provided at the time by the college where I taught) and flunked, dropped out, etc. Some got Cs, most got Bs and As, in that order. And I got to see the improvement in most of the students (I had my ways!).

A peculiar point: I most often assigned an I ("Incomplete") when a student didn't show, or stopped short of meeting minimum requirements for a C. These same students often bitched about the I. Why? Because they were on some kind of scholarship (often a GI Bill), and were willing to average in a D or even F (sometimes by design) for an over all C or above average. In that manner, they received their money. But an "I" dropped them below the minimum course load required for the money! The complaints I received were from the school's administration!! Now THAT is a problem.

[patrice]

22. I always looked at my stats from the perspective of the bell curve as just one perspective

on what was going on, but not the guiding model.

You'd want to know, at least, if there's a skew and if there is, look at your scoring to compare your application of rubrics within that set. In order to be able to make that comparison you need some method of quantification of the traits of the relevant rubric, so that the scoring can be reviewed and compared. It could be that the scoring IS consistent and what you are looking at is, depending upon the work itself, either a dominant strength or weakness in the group, both of which have implications for lesson planning and teaching. This is the only way I could think of to hold the line on the kinds of variations in my scoring that would result in grade inflation/deflation.

I agree with you about not being allowed that "I". Implying that someone did the work, but didn't do well on it, is not the same thing as saying they didn't do it. My response, in secondary teaching, to that issue was to take un-limited make-up (NO!!! Extra-credit!) that declined in value the later it was until it was an F, but which I'd take anyway if a student produced it, and if that put them over the line, fine, if not that was as it should be.

[amborin]

37. thanks for this interesting post!

there are often strict guidelines about the I grade in many places, as you yourself apparently experienced....

[patrice]

15. You see, the idea could be to reduce that dependence by documenting the dialogue according to certai

n collaboratively agreed upon criteria - and - to me the most important aspect, by making it a two-way-street at ALL levels of organizational functions.

It's the same kind of knowledge-base construction that big network ops shops depend upon, except expanded beyond just one level of organization.

[patrice]

5. It is possible to see new more authentic definitions of "performance", not just publish or perish.

Possible, that is, IF that's what the institution wants.

[amborin]

6. what would those new definitions be?

[patrice]

10. That could be determined by the participants. In much the same manner as various

kinds of databases, including click-trails, yield new internet products.

All of it could even be the course work for certain kinds of research degrees.

The only part I wonder about it how parts of it could depend upon chunks of text, e.g. something like the DU but involving certain categories of stake-holders, which if they'd want to do that the state-of-the-art in text parsers would be real important and, having been out of technical writing for a while now, I'm not sure what the best technical capacities are in that area now. But, even with relatively primitive parsing, there could still be collaborative study-groups composed of ALL stake-holders who refine the data.

My main interest in this route is my hope that it could indeed be a two-way-street, by means of which ALL participants (professors, students, administrators, funders, parents, community . . . ) engage in self-development in dialogue with themselves and with their context.

[BadgerKid]

This will just provide opps for more teaching faculty

especially for the freshman level courses.

There are profs who enjoy teaching, but the reality is that they're hired for their potential to pull in grant money for the university. If you're deemed to have pulled in enough money, don't do a terrbile job teaching, and are congenial, then you just might get tenure.

[nadinbrzezinski]

2. You beat me to it, publish or perish indeed

[patrice]

4. I think this sort of thing could be useful to a MUCH needed NEW dialogue in this country IF

All stake-holding participants are authentically involved in the construction and characteristics of all assessment processes and IF said processes are genuinely two-way-streets supported by longitudinal feedback and strategic planning.

These could be institutional self-knowledge bases supported by the logic and methods of evergreen empirical research.

Probably a little much to hope for, but this old Freire -ian CAN dream.

[SteveM]

7. A note on UT's history of evaluations...

From the time I received my Master's from UT (Class of '72), there has been a row of tables outside the Academic Center with the names of professors and their courses at the top of blank papers were voluntarily filled out by students doing evaluations. I don't know what happened to this practice. My Dad underwent more formalized evaluations when he taught at the University of Florida ('51 - '84), and I voluntarily made up a "poor - excellent" evaluation scheme for several measures of performance when I taught at Austin (TX) Community College ('73 - '76). No problema.

[patrice]

12. I constructed all of my own rubrics and processes when I taught AP Psych. It is tantalizing to think

of that process occurring in a fully interactive collaborative environment.

[SteveM]

17. Tantalizing and a bit scary! I mean, who uses poli-sci to make an everyday point?

:D

[patrice]

20. I'm not too sure what you mean by poli-sci. I needed to identify valid descriptors for the work

my students produced, descriptors that also related the defined elements of the curriculum as they were manifested in that work. The elements of the rubric itself also needed to be related to one another in somekind of scoring schema that produced, not only the grade for a given piece of work, but also how different kinds of work were going to be related to one another in order to result in an over-all grade.

These are the sorts of things that went into a student's evaluation of me as a teacher. I guess that's political to the extent that it involves multiple persons, but, then isn't all instruction not only a matter logical explication, but also a persuasion?

I always found it kind of difficult to get anyone to care enough to pay attention to the details of the process. That seems unfortunate to me, especially since practically everyone wants to complain and criticize, political actions both, and not very many wanted to do the work of participating in the creation of what they so often complained about.

I'm not sure; did I respond to your point?

[readmoreoften]

29. It is absolutely nothing like that now.

Evaluations are standardized and not always accurate or relevant to anything. For example, I teach a mandatory large section course of hundreds of students. My students run the gamut from seniors in biochemistry to freshmen in athletic training and coaching. Yet I'm evaluated by the following metric: "This course was challenging. A-E." I cannot really both equally challenge the biochemists and the personal trainers, let alone freshmen and seniors. I basically had a 4-1 ratio of challenging to not-challenging, but that isn't good enough to lower my "numbers" by a smidge. Likely, my uni doesn't take this garbage as seriously as others.

These research methods are really harming education. They're very standardized, very strict.

[DrDan]

11. aaaaaaargh - politicians need to stay out of education

[patrice]

13. Depending upon what a given institution decides the processes by means of which this sort of thing

happens can be self-governed or more authoritarian.

[patrice]

14. An empty process can be defined collaboratively and mandated, the contents of which would then

be defined by the individual institutions themselves.

Think of the kinds of software engineering that make the internet interactive; that's a model. I hope that's the direction they are headed with this stuff.

[jwirr]

18. NCLB has worked so well now we are going to destroy our colleges

with it. :sarcasm:

[amborin]

27. agree

[saras]

19. Professors are EASY to evaluate - this is a waste. I want to see performance data on administrators

I think each employee of the administration ought to have to justify their job in terms of number of students helped. If their job doesn't help students learn and graduate, they're a waste of money.

Everyone who runs the infrastructure has an easy job of it - their work is obviously necessary for the university to exist. Managers, on the other hand - if they save money in ways that harm instead of helping students - kick 'em out.

[patrice]

21. That's one of my points. If they're going to do this sort of thing, it should involve the entire

functional structure of the organization and all evaluators should also be evaluatees and vice versa.

This isn't that new an idea; Edward Deming created continuous quality improvement methodologies, just after WWII I think, very collaborative horizontal stuff, enterprise-wide, was laughed at in America, took it to Japan and set them on their course to kicking American automotive butt. There are a lot of others who built on his work, but I imagine the idea of applying this type of methods in education is relatively un-heard of. I hope that's where this Texas group is headed.

[bbgrunt]

31. ...another failed buzz-word: continuous quality improvement = CQI

[patrice]

39. Anecdotal or research based evidence? nt

[readmoreoften]

33. Trust me. They aren't headed anywhere good.

This is a political move to raise class sizes and punish faculty with any sort of realistic course load. (And to show that I am not speaking out of self-interest, here, I teach the largest course in my subject matter in the US)

[amborin]

25. +1000

[madrchsod]

23. another waste of money....well maybe.

expanding technology and improving efficiency is worth the money.

[amborin]

28. the technology part

is probably good

[readmoreoften]

24. It's not about publishing. The key for "value" and "productivity" in the first draft was class size.

A professor who teaches 400 students in a stadium course is more "productive" than a professor who mentors 40 graduate students and runs a research center. In other words, a professor who is early career and exploited is "more productive" than a full professor who has earned perks over the years. In other words, this is union busting actions except there isn't even a union. (Well, not really...)

[amborin]

26. think you nailed it...and it's probably not really about improving teaching at all...$$$$

[readmoreoften]

32. Of course it's not about improving teaching. We don't even get any TRAINING for teaching.

The vast majority of us are thrown into a classroom with little to no experience and next to no support. I don't have a Ph.D. in teaching. I have a Ph.D. in my subject matter. I happen to be a good teacher by nature, but many of my colleagues struggle with it and they don't have access to formal training to become better teachers. I deal with rape survivors, soldiers with PTSD, violent students, drunks, racists, homophobes, and students with learning disabilities and I have zero avenues for training.

This has nothing to do with "democracy" and nothing to do with "community members" knowing the "value" of teachers. This has to do with tenure-busting, increasing class-size, and opening controversial courses to populist critique (They teach a course in LGBT history?! Marxist economics?! Tar and feather them!) This country has a perverse obsession with teachers.

If parents want to home-school their college age "children" (aka, the adults of pre-1960s America) then they should do it themselves and leave me to teach students who want to be school-schooled.

[amborin]

35. +1000

yep, first step toward eliminating tenure and stifling any voices that don't conform

jr faculty already have huge teaching load, huge class sizes, more and more committee work, etc....

[readmoreoften]

38. By the way, performance data is already published on every professor in a public university in Texas

By law, our data is already public. In fact, I found my name, age, salary, and gender (so fun for transgendered folks!) printed in a magazine. Because my students' families also need to know how much I make. Fine. They're probably laughing at me for being a sucker taking such a low-paid career.

[amborin]

42. your

teaching evals, too?

[patrice]

40. Thanks for the perspective, I guess. :-(((((

[melissaf]

43. yes, indeed

It's tragic how many college students and/or their parents really aren't looking to be school-schooled.

[girl gone mad]

44. You are right, this is an effort to eliminate smaller classes..

but I also don't think that parents are the ones pushing it. Administrators are feeling pressure from the state legislature over costs and, of course, they are scapegoating the teachers.

The focus is completely misguided. As a parent, I want my child to have access to some small classes.

[bbgrunt]

30. what bullshit. Administration initially (in the old days) was hired

to serve faculty. Eventually they turned that whole concept on its head. Now faculty are to there at the whim of administrative accountability numbers.

Once you start turning faculty jobs into bean-counting (like public school teachers are evaluated by the tests their students take) you end up with nothing but foul smelling farts and everyone's energy focused on how to outsmart the "system" instead of dedicating themselves to their jobs and enjoying the responsibility of a job well done.

.....not to mention the waste of energy and resources developing such a system, implementing it, constantly feeding it data that doesn't and can't measure what it purports to measure, and reducing faculty to animals on a hamster wheel.

Stop the de-skilling of professional jobs through bullshit "accountability" measures!

[readmoreoften]

34. +1,000,000

[amborin]

36. agree, completely

[AdHocSolver]

41. The real issue with a college education is that it costs too damn much.

When I went to college, how did I find out about the quality of courses and professors? I asked other students who had already taken the courses.

I was able to pay for my college expenses working part time, and graduated with a bachelors degree without a dime of debt.

Like NCLB and RTTT, this is all smoke and mirrors to distract the public from the real problems of education in this country.

Many other industrialized countries subsidize a college education for their young people, because those countries expect that there will be jobs for them after they graduate.

Makes you wonder what our leaders plan for the future of our young citizens.