CALS CASI

Committee on Academic Staff Issues (CASI)

---

Academic Staff Evaluation Survey Results

College of Agricultural and Life Sciences
Jan. 4, 2001
Committee: Ellen Maurer, Bryan Jensen, Tom Wright
Sub-Committee of the Dean's Committee on Academic Staff Issues

print version

EXECUTIVE SUMMARY

A survey of 605 Academic Staff members (with a response rate of one third) showed that they feel evaluations are important because they:

1. Provide an opportunity to present activities and accomplishments
2. Identify ways to enhance job satisfaction/performance
3. Identify career opportunities and develop a plan to achieve them
4. Foster closer communication between staff members and their supervisors, which can avoid problems before they develop
5. Provide a consistent opportunity to build a record on performance for use in promotion and merit recommendations

THE GOOD NEWS

Academic Staff
Most academic staff want and have annual reviews and are satisfied with their evaluators. They are more likely to be satisfied if their evaluations contain both verbal and written components.

• Of the 208 respondents, 72 % report that they have evaluations: more often than annually, annually, or every other year.
• Of the respondents, 59% are satisfied with their frequency of evaluation.
• Of those who are satisfied, 66 % have both verbal and written components as part of their evaluation procedure.
• Of the respondents, 74 % are satisfied with their evaluators.

Departments/Centers/Units
The Dean's office surveyed 41 Departments/Centers/Units with a 59% response rate. Of the respondents, an overwhelming majority had some form of Academic Staff Evaluation method.

• Of the respondents (24), 92% (22) indicated that they have some Academic Staff Evaluation procedure though they may or may not be formal written policies.
• Of the respondents, 42% said they asked Academic Staff to help develop the evaluation procedure. Two units responded that Academic Staff developed their own evaluation methods.
• Of those with a procedure, 64% (14) units have formal written evaluation procedures, ranging from a one-page list of information to submit to multiple page documents.
• The 24 respondents used the evaluations for:
  • Merit salary increases: 21 responses or 88%
  • Promotion to a higher title: 16 responses or 67%
  • Increased job security: 14 responses or 58 %
• Units listed some of the following benefits of their systems:
  • The system in place provides for self-evaluation, feedback from supervisors, being able to set goals for the upcoming year, defining significant changes in responsibilities and discussing professional development goals
  • Agreed evaluation between Academic Staff member and Supervisor and signed copy on file.
  • The document is broad enough to capture the range of activities; the summary is open ended and allows the Academic Staff member to express accomplishments and achievements in their own word. It requires feedback and dialogue with the supervisor and rebuttal by the staff member. It also requires that goals be set and discussed with the supervisor. The Academic Staff members developed the summary.
  • Simple easy procedure.
  • Provides uniform evaluation system for all Academic Staff.

THE BAD NEWS

Academic Staff
Although 90% of Academic Staff want evaluations more often than annually, annually, or every other year, more than one fourth do not receive them. Chapter 10.01 of the September 2000 Academic Staff Policies and Procedures (ASPP), "Academic staff shall be reviewed annually in a manner appropriate to their work setting and responsibilities."

More than one third of Academic Staff are insecure or unhappy with their current evaluation system.

• Of the 208 respondents, 28% don't know how often they are evaluated, have sporadic evaluations (3-18 years between reviews), or have never had an evaluation.
• Of the respondents 35% are unsure or not satisfied with their evaluation procedure.
• Of the unsatisfied staff, only 16% have both verbal and written components in their evaluation procedure while 72% of total respondents want both components.
• Of the unsatisfied staff, only 19% have annual evaluations, while 90% want annual evaluations and the Academic Staff Policies and Procedures require them.
• Sample comments reveal problem areas:
  • An Academic Staff member with more than 20 service has never received an evaluation.
  • Another has never had one in almost 10 years.
  • When asked about frequency of evaluations, "Other: we're are just told to write up our accomplishments for the year and never hear another thing. I have asked on numerous occasions about evaluations, raises, etc. and am told "1. no money, 2. not sure how to do it, 3. let me check, and more excuses by my boss. He also avoids the issue."
  • When asked about frequency of evaluations, one member said, "I just receive an annual written letter or re-appointment as director from the dean."
  • "I have had 3 verbal evaluations that have occurred over the period of 28 years. I have come to the conclusion that I am evaluated on an annual basis from the annual work report that is filed with the departmental chair. But I have never been informed either verbally or in writing that this is the case."
  • Frequency of evaluations: "Don't know, I've never seen an evaluation-maybe it's done and filed when merit raises are given, but I've never seen one."
  • "I have had one evaluation in 15 years."

Departments/Centers/Units
The Department/Center/Unit survey reflects several areas of concern.

• Although the request for information about Academic Staff Evaluation procedures came from the Dean's office to 41 Departments/Centers/Units, 41% (17) did not respond.
• Of the respondents, 38% (9) said that Department Faculty and/or the Department Chair, Center Director, or Departmental Executive Committees developed the Academic Staff evaluation procedure. But Chapter 10.04 of the Academic Staff Policies and Procedures (ASPP) says, "Academic staff shall participate in establishing the criteria and defining the methods of academic staff performance review to be used in the unit."
• It is quite apparent that the Academic Staff Evaluation procedures that are in place in the College of Agricultural and Life Sciences are very diverse and do not follow any kind of set protocol. Some have very detailed written procedures and others are using very loose and possibly ambiguous methods.
• The units listed some of the following limitations or drawbacks:
  • Many of the comments deal with the fact that the policies need to be more subjective in nature and it also fails to be uniform throughout the Academic Staff.
  • A problem exists since many staff are on "soft" money. Some members of the Executive Committee may not know Academic Staff members very well and as a result feel uncomfortable making any recommendations.
  • The system is dependent on faculty supervisor's initiative.
  • The major drawback is to find time to implement and sustain the process on a regular basis. When staff are under time limitations, the annual evaluation gets put on the back burner.
  • The present procedures are based on a document that was not intended for Academic Staff Evaluation.
  • Judgments about performance are totally the responsibility of the director.
  • The system is not fair in respect to evaluating academic staff who work in the field as opposed to those in the laboratory, classroom or in the computer lab.
  • It is still difficult to get the faculty to complete evaluations for all of their staff.
  • System lumps everyone into one category so they all get equal raises. Although this is what was wanted it is hard to reward anyone that is outstanding. Also different faculty will rate staff using a different basis thereby allowing some individuals to always end up as outstanding.
• Only 29% (7) units used the evaluation for improved performance.

HOW TO IMPROVE - RECOMMENDATIONS

• According to Chapter 10.01 of the Academic Staff Policies and Procedures (ASPP), "Academic staff shall be reviewed annually in a manner appropriate to their work setting and responsibilities." Since this is not presently happening, the Dean should mandate an annual evaluation for every academic staff member.

• The unit should invite academic staff to participate in developing the evaluation procedure.

• The evaluator should be the person's direct supervisor or someone who is very familiar with the staff member's work.

• The evaluation should include both verbal and written components.

• Based on our survey responses, academic staff feel the best evaluation system follows this method:
  • Staff member submits a written report of activities and accomplishments to the supervisor.
  • The supervisor and staff member meet for a discussion about goals, activities, accomplishments, career advancement opportunities, and job satisfaction and performance.
  • Based on the written report and verbal discussion, the supervisor provides a written evaluation/merit recommendation for the staff member to review.
  • The academic staff member may agree or ask for changes.
  • The supervisor may decide to change the evaluation.
  • If the supervisor decides not to change the evaluation, the staff member is invited to prepare written comments of disagreement with the evaluation.
  • The staff member signs and dates the evaluation or the comments of disagreement.
  • The supervisor then submits the written evaluation/merit recommendation with the academic staff member's attached comments to the Chair or Executive Committee for a final decision on evaluation/merit.
  • These materials are then saved in the staff member's permanent record.

• The Dean should send a copy of this report to each department.

• The Dean should then require each department to respond, comparing their existing processes to the above recommendations, including their academic staff's role in developing the process.

• The Dean should then require each department to present its performance evaluation procedures at the annual meeting that discusses salary recommendations.

---

Academic Staff Evaluation Survey

1. Department  name:

2. How long have you been an academic staff member in your current
department/unit/center?  (place an X next to or underline your answer)

less than 1 year
1-5 years
6-10 years
11-15 years
16-20 years
more than 20 years

3. Are your evaluations:

annual		every other year	never	don't know	other (explain)

4. If you are evaluated, what method is used?

written		verbal		both		other

5.  By whom?

Supervisor	Department Chair	Executive Committee   Peers	Student Evaluations

6. What are the advantages of the method used to evaluate you?
* written  ______________
* verbal   ______________
* both     _______________
* other ________________

7. What are the disadvantages?
* written  ______________
* verbal   ______________
* both     _______________
* other ________________

8.  Are you satisfied with the method of evaluation used for you?

	yes			no		unsure

9. What evaluation method would you prefer?

written		verbal		both		other (explain)	

10. How often would you like to be evaluated?

annual		every other year	never	don't know	other (explain)

11. Are you satisfied with the person who evaluates you?

	yes			no		unsure
 
12 Who would you like to evaluate you?

Supervisor	Department Chair	Executive Committee     Peers	    Student Evaluations

13. Do you have any other concerns about the evaluation methods in your department?

---

Academic Staff Evaluation Data

Background

605 Academic Staff received surveys via email.
208 (1/3) responded 
80% through email (3 emailings: 1st = 70 responses, 2nd =118, 3rd = 20)
20% through campus mail

Actual Frequency of Evaluation

More often than annual		=	 2%
Annual 				=	69%
Every other year 		=	 1%
Don't know			=	 9%
Never				=	11%
Sporadic (3-18 yrs.)		=	 8%

Desired Frequency of Evaluation

More often than annual		=	 <1%
Annual 				=	 79%
Every other year 		=	 10%
Don't know			=	  5%
Never				=	  3%
Whenever			=	  4%

Satisfaction with Current Evaluation Method

Satisfied 			= 	59%
Unsure				=	18%
Not satisfied			=	17%

Satisfied Academic Staff

Satisfied Academic Staff	=	119 (59%)

Verbal evaluations 		=	11% 
Written evaluations		=	18% 
Both verbal and written		=	66% 

Annual evaluations		=	89%
Every other year 		=	 3%
Other				=	 4%

Comments:
-As needed
-Weekly meetings
-2 in 6 years
-1 evaluation in 10 years

Unsatisfied Academic Staff	=	37 (18%)

Verbal evaluations 		=	16% 
Written evaluations		=	 8% 
Both verbal and written		=	16% 

Annual evaluations		=	19%
Every other year 		=	 0%
Other				=	46% 

Comments:
-4-5 years
-Don't know -Random, very informal
-3 evaluations in 28 years
-once in three years
-never had an evaluation, don't know why
-just receive annual written letter of reappointment
-no formal evaluation
-1 evaluation in 15 years
-student evaluations for each course taught
-never
-don't know
Other: they are not regular-very infrequent, only when absolutely necessary of a supervisor is unhappy about something
-3-4 times in 15 years
-don't know

Unsure Academic Staff		=	35 (17%)
Verbal evaluations 		=	26% 
Written evaluations		=	11% 
Both verbal and written		=	17% 

Annual evaluations		=	34%
Every other year 		=	 0%
Other				=	57% 

Comments:
-each semester
-never had evaluation
-at my own initiation
-never-informal evaluation
-not regular or formal
-Don't know (8x)
-never had one
-1 evaluation ever
-never
-I report my activities annually. But I never meet to discuss performance, nor do I know what goes into my personnel file.
-going to be better next year
-Annual (supposedly although it's been rather sporadic)

Advantages and Disadvantages of Each Evaluation Method

Verbal

Advantages

Disadvantages

face-to-face communication can ask questions informal can formulate my case chance to respond brief get immediate feedback can discuss sensitive areas without putting them into writing provides positive environment to understand confidentiality each other's expectations forces people to think about goals promotes discussion constructive exchange of ideas can't go into much depth can explain issues and performance more personal clearer understanding can be honest and direct opportunities to negotiate no written record of negatives allows exchange of emotions and body language feels good to get positive feedback

no written record I am verbally challenged can be negative too much "red tape" must stay focussed peers do not know your work, and you are judged during short interview uncomfortable with face-to-face objectives not clearly defined intimidated by supervisor afraid of reprisal feel awkward talking about my own improvements hard not to get angry when criticized difficult to schedule time to meet hard to remember points to make supervisor makes it a "chew out" session

Written

Advantages

Disadvantages

You can be as complete as necessary You can use whatever format fits your work Permanent written record Can monitor progress and refer to goals You can take your time to reflect before answering Allows for input by both parties. I get the opportunity to list all my activities.

Takes time Non-skilled writers are handicapped Self-evaluation is harsher Too much "red tape" compared to benefits Doesn't always reflect all you do Must stay focussed No face-to-face contact Confidentiality No appeal process May not fully communicate your Intent All skills are not considered No interpersonal communication Not a formalized process - no forms May never see final written statement Those who can "sell" themselves are at an advantage No feedback on how to improve Our form is poor Being evaluated by people who don't know your work. Can't include everything. Bias toward positive comments because they don't want negative comments in permanent record Scoring system encourages supervisors to inflate Merit raise is based on what is written

Review Preference

Verbal 		=	8%
Written 	=	13%
Both		=	72%
Other		=	7%

Are you satisfied with your evaluator?

Yes		=	74%
No		=	4%
Unsure		=	11%
No response	=	11%

---

Summary of Department Responses to Questions
about Academic Staff Evaluation Procedures

Departments were asked to respond to a survey inquiring about their policies for evaluating Academic Staff members of their department. The surveys written by the CASI subcommittee on Academic Staff Evaluations was sent out by Dean Kooistra's office. Of surveys to 41 units, 24 or 59% were returned.

QUESTION # 1
Name of your department?

QUESTION # 2
What procedures and criteria does your department/center/unit use in evaluating the performance of academic staff members? If there are different procedures and criteria for different types of academic staff members within the department/center/unit, or for different labs or divisions within the department, please describe the differences. (If your procedures and criteria are in writing, forward a copy of them and go to the next question.)

Of 24 respondents, 92% (22) indicated that they have some Academic Staff Evaluation procedure in place. These may or may not be formal written policies.

Of the 2 units without a procedure, one is in the process of developing a process and the other deemed the survey not applicable.

Of those with a procedure, 64% (14) Departments have formal written evaluation procedures, ranging from a one-page list of information to submit to multiple page documents. These procedures almost always involve the AS member's immediate supervisors. Most of these procedures were developed by the Academic Staff in consultation with the Department Chair or appointed Faculty members. The other 8 departments request some form of an "activity report" from their AS members and then discuss this with them.

QUESTION # 3
How were these procedures and criteria developed and by whom?

Of the respondents, 38% (9) said that Department Faculty and/or the Department Chair, Center Director, or Departmental Executive Committees developed the procedure.

Of the respondents, 42% (10) said committee of both Faculty and Academic Staff members developed the criteria.

Of the respondents, 8% (2) said academic staff developed their own evaluation criteria and procedures.

Two (8%) of the departments inherited current policies and do not know of the origins of these policies.

And one (4%) of the departments lets each person develop their own procedure for evaluating their staff. It is assumed that this means Faculty develops their own procedures for evaluating any Academic Staff they supervise.

QUESTION # 4
Based on your department's experience with the procedures and criteria, what are the benefits of your system?

Selected comments include:

• Simple easy procedure.
• Forces AS to plan goals, objectives, and activities rather than recap what was done the past year.
• Standardization for all AS members
• Chair uses judgment to ensure salary adjustments are equitable which requires chair to have broad experiences.
• Provides uniform evaluation system for all AS.
• The system in place provides for self-evaluation, feedback from supervisors, being able to set goals for the upcoming year, defining significant changes in responsibilities, and discussing professional development goals.
• Agreed evaluation between AS member and Supervisor and signed copy on file.
• No real benefits since current system is not fair to all AS members.
• The document is broad enough to capture the range of activities; the summary is open-ended and allows the AS member to express accomplishments and achievements in their own word. It requires feedback and dialogue with the supervisor and rebuttal by the AS member. It also requires that goals be set and discussed with the supervisor. The Academic Staff members developed the summary.

QUESTION # 5
What are the limitations or drawbacks?

The following are selected comments from the surveys returned.

• Many of the comments deal with the fact that the policies need to be more subjective in nature and it also fails to be uniform throughout the Academic Staff.
• A problem exists since many AS are on "soft" money. Some members of the Executive Committee may not know AS members very well and as a result feel uncomfortable making any recommendations.
• The system is dependent on faculty supervisor's initiative.
• The major drawback is to find time to implement and sustain the process on a regular basis. When staff are under time limitations, the annual evaluation gets put on the back burner.
• The present procedures are based on a document that was not intended for Academic Staff Evaluation.
• No limitations or drawbacks at this time (system been in place for two years only)
• Executive committee members often do not want to be constructively critical of fellow staff members. This then leads to problems in awarding merit pay since everyone is rated above average.
• Judgments about performance are totally the responsibility of the director.
• Maintain a level playing field, as some Faculty will "defend their own people."
• The system is not fair in respect to evaluating academic staff who work in the field as opposed to those in the laboratory, classroom or in the computer lab.
• Generally all the staff is above average in evaluations, which can hinder the basis for merit pay adjustment.
• It is still difficult to get the faculty to complete evaluations for all of their staff.
• Accounting for persons with different job responsibilities.
• Center Director makes the final salary decision. The AS has not chosen to evaluate themselves.
• System lumps everyone into one category so they all get equal raises. Although this is what was wanted it is hard to reward anyone that is outstanding. Also different faculty will rate staff using a different basis thereby allowing some individuals to always end up as outstanding.

QUESTION # 6
What changes would you make, if any?

Selected comments.

• We are considering a new system, which will allow the setting of goals for the coming year with eventual accountability for how well those goals were achieved.
• None (12)
• Prepare evaluation criteria in a printed form so AS will have a better idea of what is expected of them.
• Department will be developing a new evaluation procedure for AS this year.
• Executive committee should consider feedback from departments and Director. The rating system needs to be more realistic with clear constructive feedback both positive and negative.
• Could be a little less structured but do not change the end results.
• Forms could be refined or improved.
• No changes unless Academic staff suggests some but so far there have been no suggested changes in 9 years.
• Review biennially instead of annually.

QUESTION # 7
Are the evaluations sued for: (circle all that apply) 1. Salary merit evaluation, 2. Promotion to a higher title, 3. Increased job security, 4. Other______?

Of the 24 respondents,

1. Salary merit evaluations: 21 responses or 88%
2. Promotion to a higher title: 16 responses or 67%
3. Increased job security: 14 responses or 58 %
4. Other: 7 responses or 29%

• Used for improved performance.
• Evaluations are used to identify problems that the staff may have before the problems get out of hand. Also used to see if there is area where staff needs training.
• Used to evaluate programmatic contributions to the department.
• General feedback on the performance of the unit and staff.
• Performance feedback.
• Competitive compensation distribution.

CONCLUSION:
It is quite apparent that the Academic Staff Evaluation procedures that are in place in the College of Agricultural and Life Sciences are very diverse and do not follow any kind of set protocol. Although only a little over half (59%) of the surveys were returned it is apparent that most Department or Centers have developed their own system of Academic Staff evaluation. Some have very detailed written procedures and others are using very loose and possibly ambiguous methods.

As to the development of evaluation methods it is somewhat encouraging that 42% indicated that they sought Academic Staff input into developing them. Two of the responses indicated that their own Academic Staff developed their methods.

Although this survey is very unscientific there are some conclusions that can be drawn as to how the Academic Staff is evaluated and it may explain some of the responses on the Academic Staff Climate survey.

############

A few things that have been pulled from the survey.

Of the people who responded (208 out of a possible 605 AS), 59% were satisfied while 35 % were unsure (18%) or unsatisfied (17%). Of the AS who claimed they were satisfied, 89% were reviewed on an annual basis, and 66% received both a written and verbal evaluation as compared to verbal only 11%, written only 18%. So it seems that annual evaluations w/ written & verbal format were common threads w/ satisfied AS.

To help prove this point, of the unsatisfied and unsure staff, only 19% and 34 % respectively, received annual evaluations. Only 16% and 17%, respectively, received both written and verbal evaluations.

Our point, I think, should be that a fairly large % of the respondents was satisfied. However, a surprising number were not. Not having a formal evaluation process could contribute to this dissatisfaction because 20% of respondents don't know or have never had an evaluation of any kind. Although it is difficult to summarize individual responses, there were several examples of people either not having been evaluated or at a very low frequency. Even w/ numerous years of service. We must remember that these comments are "one side of the story". However, the comments listed below do indicate some potential problems w/ the AS evaluation process.

Faculty, as I understand are evaluated on an annual basis and this is used for merit evaluation. Sometimes this is the case w/ AS evaluations, sometimes not. Points were made by AS that they did not think there was a correlation.

There are some departments/units which the survey shows good acceptance by AS. Some common threads are that they are:

• Annual
• Include both written and verbal reports.
• Are done by executive committee or direct supervisor
• Evaluations provide a time for productive exchange of ideas and suggestions