An Appraisal Mechanism for Improving Teaching Service Delivery in Universities

Most universities have set up appraisal of lecturers’ class competencies to be done by students. The setbacks have been that students don’t feel free to give honest assessment, difficulty in collating the appraisal results, lecturers not trusting an appraisal system done solely by only students, and the assessment not considering positive or negative factors affecting teaching delivery. These taken together, have led to de-emphasis on teaching delivery appraisal. To solve this, questions that better reflect what defines of a good lecturer were modeled to achieve validity. The system was set up to protect anonymity of the students appraising staff. The work also presents a mathematical model that factors in the complexity of teaching to achieve fairness. An interaction of three weights (class size, credit unit and number of lecturers taking the course) was used to attain fairness and foster lecturers’ trust in this single-source appraisal. The grand or final teaching delivery score is a weighted average of the assessment scores computed from each of the courses taken by the appraised lecturer in the year under consideration. A web-based application was developed to deploy the appraisal mechanism advanced in this work. The application proved an effective platform for easy collation of appraisal outcome, roll out of questions covering multiple aspects of teaching delivery that improve learning and assessable by students. The interaction of three weights ensured that this single-source appraisal is not tilted unduly against the appraised staff while not unmasking the evaluators. The system is a module in a broader human resource management system for universities.



Abstract-Most universities have set up appraisal of lecturers' class competencies to be done by students. The setbacks have been that students don't feel free to give honest assessment, difficulty in collating the appraisal results, lecturers not trusting an appraisal system done solely by only students, and the assessment not considering positive or negative factors affecting teaching delivery. These taken together, have led to de-emphasis on teaching delivery appraisal. To solve this, questions that better reflect what defines of a good lecturer were modeled to achieve validity. The system was set up to protect anonymity of the students appraising staff. The work also presents a mathematical model that factors in the complexity of teaching to achieve fairness. An interaction of three weights (class size, credit unit and number of lecturers taking the course) was used to attain fairness and foster lecturers' trust in this single-source appraisal. The grand or final teaching delivery score is a weighted average of the assessment scores computed from each of the courses taken by the appraised lecturer in the year under consideration. A web-based application was developed to deploy the appraisal mechanism advanced in this work. The application proved an effective platform for easy collation of appraisal outcome, roll out of questions covering multiple aspects of teaching delivery that improve learning and assessable by students. The interaction of three weights ensured that this single-source appraisal is not tilted unduly against the appraised staff while not unmasking the evaluators. The system is a module in a broader human resource management system for universities.
Index Terms-Appraisal Mechanism, Teaching Service.

I. INTRODUCTION
In human resource management, development is unattainable in competencies not properly assessed. Of the three professional obligations placed upon college professors-commonly defined as teaching, research, and service-teaching is perhaps the most complex and difficult to evaluate [1].
In trying to emphasize a 'non-research' category of academic staff performance evaluation like teaching, administrators of institutions have employee training, something considered in [2] as rudimentary to the best Human Resource Management Systems (HRMSs). Institution heads, however, are not able to evaluate how much of these advantages have been gained, especially regarding the impact of employee training in teaching delivery. In [3] is presented students' evaluation of teacher's effectiveness (SETE) as a tool in assessing universities' and colleges' academic staff teaching delivery. They stated areas of teaching delivery assessment to include in-class attitudes and behaviors towards students, and ability of the lecturer to create the required conducive emotional climate. In [4] is recommended Emotional Skill Assessment Process (ESAP) as a major tool suit that predicts lecturers' teaching effectiveness in Universities. ESAP has four components: personal leadership skills, self-management skills, interpersonal skills and intrapersonal skillsthe first two being the most important predictors [4]. There has to be a way of incorporating SETE into staff appraisal system in order to underscore teaching delivery as much as research. So far, the best attempts at this has been the issuance of questionnaires to students to rate their lecturers.
There have been some setbacks. First, questions on such questionnaires administered mostly do not represent what defines a good lecturer. [5]; and [6] in line with this posit that the most crucial consideration in teaching service appraisal is that the evidence collected be useful in advancing the professional development of a lecturer being assessed and improve learning.
Secondly, students do not generally feel free enough to give honest rating. Student assessors usually have to provide their unique identifiers (e.g. registration numbers) to help with de-duplication of assessments i.e. ensuring no student appraises a lecturer more than once on a particular course, in a given appraisal year. It is the supply of this unique identifiers that makes students fear that the confidentiality of their survey response will not be protected. There is therefore the need for students' appraisal of lecturers' competencies to be confidential while also avoiding duplicate assessments.
Thirdly, too often staff justifiably lose trust in appraisal system that relies too heavily on a single measure, such as student ratings [7]. Leading [1] to include fairness as one of the conditions of achieving trustworthiness of academic staff appraisal by students. Fairness is explained as protecting the assessed by ensuring that the measures collected adequately represents complexity of teaching [1]. The scoring system adopted in appraising teaching delivery of academic staff should take into consideration factors that positively or negatively affect classroom effectiveness of lecturers. Such factors include class size, course credit unit (or hours of lectures in a week), and if or not there are co-lecturers for a particular course in which a lecturer is assessed on.
Lastly, to avoid bias, the time of administering the assessment needs to be carefully chosen. For instance, students tend to assess lecturers based on the how good their exam performances/scores were.
The system for students' appraisal of lecturers' class competencies developed/proposed in this work will therefore address the inefficiency in collecting and collating An Appraisal Mechanism for Improving Teaching Service Delivery in Universities Bliss U. Stephen, Gloria Chukwudebe, Damian O. Dike, and Nkwachukwu Chukwuchekwa teaching appraisal of academic staff. Questions that better reflect what makes a good lecturer will be modelled, and a scoring system will be developed for teaching service delivery developed. To attain credible appraisal results, a mechanism will be incorporated to protect assessors' (students') anonymity so that they can give unreserved assessment, and period for the appraisal during a semester will be recommended.

II. LITERATURE REVIEW
Identified in [4] are components of emotional intelligence that significantly impact on lecturers' teaching effectiveness for public and private universities in Malaysia. Those components of emotional intelligence include personal leadership skills, interpersonal skills, self-management skills and intrapersonal skills. Used in the research was a structured self-administered questionnaire to 316 lecturers at two public universities and two private universities. Although the work established that the impact of selfmanagement skills and personal leadership skills on teaching effectiveness for private universities is somewhat higher than public universities, it generally proved that no component of emotional intelligence is of negligible impact on teaching effectiveness. The work explained each component of emotional intelligence but did not make recommendations on how such assessment can be carried out either by formulating questions for assessments, recommending assessors, or ideal timing for such assessment.
In [8] are modelled the most valued qualities of university lecturers. Specifically, a common list of the teacher qualities most valued by on-line and on-site university students was established in the work. The qualities range from closeness to students and command of subject matter as the most valued qualities down to openness and teacher's image as the least valued qualities.
Some tertiary institutions have made provisions for students' assessment of lecturers' class effectiveness in their appraisal guidelines. In [9], for instance, is evaluation questionnaire for appraisal of teaching delivery, but the instrument lacks a little in simplicity. Students generally do not know attributes that mark a good lecturer, and therefore should not be asked directly if a lecturer is good or not.

III. METHODOLOGY
The Students' Assessment of Lecturers' Competencies (SALC) system developed in this work was required to carry out students' appraisal of lecturers' competencies in class for easy incorporation into overall appraisal in Federal University of Technology, Owerri, Nigeria. It was also a necessity for this application to protect anonymity/facelessness of the assessors while ensuring nonduplicate assessments. Therefore, the development of this application was done in two stages: question formulation for assessment of lecturers' competencies and coding of the SALC application, discussed next.
Question formulation for assessment of lecturers' competencies.
For the aspects of this assessment, questions were modelled to capture aspects of emotional intelligence highlighted in [4] and [8]. However, intrapersonal skills assessment was not considered in this work as it really isn't assessable by students. Intrapersonal skills have to do with self-esteem, work-family conflict and stress management. Also, forced choice approach (with maximum of three answer choices) was adopted to minimize as much as possible rating errors. Thus, enhancing integrity of student rating of lecturers. In furtherance of this objective, this work recommends that the assessment be done prior to period stipulated for first continuous assessment in school calendar. This is because students tend to adjudge a lecturer as good if they performed well in the continuous assessment or exams.
Next are the questions formulated for each: Answer: The next lecture/Two weeks/Longer than two weeks xvi. Does he/she look happy to be in class?
Answer: Always/Sometimes/Never xvii. Does he/she teach with enthusiasm?

Answer: Yes/No
Coding of the SALC System The system was developed with three major requirements: Mathematical model for the computation of SALC score, protecting anonymity of student/assessors, and de-duplication of assessment. These are explained next.

D. Mathematical model for the computation of SALC score
Every question has a maximum of 1 point. For two option questions, YES was assigned '1' while NO was assigned '0'. This is because the questions with YES or NO answers were formulated such that YES represents what's expected of the lecturer, and NO represents a totally unacceptable scenario. Not assigning it any fraction of the maximum score was to discourage complete lack of such competencies. The value assigned to an option is what is returned to the database.
For questions with three options, the first option in the drop down menu, was assigned '1', being the most desirable of the three. The second option was assigned '0.5', being the midpoint between most desirable and most undesirable. The last option was assigned '0'. Take for example the third question in self-management skills appraisal. ALWAYS was assigned 1 point, SOMETIMES was assigned 0.5 points and NEVER assigned 0 points.
In computing the score per assessment, the scores from all the questions are averaged as shown in (1).

= ∑ =1
(1) Expanding equation (1) gives, where n is the total number of questions on the survey form q1 is the score in first question q2 is the score in second question… up to the nth question S is the score from each student making the assessment Next, was the computation of an assessment score (course aggregate score, A) from all the student-assessors i.e.at the completion of the survey. This is given by, Expanding equation (3) gives, where A is the course aggregate score from all the studentassessors p is the number of student that participated in the assessment/survey S1 is the score from the first student S2 is the score from the second student … up to the pth (last) student Note that course aggregate score, A, is computed per course taken by the lecturer under assessment during the academic session under review.
Having computed the aggregate assessment score, A, from assessments made by all the students that did the survey, next was the computation of grand score, G, which is the final Students' Assessment of Lecturers' Competencies (SALC) score for a particular lecturer in the academic session under review. Grand SALC score, G, is the weighted average of his course aggregate scores (A1, A2, …, Ak) from all the courses assigned to him, multiplied by maximum score, T, the school assigns to teaching service delivery in the academic session under review.
An interaction of three factors were used for the weighting: i. Credit unit of all the courses taken by the lecturer during the academic year under review; ii. Class size or number of students taking the course; and, iii. Number of lecturers taking each of the courses the lecturer under assessment. These factors each have impact on the three aspects of students' assessment of lecturers' competencies considered in this work i.e. interpersonal skill, personal leadership, and self-management skills. They were introduced to achieve fairness as part of standard practice recommended in [1].
Since increment in the first two factors (credit units, w, and class size, c) tend to increase the lecturers' work load and reduce their effectiveness, they were made multiplicative. On the other hand, number of lecturers, x, taking the course was made divisive since workload tend to decrease and effectiveness step up as the number of lecturers assigned to a course increases.
The mathematical expression of this interaction of these where c is the weighting by class size categorization w is the credit unit of the course under assessment x is the number of lecturers taking the course It is important to note that class size categorization adopted is in tandem with 1:15 lecturer-to-student ratio recommended by Nigerian University Commission (NUC). Meaning that adopted standard class size was 15. The categorization is as follows: i.
Expanding equation (5) where 1 is the lecturer's aggregate score in his first course 2 is the lecturer's aggregate score in his second course… up for his kth (last) course 1 is the credit unit of his first course 2 is the credit unit for his second course . . . up to being credit unit of his kth (last) course 1 is the number of lecturers taking his first course (i.e. him and co-lecturers of the course) is the number of lecturers taking his kth (last) course T is the maximum score approved for teaching service delivery In order to make sure that teaching service appraisal score does not dovetail research appraisal score by too much, this work recommends that the value of T be equivalent to at least the maximum score attainable with one publication. One publication because appraisal guidelines in some universities stipulate that an academic staff must have at least one publication every year.

E. Protection of Assessors' Anonymity and Deduplication of assessment
The coding process for these functionality will be explained using the sequence diagram in Fig. 1. On the SALC interface, the student first provides details of the course he/she wants to assess a lecturer in, namely course code, course title, semester and name of lecturer handling the course. It is important to note that the assessment is done for every course. Next, the student inputs his registration number, which is where the sequence diagram in Fig. 1 begins from. With the supplied student registration number (Reg. No.), the SALC page links a javascript file (fujs.js) which references a function, 'getStudentDet' which then calls a php file 'getStuDetails.php'. It is the 'getStuDetails.php' file that receives the registration number keyed in. The registration number is secured from hacking using 'escape_string' function, while the encryption is done by 'encryptpasswrd' function. These two functions are called by 'getStuDetails.php' but are actually contained in an Application Programming Interface (API) named 'IntelligentPHPAPI.php'. The codes for getStuDetails.php' are shown in Fig. 2. The registration number encryption was necessary in order for the identity (registration number) not to be viewable in intelligible/readable form in the database.
The protection was necessary to make the registration number irretrievable from the database. This is how the SALC application protects the anonymity of the students/assessors.  ; // get the reg no. from the user $RegNum1 = $obj->escape_String($RegNum); //secure the reg no. from hacker injection to db $getHashPass = $obj->encryptpasswrd($RegNum1); //encrypts Reg. no. The value of $getHashPass will be sent to the db echo $getHashPass; } ?> After the password encryption and protection, what happens next is that eligibility of student/assessor is checked at two levels: first, if he/she has registered for the course, and secondly if he/she has done the assessment already for that course. If he is not registered for the course, an error message is displayed disallowing progress in the assessment process. If he is registered, the process of avoiding multiple assessment by any student is carried out. If the assessor had done the survey already, a notice of ineligibility is displayed, telling the student he/she cannot proceed with the assessment. Otherwise, the student is allowed to do the survey and the score saved in the database.

IV. RESULTS
Presented in this section are screenshots showing a walk through of the SALC app. As shown in Fig. 3, the student/assessor begins by selecting his/her faculty, department, current session of study, level and most importantly, registration number (Reg. No.). This registration number is a student's unique identifier which is used solely to check if the student is registered for the course he wants to assess a lecturer on, and restrict each student to one assessment per lecturer, per course. This registration number is encrypted and protected in the database order to protect student identity. Clicking on 'Course Details' (shown in Fig. 3) enables user to input information on the course: course code, course title, semester and name of course lecturer in Fig. 4.
Next, the assessor clicks on 'Evaluation Questions' to begin answering the SALC questions. The evaluation questions are shown in Fig. 5. User scrolls down to reveal more questions. The lower part of the questions is shown in Fig. 6. Fig. 6 also shows the 'Submit' button that is clicked on completion of the survey to return the assessment score to the database.

V. CONCLUSION
The mechanism developed in this work for students' appraisal of lecturers' teaching service delivery achieved standard requirements for students' appraisal of lecturers' class competencies. The questions formulated achieved reliability by covering multiple aspects of teaching delivery assessable by on-line and off-site students and relevant to improvement of course delivery both in private and public universities. The question also attains validity by making sure that these aspects can directly improve student learning. The interaction of three weights (of class size, credit hours and number of lecturers co-assigned the course) ensured that the use of students as the sole means of teaching service appraisal does not skew the evaluation in disfavor of the appraised staff. Protection of anonymity of assessors helped the integrity of the assessment. The work ultimately addresses problems underlying the de-emphasis of teaching service appraisal.