PERTANIKA JOURNAL OF SOCIAL SCIENCES AND HUMANITIES

This research aimed to examine higher education instructors’ performance assessment in determining the cut-off point by setting criteria on the Wright map from big data. It is followed by designing performance assessment standards and assessing their quality. A total of 603 instructors from a Thai public university were selected as participants. The researchers employed a design-based research method encompassing four phases: analyzing the results of the performance assessment, formulating the standards-setting appraisal, applying trial and quality inspection, and improving the standards-setting appraisal approach. Data were analyzed using the Rasch model and the Maximum Likelihood Estimation method. The results of the determination of the cut-off point in terms of assessing instructors’ performance indicated that there are four cut-off points in ascending order, specifically, -11.67, -2.68, 4.59, and 9.76. The standards-setting appraisal showed that the assessment criteria consisted of five score ranges converted from estimation competency parameters into the scale and raw scores, respectively. Even though the standards-setting appraisal was determined, the researchers found that the transition point with regard to determination will be accurate and consistent in terms of those instructors who are at a moderate to high competency level and not suitable for evaluating those at a low competency level. The standards-setting appraisal approach is relevant for use as a criterion for recruiting and selecting higher education instructors. It can also support the development of sustainable human capital. It implies that instructors must possess high core competencies to match the high demand for quality teaching.

• Adam, R., & Khoo, S. (1996). Quest: Interactive test analysis system version 2.1. The Australian council for educational research. https://research.acer.edu.au/cgi/viewcontent.cgi?article=1002&context=measurement

• Aliasghar, M., Seyed, A. H., & Kamran, M. K. (2017). A model for competency assessment of the faculty in Islamic Azad University. International Education Studies, 10(3), 17-23.

• Alkhafaji, S. (2013). Instructor’s performance: A proposed model for online evaluation. International Journal of Information Engineering and Electronic Business, 4, 34-40. https://doi.org/10.5815/ijieeb.2013.04.05

• Altbach, P. G., Reisberg, L., & Rumbley, L. E. (2009). Trends in global higher education: Tracking an academic revolution. UNESCO Publishing.

• American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing (6th ed.). American Educational Research Association.

• Anwar, M., Chandrarin, G., Darsona, J. T., Respati, H. (2017). Lecturer job performance study: Motivation, emotional intelligence, organizational culture, and transformational leadership as antecedents with job satisfaction as an intervening. IOSR Journal of Business and Management, 19(6), 1-9.

• Baker, F. B., & Kim, S. (2017). The basics of item response theory using R. Springer.

• Blašková, M., & Blaško, R. (2012). Dimensions and attributes of the university teacher quality. In Human potential management in a company (pp. 32-43). University of Matej Bel.

• Blašková, M., Blaško, R., Jankalová, M., & Jankal, R. (2014). Key personality competencies of university teacher: Comparison of requirements defined by teachers and/versus defined by students. Procedia – Social and Behavioral Sciences, 114, 466-475.

• Bowen, D. E., & Ostroff, C. (2004). Understanding HRM-firm performance linkages: The role of the ‘strength’ of the HRM system. The Academy of Management Review, 29, 203-221. https://doi.org/10.5465/amr.2004.12736076

• Cardoso, S., Tavares, O., & Sin, C. (2015). The quality of teaching staff: Higher education institutions’ compliance with the European standards and guidelines for quality assurance – The case of Portugal. Educational Assessment Evaluation and Accountability, 27(3), 205-222. https://doi.org/10.1007/s11092-015-9211-z

• Caruth, D. L., & Humphreys, J. H. (2008). Performance appraisal: Essential characteristics for strategic control. Measuring Business Excellence, 12(3), 24-32. https://doi.org/10.1108/13683040810900377

• Curzi, Y., Fabbri, T., Scapolan, A. C., & Boscolo, S. (2019). Performance appraisal and innovative behavior in the digital era. Frontiers in Psychology, 10, 16-59. https://doi.org/10.3389/fpsyg.2019.01659

• Embretson, S. E., & Reise, S. P. (2000). Item response theory of psychologists. Erlbaum.

• Gómez, L. F., & Valdés, M. G. (2019). The evaluation of teacher performance in higher education. Propósitosy Representaciones, 7(2), 479-515. https://doi.org/10.20511/pyr2019v7n2.255

• Hosain, M. S. (2016). Teaching workload and performance: An empirical analysis on some selected private universities of Bangladesh. International Journal of English and Education, 5(3), 1-11.

• Islam, R., Haidoub, I. M., & Tarique, K. M. (2019). Enhancing quality of education: A case study on an international Islamic university. Asian Academy of Management Journal, 24(1), 141-156.

• Junpeng, P., Marwiang, M., Chiajunthuk, S., Suwannatrai, P., Chanayota, K., Pongboriboon, K., Tang, K. N., & Wilson, M. (2020). Validation of a digital tool for diagnosing mathematical proficiency. International Journal of Evaluation and Research in Education, 9(3), 665-674. https://doi.org/10.11591/ijere.v9i3.20503

• Khon Kaen University. (2015). Criteria and methods for evaluating the performance of personnel. Khon Kaen University printing house.

• Kurtulmus, B. E., Warner, B., & Özari, Ç. (2016). Research or teaching oriented? Game theory models for the strategic decision-making process of universities with the external environment held neutral. Electronic Journal of Applied Statistical Analysis, 9(3), 469-490.

• Laei, S., Abdi, A., Karamaerouz, M. J., & Shirkhani, N. (2014). Instructors’ evaluation as an instrument to improve performance and determine competence. Universal Journal of Educational Research, 2(2), 110-118. https://doi.org/10.13189/ujer.2014.020202

• Le, L. C. (2021). Assessing lecturer competence: A case study of public universities in Ho Chi Minh City. Academy of Strategic Management Journal, 20(S2), 1-12.

• Lohman, L. (2021). Evaluation of university teaching as sound performance appraisal. Studies in Educational Evaluation, 70, Article 101008. https://doi.org/10.1016/j.stueduc.2021.101008

• Lunz, M. E. (2010). Measurement research associates test insights. https://www.rasch.org/mra/mra-01-10.htm

• Masters, G. N. (2005). Partial credit model. Encyclopedia of Social Measurement, 7-17. https://doi.org/10.1016/BO-12-369398-5/00462-X

• Molefe, G. N. (2010). Performance measurement dimensions for lecturers at selected universities: An international perspective. SA Journal of Human Resource Management, 8(1), Article 243. https://doi.org/10.4102/sajhrm.v8i1.243

• Nongna, C., Junpeng, P., Hong-ngam, J., Podjana, C., & Tang, K. N. (2021). Creating core competencies and workload-based outcome indicators of university lecturers’ performance assessment: Functional analysis. Journal of Education and Learning, 10(6), 82-91. https://doi.org/10.5539/jel.v19n6p82

• Odero, J. A., & Makori, E. M. (2018). Employee involvement and employee performance: The case of part time lecturers in public universities in Kenya. International Journal of Management and Commerce Innovations, 5(2), 1169-1178.

• Office of the Civil Service Commission. (2009). Guide to core competencies. P. A. Living.

• Office of the Higher Education Commission. (2018). Guidelines for enhancing the quality of teaching and learning management of instructors in higher education institutions. Prints.

• Prasetio, A. P., Azis, E., Fadhilah, D. D., & Fauziah, A. F. (2017). Lecturers’ professional competency and students’ academic performance in Indonesia higher education. International Journal of Human Resource Studies, 7(1), 86-93.

• Reeves, T. C. (2006). Design research from a technology perspective. In J. V. D. Akker, K. Gravemeijer, S. McKenney, & N. Nieveen (Eds.), Educational design research (pp. 52-66). Routledge.

• Rossi, R. J. (2018). Mathematical statistics: An introduction to likelihood-based inference. John Wiley & Sons.

• Selvi, K. (2010). Teachers’ competencies. Cultura International Journal of Philosophy of Culture and Axiology, 7(1), 167-175. https://doi.org/10.5840/cultura20107133

• Tan, S., Lau, E., Ting, H., Cheah, J. H., Simonetti, B., & Tan, H. L. (2019). How do students evaluate instructors’ performance? Implication of teaching abilities, physical attractiveness and psychological factors. Social Indicators Research, 146, 61-76. https://doi.org/10.1007/s11205-019-02071-6

• Turturean, M. (2013). University trainees’ key competencies – a global profile. Procedia – Social and Behavioral Sciences, 76, 801-805.

• Vongvanich, S. (2020). Design research in education. Chulalongkorn University Printing House.

• Ward, M. D., & Ahlquist, J. S. (2018). Maximum likelihood for social science: Strategies for analysis. Cambridge University Press.

• Wilson, M., Allen, D. D., & Li, J. C. (2006). Improving measurement in health education and health behavior research using item response modelling: Comparison with the classical test theory approach. Health Education Research, 2(1), i19-i32.

• Wu, M. L., Adams, R. J., Wilson, M. R., & Haldane, S. A. (2007). ACER Con Quest 2.0: Generalized item response modelling software. ACER Press.