Analysis Of Score Changes And Ceiling Effects On The Asn Integrity Training Test: Implications For The Ministry Of Transportation's Learning Evaluation
DOI:
https://doi.org/10.56457/tjm.v4i2.417Keywords:
civil servant integrity, training evaluation, score change, ceiling effect, corporate universityAbstract
Civil servant integrity training requires evaluation frameworks that distinguish recorded cognitive test performance from ethical workplace behavior. This study examines score changes and ceiling effects in the ASN Berintegritas training program and their implications for learning evaluation within Indonesia’s Ministry of Transportation. A retrospective quantitative design analyzed Learning Management System (LMS) records from March 18 to September 3, 2026. The primary analysis involved 24,174 paired participant records after excluding 38 invalid time-sequence pairs. Data were processed in R using descriptive statistics, paired t-tests, and McNemar’s test. Mean scores increased significantly from 90.25 to 95.27, representing a 5.02-point gain (95% CI: 4.85–5.19; p < .001). However, the median individual score difference was 0.00: 41.49% of participants improved, 41.10% remained unchanged, and 17.41% declined. The proportion of maximum scores (100) expanded from 43.18% at pretest to 57.30% at posttest (p<.001). High upper-end score concentration indicates a substantial ceiling effect that constrains the measurement of true learning gains. Furthermore, the lack of a control group and test-equating evidence precludes causal conclusions about training efficacy or the development of behavioral integrity. These findings highlight the need for Kemenhub Corporate University to refine question banks, implement Situational Judgment Tests (SJTs), and integrate cognitive assessments with workplace behavioral observations and Zona Integritas frameworks.
References
Alliger, G. M., Tannenbaum, S. I., Bennett, W., Jr., Traver, H., & Shotland, A. (1997). A meta-analysis of the relations among training criteria. Personnel Psychology, 50(2), 341–358. https://doi.org/10.1111/j.1744-6570.1997.tb00911.x
Barnett, A. G., van der Pols, J. C., & Dobson, A. J. (2005). Regression to the mean: What it is and how to deal with it. International Journal of Epidemiology, 34(1), 215–220. https://doi.org/10.1093/ije/dyh299
Blume, B. D., Ford, J. K., Baldwin, T. T., & Huang, J. L. (2010). Transfer of training: A meta-analytic review. Journal of Management, 36(4), 1065–1105. https://doi.org/10.1177/0149206309352880
Demir, T., Reddick, C. G., & Perlman, B. J. (2023). In search of ethics infrastructure in U.S. local governments: Building blocks or dead end? Administration & Society, 55(10), 1866–1892. https://doi.org/10.1177/00953997231190571
Febrina, R., & Ekowati, A. (2026). Facilitator competencies gap analysis in the implementation of distance learning model for CPNS training in Ministry of Transportation. Kontingensi: Jurnal Ilmiah Manajemen, 14(1), 378–387.
Hughes, A. M., Zajac, S., Woods, A. L., & Salas, E. (2020). The role of work environment in training sustainment: A meta-analysis. Human Factors, 62(1), 166–183. https://doi.org/10.1177/0018720819845988
Kementerian Pendayagunaan Aparatur Negara dan Reformasi Birokrasi. (2017). Peraturan Menteri Pendayagunaan Aparatur Negara dan Reformasi Birokrasi Nomor 38 Tahun 2017 tentang Standar Kompetensi Jabatan Aparatur Sipil Negara. https://peraturan.bpk.go.id/Download/123418/PERMENPAN%20NOMOR%2038%20TAHUN%202017.pdf
Kementerian Pendayagunaan Aparatur Negara dan Reformasi Birokrasi. (2021). Peraturan Menteri Pendayagunaan Aparatur Negara dan Reformasi Birokrasi Nomor 90 Tahun 2021 tentang Pembangunan dan Evaluasi Zona Integritas Menuju Wilayah Bebas dari Korupsi dan Wilayah Birokrasi Bersih dan Melayani di Instansi Pemerintah. https://peraturan.bpk.go.id/Details/202223/permen-pan-rb-no-90-tahun-2021
Kementerian Pendayagunaan Aparatur Negara dan Reformasi Birokrasi. (2024). Peraturan Menteri Pendayagunaan Aparatur Negara dan Reformasi Birokrasi Nomor 5 Tahun 2024 tentang Perubahan atas Peraturan Menteri Pendayagunaan Aparatur Negara dan Reformasi Birokrasi Nomor 90 Tahun 2021 tentang Pembangunan dan Evaluasi Zona Integritas Menuju Wilayah Bebas dari Korupsi dan Wilayah Birokrasi Bersih dan Melayani di Instansi Pemerintah. https://peraturan.go.id/id/permenpanrb-no-5-tahun-2024
Kementerian Perhubungan. (2025). Peraturan Menteri Perhubungan Republik Indonesia Nomor PM 4 Tahun 2025 tentang Organisasi dan Tata Kerja Kementerian Perhubungan. https://peraturan.go.id/id/permenhub-no-4-tahun-2025
Lakens, D. (2013). Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. Frontiers in Psychology, 4, 863. https://doi.org/10.3389/fpsyg.2013.00863
Perlman, B. J., Reddick, C. G., Demir, T., & Ogilby, S. M. (2021). What do local governments teach about in ethics training? Compliance versus integrity. Journal of Public Affairs Education, 27(4), 490–507. https://doi.org/10.1080/15236803.2021.1972742
Reddick, C. G., Demir, T., & Perlman, B. J. (2025). A categorical framework of ethical leadership for public organizations: Testing a multi-dimensional model of local government managers. Administration & Society, 57(1), 3–33. https://doi.org/10.1177/00953997241291833
Republik Indonesia. (2023). Undang-Undang Republik Indonesia Nomor 20 Tahun 2023 tentang Aparatur Sipil Negara. https://peraturan.go.id/id/uu-no-20-tahun-2023
Šimkovic, M., & Träuble, B. (2019). Robustness of statistical methods when measure is affected by ceiling and/or floor effect. PLOS ONE, 14(8), e0220889. https://doi.org/10.1371/journal.pone.0220889
Staus, N. L., O’Connell, K., & Storksdieck, M. (2021). Addressing the ceiling effect when assessing STEM out-of-school time experiences. Frontiers in Education, 6, 690431. https://doi.org/10.3389/feduc.2021.690431
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Rima Febrina, Afsuni Wijananto, Dyan Apriyani, Siswantoyo Siswantoyo
This work is licensed under a Creative Commons Attribution 4.0 International License.
