Statistical Tools for Measuring the Quality of Education. Part 4. Modern approach
Abstract
The article presents an overview of the statistical grounds of testology. The purpose of this paper is to explain the in experienced readers, such as teachers of economic disciplines, opportunities of improvement of quality of education with the utilization of objective andimpartial tools of students’ achievement measurement, known as tests.The first part of the article overviews the shortcomings of the traditional system of evaluation of educational achievements that is built around the use of ordinal scales. The limitations imposed to the possibilities of statistical processing of the raw data by the type of scale are shown. Basic tasks, the corresponding mathematical models, statistical characteristics and sub-test score evaluation reliability formulas are described.The second part of the article describes approaches to the determination of reliability, uniformity and resolution ofthe test, built on the analysis of the correlation between students’answers to the identical questions asked. Options of conversion of primary points to a quantitative scale are provided. Ways of lowering the probability of correctly guessed an swersare shown. The approach to processing of results of complex test structures is given and the possibility of utilization of two-factor analysis of variance (2 WayANOVA) for dichotomoustests reliability estimation is demonstrated.The third and the fourth parts of the article are devoted to the modern theory of tests(IRT).The third part provides an analysis of shortcomings of the CTT, which were the mainfocus of efforts to overcome of IRT supporters during the last 60 years. The theoretical basis for building a Rach model and its subsequent developments is described. The methodology of estimation of properties of the test by its characteristic curves and parameters of its information function is illustrated. The basic equation, the correspondent solution of which gives an estimate of the probability of obtaining a certain personal score of a test is formulated.The fourth part of the article provides various methods of finding a solution of the basicequation for the 1PL and 2PL - models and data preparation for a correct use. Several software packages, both considered to be classical tools as well as brand new ones, are overviewed. An example of ranking of NASO A students’ achievements obtained by traditionalevaluation and IRTapproach is given.
Downloads
References
2. Сіницький М. Є. Статистичні інструменти вимірювання якості освіти. Ч. 2. Класичний підхід // Науковий вісник НАСОА. - 2015. - № 1 - С. 75-86.
3. Rash G. On Objectivity and Specificity of the Probabilistic Basis for Testing [Electronic resource]. - Access mode: http://www.rasch.org/memol96x.pdf.
4. Нейман Ю. М. Введение в теорию моделирования и параметризации педагогических тестов / Ю. М. Нейман, В. А. Хлебников. - М. : Прометей, 2000. - 168 с.
5. Крокер JI. Введение в классическую и современную теорию тестов : [учебник] / JI. Крокер, Дж. Алгина; пер. с англ. Н. Н. Найденовой, В. Н. Смилкина, М. Б. Челышковой ; под общ. ред. В. И. Звонникова, М. Б. Челышковой. - М. : Логос, 2010.-668 с.
6. Аванесов В. С. Item Response Theory: Основные понятия и положения [Электронный ресурс]. - Режим доступа : testolog@mail.ru.
7. Bryce В. Reeve. An Introduction to Modem Measurement Theory. - Outcomes Research Branch Applied Research Program, Division of Cancer Control and Population Sciences [Electronic resource]. - Access mode: http://www.appliedresearch.cancer.gov/archive/.../immt.pdf.
8. Lord F. M. Application of Item Response Theory to Practical Testing Problems / F. M. Lord. - Hillsale, NJ: Lawrence Erlbaun Associates, 1980.
9. Bimbanm A. Some Latent Trait Models and Their Use in Imferring an Examinee’s Ability / A. Bimbahn // F. M. Lord, M. P. Novick. Statistical Theories of Mental Test Scores. - Reading MA : Addison-Wesley, 1968.
10. Linden W. Handbook of Modem Item Response Theory / W. Linden, R. Habbleton. - NY. : Springer-Verlag, 1997.
11. Bock R. D. Marginal Maximum Likelihood Estimation of Item Parameters: An Application of the em Algorithm / R. D. Bock, M. Aitkin // Psychometrika. -1 9 8 1 ,- No 46. - P. 443—459.
12. Item Response Theory [Electronic resource]. - Access mode : www://en.wikipedia.org/ wikHtemresponsetheory# Information.
13. Sijtsma K. Item Response Theory: Past Performance, Present Developments, and Future Expectations / K. Sijtsma, B. W. Junker // Behaviometrika. - 2006. - Vol. 33, No 1. - P. 75-102.
14. Masters G. N. A Rasch Model for Partial Credit Scoring / G. N. Masters // Psychometrika. - 1982. - Vol. 47, No 1. - P. 149-174.
15. Bartholomew D. The Analysis and Interpretation of Multivariate Data for Social Scientists / D. Bartholomew, F. Steele, I. Moustaki, J. Galbraith. - Charman & Hall: London, 2002.
16. Skrondal A. Generalized Latent Variable Modeling: Multilevel, Longitudinal and Structural Equation Models / A. Skrondal, S. Rabe-Hesketh. - Charman & Hall: Boca Ration, FL, 2004.
17. Лоусон Ч. Численное решение задач наименьших квадратов / Ч. Лоусон, Р. Хенсон; пер. с англ. - М. : Наука. Гл. ред. физ.-мат. лит., 1986. - 232 с.
18. Карданова Е. Ю. Моделирование и параметризация тестов: основы теории и приложения / Е. Ю. Карданова. - М. : Федеральный центр тестирования, 2008.-303 с.
19. Verhelst N.D. Modeling Sums of Binary Responses by Partial Credit Model / N. D. Verhelst. - H. H. F. M. Verstalen, Measurement and Research Department Reports 97-7. - Cito, Arnhem, 1997.
20. Muraki E. A. Generalized Partial Credit Model: Application of an EM Algorithm / E. A. Muraki //Applied Psychological Measurement. - 1992. -N o 16(2). - P. 159-176.
21. Елисеев И. H. Теоретические основы алгоритма расчета латентних переменных программным комплексом RILP - 1М / И. Н. Елисеев // Программные продукты и системы. - 2011. - № 2. - С. 67-72.
22. Cohen L. Approximate Expression for Parameter Estimates in the Rasch Model / L. Cohen // British Journal of Mathematical and Statistical Psychology. - 1979. - Vol. 32, No l . - P . 113-120.
23. Wright B.D. Best Test and Self-Tailored Testing / B. D. Wright, G. A. Douglas. - Research Memorandum No 19. - Chicago : MESA Press, 1975 [Electronic resource]. - Access mode : www.rasch.org/nnt/nntl02q.htm/.
24. Linacre J. M. PROX with Missing Data, or Known Item or Person Measures / J. M. Linacre // Rasch Measurement Transactions. - 1994. - Vol. 8, [Electronic resource]. -Accessmode: www.rasch.org/nnt/nntl22q.htm/.
25. Linacre J. M. Estimating Rasch Measures with Known Polytomous Item Difficulties / J. M. Linacre // Rasch Measurement Transactions. - 1998. - Vol. 12, [Electronic resource]. -Accessmode: www.rasch.org/nnt/nntl22q.htm/.
26. Linacre J. M. Dichotomous Mean-square Chi-square fit statistics / J. M. Linacre, B. D. Wright // Rasch Measurement Transactions. - 1994. - Vol. 8, No 2. - P. 360.
27. Wright B.D. Computation of OUTFIT and INFIT Statistics / B.D. Wright, G. N. Masters // Rasch Measurement Transactions. - 1990. - Vol. 3, [Electronic resource]. – Access mode: www.rasch.org/nnt/nnt34e.htm/.
28. Linacre J. M. PROX for Polytomous Data / J. M. Linacre // Rasch Measurement Transactions. - 1995. - Vol. 8, [Electronic resource]. - Access mode: www.rasch.org/nnt/nnt84q.htm/.
29. Hambleton R. K. Fundamentals of Item Response Theory / R. K. Hambleton, H. Swaminathan, H. J. Rogers. - Sage Publications Inc. London, 1991, Chapter 2, p. 9-12 and Exercise 6. - P. 29-31.
30. Tristan A. Chi-square Local Independence Meets the Rasch Model / A. Tristan // Rasch Measurement Transactions. - 2002. - Vol.16, No l . - P . 861.
31. Rizopoulos D. Ltm: An R Package for Latent Variable Modeling and Item Response Theory Analyses / D. Rizopoulos // Journal of Statistical Software. - Nov. 2006. - Vol. 17, Issue 5 [Electronic resource]. - Access mode : http://www.jstatsoft.org/.
32. Латентно-структурный анализ и теория тестов / Математические методы в социальных науках // Под ред. П. Лазарсфельда, Н. Генри; пер. с англ. - М .: Прогресс, 1973. - 297 с.
33. Bock R. D. Full-Information Item Factor Analysis / R. D. Bock, R. D. Gibbons, E. Muraki // Applied Psychological Measurement. - 1988. - No 12. - P. 261-280.
34. Gibbons R. D. Full-infonnation item bi-factor analysis / R. D. Gibbons, D. Hedeker // Psychometrika. - 1992. - Vol. 57. - P. 423—436.
35. Vermunt J. K. Latent Class Cluster Analysis / J. K. Vermunt, J. Magidson // J. A. Hagenaars, A. L. McCutcheon (eds.). Advances in Latent Class Analysis. - Cambridge University Press, 2002.
36. Glas C. A. A Bayesian Approach to Person Fit Analysis in Item Response Theory Models / C. A. Glas, R. R. Meijer // Applied Psychological Measurement. - 2003. - Vol. 27, No 3 , - P. 217-233.
37. Bayesian Modeling Using WinBUGS. - A John Viley&Sons, Inc, publication, 2009. - 506 p.
38. Wim J. van der Linden. Elements of Adaptive Testing / Wim J. van der Linden and Cees A. W. Glas. -New York-Dordrecht-Heidelberg-London: Springer, 2010. - 4 3 7 p.
39. Авраменко О. В. Моделі та методи статистичної обробки результатів тестування: огляд монографій та підручників / О. В. Авраменко // Наукові записки НДУ ім. М. Гоголя. Психолого-педагогічні науки. -2011. -№ 10. - С. 17-24.
40. Bollen К.А. Latent Variables in Psychology and Social Scinces / K. A. Bollen // Annu. Rev. Psychol. - 2002. - Vol. 53. - P. 605-634.
41. Скрытая марковская модель. - Википедия [Электронный ресурс]. - Режим доступа: https://m.wikipedia.org/wiki/CKpbn^ марковская модель.
42. Эфрон Б. Нетрационные методы многомерного статистического анализа / Сборник статей // Пер. с англ. Ю. П. Адлера, Ю. А. Кошевника, В. Н. Солнцева ; под ред. Ю. П. Адлера. - М. : Финансы и статистика, 1988. - 264 с.
Abstract views: 263 PDF Downloads: 202
