Testing the Validity and Structure of the Data Envelopment Analysis PISA Scores
Abstract
This research aims to take efficiency as a black box between input and output, or a property of the process, and suggest an alternative way to evaluate the validity of Data Enveloping Analysis (DEA) scores. Over the last ten years, DEA has become a commonly used measure of efficiency of processes. It is offered as a non-parametric method of output oriented optimization, which is solved without any assumption under the restriction of the convexity with metric input and output measurements. Alternatively, output models like Cobb-Douglas or advanced econometric models are valid under several assumptions, and efficiency requires a significance test. Similar processes with the same input level exhibit a different level of output due to efficiency. This means that output is a function of input and efficiency. The present study models the DEA scores of an education efficiency research study by applying path analysis. Efficiency is found as moderator variable instead the expectation of mediation effect of an educational input-output process. The findings and interpretation of the structure depend on the data and its coverage, which do not lead to generalization of the efficiency concept. However, the main insight of the method is to consider real processes, not simulated model data. The purposed method enables the statistical testing of calculated DEA efficiency scores. Processes might be updated through the structured interpretation of how efficiency affects the output.
Keywords: Data enveloping analysis, Path analysis, Education, efficiency, Score validity, Significance test.
References
Charnes, A, Cooper WW, Rhodes E (1978) Measuring the efficiency of decision making units. European J. Operational Research, 2:429-444.
Boussofiane A, Dyson RG, Thanassoulis E (1991) Applied data envelopment analysis. European J. Operational Research, 52:1-15.
Seiford LM (1999) A cyber-bibliography for data envelopment analysis (1978–1999), in Cooper, W.W. and Seiford, L.M. (Ed.), Data Envelopment Analysis: A Comprehensive Text with Models, Applications, References and DEA-Sover software, Kluwer Academic Publishers.
Andersen P, Petersen NC (1993) A procedure for ranking efficient units in data envelopment analysis. Management Science, 39:1261-1264.
Sexton TR, Silkman RH, Hogan AJ (1986) Data Envelopment Analysis: Critique and Extensions, in Silkman, R.H. (Ed.), Measuring Efficiency: An Assessment of Data Envelopment Analysis, Jossey-Bass, San Francisco, CA.
Li X, Reeves GR (1999) A Multiple Criteria Approach to Data Envelopment Analysis. European J. Operational Research, 115:507-517.
Charnes A, Cooper WW, Rhodes E (1981) Evaluating program and managerial efficiency: An application of data envelopment analysis to program follow through. Management Science, 27(6):668-697.
Berg S (2010) Water Utility Benchmarking: Measurement, Methodology, Performance Incentives", International Water Association.
Alwin D, Hauser R (1975) The decomposition of effects in path analysis. American Sociological Review, 40:37-47.
Graff J, Schmidt P (1982) A general model for decomposition of effects, in Joreskog K and Wold H (Ed.), Systems Under Indirect Observation: Causality, Structure, Prediction, North-Holland, Amsterdam, pp. 131-148.
Sobel M (2008) Identification of causal parameters in randomized studies with mediating variablesâ€, Journal of Educational and Behavioral Statistics, 33, pp. 230-231.
Bollen K (1989) Structural Equations with Latent Variables, John Wiley, New York.
Judd C, Kenny, DA (1981) Process analysis: Estimating mediation in treatment evaluations. Evaluation Review, 5:602-619.
Baron R, Kenny D (1986) The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. J. Personality and Social Psychology, 51:1173-1182.
Shrout P, Bolger N (2002) Mediation in experimental and nonexperimental studies: New procedures and recommendationsâ€, Psychological Methods, 7:422-445.
Muller D, Judd C, Yzerbyt V (2005) When moderation is mediated and mediation is moderated. J. Personality and Social Psychology, 852-863.
MacKinnon DP, Fairchild AJ, Fritz MS (2007) Mediation analysis. Annual Review of Psychology, 58:593-614.
Holland P (1995) Some reflections on freedman’s critiques. Foundations of Science, 1:50-57.
Pearl J (1998) Graphs, causality, and structural equation models. Sociological Methods and Research, 27:226-284.
Pearl J (2010) The Mediation Formula: A guide to the assessment of causal pathways in non-linear models, UCLA Computer Science Department, Technical Report R-363, January 2011.
Imai K, Keele L, Yamamoto T (2010) Identification, inference, and sensitivity analysis for causal mediation effects. Statistical Science, 25(1):51-71.
MacKinnon DP, Warsi G, Dwyer JH (1995) A simulation study of mediated effect measures. Multivariate Behavioral Research, 30:41-62.
Judd CM, Kenny, DA (2010) Data analysis in social psychology: Recent and recurring issues, in Fiske S, Gilbert D and Lindsay G (Ed.), The handbook of social psychology, John Wiley and Sons.
Kramer HC, Wilson GT, Fairburn CG, Agras WS (2002) Mediators and moderators of treatment effects in randomized clinical trials. Archives of General Psychiatry, 59:877-883.
Frazier PA, Tix AP, Barron KE (2004) Testing moderator and mediator effects in counseling psychology research. J. Counseling Psychology, 51:115-134.
Gupta, S, Verhoeven, M. (2001) The efficiency of government expenditure – experiences from Africa. J. Policy Modelling, 23:433-467..
Clements B (2002) How efficient is education spending in Europe? European Review of Economics and Finance, 1:3-26.
St. Aubyn M (2003) Evaluating efficiency in the Portuguese education sector, Economia, 26:25-51.
Abu-Ghaida D (2007) Education, in Gray C, Lane T, Varoudakis A (Ed.), Fiscal policy andeconomic growth. Lessons for Eastern Europe and Central Asia, The World Bank, Washington, D.C., pp. 147-178.
Afonso A, Schuknecht L, Tanzi V (2004) Public sector efficiency: An international comparison. Public Choice, 123:321-347.
Özden ÜH (2010) Veri zarflama analizi (VZA) ile Türkiye’deki vakıf üniversitelerinin etkinliğinin ölçülmesi. Istanbul University J. the School of Business Administration, 39(1):167-185.
Fakin B, Crombrugghe A (1997) Fiscal adjustment in Transition Economies: Social transfers and the efficiency of public spending, a comparison with OECD countriesâ€, Working paper 1803, World Bank, Washington.
Afonso A, St. Auby, M (2005) Non-Parametric approaches to education and health efficiency in OECD countries. J. Applied Economics, 8(2):227-246.
Sutherland D, Price R, Joumard I, Nicq C (2007) Performance indicators for public spending efficiency in primary and secondary education, Working paper 546, OECD Economics Department.
Koçak H, Çilingirtürk, AM (2011) Efficiency Analysis of OECD Public Education Spending, in Annual International Conference on Operations Research and Statistics -ORS 2011, Malaysia, 2011:83-88.
Yıldırım IE (2010) Veri zarflama analizinde girdi ve çıktıların belirlenmesindeki kararsızlık problemi için temel bileşenler analizine dayalı bir çözüm önerisi. Istanbul University & J. the School of Business Administration, 39(1):141-153.
Eugène B (2008) The efficiency frontier as a method for gauging the performance of public expenditure: A Belgian case studyâ€, Working paper 138, National Bank of Belgium, Belgium.
MacKinnon DP (2008) Introduction to Statistical Mediation Analysis, Lawrence Erlbaum Associates, New York.
Pearl J (2009) Causal inference in statistics: An overview. Statistics Surveys, 3:96-146.
Sobel M (1987) Direct and indirect effects in linear structural equation models. Sociological Methods & Research, 16(1):155-176.
