Meta-Analysis and Meta-Regression Analysis for Economics and Business

Prof. Tom Stanley
School of Business, Deakin University, Melbourne, Australia

Dr. Bianka Mey
TU Chemnitz

Workshop outline:
Day 1
Morning Session:
I) Motivation and Background
The ‘Replication/Credibility’ Crisis
What meta-analyses can tell us about contemporary research.
Publication Selection Bias (PSB): Its consequences and solutions
What’s wrong with Social Science research?
Ans: Low power, high heterogeneity, and publication selection bias
What’s the cause of the Replication Crisis? {see above}
II) What Meta-Regression Analysis (MRA) can do.MRA can detect and greatly reduce publication bias.
MRA can remove other biases and explain systematic heterogeneity.
Unlike most social science research, MRAs are often successfully replicated.
Afternoon Session:
III) Conducting Basic Meta-AnalysisMethods
Applications
Meta-analysis with STATA and/or R

Day 2
Morning Session:
IV) Meta-Regression Analysis
Methods
Problems/Remedies
Applications, Applications, Applications
Meta-regression analysis with STATA and/or R
V) Recent Advances in Meta-Analysis and Meta-Regression AnalysisExcess Statistical Significance and testing for publication selection bias
Bayesian modeling averaging across alternative methods to correct PSB
New Metrics of Misleading and Informative Evidence
Afternoon Session:
VI) Students individually replicate existing meta-analyses using STATA or R.

• Selections from Borenstein et al. (2009). Introduction to meta-analysis, Chichester, U.K. John Wiley & Sons. See class folder.
• Stanley, T. D. “Does economics add up? An introduction to meta-regression analysis.” European Journal of Economics and Economic Policy 10(2013): 207-220. Free download here.
• De Linde Leonard, M. and Stanley, T. D. (2015). Married with children: What remains when observable biases are removed from the reported male marriage wage premium. Labour Economics 33 (2015), 72-80. See class folder.
• Irsova, Z., Doucouliagos, H., Havranek, T., Stanley, T.D. (2023). Meta-analysis of social science research: A practitioner’s guide. Journal of Economics Surveys. Free download here.
• Bem D, Tressoldi P, Rabeyron T and Duggan M. Feeling the future: A meta-analysis of 90 experiments on the anomalous anticipation of random future events. F1000Research 2016, 4:1188. Available here.
• Askarov, Z., Doucouliagos A, and Doucouliagos H., and Stanley, T.D. (2023b). Selective and (mis)leading economics journals: Meta-research evidence. Journal of Economics Surveys. Available here.

Further Suggested Readings:

• Askarov, Z., Doucouliagos A, Doucouliagos H. & Stanley (2023a). The significance of data-sharing policy. J. European Economic Association, 21:1191–1226.
• Bartoš F, Maier M, Wagenmakers EJ, Doucouliagos H, Stanley TD. (2023). Robust Bayesian meta-analysis: Model averaging across complementary publication bias adjustment methods. Research Synthesis Methods,14:99-116.
• Bartoš F, Maier M, Quintana DS, Wagenmakers EJ (2022). Tutorial: Adjusting for Publication Bias in JASP and R. Adv Meth & Pract Psyc Sci, 5: 1-19 Available here.
• Havránek, T., Stanley, T.D., Doucouliagos, H. et al. (2020): Reporting guidelines for meta-analysis in economics. Journal of Economic Surveys, 34: 469-75. Available here.
• Stanley, T.D. and H. Doucouliagos. (2012). Meta-Regression Analysis in Economics and Business, Oxford: Routledge.

Doctoral candidates from other universities can earn a certificate as well.