Meta-Analysis and Meta-Regression Analysis for Economics and Business 2

LecturerProf. Tom Stanley
School of Business, Deakin University, Melbourne, Australia
Date25./26.04.2024
10:00 am to 1:15 pm (with a short break) and 2:30 pm to 4:00 pm each day
Room/AddressTU Chemnitz, Faculty of Economics and Business Administration
Seminar content2-day program consisting of lectures on meta-analysis methods, discussion of numerous actual meta-analysis applications, and interactive exercises. Each day there will be a morning session with a short break and an afternoon session.
Although not required: 2 recorded lectures from a similar class are available for you to preview and to reinforce what was covered in this class. See the class folder
Before day 1: Please view recorded lecture 1 (1:18) and read selections from Borenstein et al. (2009) as well as Stanley (2013): “Does economics add up?”

Workshop outline:
Day 1
Morning Session:
I) Motivation and Background
  1. The ‘Replication/Credibility’ Crisis
  2. What meta-analyses can tell us about contemporary research.
  3. Publication Selection Bias (PSB): Its consequences and solutions
  4. What’s wrong with Social Science research?
    Ans: Low power, high heterogeneity, and publication selection bias
  5. What’s the cause of the Replication Crisis? {see above}
II) What Meta-Regression Analysis (MRA) can do.
  1. MRA can detect and greatly reduce publication bias.
  2. MRA can remove other biases and explain systematic heterogeneity.
  3. Unlike most social science research, MRAs are often successfully replicated.
Afternoon Session:
III) Conducting Basic Meta-Analysis
  1. Methods
  2. Applications
  3. Meta-analysis with STATA and/or R

Day 2
Morning Session:
IV) Meta-Regression Analysis
  1. Methods
  2. Problems/Remedies
  3. Applications, Applications, Applications
  4. Meta-regression analysis with STATA and/or R
V) Recent Advances in Meta-Analysis and Meta-Regression Analysis
  1. Excess Statistical Significance and testing for publication selection bias
  2. Bayesian modeling averaging across alternative methods to correct PSB
  3. New Metrics of Misleading and Informative Evidence
Afternoon Session:
VI) Students individually replicate existing meta-analyses using STATA or R.
RequirementsStudents are expected to have taken Statistics and/or Econometrics classes and to know some regression analysis. Also, they should have STATA and R installed on their laptops, either by buying a 6-month STATA license for $48 or by downloading it from your university if available.
Required Readings
  • Selections from Borenstein et al. (2009). Introduction to meta-analysis, Chichester, U.K. John Wiley & Sons.
    See class folder.
  • Stanley, T. D. “Does economics add up? An introduction to meta-regression analysis.” European Journal of Economics and Economic Policy 10(2013): 207-220.
    Free download here.
  • De Linde Leonard, M. and Stanley, T. D. (2015). Married with children: What remains when observable biases are removed from the reported male marriage wage premium. Labour Economics 33 (2015), 72-80.
    See class folder.
  • Irsova, Z., Doucouliagos, H., Havranek, T., Stanley, T.D. (2023). Meta-analysis of social science research: A practitioner’s guide. Journal of Economics Surveys.
    Free download here.
  • Bem D, Tressoldi P, Rabeyron T and Duggan M. Feeling the future: A meta-analysis of 90 experiments on the anomalous anticipation of random future events. F1000Research 2016, 4:1188.
    Available here.
  • Askarov, Z., Doucouliagos A, and Doucouliagos H., and Stanley, T.D. (2023b). Selective and (mis)leading economics journals: Meta-research evidence. Journal of Economics Surveys.
    Available here.

Further Suggested Readings:
  • Askarov, Z., Doucouliagos A, Doucouliagos H. & Stanley (2023a). The significance of data-sharing policy. J. European Economic Association, 21:1191–1226.
  • Bartoš F, Maier M, Wagenmakers EJ, Doucouliagos H, Stanley TD. (2023). Robust Bayesian meta-analysis: Model averaging across complementary publication bias adjustment methods. Research Synthesis Methods,14:99-116.
  • Bartoš F, Maier M, Quintana DS, Wagenmakers EJ (2022). Tutorial: Adjusting for Publication Bias in JASP and R. Adv Meth & Pract Psyc Sci, 5: 1-19.
    Available here.
  • Havránek, T., Stanley, T.D., Doucouliagos, H. et al. (2020): Reporting guidelines for meta-analysis in economics. Journal of Economic Surveys, 34: 469-75.
    Available here.
  • Stanley, T.D. and H. Doucouliagos. (2012). Meta-Regression Analysis in Economics and Business, Oxford: Routledge.
CertificateDoctoral candidates from the Faculty of Business and Economics, TU Dresden can earn a certificate according to § 9 of the Ph.D. doctoral regulations (PromO 2018):
Doctoral candidates of Business Administration: § 9 (1) Nr. 5 or 6
Doctoral candidates of Business Information Systems: § 9 (1) Nr. 6
Doctoral candidates of Economics: § 9 (1) Nr. 6

Doctoral candidates from other universities can earn a certificate as well.
Assignment Students need to use STATA or R to do meta-analysis and meta-regression analysis calculations for an actual meta-analysis dataset. There will be a lab with individual help on the second day to ensure that each student can complete the assignment. Students need to copy and paste their findings in a MS Word document and briefly interpret their findings. That is, what do the statistical results reveal about this research and the underlying phenomenon studied.
Full credit requirement:
Students need to attend class, read and be ready to discuss the assigned material, and to demonstrate that they can use STATA or R to do meta-analysis and meta-regression analysis calculations for an actual meta-analysis dataset and briefly interpret their findings.
RegistrationParticipation is limited (max. 25).
To register, send an e-mail to Dr. Uta Schwarz: uta.schwarz@tu-dresden.de.