Below you will find more information about the workshop and a link to register!

Workshop 20 May at 13–17

Statistical inferences are often challenged because of uncontrolled bias. There may be bias due to uncontrolled confounding variables or non-random selection into a sample. We will turn concerns about potential bias into questions about how much bias there must be to invalidate an inference. For example, challenges such as “But the inference of a treatment effect might not be valid because of pre-existing differences between the treatment groups” are transformed to questions such as “How much bias must there have been due to uncontrolled pre-existing differences to make the inference invalid?”

By reframing challenges about bias in terms of specific quantities, this workshop will contribute to scientific discourse about uncertainty of causal inferences. Critically, while there are other approaches to quantifying the sensitivity of inferences (e.g., Robins and Rotnitzky, 1995; Rosenbaum and Rubin, 1983; Rosenbaum, 2000), the approaches presented in this workshop based on correlations of omitted variables (Frank, 2000) and the replacement of cases (Frank and Min, 2007; Frank et al, 2013) have greater intuitive appeal. In this sense the techniques inform a conversation among a broad set of stakeholders and help apply research evidence to practice.

Activities

This is a 4-hour workshop (including multiple breaks) with a mixture of presentation, individual hands-on exploration, group work and informal interactions. The interactive activities will be face-to-face, although the presented material can be accessed via zoom which can be recorded for asynchronous viewing. The target audience is anyone familiar with general quantitative methods including regression and analysis of variance as well as more sophisticated techniques (including multilevel models and logistic regression). 

In part I, we use Rubin’s causal model to interpret how much bias there must be to invalidate an inference in terms of replacing observed cases with counterfactual cases or cases from an unsampled population (e.g., Frank et al, 2013).  This includes application to logistic regression (Frank et al., 2021). In part II, we quantify the robustness of causal inferences in terms of correlations associated with unobserved variables or in unsampled populations (e.g., Frank 2000). The techniques can be applied to linear models and related extensions (e.g., multilevel models, mediation, propensity models). Calculations will be presented using the app https://konfound-project.shinyapps.io/konfound-it/ with links to STATA and R modules.

Presentation 21 May at 10–11

Quantifying Sensitivity to Selection on Unobservables: Refining Oster’s Coefficient of Proportionality. 

Abstract

Beginning with debates about the effects of smoking on lung cancer, sensitivity analyses characterizing the hypothetical unobserved conditions that can alter statistical inferences have had profound impacts on public policy. One of the most ascendant techniques for sensitivity analysis is Oster’s coefficient of proportionality, which approximates how strong selection into a treatment on unobserved variables must be compared to selection on observed variables to change an inference. We refine Oster’s asymptotic approximation by deriving expressions for the correlations associated with an omitted variable that reduce an estimated effect to a specified threshold, given a corresponding coefficient of determination (R2). We verify our expressions through empirical examples and simulated data. Because our calculations are exact, they apply regardless of sample size. In contrast, Oster’s approximation is likely to overstate robustness when sample size is small and observed covariates account for a large portion of an estimated effect relative to a baseline model. 

Moreover, unlike Oster’s estimator, our correlation-based expressions do not depend on the analyst’s choice of baseline model. Furthermore, our correlation-based expressions can be directly calculated from conventionally reported quantities through commands in R or Stata and an on-line app, and therefore can be applied to most published studies.

Instructor bios

Kenneth Frank received his Ph.D. in measurement, evaluation and statistical analysis from the School of Education at the University of Chicago in 1993. He is MSU Foundation professor of Sociometrics, professor in Counseling, Educational Psychology and Special Education; and adjunct (by courtesy) in Fisheries and Wildlife and Sociology at Michigan State University.  Dr. Frank has published articles across the social sciences on quantifying the robustness of causal inferences including in Sociological Methodology; Sociological Methods and Research; Journal of Educational and Behavioral Statistics; Education, Evaluation and Policy Analysis; and Journal of Clinical Epidemiology. This work has been cited in top journals in sociology (e.g., American Sociological Review; Social Forces; Sociology of Education; Journal of Marriage and Family) as well as in economics, psychology, political science, public policy, and health. See published examples under “more” at http://konfound-it.com. He has also developed the app konfound-it.com and R and STATA macros and gave the Duncan lecture at ASA in 2020 on sensitivity analysis. 

Qinyun Lin is a Senior Lecturer in the School of Public Health and Community Medicine, University of Gothenburg. Most broadly, she is a methodologist invested in humanizing quantitative research to better understand mechanisms underlying inequities, particularly health and educational disparities, and, ultimately, informing policymaking. Towards this, she seeks to advance and expand existing quantitative methods and the interpretation of inferences by incorporating human interactions, social and spatial contexts, and mediating factors within these analyses. Her works have appeared in high-impact peer-reviewed journals such as JAMA Network Open, Social Science & Medicine, and Psychological Methods.

Background readings

*Frank, K. A., *Lin, Q., *Maroulis, S., *Mueller, A. S., Xu, R., Rosenberg, J. M., ... & Zhang, L. (2021). Hypothetical case replacement can be used to quantify the robustness of trial results. Journal of Clinical Epidemiology, 134, 150-159. *authors listed alphabetically. 

Frank, K.A., Maroulis, S., Duong, M., and Kelcey, B. 2013.  What would it take to Change an Inference?: Using Rubin’s Causal Model to Interpret the Robustness of Causal Inferences.   Education, Evaluation and Policy Analysis.  Vol 35: 437-460.

Frank, K.A., Gary Sykes, Dorothea Anagnostopoulos, Marisa Cannata, Linda Chard, Ann Krause, Raven McCrory. 2008. Extended Influence: National Board Certified Teachers as Help Providers.  Education, Evaluation, and Policy Analysis.  Vol 30(1): 3-30.

*Frank, K. A. and Min, K. 2007. Indices of Robustness for Sample Representation. Sociological Methodology.  Vol 37, 349-392. * co first authors.

Frank, K. 2000. "Impact of a Confounding Variable on the Inference of a Regression Coefficient." Sociological Methods and Research, 29(2), 147-194.

Xu, R., Frank, K. A., Maroulis, S. J., & Rosenberg, J. M. (2019). konfound: Command to quantify robustness of causal inferences. The Stata Journal, 19(3), 523-550.

Register for the workshop

Please register here before 13 May if you want to attend.

Practicalities

The workshop 20 May is between 13.00 and 17.00 including a coffee break.
The presentation 21 May is between 10.00 and 11.00. Coffee and rolls will be served from 09:30.

Organisers:
Moira Nelson, Associate Professor and Senior Lecturer at the Department of Political Science, Lund University
Qinyun Lin, Senior Lecturer at School of Public Health and Community Medicine, University of Gothenburg
Kenneth Frank, Professor of Sociometrics at the Michigan State University Foundation 

We look forward to seeing you at the workshop in May!

Arranged by: Lund Social Science Methods Centre