RMME Evaluation Colloquium

The Health Profession Opportunity Grant (HPOG) Impact Study: A Behind-the-Scenes Look at Experimental Evaluation in Practice

Dr. Laura Peck
Abt Associates

Friday, March 10, at 11AM ET

https://tinyurl.com/eval-Peck

In 2010, the U.S. Department of Health and Human Services’ Administration for Children and Families awarded Health Profession Opportunity Grants (HPOG 1.0) to 32 organizations in 23 states. The purpose of the HPOG Program is to provide education and training to Temporary Assistance for Needy Families (TANF) recipients and other low-income individuals for occupations in the healthcare field that pay well and aim to meet local areas’ healthcare sector labor shortages. To assess its effectiveness, an experimental evaluation design assigned eligible program applicants at random to a “treatment” group that could access the program or a “control” group that could not. Beyond the impact analysis, the evaluation also probed questions about what drove program impacts, using various strategies. This colloquium will discuss how the HPOG 1.0 impact study was designed/implemented and introduce attendees to various design and analysis choices used by investigators, in partnership with the government funder, to address research questions. Specific topics will include: experimental design, multi-armed experimental design, experimental impact analysis, planned variation, natural variation, endogenous subgroup analysis, evaluation in practice.

Loading...

Taking too long?

Reload document

| Open in new tab

RMME/STAT Joint Colloquium

Bookmaking for Binary Outcomes: Prediction, Profits, and the IMV

Dr. Ben Domingue
Stanford University

Friday, February 24, at 11AM ET

https://tinyurl.com/rmme-Domingue

Understanding the “fit” of models designed to predict binary outcomes is a long-standing problem. We propose a flexible, portable, and intuitive metric for such scenarios: the InterModel Vigorish (IMV). The IMV is based on a series of bets involving weighted coins, well-characterized physical systems with tractable probabilities. The IMV has a number of desirable properties including an interpretable and portable scale and an appropriate sensitivity to outcome prevalence. We showcase its flexibility across examples spanning the social, biomedical, and physical sciences. We demonstrate how it can be used to provide straightforward interpretation of logistic regression coefficients and to provide insights about the value of different types of item response theory (IRT) models. The IMV allows for precise answers to questions about changes in model fit in a variety of settings in a manner that will be useful for furthering research with binary outcomes.

Loading...

Taking too long?

Reload document

| Open in new tab