Comments on: Non-experimental evidence

By: pushmedia1

pushmedia1 — Wed, 02 Jun 2010 02:16:57 +0000

Do you think its the jump from non-human to human disease that catches epi folks up?

By: Epicure

Epicure — Mon, 31 May 2010 17:58:13 +0000

Confession time:)…..I was introduced to this concept not during my general epi years but when I transitioned to health outcomes research and worked with a mentor who directs/leads cost-effectiveness projects. I find it interesting that not hearing about this when training in a field whose core tennets revolve around bias and confounding. Calls for more interaction between these various disciplines.

By: pushmedia1

pushmedia1 — Fri, 28 May 2010 15:28:17 +0000

I’m happy to hear an epidemiologist looking at IV stuff and worrying about identification in general. My significant other is an “on the bench” medical researcher and in her PhD program had to take a few epidemiology classes… from what I could tell, they didn’t cover IVs in any of her classes.

By: Epicure

Epicure — Fri, 28 May 2010 14:23:01 +0000

My recent on-going dabbling in instrumental variable analysis in medical science is what brought me to Austin Frakt’s blog (nice work!) which led me to this blog. Given my background in epidemiololgy, I find the above conversation re. study designs particularly interesting. I happen to agree that (any) evidence is evidence, not so much from ‘let’s start basing treatment decisions on these’ point-of-view but more from ‘ let’s see how these stock up against other studies as well as the inherent limitations associated with the study design’. In health service research, experimental evidence (talking clinical trials) is gold-standard when it comes to causal inference but, for the most part, has limited generalizability to populations outside of these studies. Neverthless, the enthusiasm for drawing causal inference from non-randomized, observational studies is high, and Kevin’s remark serves a good reminder.

By: Kevin Dick

Kevin Dick — Wed, 12 May 2010 22:19:17 +0000

Ahh, I misunderstood. As a card carrying Bayesian, I agree we shouldn’t ignore any evidence. So I guess my disagreement is with Angus about the position of experimental evidence in the evidence hierarchy. My prior is that experimental evidence is more likely to provide better causal inference.

By: pushmedia1

pushmedia1 — Wed, 12 May 2010 21:35:49 +0000

Kevin, I didn’t say they were on the same level. I just pointed out that we shouldn’t ignore evidence just because it harder to apply the “experimental” label to it.

There is a Cult of Identification that doesn’t just ignore evidence but whole topics of study because there’s no identifiable natural experiment.

By: ssendam

ssendam — Wed, 12 May 2010 20:36:21 +0000

On the topic of identification, the latest issue of J. Econ Perspectives has some amusing counterpoint between Angrist & Pischke and various critics.

By: Kevin Dick

Kevin Dick — Wed, 12 May 2010 19:50:28 +0000

I’m not sure I agree about evidence from experiments and cross-section data being on the same level. It all comes down to the problem of causal inference. My memory of the philosophy and statistics here is vague, but I seem to recall that it’s a shorter inferential leap to causality given a well designed experiment.

In any case, my operating hypothesis would be that the weight of evidence should be measured based on its contribution to causal inference. Certainly, a poorly designed experiment could have less inferential weight than a well performed cross-sectional analysis. But my guess is that, on average, experiments do better at establishing causality.