Causal Inference from Multiple Studies

Practice Questions



1. A pharmaceutical company wishes to establish the superiority of its pain killer over a major competitor for specific types of pain. The company funds a dozen studies comparing its pain killer to competitor for lower right quadrant chronic back pain, upper right quadrant chronic back pain, lower left quadrant chronic back pain, upper left quadrant chronic back pain, pain post-dental extraction, pain post-appendectomy, muscular pain associated with strenuous exercise, muscular pain associated with each of three difference muscular degenerative diseases, arthritic pain, and pain of migraine headache. The study on upper right quadrant chronic back pain produces a statistically significant (p=.04) difference in favor of their product, but none of the other studies show differences that could not easily be explained by random chance. The company publishes the positive study in the prestigious Journal of Pain Relief and documents it to the FDA, with a request that the labeling of their product be allowed to incorporate a claim of superiority for relief of chronic back pain.

a. Should the FDA approve the request? Justify your answer.

b. Would your answer be different if the statistically significant study were the only one that had been conducted? Why?

c. What if all the studies were conducted and came out as described, but the FDA knew only results of the statistically significant study, and didn't even know that the others had been done? How would this affect the drug approval and labeling process?


2.  A dedicated clinical investigator believes firmly in the rationale behind a new therapy, and is thus convinced that it must be superior to the conventional therapy. He conducts a comparative clinical trial in which twenty measures of different aspects of a disease outcome, relatively equally important, are obtained. A statistically significant difference (.05<Pex post facto rationales for why these measures might have been expected to respond to the therapy whereas the others should not have. Are you convinced that the treatment is superior generally, for the specific two measures for which statistical significance was found, or for any measure? Would your interpretation be different if the investigator had designated the two measures as "primary outcome variables" in a paper published on the design of the study, before the data were collected? Would your interpretation be different if the investigator's publication of results omitted mention of the eighteen non-significant variables and focussed entirely on the two for which statistical significance had been obtained? Justify your answers.


3.  In a case-control study of possible dietary correlates of cancer of the tongue, subjects were asked about their histories of consuming 50 foods. The histories of cases differed from those of controls, at a=.05, for four of these foods. Which of the following is justified?

a. This study shows that diet is definitely related to the disease but can't demonstrate which of the four foods are important contributors, so they should be looked at in further case-control studies.

b. This study justifies major expenditure on a cohort study in which those who consume these four foods are compared over time to those who don't.

c. The study is inconclusive as to whether diet is related to disease, but further case-control investigations of diet should look more closely at these four foods.

d. The study is much more conclusive on the importance of the four foods if they are almost always eaten together.

e. The study is much more conclusive on the importance of the four foods if consumption of these foods is uncorrelated in the population.


4. Which of the following is not a criterion used in assessing the evidence for causality in a body of literature?

a. consistency

b. temporality

c. reversibility

d. sensitivity

e. specificity


5.  Although an observed dose-response effect supports the hypothesis of a causal relationship, studies of a genuinely causal relationship may fail to show a dose-response effect for several reasons. Which choice below is the generally the least plausible reason?

a. all doses are too high to reveal the effect

b. all doses are too low to reveal the effect

c. a confounder obscures the effect

d. random variation obscures the effect

e. there is no real dose-response effect


6.  Publication bias includes

a. the tendency of editors to accept articles with which they agree

b. the tendency of editors to accept articles written by their friends

c. the tendency of authors to submit poorer studies to second-line journals

d. the tendency of authors to withhold negative results from publication

e. the tendency of readers to accept or reject findings of articles based on the journal in which they appear


 7.  In epidemiology, an ecological study refers to research

a. in which exposure data are available for groups but not for their individual members

b. in which geographic units define epidemiologic populations

c. on how changes in an ecosystem affect spread of a contagious disease

d. on host species and other ecologic reservoirs for infectious organisms

e. on how changing demography of a population affects the distribution of chronic disease


8.  Temporality, biologic plausibility, specificity, and analogy are three criteria used to evaluate whether an observed association is likely to be causal. However, these particular criteria are more useful to support a case for causality made on other grounds than to establish the case. Explain the meaning of these four criteria and why they are not useful as primary indicators of a causal relationship. Give three other criteria which are used in combination, when only observational data are available, as primary support for a causal hypothesis.



9. Metaanalyses that pool data from multiple studies would be unnecessary if individual studies

a. avoided bias successfully

b. avoided confounding successfully

c. avoided effect modification successfully

d. used sufficiently low a

e. had sufficient power


10.  Metaanalyses that pool data from multiple studies

a. assume that differences in aspects of study design and populations studied are not major effect modifiers.

b. analyze the pooled data as though it had been obtained from one study conducted at a single location.

c. require that the therapeutic regimens used be identical across the studies that are pooled.

d. protect against biases in study design by assuring that the biases in individual studies will cancel one another out when the studies are pooled.

e. protect against publication bias better than criterion-based metaanalyses.