At Carnegie Mellon I remember that a prerequisite for my psychology classes was to participate in a certain number of psychology experiments. We were the free subjects for the Carnegie Mellon Psychology department. Our department was not unique in this practice and Heinrich Joseph, Stephen Heine, and Ara Norenzayan from the University of British Columbia have a review paper in Behavioral and Brain Science says that the overwhelming number of psychology studies in the literature use undergraduates from the United States and Europe in their studies. The problem with that is that these undergraduates are unique in the world, typically being, Western, educated, industrialized, rich and democratic and that these WEIRDos are not representative of humanity as a whole.
The typical example of test that yields startling different results for the US and Europe vs. other countries on the world is the classic Müller-Lyer illusion where the test subject must decide which of the lines is longer, Americans and Europeans typically think that the line with the arrows pointing out is shorter, but this is not a typical answer around the world. The chart below shows the increased length of the "short" line required to make the lines seem similar. People in industrialized nations (to the right on the chart below) show the most effect.
The review paper discusses a wide range of psychology experiments designed to explore the free-rider problem and punishment from economics, through language development and family based reasoning (clumped by function - notebook-pencil) vs. rule based reasoning (notebook-magazine), to other social psychology problems.
The authors conclusions include that generalizing experimental results from WEIRD test subjects to the world population as a whole may yield very incorrect generalizations.
In the open peer commentary to the paper (found at the end of the paper at this .pdf link), other psychology researchers propose other reasons and commentary for and on the presumed outlier nature of WEIRD subjects in psychology experiments vs. the rest of the world. Some highlights are briefly listed below.
- One group proposes that the experiments are weird, not the subjects.
- Another suggests the variations within populations are not taken into account correctly.
- Some argue that brain scientists are just as guilty as psychologists in using WEIRD subjects, and that generalizing results from a narrow sample of WEIRD brain subjects to the entire species is also unwise.
- Many provide suggestions in reaching a wider sample pool. Although the one suggestion to use the Internet will then select for a worldwide population with access to the Internet - that may also be an unusual population The authors address this in their response to the commentary.
- Others caution against generalization in animal studies and coin their own acronymical description of chimpanzees in animal experiments: Barren, Institutional, Zoo, And other Rare Rearing Environments (BIZARRE) chimpanzees. This is only one of several acronyms coined in the commentary (WRONG - When Researchers Overlook uNderlying Genotypes, ODD - Observation- and Description-Deprived psychological research)
- Some point out that it is not just the subjects, but there are also too many WEIRD researchers.
- One points out that the WEIRD subjects are a harbinger of what the world is becoming and wants more research on all of the variations in human experience before we become culturally homogenized. One of the more hopeful comments I thought, though it implies a trade-off between prosperity and diversity.
I suggest that you read the review and the back and forth commentary if this peaks your interest. The take home lesson appears to be that you need to be aware of the selection of your sample subjects in social, psychological and cultural experiments, using caution when reasoning from a narrow sample to the larger population (or species) and do not assume that everyone (in the world, in your culture, in the same room as you) are all the same, except when they are.
No comments:
Post a Comment