Month: December 2010

Self-selection into online or face-to-face studies

Published on by Sanjay Srivastava4 Comments

A new paper by Edward Witt, Brent Donellan, and Matthew Orlando looks at self-selection biases in subject pools:

Just over 500 Michigan State University undergrads (75 per cent were female) had the option, at a time of their choosing during the Spring 2010 semester, to volunteer either for an on-line personality study, or a face-to-face version…

Just 30 per cent of the sample opted for the face-to-face version. Predictably enough, these folk tended to score more highly on extraversion. The effect size was small (d=-.26) but statistically significant. Regards more specific personality traits, the students who chose the face-to-face version were also more altruistic and less cautious.

What about choice of semester week? As you might expect, it was the more conscientious students who opted for dates earlier in the semester (r=.-.20). What’s more, men were far more likely to volunteer later in the semester, even after controlling for average personality difference between the sexes. For example, 18 per cent of week one participants were male compared with 52 per cent in the final, 13th week.

Self-selection in subject pools is not a new topic — I’ve heard plenty of people talk about an early-participant conscientiousness effect (though I don’t know if that’s been documented or if it’s just lab-lore). But the analyses of personality differences in who takes online versus in-person studies are new, as far as I know — and they definitely add a new wrinkle.

My lab’s experience has been that we get a lot more students responding to postings for online studies than face-to-face, but it seems like we sometimes get better data from the face-to-face studies. Personality measures don’t seem to be much different in quality (in terms of reliabilities, factor structures, etc.), but with experiments where we need subjects’ focused attention for some task, the data are a lot less noisy when they come from the lab. That could be part of the selection effect (altruistic students might be “better” subjects to help the researchers), though I bet a lot of it has to do with old-fashioned experimental control of the testing environment.

What could be done? When I was an undergrad taking intro to psych, each student was given a list of studies to participate in. All you knew was the codenames of the studies and some contact information, and it was your responsibility to arrange with the experimenter to take the experiment. It was a pain on all sides, but it was a good way to avoid these kinds of self-selection biases.

Of course, some people would argue that the use of undergraduate subject pools itself is a bigger problem. But given that they aren’t going away, this is definitely something to pay attention to.

McAdams on Bush: a psychobiography

Published on by Sanjay Srivastava2 Comments

Personality psychologist Dan McAdams has a new book out called George W. Bush and the Redemptive Dream. Dan was my undergraduate advisor, and I saw him give a provocative talk about this work at last summer’s ARP conference. I just told my wife to add the book to my Christmas list.

Most of McAdams’s research centers on personal narratives — the stories that people create and tell about themselves, and what role these stories play in identity and personality. But in the talk — and I gather in the book as well — Dan drew on a variety of theories and frameworks to understand some of Bush’s most consequential actions before and during his time in office. Here’s a brief description from an announcement I got about the book:

This short, streamlined psychological biography uses some of the best scientific concepts in personality and social psychology to shed light on Bush’s life, with a focus on understanding his fateful decision, as President, to launch a military invasion of Iraq.  The analysis draws heavily from contemporary research on Big Five traits, psychological goals and strivings, and narrative identity, as well as social identity theory, evolutionary psychology, research on motivated social cognition, research on authoritarianism and related concepts in political psychology, and Jon Haidt’s brilliant synthesis of moral intuitions.

Once upon a time, psychobiography was a pretty well-respected enterprise in personality psychology. I think it’s fallen out of favor in part because of the field’s emphasis on the Big Five traits and other discrete, fractionated variables. That emphasis has had benefits, focusing the field on constructs and theories that we can rigorously quantify and formalize.

But early personality psychologists like Gordon Allport and Henry Murray emphasized that any comprehensive study of personality must be able to account for the person as an integrated whole and a unique individual. The field has lost track of that to a substantial degree. But unlike earlier psychobiographers, who had very little and/or bad science to draw upon, McAdams has almost a century worth of theories and empirical research to bring to bear. That doesn’t mean the task is easy now. But I’m definitely looking forward to reading how Dan took it on.