Tag: accuracy

On base rates and the “accuracy” of computerized Facebook gaydar

Published on by Sanjay Srivastava2 Comments

I never know what to make of reports stating the “accuracy” of some test or detection algorithm. Take this example, from a New York Times article by Steve Lohr titled How Privacy Vanishes Online:

In a class project at the Massachusetts Institute of Technology that received some attention last year, Carter Jernigan and Behram Mistree analyzed more than 4,000 Facebook profiles of students, including links to friends who said they were gay. The pair was able to predict, with 78 percent accuracy, whether a profile belonged to a gay male.

I have no idea what “78 percent accuracy” means in this context. The most obvious answer would seem to be that of all 4,000 profiles analyzed, 78% were correctly classified as gay versus not gay. But if that’s the case, I have an algorithm that beats the pants off of theirs. Are you ready for it?

Say that everybody is not gay.

Figure that around 5 to 10 percent of the population is gay. If these 4,000 students are representative of that, then saying not gay every time will yield an “accuracy” of 90-95%.

But wait — maybe by “accuracy” they mean what percentage of gay people are correctly identified as such. In that case, I have an algorithm that will be 100% accurate by that standard. Ready?

Say that everybody is gay.

You can see how silly this gets. To understand how good the test is, you need two numbers: sensitivity and specificity. My algorithms each turn out to be 100% on one and 0% on the other. Which means that they’re both crap. (A good test needs to be high on both.) I am hoping that the MIT class’s algorithm was a little better, and the useful numbers just didn’t get translated. But this news report tells us nothing that we need to know to evaluate it.

Do people know how much power and status they have?

Published on by Sanjay Srivastava5 Comments

Do you know how much power and status you have in the important social situations in your life? Cameron Anderson and I have a chapter coming out in a few months looking at that question. The chapter is titled “Accurate When It Counts: Perceiving Power and Status in Social Groups.” (It draws in part on an earlier empirical paper we did together.) The part before the colon probably gives away a little bit of the answer. We present a case that most people, much of the time, are pretty good at perceiving their own and others’ power and status. (Better than they are at perceiving likability or personality traits.)

You can read the chapter if you want to see where the main point is coming from. I just want to briefly comment on a preliminary issue we had to develop along the way…

One of the fun things about writing this paper was working out what it means to be accurate in perceiving power and status. Accuracy has a long and challenging history in social perception research. How do you quantify how well somebody knows somebody else’s (or their own) likability, extraversion, morality, or — in our case — power or status?

We started by creating working definitions of power and status. What became clear along the way is that the accuracy question gets answered differently for power than for status because of the different definitions. For power, we adopted Susan Fiske’s definition that power is asymmetric outcome control (in a nutshell, Person A has power over Person B if A has control over B’s valued outcomes). For status, we defined it as respect and influence in the eyes of others.

Drawing on those definitions, here’s what we say about how to define accuracy in perceiving power:

The outcome-control framework is useful for studying perceptions. Outcome control is a structural property of relationships that does not depend on any person’s construal of a situation. Thus, one person may have power over another person even if one or both people do not realize it at a given time. (For example, a late-night TV host and the female intern he dates might both think about their relationship in purely romantic terms, but the fact that the host makes decisions about the intern’s salary and career advancement means that he has power over her). Because the outcome-control framework separates psychological processes such as the perception of power from power per se, it is conceptually coherent to ask questions about the accuracy of perceptions.

And here’s how accuracy is different for status:

Like power, status is a feature of a relationship (Fiske & Berdahl, 2007). Like power, status may vary from one situation to another. And like with power, it is possible for a single individual to misperceive her own status or the status of another person. However, because status is about respect and prestige in the eyes of others, at its core it involves collective perceptions – that is, status is a component of reputation. Thus status is socially constructed in a different and perhaps more fundamental way than power. Whereas it might make sense to say that an individual has power but nobody knows it, it would not make sense to say the same about status. This gives status a complicated but necessary relation to interpersonal perceptions, which will become important when we consider what it means to be accurate in perceiving status.

On a side note: egads, am I becoming a social constructivist?

Reference:

Srivastava, S. & Anderson, C. (in press). Accurate when it counts: Perceiving power and status in social groups. In J. L. Smith, W. Ickes, J. Hall, S. D. Hodges, & W. Gardner (Eds.), Managing interpersonal sensitivity: Knowing when—and when not—to understand others.