What the Heck is Research Anyway? (A guest post by Brent Roberts)

Brent Roberts recently showed me a copy of this essay he wrote to explain to family members what he does for a living. I thought it would make a neat holiday-themed entry on the blog (a link to forward in response to “it must be so nice to have almost a month off between semesters!”). So I asked him if I could put it up as a guest post, and he kindly agreed. 

Recently, I was asked for the 17th time[1] by a family member, “So, what are you going to do this summer?”  As usual, I answered, “research.”  And, as usual, I was met with that quizzical look that says, “What the heck is research anyway?”

It struck me in retrospect that I’ve done a pretty poor job of describing what research is to my family and friends.  So, I thought it might be a good idea to write an open letter that tries explaining research a little better.  You deserve an explanation.  So do other people, like parents of students and the general public.  You all pay a part of our salary, either through your taxes or the generous support of your kid’s education, and therefore should know where your money goes.

First, I should apologize if my reaction to the question “What are you going to do this summer?” has been less than positive in the past.  It is hard not to react negatively.  Because when asked this question it is hard not to interpret it as really asking “Hey, you’re a teacher, and now that you are done teaching, what the heck are you going to do with yourself?”  Since scientific research is typically the majority of the work we do in the professoriate we tend to chafe at seeing our job pigeonholed in such a way.  In fact, when we are asked “are you done for the summer?”, we typically think to ourselves “I’m going to get a boat-load of research done, like four papers, two grants, and some progress made on my book, along with starting several new projects.”  In other words, we typically think along the lines of “I’m going to work my tail off this summer because I’m finally free of those teaching and service obligations which take me away from what I love to do and for that matter what I mostly get paid to do.”

Let me expand on that latter point a little before delving into what I mean by scientific research. As a professor at a major research university I am paid to do three things: Research, teaching, and service.  On the teaching side of things, we often teach what appears to be an appallingly small number of classes.  That said much of our teaching is done in the old-fashioned artisan-apprentice fashion—one-on-one with students.  We have countless meetings throughout our week outside of the classroom working with undergraduate and graduate students, and post-doctoral researchers teaching them how to do research.  In terms of service, we are tasked with helping to run our department and university, and with running the guilds to which we belong.  I can expand on that later if you like.  That said, one thing you may not have known is that at major research universities teaching and research service constitute less than 50% of our job description, combined.  You may expect us to take summers and winter breaks off, but our universities are smiling as we apply ourselves to what they hired us to do, research—often when they are not even paying us.  There’s nothing like free labor[2].

So what is research anyway?  Let me answer a slightly different question that my wife’s aunt asked recently as it will help frame the answer.  She asked, “What purpose does research serve?” Now there is probably less consensus on the answer to this question than I’d like, but ultimately, I think the answer is knowledge.  Research is supposed to provide knowledge that can be used by others and hopefully the broader society.  To illustrate, let me describe the number of ways in which the knowledge we generate might be used.

The most common way that the knowledge we create is used is by other researchers.  This is what you’ll hear described as “basic” research because it may or may not have a direct applied purpose.  This is about all most researchers can aspire to.  We are pretty happy if other researchers not only read our work, but also draw on it to inform their research too.  This is important because the knowledge we generate is not only read, but also built upon and extended in meaningful ways by others.  The next way our knowledge is used – and ultimately the way our research will most likely influence society – is through teaching.  Yes, our research hopefully gets incorporated into the classroom because it is summarized in textbooks or our original research articles are assigned as core reading.  In this way, our research forms the material that thousands of students learn in order to make themselves better-informed citizens who hopefully go on to be productive members of society.  Finally, our research might be used for more practical aims like shaping social policy set by State and Federal authorities, informing decisions made by employers or other organizations, or helping practitioners treat illness.  For example, recently a Nobel Prize winning economist discovered our work on the personality dimension of conscientiousness (being self-controlled, responsible, and organized).  He has conducted rigorous work demonstrating the importance of this psychological attribute to human potential, and has started lobbying congress and federal funding agencies to focus on how we can teach kids to be more conscientious.  Similarly, other scholars conduct research on how best to characterize psychopathology and, in turn, how that might affect the way we treat patients.  Not many researchers ever get this level of influence, but you see it in medical breakthroughs and engineering accomplishments on a regular basis.  So, ultimately, we do research to provide usable knowledge back to society.  At least, that’s my opinion.

So what do we do when we do this thing called research?  I can’t speak for all types of scientists, but here are what I believe to be the basic phases of the generic research project:

  1. We are posed with a problem, challenge, riddle, or question that needs to be solved or answered.  For example, Teresa might ask:  “How can an employer help workers to see work as more meaningful?”
  2. We come up with a method for answering the question.
  3. We assemble the tools and resources needed to conduct our research.
  4. We run the study intended to answer our question.
  5. We analyze the data that comes from our study.
  6. We write up our findings and send the paper off to a journal where it is reviewed by several (typically three) anonymous peers who, along with the journal editor, decide whether the way we answered the question provides an adequate answer and thus provides an incremental advancement to our knowledge.  If they think we did add something to the knowledge pool, then Hallelujah, our work gets published.

I know, that all sounds a little abstract.  So let me walk you through these steps in a little more detail.  What do I mean by a problem, challenge, riddle, or question and where do these things come from?  Well, typically, these riddles come from us knowing a lot about some particular area of knowledge.  By becoming an expert in a specific area you become aware of not only what we know, but also of what we don’t know, and more importantly, what we need to know.  To get the point of being able to ask the right question requires a lot of time reading, going to conferences, and meeting with other experts.  That is why our graduate education took so long.  That is why we spend a lot of time with our noses stuck in books and journals.  We need to know.

Once we have a grasp of some issue, then comes the fun—and hard part—coming up with the question and or idea you want to test.  I think this is the fun part because it is often the most creative part of the job.  It is like solving a riddle or puzzle.  You have a bunch of disparate facts and you need to put them together in a new way.[3] This is one reason we are so often lost in thought.  It is hard to turn the thinking off and new ideas might come to us at any time and from any source.  One time I cooked up a program of research by having a discussion with my mother-in-law about the Ten Commandments. Inspiration can strike anywhere, any time.

The way we test our ideas is often tied to what kind of researcher we are.  That said, there are only so many options available.  The key to our choice of method is that anything we do should be transparent to others, replicable (i.e., we or someone else should be able to do what we did again and get the same results), and systematic so that other researchers can duplicate our efforts.  Our methods range from simple observation and documentation—this might result in a book or case study—to surveys where we see if two things go together (e.g., age and maturity)—to experiments where we obsessively control all extraneous variables so that we can get an idea of something causes something else (e.g., does increasing empathy for others increase cooperation?).  Often our choices are determined by our question—for example, personality change is hard to study using experiments.  Changing someone’s mood is relatively easy and easily tested with an experiment.  Some of our choices are determined by technology—we can now take pictures of the brain in action, something not available to researchers in previous generations.  We like technology, especially if it is new.[4]

Assembling our tools ranges from the simple—like going to a library—to the complex, like ordering a great big atom smasher or a satellite to be delivered to high earth orbit.[5]  Many of us will populate a lab space with necessary equipment. Some of us will work solo on a computer in our office.  Usually, we work in teams of 2 or more people and we sometimes work with researchers from other universities.  Graduate students are often part of the team, but it can also include undergraduates, post docs, and administrative staff.  Remember when I said we spend a lot of time teaching students one-on-one?  This is a good example. For some of us, our labs become very much like a small business.  Of course, to get to that stage, we typically need grant money, but that’s a topic for a different letter.

After we come up with our idea and assemble our team to work on it, we run the study.  This can be as simple as borrowing other people’s data—economists seem to do this a lot—or more likely, we’ll run the study with people or animals, or things, in our labs.  Sometimes this goes fast.  Running a small experiment with undergraduate students can take as little as a few days.  Sometimes this goes slow.  I’ve been running several longitudinal studies now for 10 years.  I may never stop.  Some researchers become famous because at this stage they are either very creative in how they test their ideas or ingenious in how they develop their techniques.  This type of technical skill is an underappreciated aspect of the job.

After we’ve collected our data, we analyze it.  This is where that dreaded concept of statistics rears its ugly head.  To be honest, some of us get really excited at this stage.  Okay, to be really honest, I get excited at this stage.  Call me a nerd.  I’m okay with that.  This is also where we lose our audience.  You’ve probably heard us invoke statisticalese in describing our work or some other finding.  It has a universal effect on the neurobiology of human brains—it puts 99% of them to sleep[6].  Again, please accept our apologies. If we start down this path, ask us to explain it in language that normal people can understand.

Finally, we write.  This stage would be great for us, and others, if we could write like normal writers, but we can’t.  We have to write for an academic audience.  This means that most of the rhetorical techniques used by creative writers to keep readers engaged are off limits.  We must choose our words carefully, be painfully consistent with those words, and hedge most everything we say.  This doesn’t mean it is bad writing, just typically not that exciting—closer to a user’s manual than pulp fiction- and it’s full of arcane terms that only people in our field are likely to understand.

Of course, once we’ve written our research article we need to submit it to a scientific journal and have it reviewed.  This step is what makes our work different from magazine writers, pundits, or reporters.  We can’t just spout off.  Our ideas need to be vetted by other knowledgeable researchers.  More often than not, our papers get rejected, or at best rejected with an invitation to make revisions along the lines of the criticisms laid out by the reviewers.  In other words, you have anonymous people ripping your precious ideas, hard work, and painful writing to shreds.  It hurts.  You will see us at our most depressed following rejections of our work.  Considering the fact that a typical research project can often take upwards of three years from inspiration to rejection, a little depression is warranted.  Eventually, some of our work gets published. Then, hopefully, somebody uses it, somehow.

Now, multiply this process several times over and you get an idea of our research lives.  Most of us work on several projects simultaneously.  It keeps us busy and off the streets at night.  For that matter, it keeps us off the streets during the day too—thus the pasty complexion.

So, that’s research.  Sorry to be long winded.  That’s one reason we don’t elaborate on our answer to your questions concerning our summertime activities.  Your eyes would be glazed over before we got to the second paragraph.  Keep your questions coming, though. Right now, I gotta go do some research.

[1] I’ve been doing this professor thing for 17 years now.

[2] Our employers, Universities, typically pay us on a 9-month contract.  They like research because it makes our institutions famous.  The more famous the institution, the more likely students will come and the more likely granting agencies will give us money.   Teaching and service are important, but research brings in the dough.

[3] I often find ideas come to me in the bathroom, which has led to the “proximity to porcelain” hypothesis.  Being in contact or near porcelain acts like a catalyst for new ideas.  Or, alternatively, it is the only time you are left alone for long enough to think.

[4] The ideas described here would offend our colleagues who consider themselves “post modern” or “deconstructivists”.  They don’t believe in replicable knowledge or anything roughly thought of as the scientific method.  We mostly humor them as they eat their own departments and fields from the inside out and then we take their faculty lines for our own.

[5] No kidding.  Some physicists at Berkeley did this while I was there.

[6] Another fMRI study begging to be done….

A job for a personality psychologist

August has been a busy month for me, with lots of travel and now preparations for a sabbatical. Hoping to pick up the blogging pace after I’m resettled. In the meantime (and apropos of an earlier post and commenter’s request for info about finding academic jobs in personality psychology), here is an ad for a job at Washington University at St. Louis:

WASHINGTON UNIVERSITY IN ST. LOUIS, Department of Psychology is seeking candidates for an Assistant Professor position in the area of personality psychology with strong quantitative skills, to teach psychology or related courses, conduct research, publish in peer-reviewed journals, advise students and participate in faculty meetings.  A primary teaching responsibility for the candidate will be in the graduate statistics curriculum in the psychology department. Any area of personality psychology will be considered, though preference will be given to candidates with a strong quantitative background.  The primary qualification for this position is a doctorate and a demonstrated excellence in research and teaching.  We especially encourage applications from women and members of minority groups.

Please email a curriculum vitae, reprints, a short statement of research interests and teaching experience, and three letters of reference to Cheri B. Casanova at cbcasano@wustl.edu.  The Search Committee will begin the formal review process Nov. 1, 2010; applications will be accepted until the position is filled.  Washington University is an equal opportunity/affirmative action employer.  Employment eligibility verification is required upon hire.

Note that WUSTL has another job in behavior genetics, which may overlap substantially with personality psychology.

Making progress in the hardest science

The name of this blog comes from a talk I gave at the SPSP conference.

The talk was in a training symposium for people starting out in academic psychology. People at various stages of their careers were invited to talk about how we approach research. I titled my talk “Making Progress in the Hardest Science,” and the first third of the talk was a half-serious, half joking explanation of the title.

The idea is that you often hear people arrange the sciences on a continuum from “hard” to “soft,” with physics at the hard end and psychology at the soft end. The implicit message is that the “hard” sciences are more scientific. But that’s not based on anything fundamental or substantive. As best as I can tell, it’s about scienciness. We have these preconceptions and stereotypes about what science is supposed to be about — big fancy equipment, lab coats, etc. But that’s not science. That’s the superficial sheen of scienciness.

What science is is a method of inquiry. In a nutshell, it’s the application of logic to empirical evidence. And by that measure, physics and biology and psychology are equally and fully scientific, because we’re all trying to figure stuff out by systematically gathering evidence and applying logic to it. (Or at least our respective academic versions are. I offer no defense of the Doctor Firstnames in your local bookstore.)

So it doesn’t make sense to try to determine who is more scientific. Instead, what differs is what we are trying to figure out — the phenomena we strive to understand. And here I think the other meaning of “hard” is useful.

What are the “hard” — as in difficult — problems in science? Hard problems in science are those that are embedded in complex systems; they are hard because to study something well you often need to isolate it from outside influences. Hard problems are those that vary by local conditions — science seeks to identify general laws, and when something is locally dependent, you need to sniff out the complex interactions that make it so. And hard problems are those that are difficult to quantify — science rests upon formalization and quantification, and you need to get traction at that initial step of quantification (i.e., measurement) before you can test theories. So… by these measures, if we are going to differentiate areas of science, the continuum of scientific problems should go from “hard” to “easy,” and psychology is clearly a science that deals with hard problems. Perhaps the hardest.

This was mostly intended as cheerleading to an audience of budding academic psychologists, revolving around a debatably clever pun. (I don’t really mean to suggest that physicists and chemists are picking the low-hanging fruit of the scientific disciplines. The low branches were picked clean centuries ago.) But the human mind is an amazingly complicated thing to study, which is what makes psychology so much fun.