No, Twitter isn’t killing the research biz; reactivity is doing that job

According to Fabio Rojas (@fabiorojas), Associate Professor of Sociology at Indiana University, the arrival of “digital democracy” and the pervasive use of social media comprise a “new world [that] will undermine the polling industry,” and “put … campaign professionals out of work.”

So says an opinion piece in The Washington Post written by Rojas, which caused a small stir this morning on the AAPORnet listserv. Discussing findings from an impending conference paper which explores the use of Twitter analyses in predicting election results, Rojas claims:

For nearly a century, conventional wisdom has argued that we can only truly know what the public thinks about an issue if we survey a random sample of adults. An entire industry is built on this view. Nearly every serious political campaign in the United States spends thousands, even millions, of dollars hiring campaign consultants who conduct these polls and interpret the results.

Digital democracy will put these campaign professionals out of work.

Pollster Steve Koczela (@skoczela), who first noted the Rojas op-ed in the AAPOR discussion, dryly warned, “Well … looks like we are doomed.”

5-14-11

Humor aside, another user, Casey Tesfaye (@FreeRangeRsrch), summarized the source of some frustration with such grandiose pronouncements about social media analytics supplanting, rather than supplementing, the role of more traditional research techniques:

It is so easy for people to forget what they’re measuring when they discuss social media research. This isn’t a study that is representative of any general population. This study simply reflects the success of a collection of social media strategies. A successful social media strategy has some influence. … The findings in this study become much more meaningful in the context of other, more traditional studies of election dynamics.

None of that spells the end of polling as we know it, or amounts to a replacement for polling, which is focused on representing populations. But that doesn’t mean that people aren’t beating down the door of traditional polling for a cheaper and faster alternative. As others have mentioned on AAPORnet recently, this is an important time for us to publicly emphasize our unique value.

For me, the most important point is in Tesfaye’s second paragraph: In the opinion and marketing research communities, resource constraints are resulting in social media measurement being substituted for more rigorous and projectable methods of research. Most advocates of the swap probably don’t understand the implications of their advocacy or, if they do, don’t particularly care whether the resulting research product is of quality.

Not merely bad research technique, that is an abdication of the professional responsibility that researchers hold to help clients and other users of research make informed decisions on methodology and data interpretation. Individual researchers, their companies and our industry organizations need to educate our constituencies about how to embrace new sources of information like social media in professionally and ethically responsible ways.

So far, though, I don’t see a lot of leadership in this area, especially from industry groups. It’s almost as if the MRAs, CASROs and AAPORs are more worried about being cast as against innovation than concerned with using resources appropriately to reach sound policy and business decisions. So, we as an industry remain relatively silent in the face of silly assertions like the one from Rojas — which, while not novel, carries an imprimatur of academic authority — or come out with reactive defenses of the status quo, rather than staying ahead of the opinion curve (pun intended).

It’s up to the profession as a whole to help decision-makers understand that bad research is worse than no research at all. If we can’t muster the backbone to do that, we don’t deserve to be considered professionals.

2 comments

  1. Chris

    Funnily enough, I just wrote about managers using social comments more than social analytics in business decisions. The data came from some doctorate work I’m doing on data use in business decisions where I’m also finding that market research tools (surveys, customer feedback) are still high on ‘use’ list by business leaders. The important point from your article is that insights from needs to supplement not supplant research – social media data is one input in what should be a data triangulation leading to insights for business decisions.

    • Mike Donatello

      Chris, thanks for the comment.

      To be sure, I’m a big user of social data and wouldn’t for a minute suggest ignoring any source of appropriate information. And, if we can demonstrate predictive validity of social indicators in the estimation of election results, that’s great. My issue is with the way the study is framed as spelling the end of broad polling efforts.

      Having not read the study, all I have to go on is the author’s representations in the media; maybe the paper itself is more circumspect. Even so, I am really surprised that a tenured faculty member at a major university would be so cavalier in overstating the implications of his work. And, I’m even more bent out of shape that those of us who know better will probably let him get away with it.

Post a comment

You may use the following HTML:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>