La reacción por Twitter no siempre es la del público

Twitter Reaction to Events Often at Odds with Overall Public Opinion

The reaction on Twitter to major political events and policy decisions often differs a great deal from public opinion as measured by surveys. This is the conclusion of a year-long Pew Research Center study that compared the results of national polls to the tone of tweets in response to eight major news events, including the outcome of the presidential election, the first presidential debate and major speeches by Barack Obama.

At times the Twitter conversation is more liberal than survey responses, while at other times it is more conservative. Often it is the overall negativity that stands out. Much of the difference may have to do with both the narrow sliver of the public represented on Twitter as well as who among that slice chose to take part in any one conversation.

A More Liberal Twitter Reaction to Some Events

In some instances, the Twitter reaction was more pro-Democratic or liberal than the balance of public opinion. For instance, when a federal court ruled last February that a California law banning same-sex marriage was unconstitutional – a case that is now coming before the Supreme Court – the reaction on Twitter was quite positive. Twitter conversations about the ruling were much more positive than negative (46% vs. 8%). But public opinion, as measured in a national poll, ran the other direction: Of those who had heard about the ruling, just 33% were very happy or pleased with it, while 44% were disappointed or angry.

And this was also evident when it came to the fall presidential campaign. For example, while polls showed that most voters said Mitt Romney gave the better performance in the first presidential debate, Twitter reaction was much more critical of Romney, according toan analysis of social media reaction to the debate.

And when Obama won the election on Nov. 6, the post-election conversation on Twitter was very positive about his victory. The analysis showed an overwhelming majority (77%) of post-election Twitter comments about the outcome were positive about Obama’s victory while just 23% were negative. But a survey of voters in the days following the election found more mixed reactions to the election outcome: 52% said they were happy about Obama’s reelection while 45% were unhappy.

This tilt to the Twitter conversation was evident throughout the fall campaign. In nearly every week from early September through the first week of November, the Twitter conversation about Romney was substantially more negative than the conversation about Obama.

Still, the overall negativity on Twitter over the course of the campaign stood out. For both candidates, negative comments exceeded positive comments by a wide margin throughout the fall campaign season. But from September through November, Romney was consistently the target of more negative reactions than was Obama.

Twitter Reactions Not Always More Liberal

The pro-Democratic or liberal tilt of tweets was not always apparent in the Pew Research Center case studies. The reaction on Twitter to Obama’s second inaugural address and his 2012 State of the Union was not nearly as positive as public opinion.

The contrast was particularly striking in assessments of last year’s State of the Union. The president’s speech was generally well-received by the public: 42% said they had a positive reaction while 27% had a negative reaction. On Twitter, however, the conversation about Obama’s speech was far more negative (40%) than positive (21%).

More recently, Obama’s second inaugural address received more positive than negative assessments in a national survey conducted after the speech. But the conversation about the speech on Twitter tilted more toward criticism than praise.

Of the eight events that the Pew Research Center tracked since the beginning of last year, there were two – Mitt Romney’s selection of Paul Ryan as his vice presidential running mate and the Supreme Court’s ruling on the 2010 Affordable Care Act – when the reaction on Twitter paralleled public opinion.

When Mitt Romney tapped Ryan as his running mate, it received a more negative than positive reaction both from the general public and in the conversation on Twitter. And when the Supreme Court handed down its ruling upholding the health care law in June 2012, public reaction was split: A national survey found 36% approving and 40% disapproving of the Court’s decision. The reaction on Twitter was about the same: Among those offering a viewpoint, roughly half were positive comments and half were negative.

Why Twitter May Be Different at Times

The lack of consistent correspondence between Twitter reaction and public opinion is partly a reflection of the fact that those who get news on Twitter – and particularly those who tweet news – are very different demographically from the public.

The overall reach of Twitter is modest. In the Pew Research Center’s 2012 biennial news consumption survey, just 13% of adults said they ever use Twitter or read Twitter messages; only 3% said they regularly or sometimes tweet or retweet news or news headlines on Twitter.

Twitter users are not representative of the public. Most notably, Twitter users are considerably younger than the general public and more likely to be Democrats or lean toward the Democratic Party. In the 2012 news consumption survey, half (50%) of adults who said they posted news on Twitter were younger than 30, compared with 23% of all adults. And 57% of those who posted news on Twitter were either Democrats or leaned Democratic, compared with 46% of the general public. (Another recent Pew Research Center survey provides even more detail on who uses Twitter and other social media.)

In another respect, the Twitter audience also is broader than the sample of a traditional national survey. People under the age of 18 can participate in Twitter conversations, while national surveys are limited to adults 18 and older. Similarly, Twitter conversations also may include those living outside the United States.

Perhaps most important, the Twitter users who choose to share their views on events vary with the topics in the news. Those who tweeted about the California same-sex marriage ruling were likely not the same group as those who tweeted about Obama’s inaugural or Romney’s selection of Paul Ryan.

This is clear when we look at the volume of Twitter discussion on each of the events studied. In the two days following Obama’s re-election on Nov. 6, there were nearly 14 million Tweets from people expressing their reaction. And more than five million expressed their reactions to the first presidential debate. But other events, particularly the federal court ruling on same sex marriage in California last February and Obama’s nomination of John Kerry in December, drew a much lower volume of tweets.

Overall, the reaction to political events on Twitter reflects a combination of the unique profile of active Twitter users and the extent to which events engage different communities and draw the comments of active users. While this provides an interesting look into how communities of interest respond to different circumstances, it does not reliably correlate with the overall reaction of adults nationwide

Methodology

Data regarding the tone of conversation on Twitter were derived by the Pew Research Center’s Project for Excellence in Journalism from a combination of traditional media research methods, based on long-standing rules regarding content analysis, along with computer coding software developed by Crimson Hexagon. That software is able to analyze the textual content from millions of posts on social media platforms. Crimson Hexagon(CH) classifies online content by identifying statistical patterns in words. The parallel opinion surveys were conducted by the Pew Research Center for the People & the Press during the same general time period as the Twitter data were aggregated.

The data on Twitter comes from an analysis of all publicly available Tweets. The time period for each event varied, but none included more than three days worth of reaction. For each subject, multiple search terms were used to identify appropriate tweets. For example, to find messages commenting on President Obama’s 2013 State of the Union Speech, Tweets were included if they appeared in the four hours following the start of his speech and used the words “state” and “union,” or “Obama,” or “SOTU.”
Unlike most human coding, CH does not measure each post as a unit, but examines the entire discussion in the aggregate. To do that, the algorithm breaks up all relevant texts into subsections. Rather than dividing each Tweet, paragraph, sentence or word, CH treats the “assertion” as the unit of measurement. If 40% of a story fits into one category, and 60% fits into another, the software will divide the text accordingly. Consequently, the results are not expressed in percent of Tweets, but rather the percent of assertions out of the entire body of stories identified by the original Boolean search terms.

Extensive testing by Crimson Hexagon has demonstrated that the tool is more than 90% reliable, that is, in more than 90% of cases analyzed, the technology’s coding has been shown to match human coding. Pew Research spent more than 12 months testing CH and its own tests comparing coding by humans and the software came up with similar results.

In addition to validity tests of the platform itself, PEJ conducted separate examinations of human intercoder reliability to show that the training process for complex concepts is replicable..

Pew Research Center