Data Driven

“What is the continuing role of the qualitative in an era devoted to data?” – Gideon Lewis-Kraus The Trials of Alice Goffman

Data vs Metrics: Sacrificing depth?

If an accent challenge video has more dislikes than likes what does that tell us? That more people responded negatively to the video’s content? But what is the sample? The sum of the likes and dislikes? The number of views? Is it possible to determine how many of those views are unique views? Does any of this help us to determine whether approval or disapproval is the general response to the video?

While the lure of vast amounts of data available through the internet may be enticing for researchers, they run the risk of limiting their research agenda or research methods by the availability of data or by the metrics used on online platforms. The superficial data collection tools of online platforms (comment sections, likes and dislikes, share buttons etc.), whose primary function is to better inform marketing strategies, should not be the most influential factor in determining the research questions being asked.

Starting with the question or starting with data? Where to look?

In their article on the commodification of linguistic stereotypes in Volkswagen’s 2013 Super Bowl commercial, Quiana Lopez and Lars Hinrichs undertake a research project whose approach could be considered some form of Citizen Sociolinguistics. One of their research questions sought to understand how the commercial was interpreted by American and Jamaican viewers and they collected comments from websites as shown below: 

The researchers determined commenters’ “right to speak” by their self identification as Jamaican or as “an American of color”  or by claims of having family members that belonged to these groups. They then used discourse analysis to determine the overall reception of the commercial by Jamaicans and Americans.  

When I first reviewed this list, it was striking to me that there are no Jamaican websites listed, despite the fact that the debate about the commercial was taken up by Jamaican media, with one outlet going as far as interviewing the main actor. For me, this presents issues of validity and ethics. How does the choice of websites impact the sample of comments analyzed and ultimately the findings of Lopez and Hinrichs’ study? Additionally, how reliable is anonymous self-identification? Can the findings be considered a fair portrayal of American and Jamaican reactions based on the websites used?

The study’s findings stated that, although the majority of Jamaicans did not think that the commercial was racist, the inauthentic use of a non-American stereotype by an American production company to create a commercial for an American audience, meant that the commercial could be considered “a vehicle for indirect racism”. They also add that from a methodological standpoint, “pronouncements on the social indexicality of linguistic forms have often leaned too heavily on subjective interpretive reading alone” and call for additional analysis of data from those involved on the production side of mediatized speech (Lopez & Hinrichs 2017 p. 153).

The phrase "subjective interpretive reading" has stuck with me and I think the researchers' criticism comes from their inability to deal with the complexity offered by the perspectives of netizens. By treating the comments as decontextualized data, instead of perspectives of real people, the researchers were limited in their ability to interpret the comments beyond their face value. I find it difficult to conceive of a similar study, done with participants in real life, where such short fragments of text/speech would be deemed independently sufficient for discourse analysis and/or representative of perspectives of entire nations. Culling online data of this kind, without using complementary data collection methods such as interviews should be an immediate red flag. It is unacceptable to treat metrics or online artifacts as reliable independent sources when it is nearly impossible to verify online authorship. 

Questions

Should Citizen Sociolinguistics be aligned with qualitative or quantitative methodologies exclusively? What would be the pros and cons for either choice?

Is Citizen Sociolinguistics amenable to all traditional research methods? What would a CS ethnography look like?

Markham & Buchanan call for a process-based approach to ethics for internet research, what guidelines would you propose as a starting point for Citizen Sociolinguistics?  

References 

Lopez, Q. & Hinrichs, L. (2017). C’mon, Get Happy: The Commodification of Linguistic Stereotypes in a Volkswagen Super Bowl Commercial. Journal of English Linguistics. 45(2) 130- 156.