All data are highly contextual. They originate from particular settings and are generated through particular processes. This is also true for `big data' and is illustrated well in critical data studiesâ scholarship. Already in 2012, Boyd and Crawford (2012) stated that
Data are not generic. There is value to analyzing data abstractions, yet retaining context remains critical, particularly for certain lines of inquiry. Context is hard to interpret at scale and even harder to maintain when data are reduced to fit into a model. Managing context in light of Big Data will be an ongoing challenge.
What they mean is that there is no objective data. Rather, data can have many factors that influence how it should be used or what kind of valid interpretations can be made from it. The challenge they seek to address is that while context influences the shoulds, it does not limit the cans. It is easy to draw the wrong kinds of conclusions from data if the settings and processes in data generation are not properly accounted for.
One such context is the platform under examination. Digital communication is always bounded by the platforms, which can have different opportunities for interaction (or affordances), different user cultures or even a different user population. Research on cross-platform studies have shown that styles of interaction have platform-related implications (e.g. Griggio et al., 2019; Alhabash and Ma, 2017; Waterloo et al., 2018; Nelimarkka et al., 2020). Even interpretations of emojis may differ based on their visual representation, which is specific to platforms (Miller et al., 2016). However, to broaden the context beyond digital platforms, the venue where communication takes place is highly relevant in the analysis. Yu et al. (2008) tried to classify the party (Republican or Democrat) using speech data from the members of the Senate in the United States based on previously classified speech data from the House (and vice versa). They show that the classification accuracy across venues is just marginally better than tossing a coin. Similarly, Hoffman et al. (2017) examined a pre-made classification tool used often to measure politeness from textual data. Their results suggested that the classification tool did not generalise beyond its original context, and the results in their novel data set were clearly biased. The system was not valid for the analysis. They end their paper by highlighting that the tool has been used in over 40 studies and argue that
[- -] the results of prior studies that have relied on the Danescu-Niculescu-Mizil et al. tool should be carefully reconsidered. In particular, studies that make claims about the neutral and impolite portions of the scale probably suffer from serious inaccuracy issues relative to what humans would actually say about those texts.
Their analysis highlights how context, such as platform or venue of the study can have drastic implications for the results. The literal site of research is not the only context that might relate to the validity of research findings. Similarly, temporal aspects may change and lead to differences in any interpretations. For example, Yu et al. (2008) illustrated how the accuracy of classification changes over time. Their models were trained using speech data from 2005. Their models were rather accurate to predict the party in the early 2000s. The accuracy dropped the further in history the model was applied.
Beyond the background of big data (i.e. where and when it was created), it is critical to understand the social practices of the platform and what forms of human behaviour emerge. Social media is, for example, described as mass-personal communication (O'Sullivan and Carr, 2018), combining elements of traditional interpersonal communication with more public mass communication. More broadly, the challenge is that people engage in self-presentation and need to consider how they express themselves to their different audiences. Social media posts can be carefully crafted content to support profile work (Uski and Lampinen, 2016). They might also be more biased towards positive expressions (Reinecke and Trepte, 2014), at least for some of the platforms where digital communication takes place (, ). Similar problems are also visible in analysing social networks using digital trace data. Onnela et al. (2007) examined large-scale social networks using phone call logs. This might have been a valid choice in the early 2000s, but already at that point teens used different communication tools depending on the depth of friendship (Van Cleemput, 2010). Therefore, different data sources (or media channels) would lead to highlight different understandings of the social network (Yang et al., 2013; Karikoski and Nelimarkka, 2010). To understand ties in a social network, one needs to understand the social practices that relate to different platforms. The same problems may occur in various settings. When I was examining collaboration relationships in a company, it originally seemed that everyone was connected to everyone. More closely examining the hourly reporting data I was using, it became obvious that this was artifactually caused by their ways of reporting hours. When employees did not know which customer a task should be billed to, it was classified as an internal miscellaneous task. As I had not worked in the organisation, I did not know this detail, but asking from a few employees, it became obvious that this practice was the reason for the networks to be as dense as they were.
These were examples where the context related to situational aspects of behaviour. This directly influences what kinds of digital trace data were left from the behaviour and thus any analysis drawing from that data. However, another critical factor in context is to understand social production of data. For example, data often include classification such as questions about race or gender. Bowker and Star (2000a) show how classifications are re-establishing particular assumptions from the data. A good example in the social sciences is the U.S. census, which has changed its formulation of responders' race over time to become more inclusive. Similarly, the way we ask people about their gender is currently being debated extensively. Outside survey data - where such classifications are prominent â and other data sources may have similar issues. For example, studies about conditions for democratic societies (which we discussed in Section 3.5, see directly Jurek and Scime, 2014) depend heavily on what constitutes a democratic society. Therefore, the results from classifications replicate the beliefs and politics that go into the classification of data. The data is not context free.
While I have discussed challenges that relate to interpreting data, the challenge of research context is by no means only limited to computational methods. Social scientists might see similarities in quantitative and qualitative methods that they use every day. For example, a common critique for survey research is how the responder understands and reads the questions or, using terms I have used, in what context the respondent answers questions. Similarly, analysis of interview data similarly demands that the researchers understand what an interviewee says and why they speak in a particular manner. With `big data' these challenges have reappeared in our scholarship and have required comprehensive discussion among academics. Therefore, when using any form of data, I recommend caution when and how to draw conclusions from them.