Interactive systems can be used as research instruments. Most people use many interactive systems, such as social media websites, document editing software or mobile phone applications, on a daily basis. These kinds of data are often observational. We know what people have done, but we do not have any control of the system where these interactions take place. To challenge this, the experimental approach invites researchers to be involved and plan different cases and test specific hypotheses through these cases. Classical and famous studies include experiments on kinds of social pressures that people obey from external authorities (Milgram, 1963) or under what conditions participants follow the majorityâs opinions (Asch, 1951). Social scientists have used the experimental paradigm from highly controlled laboratory experiments to natural experiments across various disciplines of social sciences. Using an experimental paradigm, researchers create a situation where participants are asked to make choices and interact. Thus, it is not purely observational but rather can identify the impact of various treatment conditions of participants' behaviours. Constructive research provides tools to develop our own interactive systems and therefore run experiments to develop different conditional structures on digital platforms.
The obvious benefit from digital platforms is the scale. Running a study with 15,000 Salganik et al. (2006) or 61 million participants Bond et al. (2012) gives a larger sample size () and, therefore, stronger statistical power. Traditional laboratory experiments do not scale easily as participants are often pooled from nearby areas and invited to the laboratory to participate in the experiment. Reaching a large number of participants can be difficult as people tend to be busy, but researchers often compensate for participants' time. (In Helsinki, the current rate is two to four movie tickets for one hour of participants' time.) Digital crowdwork platforms, like Amazonâs Mechanical Turk, increase the pool of potential subjects available to researchers. Crowdworkers can be asked to participate in survey research, use applications specially designed for research or even conduct activities outside the crowdwork applications. It seems that crowdwork populations resemble closely enough traditional research populations, suggesting that the findings are likely to replicate (e.g. Crump et al., 2013).
A common approach is to use survey experiments with crowdworkers or through a polling company if there are specific concerns about ensuring representativeness of the target population. In a survey experiment, participants respond to a survey. However, the survey is not the same for all participants but rather is randomly assigned to different treatment conditions. Survey experiments are commonly used to study effects of media framing. Some of the responders will see a news vignette manipulated to convey framing A, some will see framing B and some belong to a control condition (i.e. they do not see any vignette). Next, participants answer additional survey questions, and researchers examine if the responses are different between treatment groups A and B and control group C even when the populations are similar before the treatments. The survey platform, such as LimeSurvey or Qualtrics, is an interactive system that is controlled by researchers to act differently for users to deliver the experimental conditions.
However, this setting is rather naive and limits our opportunities when conducting experimental studies. We are only able to provide different stimuli to individual users. Researchers can be interested in more versatile settings, like studying the social settings where interactions take place, or control of these interactions - like Milgram (1963) and Asch (1951) have done. With different kinds of interactive systems, we can capture such aspects. Coetzee et al. (2015) studied what kinds of interactions ensure that a group discussion leads to learning gains. They developed a chat platform that allowed study participants to discuss a learning problem and invited participants from Amazon Mechanical Turk to participate for approximately 15 minutes of discussion. They logged all interactions and examined differentiated discussions, where participants learned from those where learning gains were not present. Therefore, their own platform allowed moving away from the limited approach of survey experiments and illustrated the possibilities available for those willing to invest in project development.
Further direction with such systems is to develop interactive systems as real consumer-facing services - intended to be used outside the laboratory by real people. These in-the-wild studies or field experiments (Brown et al., 2011) tackle the major challenge of any laboratory experiments. In-the-wild deployments ensure that the study takes place among the people and not in an artificial situation. Both Salganik et al. (2006) and Bond et al. (2012) demonstrated this research approach. Developed systems were used [MusicPlanet for Salganik et al. (2006) and Facebook for Bond et al. (2012)]. These studies are almost similar to natural experiments. They take place in real life. However, as these systems are governed by researchers, they also can control what takes place in these systems and what experimental conditions exist.
A fifth and final opportunity to use interactive systems as research instruments is augmenting traditional laboratory studies. For example, virtual reality headsets and actuators may be used to create immersive experiences for laboratory study participants. Therefore, researchers may have high control of the environments and what takes place in them, ensuring that each participant gets exactly the same kind of treatment. Therefore, researchers can address topics that might be difficult to approach otherwise. For example, Ravaja et al. (2017) studied the emotional reactions created when the test subject is touched by a person. This would be a difficult experimental setting to achieve correctly. Controlling how people behave in social situations and different styles of touching another person - such as the length of the touch - is difficult. However, thanks to virtual reality and a small motor actuator, they could control the intensity of the touch (speed of motor creating vibrations) and the emotional expression of the virtual avatar created. Sometimes real robots could be used as well. For example, Axelsson et al. (2019) developed a robot Momo to help autistic children learn sign language. These kinds of robots could be used in social sciences to ensure that the interactive situation is always executed in the same fashion. As these were computationally created, they were consistent across conditions to ensure a proper analysis could be executed. Similar tools are becoming available to change how laboratory experiments are conducted in the future, based on the idea that we can digitally construct interactive systems used for experimental studies.
All five opportunities for experimental studies through interactive systems still follow ideas of traditional experimental studies. Researchers have tight control on how the research environment behaves and what participants can do in that research environment. Furthermore, there are approaches available to assign participants to different conditions. For both of these, we discuss how to take them into account when developing interactive systems.