This far, we have discussed participants and their choices, identified opportunities and challenges when interactive systems are used and discussed assignment strategies. The remaining challenge is to identify if there are differences between treatments.
Commonly, we look at the difference through explicit measures. Even with interactive systems we may ask for survey responses fill in some output surveys or to examine how they acted in the interactive systems during the experiment. Another dimension of experiments is to move away from laboratory settings into the wild, so more opportunities become available. Surveys can still be used, but they can benefit from longer-term exposure analysis such as traditional survey panels (Bail et al., 2018). However, scholars can move forward and examine if the treatment had longer-term effects on the participantsâ behaviour, such as tweets they produce (Munger, 2017) where he tried to understand how the tweets produced by test participants changed in response to the intervention. As this example shows, the analysis can benefit from other computational social science tools as well.