Security designs that presume enacting secure behaviors to be beneficial in all circumstances discount the impact of response cost on users' lives and assume that all data is equally worth protecting. However, this has the effect of reducing user autonomy by diminishing the role personal values and priorities play in the decision-making process. In this study, we demonstrate an alternative approach that emphasizes users' comprehension over compliance, with the goal of helping users to make more informed decisions regarding their own security. To this end, we conducted a three-phase redesign of the warning notifications surrounding the authentication ceremony in Signal. Our results show how improved comprehension can be achieved while still promoting favorable privacy outcomes among users. Our experience reaffirms existing arguments that users should be empowered to make personal trade-offs between perceived risk and response cost. We also find that system trust is a major factor in users' interpretation of system determinations of risk, and that properly communicating risk requires an understanding of user perceptions of the larger security ecosystem in whole.
Data coming soon
For the first user study, 15 pairs of participants (90 participants total) interacted with each treatment (a different warning flow when Signal is under attack). Each participant was paid $15. Each study lasted approximately 40 minutes.
Study Coordinator Guide (note the study guide references a fourth treatment but this was not pursued)
Survey (note, survey logic is not shown, and some questions were skipped based on the participant answer to a prior question)
We conducted an icon survey in which 50 participants rated potential icons for use in the re-design. Each participant was paid 10 cents and the task was estimated to take about 30 seconds to complete.
We conducted a simulation of the message-not-delivered and messaged-blocked notification flows and then had users complete a survey. We also simulated the blue banner element and an introduction screen explaining our indicators using the shield icon selected from the above icon survey. Each participant viewed a single simulation and was paid $1.00. The task was estimated to take about 5 to 10 minutes to complete. A total of 223 participants interacted with the simulations.
For the second user study, 15 pairs of participants (30 participants total) interacted with the re-designed Signal with a simulated attack. Each participant was paid $15. Each study lasted approximately 40 minutes.
The study materials were identical to the first flow. Since the re-design used a blocking flow similar to treatment 3, we used the Treatment 3 role-play.