Unknowable Manipulators: Social Network Curator Algorithms

For a social networking service to acquire and retain users, it must find ways to keep them engaged. By accurately gauging their preferences, it is able to serve them with the subset of available content that maximises revenue for the site. Without the constraints of an appropriate regulatory framework, we argue that a sufficiently sophisticated curator algorithm tasked with performing this process may choose to explore curation strategies that are detrimental to users. In particular, we suggest that such an algorithm is capable of learning to manipulate its users, for several qualitative reasons: 1. Access to vast quantities of user data combined with ongoing breakthroughs in the field of machine learning are leading to powerful but uninterpretable strategies for decision making at scale. 2. The availability of an effective feedback mechanism for assessing the short and long term user responses to curation strategies. 3. Techniques from reinforcement learning have allowed machines to learn automated and highly successful strategies at an abstract level, often resulting in non-intuitive yet nonetheless highly appropriate action selection. In this work, we consider the form that these strategies for user manipulation might take and scrutinise the role that regulation should play in the design of such systems.