Recommender systems and their ethical challenges

Rethinking Health Recommender Systems for Active Aging: An Autonomy-Based Ethical Analysis

Health Recommender Systems are promising Articial-Intelligence-based tools endowing healthy lifestyles and therapy adherence in healthcare and medicine. Among the most supported areas, it is worth mentioning active aging. However, current HRS supporting AA raise ethical challenges that still need to be properly formalized and explored. This study proposes to rethink HRS for AA through an autonomy-based ethical analysis. In particular, a brief overview of the HRS' technical aspects allows us to shed light on the ethical risks and challenges they might raise on individuals' well-being as they age. Moreover, the study proposes a categorization, understanding, and possible preventive/mitigation actions for the elicited risks and challenges through rethinking the AI ethics core principle of autonomy. Finally, elaborating on autonomy-related ethical theories, the paper proposes an autonomy-based ethical framework and how it can foster the development of autonomy-enabling HRS for AA.

Feedback Loop Failure Modes in Medical Diagnosis: How Biases Can Emerge and Be Reinforced

BACKGROUND

Medical diagnosis in practice connects to research through continuous feedback loops: Studies of diagnosed cases shape our understanding of disease, which shapes future diagnostic practice. Without accounting for an imperfect and complex diagnostic process in which some cases are more likely to be diagnosed correctly (or diagnosed at all), the feedback loop can inadvertently exacerbate future diagnostic errors and biases.

FRAMEWORK

A feedback loop failure occurs if misleading evidence about disease etiology encourages systematic errors that self-perpetuate, compromising future diagnoses and patient care. This article defines scenarios for feedback loop failure in medical diagnosis.

DESIGN

Through simulated cases, we characterize how disease incidence, presentation, and risk factors can be misunderstood when observational data are summarized naive to biases arising from diagnostic error. A fourth simulation extends to a progressive disease.

RESULTS

When severe cases of a disease are diagnosed more readily, less severe cases go undiagnosed, increasingly leading to underestimation of the prevalence and heterogeneity of the disease presentation. Observed differences in incidence and symptoms between demographic groups may be driven by differences in risk, presentation, the diagnostic process itself, or a combination of these. We suggested how perceptions about risk factors and representativeness may drive the likelihood of diagnosis. Differing diagnosis rates between patient groups can feed back to increasingly greater diagnostic errors and disparities in the timing of diagnosis and treatment.

CONCLUSIONS

A feedback loop between past data and future medical practice may seem obviously beneficial. However, under plausible scenarios, poorly implemented feedback loops can degrade care. Direct summaries from observational data based on diagnosed individuals may be misleading, especially concerning those symptoms and risk factors that influence the diagnostic process itself.

HIGHLIGHTS

Current evidence about a disease can (and should) influence the diagnostic process. A feedback loop failure may occur if biased "evidence" encourages diagnostic errors, leading to future errors in the evidence base.When diagnostic accuracy varies for mild versus severe cases or between demographic groups, incorrect conclusions about disease prevalence and presentation will result without specifically accounting for such variability.Use of demographic characteristics in the diagnostic process should be done with careful justification, in particular avoiding potential cognitive biases and overcorrection.

Ethics and responsible AI deployment

As Artificial Intelligence (AI) becomes more prevalent, protecting personal privacy is a critical ethical issue that must be addressed. This article explores the need for ethical AI systems that safeguard individual privacy while complying with ethical standards. By taking a multidisciplinary approach, the research examines innovative algorithmic techniques such as differential privacy, homomorphic encryption, federated learning, international regulatory frameworks, and ethical guidelines. The study concludes that these algorithms effectively enhance privacy protection while balancing the utility of AI with the need to protect personal data. The article emphasises the importance of a comprehensive approach that combines technological innovation with ethical and regulatory strategies to harness the power of AI in a way that respects and protects individual privacy.

Auditing YouTube's recommendation system for ideologically congenial, extreme, and problematic recommendations

Algorithms of social media platforms are often criticized for recommending ideologically congenial and radical content to their users. Despite these concerns, evidence on such filter bubbles and rabbit holes of radicalization is inconclusive. We conduct an audit of the platform using 100,000 sock puppets that allow us to systematically and at scale isolate the influence of the algorithm in recommendations. We test 1) whether recommended videos are congenial with regard to users' ideology, especially deeper in the watch trail and whether 2) recommendations deeper in the trail become progressively more extreme and come from problematic channels. We find that YouTube's algorithm recommends congenial content to its partisan users, although some moderate and cross-cutting exposure is possible and that congenial recommendations increase deeper in the trail for right-leaning users. We do not find meaningful increases in ideological extremity of recommendations deeper in the trail, yet we show that a growing proportion of recommendations comes from channels categorized as problematic (e.g., "IDW," "Alt-right," "Conspiracy," and "QAnon"), with this increase being most pronounced among the very-right users. Although the proportion of these problematic recommendations is low (max of 2.5%), they are still encountered by over 36.1% of users and up to 40% in the case of very-right users.

Depolarization of opinions on social networks through random nudges

Polarization of opinions has been empirically noted in many online social network platforms. Traditional models of opinion dynamics, based on statistical physics principles, do not account for the emergence of polarization and echo chambers in online network platforms. A recently introduced opinion dynamics model that incorporates the homophily factor-the tendency of agents to connect with those holding similar opinions as their own-captures polarization and echo chamber effects. In this work, we provide a nonintrusive framework for mildly nudging agents in an online community to form random connections. This is shown to lead to significant depolarization of opinions and decrease the echo chamber effects. Though a mild nudge effectively avoids polarization, overdoing this leads to another undesirable effect, namely, radicalization. Further, we obtain the optimal nudge probability to avoid the extremes of polarization and radicalization outcomes.

Information cocoons in online navigation

Social media and online navigation bring us enjoyable experiences in accessing information, and simultaneously create information cocoons (ICs) in which we are unconsciously trapped with limited and biased information. We provide a formal definition of IC in the scenario of online navigation. Subsequently, by analyzing real recommendation networks extracted from Science, PNAS, and Amazon websites, and testing mainstream algorithms in disparate recommender systems, we demonstrate that similarity-based recommendation techniques result in ICs, which suppress the system navigability by hundreds of times. We further propose a flexible recommendation strategy that addresses the IC-induced problem and improves retrieval accuracy in navigation, which are demonstrated by simulations on real data and online experiments on the largest video website in China. This paper quantifies the challenge of ICs in recommender systems and presents a viable solution, which offer insights into the industrial design of algorithms, future scientific studies, as well as policy making.

An Overview of Innovative Approaches to Support Timely and Agile Health Communication Research and Practice

Innovative approaches are needed to make health communication research and practice more timely, responsive, and effective in a rapidly changing information ecosystem. In this paper we provide an overview of strategies that can enhance the delivery and effectiveness of health communication campaigns and interventions, as well as research approaches that can generate useful data and insights for decisionmakers and campaign designers, thereby reducing the research-to-practice gap. The discussion focuses on the following approaches: digital segmentation and microtargeting, social media influencer campaigns, recommender systems, adaptive interventions, A/B testing, efficient message testing protocols, rapid cycle iterative message testing, megastudies, and agent-based modeling. For each method highlighted, we also outline important practical and ethical considerations for utilizing the approach in the context of health communication research and practice, including issues related to transparency, privacy, equity, and potential for harm.

Investigating Employees’ Concerns and Wishes for Digital Stress Management Interventions with Value Sensitive Design: Mixed Methods Study (Preprint)

BACKGROUND

Work stress places a heavy economic and disease burden on society. Recent technological advances include digital health interventions for helping employees prevent and manage their stress at work effectively. Although such digital solutions come with an array of ethical risks, especially if they involve biomedical big data, the incorporation of employees' values in their design and deployment has been widely overlooked.

OBJECTIVE

To bridge this gap, we used the value sensitive design (VSD) framework to identify relevant values concerning a digital stress management intervention (dSMI) at the workplace, assess how users comprehend these values, and derive specific requirements for an ethics-informed design of dSMIs. VSD is a theoretically grounded framework that front-loads ethics by accounting for values throughout the design process of a technology.

METHODS

We conducted a literature search to identify relevant values of dSMIs at the workplace. To understand how potential users comprehend these values and derive design requirements, we conducted a web-based study that contained closed and open questions with employees of a Swiss company, allowing both quantitative and qualitative analyses.

RESULTS

The values health and well-being, privacy, autonomy, accountability, and identity were identified through our literature search. Statistical analysis of 170 responses from the web-based study revealed that the intention to use and perceived usefulness of a dSMI were moderate to high. Employees' moderate to high health and well-being concerns included worries that a dSMI would not be effective or would even amplify their stress levels. Privacy concerns were also rated on the higher end of the score range, whereas concerns regarding autonomy, accountability, and identity were rated lower. Moreover, a personalized dSMI with a monitoring system involving a machine learning-based analysis of data led to significantly higher privacy (P=.009) and accountability concerns (P=.04) than a dSMI without a monitoring system. In addition, integrability, user-friendliness, and digital independence emerged as novel values from the qualitative analysis of 85 text responses.

CONCLUSIONS

Although most surveyed employees were willing to use a dSMI at the workplace, there were considerable health and well-being concerns with regard to effectiveness and problem perpetuation. For a minority of employees who value digital independence, a nondigital offer might be more suitable. In terms of the type of dSMI, privacy and accountability concerns must be particularly well addressed if a machine learning-based monitoring component is included. To help mitigate these concerns, we propose specific requirements to support the VSD of a dSMI at the workplace. The results of this work and our research protocol will inform future research on VSD-based interventions and further advance the integration of ethics in digital health.

The Psychology of Personalization in Digital Environments: From Motivation to Well-Being – A Theoretical Integration

The personalization of digital environments is becoming ubiquitous due to the rise of AI-based algorithms and recommender systems. Arguably, this technological development has far-reaching consequences for individuals and societies alike. In this article, we propose a psychological model of the effects of personalization in digital environments, which connects personalization with motivational tendencies, psychological needs, and well-being. Based on the model, we review studies from three areas of application—news feeds and websites, music streaming, and online dating—to explain both the positive and the negative effects of personalization on individuals. We conclude that personalization can lead to desirable outcomes such as reducing choice overload. However, personalized digital environments without transparency and without the option for users to play an active role in the personalization process potentially pose a danger to human well-being. Design recommendations as well as avenues for future research that follow from these conclusions are being discussed.

Social influence for societal interest: a pro-ethical framework for improving human decision making through multi-stakeholder recommender systems

In the contemporary digital age, recommender systems (RSs) play a fundamental role in managing information on online platforms: from social media to e-commerce, from travels to cultural consumptions, automated recommendations influence the everyday choices of users at an unprecedented scale. RSs are trained on users’ data to make targeted suggestions to individuals according to their expected preference, but their ultimate impact concerns all the multiple stakeholders involved in the recommendation process. Therefore, whilst RSs are useful to reduce information overload, their deployment comes with significant ethical challenges, which are still largely unaddressed because of proprietary constraints and regulatory gaps that limit the effects of standard approaches to explainability and transparency. In this context, I address the ethical and social implications of automated recommendations by proposing a pro-ethical design framework aimed at reorienting the influence of RSs towards societal interest. In particular, after highlighting the problem of explanation for RSs, I discuss the application of beneficent informational nudging to the case of conversational recommender systems (CRSs), which rely on user-system dialogic interactions. Subsequently, through a comparison with standard recommendations, I outline the incentives for platforms and providers in adopting this approach and its benefits for both individual users and society.

Blue collar with tie: a human-centered reformulation of the ironies of automation

When Lisanne Bainbridge wrote about counterintuitive consequences of the increasing human–machine interaction, she concentrated on the resulting issues for system performance, stability, and safety. Now, decades later, however, the automized work environment is substantially more pervasive, sophisticated, and interactive. Current advances in machine learning technologies reshape the value, meaning, and future of the human workforce. While the ‘human factor’ still challenges automation system architects, inconspicuously new ironic settings have evolved that only become distinctly evident from a human-centered perspective. This brief essay discusses the role of the human workforce in human–machine interaction as machine learning continues to improve, and it points to the counterintuitive insight that although the demand for blue-collar workers may decrease, exactly this labor class increasingly enters more privileged working domains and establishes itself henceforth as ‘blue collar with tie.’

Recommender systems for mental health apps: advantages and ethical challenges

Recommender systems assist users in receiving preferred or relevant services and information. Using such technology could be instrumental in addressing the lack of relevance digital mental health apps have to the user, a leading cause of low engagement. However, the use of recommender systems for digital mental health apps, particularly those driven by personal data and artificial intelligence, presents a range of ethical considerations. This paper focuses on considerations particular to the juncture of recommender systems and digital mental health technologies. While separate bodies of work have focused on these two areas, to our knowledge, the intersection presented in this paper has not yet been examined. This paper identifies and discusses a set of advantages and ethical concerns related to incorporating recommender systems into the digital mental health (DMH) ecosystem. Advantages of incorporating recommender systems into DMH apps are identified as (1) a reduction in choice overload, (2) improvement to the digital therapeutic alliance, and (3) increased access to personal data & self-management. Ethical challenges identified are (1) lack of explainability, (2) complexities pertaining to the privacy/personalization trade-off and recommendation quality, and (3) the control of app usage history data. These novel considerations will provide a greater understanding of how DMH apps can effectively and ethically implement recommender systems.

Leveraging Artificial Intelligence in Marketing for Social Good-An Ethical Perspective

Artificial intelligence (AI) is (re)shaping strategy, activities, interactions, and relationships in business and specifically in marketing. The drawback of the substantial opportunities AI systems and applications (will) provide in marketing are ethical controversies. Building on the literature on AI ethics, the authors systematically scrutinize the ethical challenges of deploying AI in marketing from a multi-stakeholder perspective. By revealing interdependencies and tensions between ethical principles, the authors shed light on the applicability of a purely principled, deontological approach to AI ethics in marketing. To reconcile some of these tensions and account for the AI-for-social-good perspective, the authors make suggestions of how AI in marketing can be leveraged to promote societal and environmental well-being.

Data usage-based privacy and security issues in mobile app recommendation (MAR): a systematic literature review

Purpose

Similarly, Zhu et al. (2014) and Zhang et al. (2014) stated that addressing privacy concerns with the recommendation process is necessary for the healthy development of app recommendation. Recently, Xiao et al. (2020) mentioned that a lack of effective privacy policy hinders the development of personalized recommendation services. According to the reported work, privacy protection technology methods are too limited for mobile focusing on data encryption, anonymity, disturbance, elimination of redundant data to protect the recommendation process from privacy breaches. So, this situation motivated us to conduct a systematic literature review (SLR) to provide the viewpoint of privacy and security concerns as mentioned in current state-of-the-art in the mobile app recommendation domain.

Design/methodology/approach

In this work, the authors have followed Kitchenham guidelines (Kitchenham and Charters, 2007) to devise the SLR process. According to the guidelines, the SLR process has three main phases: (1) define, (2) conduct the search and (3) report the results. Furthermore, the authors used systematic mapping approach as well to ensure the whole process.

Findings

Based on the selected studies, the authors proposed three main thematic taxonomies, including architectural style, security and privacy strategies, and user-usage in the mobile app recommendation domain. From the studies' synthesis viewpoint, it is observed that the majority of the research efforts have focused on the movie recommendation field, while the mainly used privacy scheme is homomorphic encryption. Finally, the authors suggested a set of future research dimensions useful for the potential researchers interested to perform the research in the mobile app recommendation domain.

Originality/value

This is an SLR article, based on existing published research, where the authors identified key issues and future directions.

Algorithmic abstractions of ‘fashion identity’ and the role of privacy with regard to algorithmic personalisation systems in the fashion domain

This paper delves into the nuances of ‘fashion’ in recommender systems and social media analytics, which shape and define an individual’s perception and self-relationality. Its aim is twofold: first, it supports a different perspective on privacy that focuses on the individual’s process of identity construction considering the social and personal aspects of ‘fashion’. Second, it underlines the limitations of computational models in capturing the diverse meaning of ‘fashion’, whereby the algorithmic prediction of user preferences is based on individual conscious and unconscious associations with fashion identity. I test both of these claims in the context of current concerns over the impact of algorithmic personalisation systems on individual autonomy and privacy: creating ‘filter bubbles’, nudging the user beyond their conscious awareness, as well as the inherent bias in algorithmic decision-making. We need an understanding of privacy that sustains the inherent reduction of fashion identity to literal attributes and protects individual autonomy in shaping algorithmic approximations of the self.

Artificial intelligence methods for a Bayesian epistemology-powered evidence evaluation

RATIONALE, AIMS AND OBJECTIVES

The diversity of types of evidence (eg, case reports, animal studies and observational studies) makes the assessment of a drug's safety profile into a formidable challenge. While frequentist uncertain inference struggles in aggregating these signals, the more flexible Bayesian approaches seem better suited for this quest. Artificial Intelligence (AI) offers great promise to these approaches for information retrieval, decision support, and learning probabilities from data.

METHODS

E-Synthesis is a Bayesian framework for drug safety assessments built on philosophical principles and considerations. It aims to aggregate all the available information, in order to provide a Bayesian probability of a drug causing an adverse reaction. AI systems are being developed for evidence aggregation in medicine, which increasingly are automated.

RESULTS

We find that AI can help E-Synthesis with information retrieval, usability (graphical decision-making aids), learning Bayes factors from historical data, assessing quality of information and determining conditional probabilities for the so-called 'indicators' of causation for E-Synthesis. Vice versa, E-Synthesis offers a solid methodological basis for (semi-)automated evidence aggregation with AI systems.

CONCLUSIONS

Properly applied, AI can help the transition of philosophical principles and considerations concerning evidence aggregation for drug safety to a tool that can be used in practice.

Intelligent environments for all: a path towards technology-enhanced human well-being

Emerging intelligent environments are considered to offer significant opportunities to positively impact human life, both at an individual and at a societal level, and in particular to provide useful means to support people in their daily life activities and thus improve well-being for everybody, especially for older people and for people with limitations of activities. In this context, accessibility and usability, although necessary, are not sufficient to ensure that applications and services are appropriately designed to satisfy human needs and overcome potential functional limitations in the execution of everyday activities fundamental for well-being. This position paper puts forward the claim that, in order to achieve the above objective, it is necessary that: (i) the design of Assistive Intelligent Environments is centered around the well-being of people, roughly intended as the possibility of executing the (everyday) human activities necessary for living (independently), thus emphasizing usefulness in addition to usability; (ii) the technological environment is orchestrated around such activities and contains knowledge about how they are performed and how people need to be supported to perform them; (iii) the environment makes use of monitoring and reasoning capabilities in order to adapt, fine-tune and evolve over time the type and level of support provided, and this process takes place considering ethical values; (iv) the applications must also support the possibility of contact with other people, who in many cases may be the only effective help. Moving forward from the Design for All paradigm, this paper discusses how the latter can be revisited under the perspective of technology's usefulness and contribution to human well-being. Subsequently, it introduces a practical notion of well-being based on the ICF classification of human functions and activities and discusses how such notion can constitute the starting point and the focus of design approaches targeted to assist people in their everyday life mainly (but not exclusively) in the home environment. As a subsequent step, the need for integrating Artificial Intelligence capabilities in assistive intelligent environments is discussed, based on the complexity of the human problems to be addressed and the diversity of the types of support needed. The proposed approach is exemplified and illustrated through the experience acquired in the development of four applications, addressing vital aspects of human life, namely nutrition, stress management, sleep management and counteracting loneliness. Finally, based on the acquired experience, the need to take into account ethical values in the development of assistive intelligent environments is discussed.

The ethics of algorithms: key problems and solutions

Research on the ethics of algorithms has grown substantially over the past decade. Alongside the exponential development and application of machine learning algorithms, new ethical problems and solutions relating to their ubiquitous use in society have been proposed. This article builds on a review of the ethics of algorithms published in 2016 (Mittelstadt et al. Big Data Soc 3(2), 2016). The goals are to contribute to the debate on the identification and analysis of the ethical implications of algorithms, to provide an updated analysis of epistemic and normative concerns, and to offer actionable guidance for the governance of the design, development and deployment of algorithms.