Ethics of Algorithms. Ethical Challenges and Outcomes of Algorithmic Decision-Making

Table of Contents

1. Introduction

2. Algorithmic decision making
2.1. Algorithmic selection & algorithmic decision making
2.2. Use of algorithmic decision making and their risks

3. Algorithmic decision making, ethical challenges and outcomes in the context of ethical decision making
3.1. Moral awareness and ethical challenges of ADM
3.2. Moral decision making and ethical outcomes
3.3. Amoral decision making and ethical outcomes

4. Conclusion

List of references

1. Introduction

Algorithms¹ and algorithmic decision making (abbreviated ‘ADM' in the following) simplify and improve peoples' lives and create new values for our society (Mannino et al. 2015: 1 ff; Bundesministerium für Wirtschaft und Energie 2016: 4 ff.; Goffey 2008: 16). ADM is used by banks to establish creditworthiness or by the police to calculate the likelihood of future crimes. It is used in agriculture, aviation, the insurance industry and many other business areas (Saurwein et al. 2017: 1 f.; Latzer et al. 2016: 401 f.; Barth 2013: 6). ADM is becoming increasingly essential and plays a crucial role in our daily life (Heise 2016: 203).

However, the use of algorithms is not just about opportunities and benefits. ADM hides significant risks and creates further ethical challenges (Hoffmann-Riem 2018: 13 f.; Saurwein et al. 2017: 4). The ethics of algorithms is a provocative topic that causes a lot of controversies (Lepri 2018; Martin 2018; O'Neil 2017). As well as the application of ADM in the ethical context (ibid.). Does is it ethical to use algorithmic decision making? Is the decision maker ‘moral aware' applying ADM? (Tenbrunsel/ Smith-Crowe 2008; Newell/ Marabelli 2015). Does moral awareness about ethical implications of ADM always lead to ethical decisions?

This study tries to answer these questions and investigates the ethical challenges of ADM arisen in the context of ethical decision making, and their ethical outcomes. The research is based mostly on the Model of Tenbrunsel and Smith-Crowe (Tenbrunsel/ Smith-Crowe 2008). Section 2 tells about ADM. Section 2.1. specifies theoretical constructs such as algorithmic selection and ADM. Section 2.2. presents the areas of use and the risks of ADM. Section 3 narrates about ethical challenges and outcomes in the context of ethical decision making. Section 3.1. explains the term moral awareness and the ethical problems of ADM in that context. Section 3.2. and section 3.3. describes the moral and amoral decision making processes and their ethical outcomes using ADM. Section 4 presents a summary of this study and includes ideas for further research in the context of ADM and ethics.

2. Algorithmic decision making

2.1. Algorithmic selection & algorithmic decision making

Our daily activities, business life, media consumption, etc. are digitized and influenced by algorithms (Latzer et al. 2016: 395 f.). They decide what search results are shown on Google or Yandex (ibid.). They automatically suggest friends, movies, travel attractions and other products (ibid.). Algorithms create the new reality: they do not only observe the people's actions and needs but also forecast and control their behavior and interests. (Hoffmann-Riem 03/2017: 11 f.; Latzer et al. 2016: 395 f.). Algorithms are used behind all these actions in the context of algorithmic selection (Saurwein et al. 2017: 1 f.; Latzer et al. 2016: 395 f.).

Algorithmic selection is a sociotechnical construct that combines (1) algorithms, (2) ‘technical components (e.g., software, platforms, infrastructure) and (3) human [factors] (e.g., designs, intents, audiences, and uses)' (Just/ Latzer 2018: 12). Algorithmic selection helps to filter and select the appropriate desired results from the growing volume of information according to specific predefined criteria (Saurwein et al.: 1 f.). Such selection allows a reduction in asymmetric information, a reduction in search costs, and improved market transparency (Saurwein et al.: 2). Without algorithmic selection, it would be impossible to handle the information overload of our time (ibid.).

Algorithmic selection is inextricably linked with data-driven or algorithmic decision making process (Lischka/ Stöcker 2017: 17; Zweig/ Bertelsmann Stiftung 2018: 12; Saurwein et al. 2017: 2 f.). ADM is automatic decision making (Brkan 2019: 1) that ‘is based on collecting and analyzing large quantities of data [...] to make strategic decisions' (Newell/ Marabelli 2015: 4).

Two general types of ADM could be marked out: fully-automated and part- automated decision-making (Wagner 2019: 104 ff.; Vieth/ Wagner 2017: 10; Zweig/ Bertelsmann Stiftung 2018: 12 f.).

Fully-automated ADM² means that no human control for the decision-making required (Wagner 2019: 104 ff.). During ADM process algorithms make independent decisions based on data evaluation and assessment, which will have a direct impact on human life (Zweig/ Bertelsmann Stiftung 2018: 12 f; Vieth/ Wagner 2017: 10). For example, a drone or other machine automatically compare faces of people with a terrorist database and, in the case of a high match, activates a weapon (Zweig/ Bertelsmann Stiftung 2018: 13).

Part-automated ADM means that the final decision is made by a human being based on ADM (Zweig/ Bertelsmann Stiftung 2018: 12 f; Vieth/ Wagner 2017: 10). Algorithms evaluate situations or people and predict the likelihood of an event occurring (Zweig/ Bertelsmann Stiftung 2018: 12). In such cases, algorithms support human resolutions (Zweig/ Bertelsmann Stiftung 2018: 13). For example, a landlord decides about the conclusion of the lease based on the SCHUFA score compiled by algorithms.

For this study, only part-automated ADM is of interest, since it implies the participation of a human being as ‘a moral agent [...] who makes a moral decision' (Jones 1991: 367).

2.2. Use of algorithmic decision making and their risks

Before examining the algorithmic decision making in the context of ethical decision making and establishing ethical challenges, it is important to understand where ADM is used and which risks arise during its application. The context of ADM plays an important role in fully understanding the research issue (Jaume-PalasF Spielkamp 2017: 5).

ADM is used in manifold life areas. Nowadays it is hard to imagine a field where an ADM is not implemented (Saurwein et al. 2017: 3 f.; Hoffmann-Riem 03/ 2017: 4 f.).

The use and application of ADM could be outlined by function (Saurwein et al. 2017: 3 f.; Hoffmann-Riem 03/ 2017: 4 f.): (1) search (e.g. Google, Shutterstock), (2) recommendation (e.g. Spotify, Netflix), (3) content creation (e.g. algorithmic journalism like Narrative Science), (4) aggregation (e.g. nachrichten.de), (5) allocation (e.g. targeted advertising like Google Adsense or algorithmic trading like Quatopian), (6) filtering (e.g. spam filters like Norton AntiSpam and child protection filters like Klicksafe), (7) forecasting (e.g. predictive policing like Precobs), (8) monitoring (e.g. surveillance programs like Prism), (9) rating/ scoring (e.g. credit scoring like Schufa and reputation systems like Amazon seller rating) (Saurwein et al. 2017: 2.; Latzer et al. 2016: 401 f.).

The application of ADM facilitates life and enables significant development for society and business (ibid.). Nevertheless, it involves considerable risks and ethical challenges that must be taken into account by using and implementing ADM.

Several risk categories of algorithms can be derived from nine functional fields of ADM. (Latzer et al. 2014: 3 ff.; Hoffmann-Riem 2018: 13 f.; Saurwein et al. 2017: 4; Saurwein et al. 2015: 37 f.; Lischka/ Stöcker 2017: 18 f.; Deutscher Bundestag 2017: 9; Martini 2017: 1017 ff.). These are the monopolization of market power and power of opinion, lack of transparency, discrimination, data protection breaches and the right to privacy, infringement of property rights, negative effects of algorithm-based automation on human cognitive abilities and the danger of increasing dominance and uncontrollability of algorithms (ibid.).

This study addresses the most suitable areas of the use of ADM with an ‘individual impact' (Jaume-Palasi/ Spielkamp 2017: 15): filtering, forecasting, monitoring and rating/ scoring (Zweig/ Bertelsmann Stiftung 2018: 29; Jaume-Palast Spielkamp 2017: 13 ff.) As well it refers to the associated risks which directly influence people's lives: lack of transparency, discrimination, right to privacy (ibid.).

For example, currently, more and more companies use ADM for hiring process (O'Neil 2017: 90 ff.; Lischka/ Klingel 2017: 22 ff.). Human Resources (abbreviated ‘HR' in the following) departments rely on algorithmic-based programs to filter candidates (ibid.). They reduce costs and optimize hiring processes (ibid.).

The ethical problem lies in functional risks of such programs, e.g., discrimination filtering by geography and poor neighborhoods, race, and ethnicity, gender (mostly women affected), etc. (ibid.). More than 70 ‘percent of résumés are never seen by human eyes. Computer programs flip through them, pulling out the skills and experiences that the employer is looking for. Then they score each résumé as a match for the job opening. It's up to the people in the human resources department to decide where the cutoff is [...]' (O'Neil 2017: 95).

The narrower the search parameters, the greater the likelihood of discrimination. If the company is in an expensive area, and the HR department looks for a candidate who geographically lives near the place of work, then people from poor areas do not even have a chance to get an interview (O'Neil 2017: 90 ff.). Or it looks for a candidate for the position of a senior manager with many years' experience who has never had a break in work for more than five months. It is very likely that a woman³ who has been on paid maternity leave (1 year) will also be subject to discrimination. Thereby, any non-standard case, unforeseen by the algorithm, can lead to discrimination in case of improper use of the system (Martin 2018: 9).

Risks of ADM and their implications are inseparably tied with ethical behavior and arise during an ethical⁴ decision making process and after its completion (Mittelstadt et al. 2016: 1 ff.). The next chapter examines the ADM in the context of ethical decision making and considers its ethical challenges.

3. Algorithmic decision making, ethical challenges and outcomes in the context of ethical decision making

Ethical decision making is a dynamic and lively construct without a distinct and formal definition (Tenbrunsel/ Smith-Crowe 2008: 593). Various research studies were conducted and offer different approaches towards ethical decision making in a business context (Loe et al. 2000; Trevino et al. 2006; Tenbrunsel/ Smith-Crowe 2008; Martin/ Parmar 2012).

This study mostly relies on the Model of Tenbrunsel and Smith-Crowe (Tenbrunsel/ Smith-Crowe 2008: 552). According to them, ethical decision making embraces three parts: moral awareness, moral decision making, and amoral decision making (ibid.). Further, ADM will be considered from the perspective of all three components.

3.1. Moral awareness and ethical challenges of ADM

Moral awareness is ‘a critical component of ethical decision making' and indicates ‘whether decision makers are morally aware' (Tenbrunsel/ Smith-Crowe 2008: 555). Existence or absence of moral awareness does not tell if the decision maker is a do-gooder (Tenbrunsel/ Smith-Crowe 2008: 553). It tells if he is aware of ethical implications or not (ibid.). This component determines if the decision maker is involved in a ‘moral decision making' or ‘amoral decision making' process (ibid.).

Moral awareness is directly connected with the ‘notion of attention' (Reynolds 2006: 233). Identifying an ‘issue as a moral issue' (Reynolds 2006: 234) or ‘an ethical dilemma' (Tenbrunsel/ Smith-Crowe 2008: 556) distinguishes an ordinary decision maker from a ‘moral aware' decision maker (Reynolds 2006: 234; Tenbrunsel/ Smith- Crowe 2008: 556; Jones 1991: 367). He⁵ recognizes that his conscious choice could have harmful consequences for others (Jones 1991: 367; Reynolds 2006: 233 f.). ‘The presence of harm and the violation of a behavioral norm' are inherent components of moral awareness (Reynolds 2006: 234).

In case of part-automated ADM, the decision maker, e.g., an HR Manager, does not violate the behavioral rules by using an ADM during his work (Martin/ Parmar 2012: 295; Reynolds 2006: 234). He uses an algorithmic-based software for hiring process, which saves costs and time (O'Neil 2017: 90 ff.; Vieth/ Wagner 2017: 16). The lack of doubt in the correctness of ADM lies in the connection between rationalization reasoning and organizational context (Martin/ Parmar 2012: 300). His behavior is ‘contextually situated endeavor' (Martin/ Parmar 2012: 297) explained by his social embeddedness (ibid). Why should he distrust an ADM, which is commonly used by the organization?

He also does not think about the second component of moral awareness: the harm caused by ‘digital discrimination' (Wihbey: 2015; Zliobaite 2017: 1063; Jones 1991: 367; Reynolds 2006: 233 f.). He bases his decision on the algorithmic score (O'Neil 2017: 90 ff.; Vieth/ Wagner 2017: 16) and regards it as rational (Martin 2018: 2; Zarsky 2016: 121).

The erroneous idea that algorithms ‘as a mathematical construct' (Mittelstadt et al. 2016: 2) are objective is due to lack of knowledge and understanding, what ADM is (Barth 2013: 1; Zarsky 2016: 121). According to a study by the Bertelsmann Stiftung, 90 percent of respondents cannot define the term ‘algorithms' and explain their functionality and risks (Fischer/ Petersen 2018: 13 ff.). Complexity and opacity of ADM make it difficult for the decision maker to be ‘moral aware' using ADM (Mittelstadt et al. 2016: 3; Zarsky 2016: 127). As he does not know and identify any ethical implications of ADM (Reynolds 2006: 234; Tenbrunsel/ Smith- Crowe 2008: 556).

Thus, a regular ADM user will not be morally aware when using ADM due to the complexity and peculiarities of algorithmic processes (Mittelstadt et al. 2016; Zarsky 2016).

[...]

¹ An algorithm is ‘a detailed work instruction' (Zirn 2004: 9) or a systematic approach to solving a problem (Sedgewick/ Wayne 2012: 3). An algorithm is a sequence of computational rules that consists of finitely many steps and ultimately finds an ideal solution to a dilemma (Sedgewick/ Wayne 2012: 4).

² Since May 25th 2018, the use of full automated ADM is restricted in accordance with paragraph 1 article 22 of General Data Protection Regulation (abbreviated ‘GDPR' in the following), but certain exceptions are allowed according to paragraph 2 article 22 GDPR: ‘Paragraph 1 shall not apply if the decision: (a) is necessary for entering into [...] a contract [...]; (b) is authorised by Union or Member State law to which the controller is subject [...]; or (c) is based on the data subject's explicit consent.'

³ ‘While mothers expected to receive 13.8 months parental allowance in 2017, it was only 3.7 months for the men' (Nier 2018).

⁴ Due to time and page number constraints, the limitations are necessary. Therefore, in the context of this study it will not be discussed in detail, what ‘ethical' means. The term ‘ethical' will be viewed as ‘generally accepted moral norms of behavior' (Trevino et al. 2006: 952) ‘that legal and morally acceptable to the larger community' (Jones 1991: 367).

⁵ The masculine gender is used in this study for the sake of brevity only, with no discrimination intended.