Applied Artificial Intelligence and Trust - The Case of Autonomous Vehicles and Medical Assistance Devices+ Technological

Forecasting & Social Change

- Monika Hengstler et al.

/ 김민준x 2016 Spring

Table of Contents

1. Paper 11. Background2. Trust3. Trust in the Technology4. Case Studies5. Results

2. Paper 21. Perceived Safety2. Anthropomorphism3. Experimental Results

3. Takeaways

BackgroundADAS 및 자율주행의 인지부하Google Self-driving Car

• Starting in 2012, Google’s self-driving cars logged 1,600,000 km by June of 2015

• A total of 14 accidents recorded by July 2015 — no accidents caused by Google

14 accidents on 1,600,000 km? Pretty good!!

BackgroundADAS 및 자율주행의 인지부하Google Self-driving Car

14 accidents on 1,600,000 km? Pretty good!!People complained that Google’s self-driving cars were “unpredictable”

• Starting in 2012, Google’s self-driving cars logged 1,600,000 km by June of 2015

• A total of 14 accidents recorded by July 2015 — no accidents caused by Google

Trust

“In interpersonal relationships, the essence of trust is the willingness to be vulnerable to the actions of another person” (Mayer et al., 1995)

“Trust provides a valid foundation for describing the relationship between humans and automation” (Lee and See, 2004)

1. Trust in the technology2. Trust in the innovating firm and its communication

Trust, Perceived Risk, and Trust in Innovation

Trust and Perceived Risk

Perceived Risk : a combination of uncertainty plus the

seriousness of the outcome involved

Perceived Risk in Innovation?: uncertainty about the possibility of failure of a new

product, or the likelihood that the product will not work properly

directly linked with adoption

Trust, Perceived Risk, and Trust in Innovation

Trust in the Technology

1. Trust is mainly driven by the predictability of the technology2. Over time, the driver of trust becomes dependability (consistent)3. Ultimately, the relationship shifts to faith (reliability)

Trust, Perceived Risk, and Trust in Innovation

Trust in the Innovating Firm and its Communication

“Trust in the innovating firm and its communicationinfluence the adoption decision…” (Sternthal et al. 1978)

Trust, Perceived Risk, and Trust in Innovation

Trust in the Innovating Firm and its Communication

“… Particularly in the early stage of commercialization, when customer knowledge is low and an extensive public discourse have not yet emerged…”

Trust, Perceived Risk, and Trust in Innovation

Trust in the Innovating Firm and its Communication

“… Particularly in the early stage of commercialization, when customer knowledge is low and an extensive public discourse have not yet emerged…”

Trust, Perceived Risk, and Trust in Innovation

Case Studies

Selection Criteria

1. Identified industries in which applications of AI have provoked skepticism

2. Selected cases that satisfied four conditions:1. the application contains a component of AI2. the application compensates for human flaws or

supplements human decision making3. the application is on the market or is close to

market introduction4. the application requires user involvement

Selection Criteria

Case StudiesHealthcare Automation

Medical TechnologyIBM Watson

HP

AiCure

Fraunhofer IPA Care-O-Bot 3

Case StudiesTransportation Automation

Transportation IndustryBMW

Deutsche BahnZF Friedrichshafen

AGDaimler -Future Trucks 2025

VAG Nürnberg

Data CollectionSources

1. Semi-structured Interviews

2. Informal follow-ups via emails and phone calls

3. Archival data

Pilot Interviewsmanagers & engineers from the target

industries+ executives from outside the target

industries

ResultsInsights Gained from Target Analysis

1. Operational safety is necessary to initiate performance trust

2. Data security is an eminent factor influencing trust

3. Cognitive Compatibility, Trialability, Usability

• A fundamental determinant of trust in the technology• A flawed system will not be trusted• Operational safety is necessary, but not sufficient for acceptance

ResultsInsights Gained from Target Analysis

1. Operational safety is necessary to initiate performance trust2. Data security is an eminent factor influencing trust

3. Cognitive Compatibility, Trialability, Usability

• In transportation, data security = operational safety• In healthcare, privacy protection is a major issue since

all applications deal with sensitive personal data

ResultsInsights Gained from Target Analysis

1. Operational safety is necessary to initiate performance trust2. Data security is an eminent factor influencing trust

3. Cognitive Compatibility, Trialability, Usability• Cognitive compatibility : compatibility between what

people feel or think about an innovation - Understandability

• Trialability : a strategy to enhance understanding. Displaying technological progress and providing a visualization of the concrete benefits, and thereby reducing perceived risk

• Usability : ease of use, an intuitive interface, balance between control and autonomy

Can Autonomous Vehicles be Safe and Trustworthy? Effects of Appearance and Autonomy of Unmanned Driving Systems+ Int’l Journal of HCI- Jae-Gil Lee et al.

/ 김민준x 2016 Spring

Perceived SafetyReducing Human Error

Automation is believed to be an effective way to reduce the human errors that contribute to negative, often life-threatening outcomes

Unmanned driving systems may increase perceived safety by explicitly removing the source of error (the human driver)

Perceived SafetyReducing Human Error

Automation is believed to be an effective way to reduce the human errors that contribute to negative, often life-threatening outcomes

Unmanned driving systems may increase perceived safety by explicitly removing the source of error (the human driver)

Not Currently Possible

Anthropomorphism and Social PresenceIncreasing Humanness

Anthropomorphism : the attribution of humanlike characteristics to

inanimate, artificial agents such as computers and robots

Individuals mindlessly apply social rules and expectations when they interact with computers that exhibit anthropomorphic cues, as if they were interacting with human beings

Anthropomorphism and Social PresenceIncreasing Humanness

Anthropomorphism and Social PresenceIncreasing Humanness

Assigning anthropomorphic cues such as agency, gender, and personality is one way to make the quality of human-robot interaction more socially meaningful = increasing trust

Human-like appearance and high autonomy in an unmanned driving system enhance the system’s similarity to humans

Anthropomorphism and Social PresenceIncreasing Humanness

Assigning anthropomorphic cues such as agency, gender, and personality is one way to make the quality of human-robot interaction more socially meaningful = increasing trust

Human-like appearance and high autonomy in an unmanned driving system enhance the system’s similarity to humans

Experiment SetupThe Effects of Appearance and Autonomy on User Perception

ResultsThe Effects of Appearance and Autonomy on User Perception

Pearson’s r correlation analyses conducted to examine the relationship between social presence and the measure variables

TakeawaysFindings on Semi-autonomous Vehicles and their Perceptions

How do we make semi-autonomous driving appear more human than it is?

There needs to be a trialability factor : how do we allow other drivers to predict, and

ultimately have faith in semi-autonomous vehiclesFinding the balance between control and autonomy