Virtual Reality (VR)/ Augmented Reality (AR) Applications in the Airline Industry. Feasibility and Effects

TABLE OF CONTENTS

List of Abbreviations

List of Figures

Abstract

1 Introduction
1.1 Problem
1.2 Objective
1.3 Methodology

2 Virtual Reality
2.1 Visual Feedback
2.2 Aural Feedback
2.3 Haptic Feedback

3 Augmented Reality

4 Other Realities

5 Drivers for VR/AR Implementation

6 Implementation of VR in the Airline Industry
6.1 On-board Applications
6.1.1 In-flight Entertainment
6.1.2 Acceptability towards VR Usage in Flight
6.1.3 VR for Cabin Crew
6.1.4 VR for Flight Deck Crew
6.2 Off-board Applications
6.2.1 Pre-flight Service
6.2.2 Marketing
6.2.3 Maintenance
6.2.4 Training
6.2.4.1 Maintenance
6.2.4.2 Flight Crew
6.2.4.3 Cabin Crew

7 Conclusion

References

List of Abbreviations

AR Augmented Reality

AV Augmented Virtuality

BITD Basic Instrument Training Device

FFS Full Flight Simulator

FTSD Flight Simulator training Device

GPS Global Positioning System

HMD Head Mounted Display

HUD Heads Up Display

IATA International Air Transport Association

ICAO International Civil Aviation Organization

MR Mixed Reality

VR Virtual Reality

List of Figures

FIgure 2 and 5 have been removed by grin for copyright reasons.

Abbildung in dieser Leseprobe nicht enthalten

Abstract

The following seminar paper explores the effectiveness and feasibility of virtual reality applications in the airline industry. Virtual reality offers numerous advantages, especially when compared to traditional media. This can be used by airlines to safe costs and improve the overall safety of processes, which are regarded as the two main drivers for implicating virtual reality. The technology is used broadly around the airline industry but does not lead to the desired success in every aspect, where the research reveals that there should be a clear distinction between replacing and supporting traditional applications. It is of importance to keep track of technological progress advanced by other industries, taking advantage of the high interest generated for the technology to derive practical applications for airlines.

1 Introduction

1.1 Problem

Virtual reality has found its way into mainstream-use with companies like Google, Microsoft and Samsung elevating the previously for technology enthusiasts reserved technology into the everyday life of people. With a multitude of diverse areas of society utilizing virtual reality, this seminar paper focuses on the airline industry and how it has fared thus far trying to utilize the technology. The increasing technological progress spawns new forms and enhancements of virtual reality with an increasing number of possibilities for applications available across the board.

1.2 Objective

The aim of this research is to determine the effects of virtual reality on the airline industry and to evaluate the feasibility of implying the technology throughout the different areas.

Through the large amount of interest generated by this technology the need to clarify feasible approaches is given, discerning between realistic applications with practical advantages and solutions that could negatively influence processes in the airline industry.

1.3 Methodology

Apriori in chapter 2-4 virtual reality is explored by comparing it to similar technologies, defining its characteristics in the process. Following this, the incentive of implementing virtual reality will be worked out in chapter 5 in order to create an understanding of what moves airlines to embrace virtual reality as a tool to reach desired outcomes. Subsequently, the succeeding chapters are dedicated to examining virtual reality solutions for airlines brought forward thus far, assessing the feasibility and effectiveness while a comparative spectrum is created to view the approaches in a context.

This strategy is used to overcome the scarce amount of empirical data available, as it is common with new technologies like these, when necessary.

2 Virtual Reality

While the term virtual reality (VR) is commonly referred by the media to as being a digital imaginary world that exists solely in computers and our minds, this statement lacks the deeper nature of the technology as defined by `www.merriam-webster.com` which states: “An artificial environment which is experienced through sensory stimuli provided by a computer and in which one's actions partially determine what happens in the environment”. (merriam-webster, 2020) (see Jason Jerald, 2016, p.9)

The vision of virtual reality being more than only looking at a screen was pioneered by Ivan Sutherland who stated “The ultimate display would, of course, be a room within which the computer can control the existence of matter. A chair displayed in such a room would be good enough to sit in. Handcuffs displayed in such a room would be confining, and a bullet displayed in such a room would be fatal”. (Ivan E. Sutherland, 1965, p.509)

The main implication derived from these quotes is the term immersion which in itself can be used in two ways, physical and mental immersion.

Mental immersion meaning that a sense of engagement and involvement is conveyed and physical immersion describing the artificial stimulus of senses using technology.

While mental immersion has been the primary focus for traditional media, physical immersion is the key to elevate VR above other media experiences regarding immersion.

VR enables the user to alter their perspective through body positioning and to affect the virtual environment helping the human to accept the virtual world as a real one. Critical for this result is sensory feedback which is achieved by using various means of technology. (see William R. Sherman. and Alan B. Craig, 2019, pp.10–11)

To achieve the illusion of being in a real environment, sensory feedback is an essential factor and varies in its implementation through different devices. (see William R. Sherman. and Alan B. Craig, 2019, p.11) (see Mel Slater and Maria V Sanchez-Vives, 2016, p.4)

2.1 Visual Feedback

The most common form of feedback when using VR is visual feedback. To result in interactive feedback, motion tracking is used to align the sensory output of a VR system with the position of the user. In most cases, this means that the head movement is tracked as well as at least one hand or object held by the user. More complex systems are not limited to these body parts and may even track major body joints as well. (see William R. Sherman. and Alan B. Craig, 2019, pp.11–12)

Visual displays can have different forms, the most common of these is the Head-Mounted Display (HMD). HMDs are fixed to the users head and should ideally track the head movement to manipulate perspective and are regarded as the main component of achieving immersion but rely heavily on quality equipment which shall ensure that the user is provided with a high resolution, low latency and good calibration of the system. (see William R. Sherman. and Alan B. Craig, 2019, pp.16–17)

In recent years the technological advancement of smartphone computing power made it possible to use smartphones as visual displays when combined with an appropriate casing the user can experience a virtual world through a magnified smartphone screen which tracks head movement through gyroscopic sensors already present in the smartphone. (see Anand Nayyar et al., 2018, p.158) (see Jason Jerald, 2016, pp.32–33)

Another form of display is the World-Fixed Display, which ideally encompasses the user wholly, forming in essence a fully surrounding screen realized by multiple projectors. (see William R. Sherman. and Alan B. Craig, 2019, pp.18–19)

While head tracking must also be used for immersion purposes, this aspect is not as critical as it is with HMDs because head movement stimuli do not depend as much on direct head motion as they do when a device is sitting on the users head directly. (see Jason Jerald, 2016, pp.33–34)

2.2 Aural Feedback

Audio is a crucial component of VR systems as it serves the purpose of adding awareness of surroundings without straining the visual sense. Stripping a VR system of aural feedback would essentially mean to leave the user deaf, which would be a thoroughly unnatural experience. (see Jason Jerald, 2016, pp.239–240)

To convey a sense of special awareness, sound has to be supplied in 3D, this can be realized by several speakers surrounding the user but a more common and effective method is the use of headphones, which isolate the user from the real environment. With ever-increasing advances in the sound department with introductions like active noise cancelling, this method is becoming even more viable and cheaper at the same time as it advances further and further into the consumer market. (see William R. Sherman. and Alan B. Craig, 2019, pp.353–355)

2.3 Haptic Feedback

A basic human understanding is that making physical contact with an object verifies its existence. (see Jason Jerald, 2016, p.357)

Cost and technology have forced VR to differentiate between two forms of haptics, passive and active. Passive haptics is defined by creating realistic physical objects in the form of props that can be touched. By touching props everything else inside the virtual environment can seem more realistic even though the nature of this method is completely static. Active haptics on the contrary give physical feedback which is controlled by the computer. Though more complex and costly in its implementation this methods inherits the advantage of giving the user dynamic control for an extensive range of simulated virtual objects. Gear that functions on the method of active haptics include controllers that use electrotactile stimulation that induce haptic feedback through an electric current passing through the user's skin, a physical glove that utilizes finger tracking and force feedback or motion platforms that move with the entire body with the result of giving the user a sense of motion and gravity. (see William R. Sherman. and Alan B. Craig, 2019, pp.37–42)

3 Augmented reality

Augmented reality (AR) can be regarded as a form of VR, nevertheless the are some differences that shall be mentioned. (see Anand Nayyar et al., 2018, p.157)

Whereas VR refers to a completely virtual world in which everything is computer-generated, AR refers to the real world where virtual objects are interpolated. This can have the form of icons, text or entire objects and graphics. (see Jason Jerald, 2016, p.29)

Ideally, the user would not be able to distinguish between virtual and real objects thus giving the user additional information which would otherwise not be present by synchronizing virtual objects with real ones and running the displayed information in real-time 3D. Inherent to the use of AR gear is that it requires a portable visual display, this can have the form of a smartphone/tablet or HMDs and a combination thereof. The primary design element of such gear is that the user is able to see the real environment, either through see-through glasses or through the use of camera technology, capturing the real environment and overlaying the digital information on it. (see William R. Sherman. and Alan B. Craig, 2019, pp.22–24) (see Mana Farshid et al., 2018, p.3)

AR can take different forms, as described briefly:

- Marker-based AR: This method uses a marker, which can mean anything from 2D images with visual codes to the real environment in order to anchor the digital information on it and display it in real-time.
- Location-based AR: Uses technologies like Global Positioning System (GPS) or digital compasses to provide information on location and orientation, used primarily for location-centric tasks, this method is often used in smartphones and cars.
- Projection-based AR: Projection-based AR functions by projecting information onto a surface. This method is also interactive as the projection reacts to the user's touches and can be either 2D or 3D.
- Superimposition-based AR: By using this method the view of an object is either partially or fully replaced through an augmented view of that object. As with marker-based AR, object recognition is a major aspect of this method as it relies on recognizing the object before the virtual information an be displayed onto it.(see Anand Nayyar et al., 2018, p.157)

4 Other Realities

To put the terms of VR and AR into perspective it is necessary to mention that these terms need some supportive terms in order to create a context. It needs to be clarified that VR and AR can not always be clearly distinguished from one another, this is where the term of mixed reality (MR) is useful as it can describe hybrid systems as well. MR encompasses everything in between the real environment and a completely virtual environment and should be regarded as a continuum as described by Milgram&Kishino. (see Mana Farshid et al., 2018, pp.4–5)

Figure 1: The Virtual Continuum

Abbildung in dieser Leseprobe nicht enthalten

Source: (see Mana Farshid et al., 2018, p.5)

Augmented virtuality (AV) is an example of a hybrid step between AR and VR as it describes the act of capturing real environment content and incorporating it into VR, this can take the form of 360° or interactive and immersive film for example while virtual environments describe artificially created ones which is a main aspect of VR as explained before. (William R. Sherman. and Alan B. Craig, 2019, pp.18–19)

While VR often recreates the real environment in some form, it is by definition not limited to that and is able to incorporate elements into the virtual world, that would otherwise not be feasible in a real environment, drawing the user into the virtual environment with the ultimate goal to temporarily replace the real world with the virtual one. Everything within the mixed reality incorporates the real environment in some form or function and thus does not share the same ultimate goal as VR. (see Carlos Flavián et al., 2019, p.549) (see Jason Jerald, 2016, p.30)

This seminar paper will continue to use VR as an encompassing term that includes the definitions of AR and MR unless otherwise stated.

5 Drivers for VR Implementation

With the establishment of definitions and an overview of the technologies of VR/AR/MR, the next step is to examine the drivers for implementation of VR focusing on the aviation industry. By creating an overview of what VR offers in terms of business advantages and comparing it to the general needs of the aviation industry it is possible to locate these drivers.

The advantages of using VR as a medium can be numerous, some examples include:

- Improving safety
- Saving cost
- Enhance marketing
- Enable simulation techniques
- Enable ability to examine in 3D

(see William R. Sherman. and Alan B. Craig, 2019, pp.725–726)

Though VR can result in many advantages, one must first consider if it is an appropriate medium for the task at hand.

Inherent to VR is its 3D nature, therefore any task that is sufficiently solved through the use of a 2D medium may not be appropriate for the use of VR an example to this would be the depiction of a stock market development, which does not need to be portrayed in 3D to avoid overloading on information. Contrary VR is well suited for visualizing a spatial environment that profits from being displayed in 3D.

Once this circumstance is established, one must evaluate on the desired outcome which shall be achieved by using VR. (see Daniel A. Guttentag, 2010, p.640) (see William R. Sherman. and Alan B. Craig, 2019, pp.725–730)

In order to examine this, a closer look at the general needs of the airline industry is required.

Though the airline industry is highly susceptible to external factors, a pre-coronavirus trend was one of good growth of demand. This could lead to the assumption that profits would be equally rising, but the profitability inside the airline industry remained marginal mainly due to small yields being achieved through declining fares and a nature of high operating cost inside the airline industry. A vital aspect of financial success for airlines is derived from this circumstance which is to ensure that unit revenues exceed unit costs. (see Rigas Doganis, 2019, pp.8–9)

This becomes more apparent when looking at data supplied by the International Air Transport Association (IATA) and the International Civil Aviation Organization (ICAO) from 2017 and 2018:

Figure 2: Air Transport Yields

https://​www.iata.org​/​en/​iata-​repository/​publications/​economic-​reports/​airline-​industry-​economic-​performance---​2018-​mid-​year---​slides/​

This figure has been removed by GRIN for copyright reasons.

Source: (IATA, 2018, p.20)

Figure 3: Block Hour Costs Boeing 737-500

Abbildung in dieser Leseprobe nicht enthalten

Source: (ICAO, 2017, p.11)

Based on this data we can assume that a crucial driver for the airline industry would be that in addition to exploring methods for generating higher revenues an incentive to minimize costs is vital to increase yields.

Something that can be described as a historical foundation of the airline industry is the aspect of safety that plays a crucial role through all of its branches. Any economic incentive put forward by airlines can become nullified if disaster as a consequence of breached safety strikes. Therefore an improvement of safety or even the avoidance of additional risk is key to the airline and aviation industry as a whole. (see Peter Belobaba et al., 2009, p.313)

Since during the 1970s technical problems which had previously arisen through the beginning of the jet age had been in drastic decline, leaving the human factor as one of the biggest issues regarding incidents. (see Thomas Biermann, 2015, pp.70–71)

This becomes apparent when looking at data involving accidents across different types of aircraft:

Figure 4: Causes Of Accidents Type Of Aircraft 1990-2011

Abbildung in dieser Leseprobe nicht enthalten

Source: (see Clinton V. Oster et al., 2013, p.157)

Across the board it is the human factor, which next to pilot errors, also encompasses the range of air traffic control, ground/cabin crew, other aircraft and equipment in the form of mechanics to some extent, which is the dominant factor for the cause of accidents.

6 Implementation of VR in the Airline Industry

Now that the two most crucial drivers of the airline industry have been explored, the next step is to get an overview of how the airline industry has implemented the technology of VR thus far, being driven by either the aspect of safety or an economical incentive as discussed.

For approaching the implementation of VR in the airline industry, a segment definition is necessary to structure an overview. This paper will base the segment definition on VR markets by application in the form of an on-board segment, which is further specified into subsegments regarding in-flight-entertainment and flight deck/cabin systems as well as an off-board segment encompasses pre-flight service, marketing maintenance and training processes.

6.1 On-board Applications

On-board tasks describe applications of VR used by passengers, pilots and crew. Thus the participants of airborne aircraft operations.

6.1.1 In-flight Entertainment

In-flight entertainment is an essential tool for airlines to differentiate themselves and its product in an ever more competitive market for the sake of gaining more market share.

Airlines connect in-flight entertainment with great importance when it comes to ticket purchases, suggestions even go as far as ranking in-flight entertainment second only to seating and legroom indicating that demand for high-quality in-flight entertainment is present. Adding to the importance of in-flight entertainment is the condition of the evolution of cabin design, which strives to accommodate more and more passenger seats, sacrificing on personal space inside the cabin. Traditional in-flight entertainment methods like seat-back displays and the increasing number of personal devices like smartphones and tablets have revealed major disadvantages in the environment of an aircraft cabin, as they offer very limited options for immersion to disconnect from the stressful environment passengers are confronted with. With these drawbacks in mind, VR seems to be an appropriate solution with its ability to isolate and immerse the user and free him from the strain put on the visual sense. (see Julie R. Williamson et al., 2019, pp.2–4)

Applications of VR for the in-flight entertainment have picked up significantly during the last decade, mostly thanks to the market growth of VR inside the entertainment industry as a whole, which can be closely related to the segment of in-flight entertainment as it borrows many attributes directly from the entertainment industry and makes use of the technology developed by this industry. (see Julie R. Williamson et al., 2019, p.3)

A main driving force behind the development of VR technology since the dawn of the 21st century was the entertainment industry and especially the sector of the gaming industry mainly geared towards home use. Looking at this graph it becomes apparent that, investments in the entertainment industry were leading when compared to other sectors. (see Karen McMenemy and Stuart Ferguson, 2007, pp.30–31)

Figure 5: VR/AR/MR Investments Worldwide https://www.statista.com/statistics/829729/investments-focus-vr-augmented-reality-worldwide/

This figure has been removed by GRIN for copyright reasons.

Source: (Statista, 2019)

While it shows that in the past, investments inside the entertainment industry were higher relative to the other sectors, a trend developed that these mentioned investments were also declining throughout the years 2016-2019, indicating a maturation of the market.

In-flight entertainment takes advantage of this development and comparatively high level of maturation and implements it in the aircraft cabin for passengers.

Airlines have implemented VR for purposes of in-flight entertainment for some years now, some more aggressively than others.

The first to introduce VR in the cabin was Qantas, who partnered with Samsung in 2015 to offer not only movies as traditional entertainment but also sights and attractions of network destinations for the first class. (see Qantas News Room, 2020)

Lufthansa trialled VR in an Airbus A330 flying from Frankfurt to Dubai in 2018 offering passengers a 3D moving map, showcasing destinations with attractions being overflown. (vrnerds, 2018)

AirFrance partnered with the company Skylights in 2017 to offer passengers a VR experience through a selection of 2D and 3D movies in the business class as a trial. (see AirFrance, 2020)

Skylights is a company which specializes in providing “immersive entertainment to airline customer experience managers seeking to upgrade and differentiate their premium passenger experience”. (SkyLights, 2020a)

This includes various 2D and 3D movies to experience mid-flight using their own promoted hardware called Allosky Headset which is marketed as being lightweight, with a Full-HD resolution and a wide field of view as a standalone piece of hardware not requiring any additional periphery such as smartphones or tablets.

Partnering with Skylights are British Airways, Garuda Indonesia and XL Airways to offer in-flight entertainment. (see SkyLights, 2020b)

In-flight entertainment use of VR can also be applied in airport lounges and before conducting flights in general which is also offered as a product by Skylights and trialled by other airlines, these applications fall under the category of off-board applications and will be discussed later on.

[...]