DevOps maturity in an IT service environment

Tabl e of Contents

List of Appendices

List of Figures

List of Tables

List of Abbreviations

1. Introduction
1.1. Motivation
1.2. Research Questions
1.3. Research Design
1.4. Outline / Remainder

2. Research Methods
2.1. Literature Review Methods
2.2. Empirical Methods

3. Literature Review
3.1. Scope
3.2. Research Process
3.2.1. Systematic Literature Review
3.2.2. Targeted Review
3.3. Results
3.3.1. Systematic Literature Review
3.3.2. Targeted Review

4. Empirical Research
4.1. Case Description
4.2. Research Process
4.3. Results
4.3.1. Practitioner Perception of DevOps
4.3.2. Model Revision, Iteration
4.3.3. Model Revision, Iteration

5. The DevOps Assessment Model
5.1. Model Presentation
5.1.1. The DevOps Mindset Assessment
5.1.2. The DevOps Capability Assessment
5.2. Model Development
5.3. Managerial Implications on Using the DevOps Assessment Model

6. Evaluation
6.1. Summative
6.2. Formative

7. Discussion, Limitations, Future Research

Appendix.

References

Abstract

Businesses increasingly depend on IT in order to sell their products and services. Shorter time-to-market is essential and agile software development approaches have emerged, improving collaboration and planning between business and software development. However, while software can be developed iteratively with higher performance and efficiency, these efforts are in vain when IT operations are left out of these changes. Insufficient communication between teams often leads to unknown or falsely documented requirements, resulting in expensive rework and project delay. Furthermore, iterative development techniques cannot tap the full potential and shorter time-to-market is not achieved while deployments are only possible once every six months. DevOps addresses these problems by promoting shorter release cycles and improved collaboration between development and operations professionals. However, practitioners long for a clear comprehension of the approach, trying to understand where they stand and identify potential areas of improvement. This thesis uses a systematic literature review to characterize DevOps and determine implementation criteria. Using Action Design Research and interviews at an organization, a maturity model is created to visualize these learnings and give practitioners the ability to assess their organization's DevOps performance. Consecutively, the model is applied to two environments of different DevOps maturity in an IT service environment.

Keywords: DevOps, Maturity Model, IT Service Management, Continuous Delivery

List of Appendices

Appendix A: PCI Short Questionnaire

Appendix B: PCI Guidelines

Appendix C: Literature Review First Search

Appendix D: Literature Review Concept Table

Appendix E: Concept Grouping with Index Cards

Appendix F: List of Interview Partners

Appendix G: DevOps Assessment Model, Version

Appendix H: DevOps Assessment Model, Version

Appendix I: Workshop Attendance Lists

List of Figures

Figure 1: The applied research design

Figure 2: The applied ADR process

Figure 3: Literature review results for DevOps outcomes

Figure 4: Literature review results for DevOps culture / mindset

Figure 5: Literature review results of the organizational environment in DevOps

Figure 6: Literature review results for DevOps automation

Figure 7: Literature review results for DevOps tools

Figure 8: Company divisions of interview partners

Figure 9: The DevOps loop

Figure 10: DevOps Outcomes for the DevOps Assessment Model

Figure 11: Culture of Collaboration Category

Figure 12: Lean Principles Category

Figure 13: Systems Thinking Category

Figure 14: Consecutive Maturity Levels in the DevOps Capability Assessment

Figure 15: Organizational Environment Category

Figure 16: Automation & Technology Category

Figure 17: Model Development Phases

Figure 18:Workshop Results for ‘Culture of Collaboration’

Figure 19:Workshop Results for 'Lean Principles'

Figure 20:Workshop Results for 'Systems Thinking'

Figure 21:Workshop Results for 'Organization Environment'

Figure 22:Workshop Results for 'Automation & Technology'.

List of Tables

Table 1: Taxonomy of the literature review

Table 2: Keyword Search

Table 3: Evaluation Results for the DevOps Mindset Assessment

Table 4: Evaluation Results for the DevOps Capability Assessment

List of Abbreviations

Abbildung in dieser Leseprobe nicht enthalten

List of Figure

Figure 2: The applied ADR process

Figure 3: Literature review results for DevOps outcomes

Figure 4: Literature review results for DevOps culture / mindset

Figure 5: Literature review results of the organizational environment in DevOps

Figure 6: Literature review results for DevOps automation

Figure 7: Literature review results for DevOps tools

Figure 8: Company divisions of interview partners

Figure 9: The DevOps loop

Figure 10: DevOps Outcomes for the DevOps Assessment Model

Figure 11: Culture of Collaboration Category

Figure 12: Lean Principles Category

Figure 13: Systems Thinking Category

Figure 14: Consecutive Maturity Levels in the DevOps Capability Assessment

Figure 15: Organizational Environment Category

Figure 16: Automation & Technology Category

Figure 17: Model Development Phases

Figure 18:Workshop Results for ‘Culture of Collaboration’

Figure 19:Workshop Results for 'Lean Principles'

Figure 20:Workshop Results for 'Systems Thinking'

Figure 21:Workshop Results for 'Organization Environment'

Figure 22:Workshop Results for 'Automation & Technology'.

1. Introduction

This thesis acknowledges DevOps as a relevant trend in information systems (IS) research as well as IT organizations. DevOps is examined in general and with a focus on characteristics and aspects of implementation and maturity. A maturity model is developed in the context of an organization, based on literary and empirical research. The following chapter depicts the motivation and interest in this topic, the main research questions defined for this thesis as well as the methods the research was based on. Finally, the chapter concludes with an outline of the contents of the thesis.

1.1. Motivation

Global digitalization still has a major influence on the worldwide economy. While new business models and start-ups are on the rise, larger enterprises face a whole variety of challenges on their way of modernizing existing business processes while targeting on being innovative. There is little doubt IT has become inevitable to business processes of any kind and thus is a key factor for achieving this innovation.

In the process of digitalization, a company’s way of creating and delivering software is one of the most important factors (Boehm 2008). Since it is of the upmost importance to be able to respond to market quickly, it is figuratively ‘no longer the big fish that swallows the small fish. It is the fast fish that eats the slow fish’ (Jones 2016). Simultaneous to this development, new approaches to how software projects are managed and organized found their way into the industry. The need to better integrate the customer’s requirements into software development processes has led to the rise of the Agile approach and, ultimately, to the use of frameworks like Scrum and Kanban within IT projects (Airaj 2017).

The same need for integration and silo-breaking can be found in DevOps. The word ‘DevOps’ first appeared during the ‘DevOpsDays’ conference in 2009 (Debois 2009) and describes a set of values, principles and practices for higher cooperation between development and operations professionals (Perera et al. 2016). Over the first few years, the DevOps approach was created and defined by a grassroots movement of practitioners of both development and operations (Davis and Daniels 2016). DevOps practices have emerged as a means to achieve faster time-to-market (Ebert et al. 2016) and software with higher quality and reliability (Perera et al. 2016).

The DevOps approach as a movement differs from other approaches and frameworks in information systems. DevOps is often confused with agile development techniques and frameworks like Scrum and Kanban. Unlike these, DevOps has no group of authors, no certification entities and therefore many different definitions. One major motivation for this master thesis about DevOps is the fact that many companies and even IT professionals are unable to decide whether to implement it because they simply do not know what it is (Hussain et al. 2017). Thus, shedding light on DevOps and contributing to further comprehension of the matter seems profitable.

Furthermore, DevOps has many different aspects and principles including continuous integration, information sharing, tool sharing, continuous delivery and inherent cultural change. This has led to companies and their managers associating DevOps with different tools and creating a widespread misconception, DevOps could be integrated via software and tools alone. The reality is that most identify the cultural changes as the most important factor (Walls 2013). The famous quote attributed to Peter Drucker fits the DevOps approach perfectly: ‘Culture eats strategy for breakfast’ (Hyken 2015).

In addition to the variety of different approaches to DevOps from practitioners and vendors, Erich et al. discovered there is not much scientific literature available on DevOps (Erich et al. 2014). Furthermore, even with the literature that is available ‘the DevOps definition and scope remains unclear’ (de França et al. 2016). DevOps and Continuous Delivery practices originated within start-ups and large Web 2.0 companies like Amazon and Flickr which are often described as ‘unicorns’ of DevOps (Hussain et al. 2017) given their extraordinary culture and flexible management structure. Unfortunately, this leads many people to believe DevOps does not work for a traditional enterprise because of cultural or systemic obstacles. Several success stories of DevOps implementations have proven this to be false (Gruver and Mouser 2015; Humble and Molesky 2011). Debois points out that ‘the term “devops” is only a stub for more global company collaboration’ (Debois 2011, p. 5) and is thus suitable for almost any environment.

Another major challenge for companies considering DevOps is the implementation. Since there is no certification and many different definitions of DevOps exist, the guidelines regarding the implementation of the DevOps approach differ as well. Companies have difficulties knowing where they stand, where to start the implementation and which parts of the approach will help them gain business value.

As measuring the current performance is a prerequisite of being able to improve (Forsgren et al. 2017b), assessments can be a useful tool. Forsgren et al. (2017b) point out that commercial assessments are expensive and subject to bias which lead them to create a commercial SaaS assessment platform for DevOps. Openly available assessment models which are based on research still do not exist.

1.2 . Research Questions

This thesis attempts to gain a deeper understanding of DevOps and identify central aspects of DevOps implementation. Furthermore, a maturity model for DevOps is developed in the context of an organization. The thesis addresses the aforementioned motivations and academic voids. Five research questions were established to ensure a structured approach in achieving the required findings.

R1: What are the key characteristics of the DevOps approach?

Due to the lack of a universal definition for DevOps both among practitioners and in scientific literature this thesis identifies the definitions and characteristics of the DevOps approach and the DevOps implementation through a literature review. This process results in several criteria for the assessment of DevOps maturity.

R2: How can the key characteristics of DevOps be assigned to respective indicators for measurement of DevOps maturity?

The thesis creates a maturity model for DevOps to help practitioners assess their own performance and identify areas to improve as well as processes to implement next. The criteria identified in R1 are used to create a DevOps Assessment Model displaying key performance indicators and correlated levels of maturity. The model is created through an iterative approach involving qualitative research at an IT service organization.

R3: What key aspects are identified by stakeholders within a corporation performing a DevOps implementation?

Through empirical research in the form of Problem-centered Interviews, the key characteristics of DevOps implementations as identified by stakeholders in a corporation are conducted. This ensures an active involvement of experience from actual practitioners within development and operations environments.

R4: How can the obtained information and learnings change and improve the designed model?

Data and learnings from literature as well as empirical methods are used to iteratively redefine and improve the created DevOps Assessment Model. The ongoing symbiosis of aspects from existing literature and knowledge from real practitioners provides a broad spectrum of sources to ensure quality as well as adaptability.

R5: Can the designed model be used for accurate assessment of DevOps maturity?

The created and redefined model is applied to two unequally mature environments in workshops and is evaluated in the process. This generates further insights into the applicability of the model. Furthermore, it produces a notion of which aspects of DevOps are applicable to which kinds of environments.

1.3 . Research Design

The master thesis uses Action Design Research (ADR). This method is an appropriate approach for the topic at hand since it was designed to address ‘a problem situation encountered in a specific organizational setting by intervening and evaluating; and [...] constructing and evaluating an IT artifact that addresses the class of problems typified by the encountered situation’ (Sein et al. 2011, p. 40). In this case, the encountered problem situation was the lack of tangibility of the DevOps term within the company at hand and the resulting uncertainty about the current performance as well as appropriate improvement initiatives. The goal was to create an artefact addressing these problems in a way other organizational environments could also benefit from it. This means constructing a model which can help understand DevOps as well as assessing the current DevOps performance and ways to improve.

To identify DevOps characteristics and implementation criteria in relevant literature, this thesis uses a literature review as defined by Webster and Watson (2002). Relevant papers and conference proceedings are reviewed and statements about DevOps and its implementation are collected and documented. The obtained statements are later grouped by area of implementation while equivalent and redundant aspects are aggregated. Statements with only one supporting source are disregarded. Thereby gained structured DevOps characteristics build the basis of the initial model design (see Figure 1).

Abbildung in dieser Leseprobe nicht enthalten

Figur e 1: The applied research design

To comply with ADR in enabling practitioners to influence the design throughout the building, intervention and evaluation stage, the thesis uses Problem-centered Interviews (PCI) as defined by Witzel (2000) to include qualitative data. Eight interviews were conducted in two phases (four interviews each) with stakeholders at an IT service organization. Knowledge gathered from these interviews was implemented into the design of the DevOps Assessment Model to iteratively shape its form and improve its quality.

The model is later evaluated through workshops where it is applied to two separate environments of different DevOps maturity. This provides the summative evaluation while the formative evaluation is an ongoing process implemented in ADR.

1.4 . Outline / Remainder

The thesis begins by illustrating the applied research methods in greater detail within chapter 2. Research Methods. ADR, the literature review process and the applied interview method are thoroughly described. Furthermore, the adequacy of each method applied in this thesis is justified.

Chapter 3. Literature Review gives insights into the procedure of the literature review and its taxonomy. The identification of databases used for the keyword search and which parameters were set for these databases are described. The results of the review are listed along with the supporting sources.

The empirical research in the form of Problem-centered Interviews is illustrated in Chapter 4. Empirical Research. The organization where the empirical research was conducted is characterized in a case description. This contains all economic and social features with an important impact on DevOps.

The chapter 5. Design describes the symbiosis of findings from literature and knowledge gained through the conducted interviews. It depicts the procedure of the iterative model design as proposed by ADR. This includes initial design decisions as well as relevant changes to the model throughout the building process. Furthermore, this chapter derives implications for practitioners facing the same class of problems as addressed in this thesis.

To describe the evaluation process of the model, the chapter 6. Evaluation illustrates the conducted workshops as well as the two company divisions they were conducted in. The chapter further describes the formative evaluation that accompanied the model design phases. Furthermore, the most important learnings, limitations and identified possibilities for future research are specified in chapter 8. Discussion, Implications, Future Research.

2. Research Methods

This chapter described the underlying research method ADR and how it was applied in this instance (see Figure 2). The following chapters 2.1. and 2.2. illustrate the methods used for the literature review and the empirical research.

The underlying research method of this thesis is Action Design Research as defined by Sein et al. (2011). The method uses an iterative approach to let the artifact emerge from interaction within an organizational context while the ‘initial design is guided by the researchers’ intent’ (Sein et al. 2011, p. 40). While ADR focuses on IT artefacts defined as ‘features that are socially recognized as bundles of hardware and/or software’ (Sein et al. 2011, p. 38) this thesis applies the same iterative process to create a maturity model for organizational change.

Stage one of Action Design Research deals with the problem formulation which is in this case triggered by several problems perceived in practice (Sein et al. 2011) and named in chapter ‘1.1. Motivation’. This stage determines the initial scope for the grade of participation as a practitioner (in this case all data gathered at the company) and the initial research questions (Sein et al. 2011) which are named in chapter ‘1.2. Research Questions’.

Stage two of Action Design Research discusses building, intervention and evaluation (Sein et al. 2011, p. 41). In this stage, the initial design of the IT artefact – in this case the DevOps Assessment Model – is created and then redefined by organizational use and evaluation. The thesis follows this principle and creates the initial model based solely on theoretical knowledge before redefining and evaluating it through organizational input with the help of semi-structured interviews in an iterative process. The initial creation is based on knowledge gathered through a literature review as described by Webster and Watson (2002) and vom Brocke et al. (2009) while the following iterative refinement and evaluation of the design is conducted through empirical research following the definition of the Problem-centered Interview (Witzel 2000). Additionally, the model is applied by potential end-users in two different divisions at the aforementioned organization (see Figure 2). The application is performed in two workshops with stakeholders.

Abbildung in dieser Leseprobe nicht enthalten

Figur e 2: The applied ADR process (Sein et al. 2011)

Stage three reflection and learning (Sein et al. 2011, p. 44) is about continuously ensuring that the acquired learnings are generalizable and applied to a broader class of problems. This stage is meant to accompany stages one and two. These principles are implemented in the analysis of both literary and empirical results as well as continuously guiding the process and ensuring a balance between the preliminary design of the artefact and its shaping through use in the organizational context. Sein et al. (2011, p. 44). call this principle the ‘guided emergence’.

Formulization and learning (Sein et al. 2011, p. 44) as described in stage four of Action Design Research ensures that valuable learnings achieved in the first three stages are ‘developed into general solution concepts for a class of field problems’ (Sein et al. 2011, p. 44). Furthermore, the learning achieved through the successful creation of the artefact is formalized. This is achieved through stating implications of the achieved learnings for management as well as future research. These are valid not only for the organization at hand but for other organizations with similar problems of the same class.

2.1 . Literature Review Methods

The literature review conducted in the context of this thesis is based on the works of vom Brocke et al. (2009) as well as Webster and Watson (2002). They consider the rigorous documentation of the literature search process substantial. Furthermore, they propose a guideline for the process of reviewing the literature and documenting it adequately at the same time. The guideline suggests starting the review with defining the review’s scope. This also helps to clarify the purpose and perspective of the review. In order to define the scope in greater detail, the taxonomy by Harris M. Cooper (1988) was used. Vom Brocke et al. (2009) focus the second phase of their framework on achieving an overview of the key issues of the topic to be able to identify keywords. Identifying key concepts can be done with encyclopedias or handbooks. Following the conceptualization of the topic is the literature search itself which consists of the sub phases journal search, database search, keyword search and/or backward/forward search (vom Brocke et al. 2009). The journal search focuses on identifying relevant journals of high quality. Subsequently, respective databases with access to the identified journals are determined. The following keyword search tries to exclude irrelevant contributions by using precise search phrases. Vom Brocke et al. (2009) consider the fourth sub phase – the backward/forward search – optional.

The structuring of the literature review results as well as their analysis and synthesis is only briefly discussed by vom Brocke et al. (2009) which is why this thesis relies on Webster and Watson (2002) for these aspects. They describe literature reviews as concept-centric which plays an important role in preparing the data for later use in the research process. In order to structure the review, a concept matrix is described as a useful tool to document and visualize the findings (Webster and Watson 2002).

2.2. Empirica l Methods

The interviews were conducted as PCIs which use an interplay of inductive and deductive thinking in order to neutralize the contradiction of direction by theory on the one hand and being open-minded on the other (Witzel 2000). This is achieved by a highly subjective approach which considers the interviewees as experts of their own views and opinions. The PCI relies on the three principles problem-centered orientation, object orientation and process orientation and is supported by the four PCI instruments short questionnaire, interviewing guidelines, tape recording and postscript. The most important principle for this thesis is the problem-centered orientation since all interviews included DevOps and the DevOps implementation in the interviewee’s organization as the primary topic in the conversation. The principle compels the interviewer to gain preliminary knowledge in order to better connect with the interviewee. Additionally, the interviewer is able to take further action within the interview by problem-centered questioning. Object orientation enables the interviewer to apply different conversation techniques depending on the objects being observed. Since the PCI demands the communication situation to be focused on the interviewee, this flexibility allows to use narratives or recurrent questioning more or less frequently depending on the individual respondent (Witzel 2000). The principle of process orientation focuses the conversation on the reconstruction of events without attaching any value. Instead, the interviewer needs to concentrate on building trust and acceptance. This motivates self-reflection and heightens the interviewee’s capability to remember. Ideally, the interviewee can be motivated to biographical story-telling which can help overcome the artificial nature of a research situation. All four of the PCI instruments were used to conduct the interviews in the organization (Witzel 2000).

Short questionnaires (see Appendix A) were used only to collect the most important data, in this case name, age and the role they fulfilled at the organization as seen by the interviewees themselves. The interviews were prepared through guidelines (see Appendix B) which contained several open questions as well as a series of questions used only to restart the conversation or lead the interviewee back to the main problem. In addition to the PCI instruments, the research process also included field notes as ‘the classic medium for documentation in qualitative research’ (Flick 2014, p. 296).

The interviews were all recorded on tape to ensure accurate documentation for the entire research process. Due to the fact that transcription is an interpretive activity, it is argued that ‘the very notion of accuracy of transcription is problematic given the intersubjective nature of human communication’ (Poland 1995, p. 292). Therefore, the tape recordings, not their transcription, are considered the most accurate documentation of verbal data. On this account, the tape recordings were not transcribed but used to establish high-quality postscripts based on the researcher’s field notes, ensuring their integrity and correctness. These postscripts were created immediately after the interview. They contain an outline of all discussed topics, important statements and comments as well as nonverbal aspects of the conversation.

Since no transcription has been performed, a sentence-by-sentence interpretation and further coding as suggested by Witzel (2000) is not possible. Instead, the postscripts are replenished with the help of the tape recordings into a state of ‘concise statements’ (Witzel 2000) as suggested for the development of case-specific main topics. These refurbished postscripts build the foundation of the following analysis by ‘systematic contrasting through case comparisons’ (Witzel 2000) illustrated in ‘4.3. Results’.

3. Literature Review

This chapter describes the process of the literature review that was conducted in order to synthesize existing knowledge about DevOps and identify the key characteristics of the DevOps approach (R1). The data is also required to create a basis for the initial model design. Additionally, this chapter illustrates the relevant findings of the literature review.

3.1. Scope

In preparation of the literature review, its scope and purpose as well as the flavor was defined according to Brock et al. (2009). Since the first mentions of DevOps started at the 2009 DevOpsDays conference (Debois 2009), the time period for the review was defined as 2009 until today. In order to further characterize the review, the taxonomy for literature reviews by Cooper (1988) was used (see Table 1).

Tabl e 1: Taxonomy of the literature review (following Cooper 1988, p. 109)

Abbildung in dieser Leseprobe nicht enthalten

The focus of the literature review is described by Cooper (1988) as the material which is of central interest to the reviewer. As this review tries to accumulate all characteristics, definitions, implementation guidelines, principles and practices of DevOps across research outcomes, theoretical approaches and applications alike, the focus includes all but research methods.

When it comes to the review’s goal, Cooper (1988) admits that almost all reviews try to integrate past literature and therefore will be categorized under integration. He does however propose to identify the different integrative activities Strike and Posner (1983) identified. The literature review at hand clearly formulates ‘general statements from multiple specific instances’ (Cooper 1988, p. 108) which is why its activity can be classified as generalization. Interestingly, since it is of interest in identifying the problems that have emerged in the past for papers to define DevOps and respective implementation criteria, the thesis also has the goal to identify central issues.

When characterizing a review, the reviewer’s perspective on the reviewed literature plays an important role. Since this thesis does not try to accumulate literature in order to express a particular point of view but attempts to synthesize past literature without value or agenda, it is described as a neutral representation.

Coverage is identified as ‘probably the most distinct aspect of literature reviewing’ (Cooper 1988). He describes it as the result of the reviewer’s decisions about which sources are considered adequate and of sufficient quality. Since DevOps literature is scarce in scholarly journals but numerous in other sources like web and news articles, the review did not include the entirety of the relevant literature and can therefore not be characterized as exhaustive or exhaustive and selective. The review intentionally did not only include central/pivotal literature in order to gain knowledge about how DevOps is perceived generally. Instead, it tries to review a ‘sample illustrative of the larger group’ (Cooper 1988, p. 111) and is therefore defined as representative. Notably, the results provided by the addressed databases were reduced only by relevance while all relevant sources found through the documented parameters were included in the review and documented in the thesis. Therefore, it might be valid to consider the review exhaustive to a certain extent.

The characteristic organization describes the order and association in which the topics are introduced. In this case, they neither appear together based on chronological order (historical) nor according to employed methods (methodological). Instead, the different topics are documented as DevOps characteristics, definitions, principles and practices while equivalent statements are aggregated in the process. This characterizes the review’s organization as conceptual (Cooper 1988).

The intended audience (Cooper 1988) of this review is twofold. Firstly, due to the nature of a master thesis to contribute to research, the primary audience of the review are specialized scholars with an interest in DevOps. Secondly, the review addresses practitioners considering DevOps implementations. This is additionally influenced by the fact that this thesis was created in an organizational environment.

3.2. Researc h Process

The following chapter describes the research process of both the systematic literature review performed in order to synthesize existing knowledge and perceptions of DevOps and identify DevOps characteristics. Furthermore, the methods this research process is based on and their application will be illustrated. The undertaken targeted review performed to generate further, more specified knowledge about individual characteristics and principles will be described as well.

3.2.1. Systematic Literature Review

After successful definition of the review’s scope, vom Brocke et al. (2009) name conceptualization as the second step of their framework. In this case, the conceptualization of what is known about the topic is heavily influenced by the purpose of the literature review. Since the lack of a unified definition is the very motivation for this thesis and the review, the conceptualization intentionally does not contain ‘working definitions of the key terms’ (vom Brocke et al. 2009). Additionally, synonyms and homonyms addressed by this step can be disregarded when trying to synthesize existing knowledge about a fixed term.

The conceptualization’s main goal is the identification of ‘key issues relevant to the subject’ (vom Brocke et al. 2009) in order to identify search terms. For the purpose of synthesizing knowledge about what is considered ‘DevOps’ and which items are identified as implementation criteria, the decision was made to use ‘devops’ as the key search term.

Journal Search

The search process, phase III of the framework by vom Brocke et al. (2009), begins with the identification of journals. Webster and Watson (2002) state that ‘the major contributions are likely to be in the leading journals. It makes sense, therefore, to start with them’ (Webster and Watson 2002, p. 16). Following this approach, two sources were consulted to identify leading journals in information systems. Peffers and Ya (2003) provide a list of the top 25 journals in information systems research while Willcocks et al. (2008) divide the journals into different groups. For this review, the lists ‘IS journals’ and ‘Practitioner journals’ (Willcocks et al. 2008, pp. 165-166) were considered. An intersection of the three lists, subject to availability, was created containing 13 journals which build the basis of the literature review (see Appendix C).

Unfortunately, there were virtually no results for the term ‘devops’ in the journals considered leading journals in information systems research (see Appendix C). In order to widen the search parameters and generate more results, the approach by Webster and Watson (2002) was disregarded along with the journal search phase in the framework by vom Brocke et al. (2009). Instead, for the purpose of an identification of databases, three existing literature reviews of DevOps were consulted.

Databases

Erich et al. (2014). conducted a literature review for ‘devops’, ‘continuous delivery’ and ‘development and operations’ in the following databases: Scopus, Web of Science, IEEE Xplore and ACM. Smeds et al. (2015). used ‘DevOps’ as the only search string and performed the search in the ACM Digital Library, DBLP, EBSCO Academic Search Premier, Google Scholar, IEEE Explore, Springer Link and Web of Science. Jabbari et al. (2016) also used ‘DevOps’ as the only search string, but added additional parameters with different databases (IEEE, ACM, Inspec, Scopus, Wiley Online Library, Web of Science).

Based on the databases used in these existing literature reviews on DevOps and the number of results they gathered from each, the following databases were identified and used in a second keyword search:

- ACM Digital Library
- IEEE Xplore
- Web of Science
- EBSCO (Business Source Complete)

Keyword Search

The aforementioned databases were then searched for the term ‘devops’. Since the search created sufficient results, it was decided not to further narrow it with additional terms (see Table 2). The backward/forward search regarded as optional by vom Brocke et al. (2009) was omitted for the same reason. Additionally, several parameters were set for the different database queries. While the Web of Science database offers searches by topic, IEEE Xplore and ACM Digital Library were searched by metadata. The EBSCO database was even used with a full text search. All databases were searched for sources for 2009 and after (see Table 2).

In a first step, the sources were filtered to exclude:

- sources without stated authors
- news articles
- public announcements
- sources in languages other than English

This was done for each database result individually (see Table 2).

Tabl e 2: Keyword Search

Abbildung in dieser Leseprobe nicht enthalten

The resulting 289 sources were reduced by duplicate detection, first with the EndNote Web software and afterwards manually. The remaining sources were further reduced by irrelevant sources. This was done with a title review and a subsequent review of the sources’ abstracts. During the title review, several basic principles were followed. Titles containing the word ‘DevOps’ were not excluded. Furthermore, the same was applied for titles containing concepts and principles closely related to DevOps. These concepts were based on the information gained from the existing literature reviews by Erich et al. (2014), Smeds et al. (2015) and Jabbari et al. (2016). The following concepts and principles were identified:

- Continuous Delivery
- Continuous Deployment
- Infrastructure as Code
- Collaboration
- Agile

All titles containing at least one of these keywords were not excluded by the title review. The same concepts were used to exclude irrelevant sources in the abstract review. Additionally, the approach consisted of excluding sources unlikely to contain general information and characterization of DevOps. After the title and abstract review, a total of 62 sources was left to be reviewed completely (see Table 2).

Literature Analysis and Synthesis

In order to analyze and synthesize the collected literature, Webster and Watson (2002) propose a concept matrix which can be used to document the concepts found in each source while reading. The concept matrix lists all collected sources allocating the supporting sources.

The literature analysis was conducted by documenting concepts relevant to the thesis motivation and the review conceptualization. The following requirements were defined to decide which concepts are relevant:

- Concepts widely acknowledged outside of DevOps (i.e. ‘defining requirements’) are irrelevant.
- Only general statements about DevOps are relevant. Specific models, instances or variations of DevOps (i.e. ‘SecDevOps’) are considered irrelevant.

The concept matrix (Webster and Watson 2002) was used for this thesis with a slight alteration. Due to the high number of concepts and sources, the visualization of the data in an actual matrix was not possible. Instead, the same data was organized in a concept table (see Appendix D) which underwent the following synthesis and editing:

- Equivalent concepts and statements were aggregated.
- Vaguely phrased concepts were disregarded.

Webster and Watson (2002) recommend developing an approach for grouping the key concepts before discussing the identified ones individually. After the concept table was created (see Appendix D), the identified concepts were grouped according to the context and their relation within the supporting sources. This step was performed with index cards (see Appendix E). Consequently, the identified concepts will be discussed in the following chapter.

3.2.2. Targeted Review

Additional sources have been considered in order to achieve further knowledge from existing sources for the model’s criteria, levels and items. In case of unconstructive feedback about individual parts of the model, this approach aloud to fill level gaps and generate further knowledge about the consecutive steps to implement DevOps principles and practices. Furthermore, knowledge about the difficulty of these steps was gathered from the sources when needed. The approach applied the following process: In order to gather the necessary information, the 62 gathered papers gathered in the systematic literature review (see 3.2.1. Systematic Literature Review) were searched for adequate keywords. Only if the necessary could not be found, wider searches were conducted as literature searches through Google Scholar and the consideration of widely acknowledged DevOps literature in the form of books and news articles as well as web sources.

3.3 . Results

The following chapter describes the results of both the systematic literature review and the targeted review. The acquired findings are structured according to identified groups and categories. The key characteristics of the DevOps approach are identified (R1).

3.3.1. Systematic Literature Review

The first notable insight generated by the literature review is the fact that overall coverage in information systems research remains a small one (de França et al. 2016) whereas the high-quality papers in IS still do not cover the topic at all (see Appendix C). Furthermore, of the 62 sources gathered by the literature review only 17 are journal articles while the remaining 45 are conference papers and proceedings. As DevOps is still widely undefined but gains momentum in adoption and application by practitioners (Forsgren et al. 2017a), a contribution to DevOps research seems to be more about time than scale.

DevOps Definitions

The definitions of DevOps vary distinctly in content and specificity between the sources of the literature review. Many sources name this circumstance explicitly. For example, Tony Clear states that DevOps as a term was ‘described as ambiguous, difficult to define and multifaceted’ (Clear 2017). The part of the definition most sources agree on is the goal to ‘bridge the gap between development and operations’ (Barna et al. 2017, p. 65). The described means DevOps applies to achieve this goal often include increased communication and collaboration between workers of these different disciplines (Waller et al. 2015). Some sources go further and suggest these disciplines to work together within the product lifecycle ‘as early as possible’ (Woods 2016, p. 20). The groundwork for these collaboration and communication improvements are of course cultural changes which are described to be a vital part of DevOps (Chung and Bang 2016). Some sources even consider the cultural values associated with DevOps so unique that they name this set of values the DevOps mindset (Chen et al. 2015).

Another goal many sources ascribe to DevOps is the ability to develop software of higher quality (Perera et al. 2016) and in less time (Soni 2015) with increased stability (Betz et al. 2016) and scalability and in a more efficient process (Spinellis 2016). This is achieved by establishing a ‘quick flow of changes to a production environment’ (Gottesheim 2015, p. 3) through smaller batch sizes (Callanan and Spillane 2016), build, test and deployment automation (Clear 2017). As described by Zhu et al. DevOps tries to apply these techniques to the entire product lifecycle to ‘reduce the time between committing a change to a system and the change being placed into normal production’ (Zhu et al. 2016, p. 33). Altogether, DevOps is described as a means for better collaboration between IT and the business by bringing ‘innovative product and features faster to the market’ (Ebert et al. 2016, p. 98).

The outcome of reduced time to market is often contributed to agile software development methodologies (Airaj 2017) and the similarities do not stop there. In fact, DevOps’ relation to the agile movement principles (Beck et al. 2001) seems to be hard to define as many sources describe it in different ways (see Appendix D). For example, Airaj claims DevOps ‘extends agile processes to include operations’ (Airaj 2017, p. 1), a view shared by several other sources, but also describes DevOps itself as ‘an agile method’ (Airaj 2017, p. 2). The literature review came to the conclusion that agile principles can be seen as a prerequisite of DevOps since many of these principles are also contributed to DevOps but none of them contradict it. This thesis shares the notion expressed by Christensen that DevOps ‘can be viewed as the natural next step of the agile movement, which emphasized working software, collaboration, speed, and responding to change’ (Christensen 2016, p. 174).

Apart from the agile principles (Beck et al. 2001), the cultural and organizational values associated with DevOps are often contributed to the lean startup methodology (Ries 2011) and the lean manufacturing principles it is based on (see Appendix D). De França et al. even claim ‘DevOps requires a lean process’ (de França et al. 2016, p. 57) since it relies on a continuous flow of changes and fast feedback not only from the customer but between development and operations while Ebert et al. state that DevOps is ‘building on lean and agile practices’ (Ebert et al. 2016, p. 100). In the end, there seems to be more consensus in past literature on what DevOps does, what belongs to DevOps, which circumstances created DevOps and what its goals are than what it actually is. Different sources use different terms to define DevOps:

- A set (of principles, practices, …) (Zhu et al. 2016)
- A framework (Diel et al. 2016)
- An approach (Artač et al. 2016)
- A method or methodology (Airaj 2017)

Erich et al. (2014) aimed at defining DevOps clearly and proposed to look at it from multiple perspectives. However, as de França et al. observed, they defined DevOps as a framework which ‘would require clear instantiation steps or rules, which is not available for DevOps’ (de França et al. 2016). The definition proposed by de França et al. describes DevOps as ‘a neologism representing a movement of ICT professionals addressing a different attitude regarding software delivery through the collaboration between software systems development and operation functions, based on a set of principles and practices, such as culture, automation, measurement and sharing’ (de França et al. 2016, p. 56). While this definition is certainly the most sophisticated within all sources of the literature review, it still bears a problem. Understanding DevOps as the representation of a movement contradicts with it being something one could apply. An adoption of DevOps, a term which de França et. al use themselves (de França et al. 2016, p. 56) is illogical with this definition.

Many of the reviewed sources consider continuous improvement central to DevOps (see Appendix D). Since continuous improvement in DevOps does not just focus on improving the product or service but intends ‘to continuously improve its processes and tools’ (Airaj 2017, p. 3), it implies a never-ending process as well as an indefinite nature. Therefore, this thesis refrains from defining DevOps as a term but considers the following to be true:

DevOps is an indefinite collection of values, practices and principles aiming to improve the creation and delivery of IT systems, subject to change at any given time.

Respectively, the adoption of DevOps will, in the context of this thesis, be interpreted as the adoption of those values, practices and principles in their current perception in literature.

DevOps Outcomes

As mentioned before, the reviewed sources often base their definitions of DevOps on the goals it allegedly tries to achieve (see Figure 3). On this account, the outcomes ascribed to DevOps show less variety than other concepts in the literature review.

The main objective for the higher collaboration between development and operations seems to be shortening the release cycles and, by that, deploying to production more frequently to improve overall time-to-market (Ebert et al. 2016). Naturally, as release cycles are shortened and releases are reduced in size, the risk of each release will decrease dramatically (Soni 2015). Consequently, this reduces the risk of the entire deployment process (Lwakatare et al. 2016), enabling teams to ‘release frequently and with confidence’ (Callanan and Spillane 2016, p. 59). The approach of smaller releases and the automation of large parts of the infrastructure also leads to faster recovery after system failure (Laukkarinen et al. 2017).

Abbildung in dieser Leseprobe nicht enthalten

Figur e 3: Literature review results for DevOps outcomes

Another alleged DevOps outcome reported by many sources is the improvement in quality of the software itself (Rajkumar et al. 2016). Additionally, the product or service provided by the system can be improved through continuous feedback (Hussain et al. 2017). Furthermore, improved stability and reliability is possible through increased automation (Oliveira et al. 2016). Through the use of virtualization and cloud environments, the delivered software can also be built in a way it is able to ‘self-adapt to external stimuli and regulate its own behavior’ (Barna et al. 2017, p. 65) which means increased scalability (Spinellis 2016).

DevOps Mindset

As mentioned before, many sources contribute a certain set of cultural values and social aspects to DevOps (see Figure 4), even labeling these the DevOps culture (Airaj 2017) or the DevOps mindset (Chen et al. 2015). As established, DevOps cultural values are heavily influenced by foregone movements and approaches such as lean principles and the agile movement. Unfortunately, these predecessors are often named without naming their individual values and principles. However, more detailed descriptions of lean principles and agile values were still identified in some sources.

Laukkarinen et al. claim that part of the spirit of lean software development is that ‘parts of the development and deployment chain that to not add value are swiftly eliminated’ (Laukkarinen et al. 2017, p. 15). De França et al. see the connection between DevOps and lean thinking principles in a willingness to foster ‘constant and fast feedback […] with customers’ (de França et al. 2016, p. 57). They add the principles of continuous flow, continuous improvement, eliminating waste and simplicity Additionally, employees adopting a DevOps mindset may gain a ‘holistic or systemic view’ (de França et al. 2016, p. 57). The latter shows a resemblance between lean principles and the systems thinking approach according to Fitzgerald and Stol (2017). They go further and attribute ‘self-organization and empowerment of people within an organization’ (Fitzgerald and Stol 2017, p. 178) to the lean philosophy as well as DevOps. An example of this is their claim that sharing in DevOps happens at different levels and includes ‘sharing in celebrating successful releases to bring development and operations teams closer together’ (Fitzgerald and Stol 2017, p. 178). They conclude that DevOps tries to change perspectives of responsibility within an organization.

Abbildung in dieser Leseprobe nicht enthalten

Figur e 4: Literature review results for DevOps culture / mindset

The connection to values of the agile movement comes natural due to the fact that DevOps was created by a community of practitioners expressing the need for ‘agile infrastructure’ (Debois 2008). Bayser et al. claim that a great part of the connection is based on exploring ‘what other elements of developer culture could be applied to operations’ (de Bayser et al. 2015, p. 1399). Furthermore, they argue that the DevOps mindset fosters personal responsibility for features and their implementation instead of handovers between different functional silos. They state: ‘Agile methods value collaboration, response time to changes, business understanding, simplicity and agility’ (de Bayser et al. 2015, p. 1398).

DevOps-specific values which could be identified in the literature review include the concept ‘faster you fail, faster you recover’ (Soni 2015) which fosters a culture contrary to blame. This is often described as a culture of continuous experimentation and learning (Clear 2017). Additionally, many sources contribute cultural traits to DevOps which aim to overcome the negative effects of functional silos and their contrary goals and incentives. Therefore, creating a culture of collaboration depends on effective communication and openness to change (de Fran<;a et al. 2016). This includes sharing values like respect and trust between development and operations as well as instigating an environment where feedback is shared without blame (Perera et al. 2016).

In conclusion, one can observe many similarities and even intersection between the agile movement, lean principles and DevOps. Notably, while many of the sources contribute the same collection of values to the three approaches in different configurations, they were no evident contradictions perceived between them.

Organizational Environment

The organizational environment in connection with DevOps depends on the team structure in which development and operations are organized (see Figure 5). As observed before, one of DevOps' main goals is to 'break down the silos wherein development and operations are separately confined' (Furfaro et al. 2016). Unfortunately, some ofthe sources do not provide a specific answer to the question if this means organizing development and operations personnel in joined teams. Airaj (2017) clearly promotes this practice and suggests bringing the two disciplines together in teams as well as physically. Some sources even go further and suggest including all workers of different functions who contribute to the product within cross-functional teams (Lwakatare et al. 2016). The organization may even choose to go beyond DevOps and break down their applications to a more granular level, enabling a microservices architecture (Ebert et al. 2016). Microservices are a variant of service-oriented architecture which Ebert et al. describe as beneficial but not required for DevOps. The advantages of a microservices architecture in conjunction with a DevOps approach are bidirectional. DevOps benefits from the separation of the system into smaller services which are independently deployable and microservices profit by the DevOps practice of small, cross-functional teams (Balalaie et al. 2016). In order to further tear down the separation between development and operations, Spinellis (2016) suggests the workers need to try to learn from each other with the operations side implementing successful development practices such as continuous integration and pair programming.

Abbildung in dieser Leseprobe nicht enthalten

Figur e 5: Literature review results of the organizational environment in DevOps

In addition to the composition of teams in a DevOps environment, the reviewed sources pay attention to the amount of responsibility and autonomy the individual teams should own. Most sources see the team’s full responsibility of the entire product’s or service’s lifecycle as a key practice for DevOps aiming to overcome the traditional function silos. This includes the definition of requirements and development as well as testing, deployment and monitoring (Christensen 2016). Additionally, all parties involved in the product need to share the responsibility for its success or failure in order to prevent blame and to increase the product’s quality (McCarthy et al. 2015). Perera et al. even include the shared responsibility in their definition of DevOps: ‘[DevOps] is the sharing of tasks and responsibilities within a team empowered with full accountability of their service and its underlying technology stack; from development, to deployment and support’ (Perera et al. 2016, p. 281). When it comes to autonomy, Lwakatare et al. (2016) promote self-organizing agile development feature teams while Fitzgerald and Stol (2017) see the continuous planning activities between the business and development functions which they call BizDev.

[...]