This thesis investigates the project management approach for big data projects for industry partner Red Rocks Company. The aim of this project is to understand best practice project management for big data initiatives and to develop a framework to help such projects to deliver the expected advantages. A brief literature review is undertaken to find out how big data projects are managed. From this, a Big Data Analytics Framework is derived which is based on CRISP-DM. The framework is validated through interviews with stakeholders from the corporate sector. For this case study, the first three phases of the Business Process Management Lifecycle are applied: process discovery, analysis and design.
Key findings of the case study are that literature recommends an agile project management approach for big data initiatives. On the contrary, the majority of interviewed industry stakeholders confirmes a waterfall approach is conducted more often to deliver such projects. The developed Big Data Analytics Framework will add significant benefits to Red Rocks Company as it will help to successfully deliver big data initiatives in future. Big data is considered a key enabler for future decision making and process automation. The topic is however very new and not well understood yet. Hence 50% of big data projects are not delivering the expected benefits and are costing more than initially planned.
Table of Contents
A. Introduction
Background
Industry partner
Project objective
Significance
B. Research methodology
Business Process Management Lifecycle
Method overview
Literature review
Phase 1 - Extraction of literature
Phase 2 – Organization and preparation for analysis of artefacts
Phase 3 – Coding and analysis
Phase 4 – Write up and presentation
Interviews
Project management approach for this research project
C. Results
Summary of literature review results
Waterfall approach
Agile
CRISP-DM
Hybrid agile and waterfall approach
Summary of interview findings
Big data Analytics Framework
1. Business Understanding
2. Understand and Prepare Data
3. Validate Business Understanding
4. Design Solution
5. Evaluate Solution
6. Validate Business Understanding
7. Deployment
D. Discussion
E. Conclusion
Project Objectives and Thematic Focus
The primary aim of this research is to identify best practice project management techniques for big data initiatives and to develop a structured framework that assists organizations in delivering successful big data projects. The research specifically addresses the high failure rate and unexpected costs currently associated with such projects by investigating industry-standard methodologies and validating them through corporate case study interviews.
- Analysis of project management methodologies for big data (Agile vs. Waterfall).
- Evaluation of the applicability of the CRISP-DM process in modern big data environments.
- Investigation into the effectiveness of hybrid project management approaches.
- Development of a standardized Big Data Analytics Framework.
- Validation of project management best practices through expert interviews in the corporate sector.
Excerpt from the Book
Hybrid agile and waterfall approach
Hayata et al (2011) states that a hybrid agile and waterfall approach is an evolving trend within organisations. Organisational change takes time and as technology teams are accustomed to their traditional way of working in a waterfall approach, the transition to an agile organisation can take many years. By using an agile and waterfall approach, it allows the organization to practice some of the agile techniques while remaining in waterfall-based world (Hotle et al, 2018). The initial literature review did not find supporting evidence for a hybrid agile and waterfall project management approach. The majority of the documents were very clear to validate an agile approach. However, the interviews confirmed that a hybrid approach was used although sometimes not prescribed and rather unknowingly. Two of the interviews operated in a waterfall approach. However, when questioned on the techniques and processes like CRISP DM used for the data analytics, it was confirmed that unknowingly this process has been followed. Further, interview 1 used a very pure agile approach which enabled the team to quickly commission a working solution to the organisation. Although one of the challenges encountered was insufficient licensing which could have been prevented by using a traditional waterfall approach. This suggests that a hybrid agile and waterfall approach would be more suitable for this organisation.
Summary of Chapters
A. Introduction: Provides background on big data, outlines the project objective, and details the significance of managing big data initiatives for the industry partner, Red Rocks Company.
B. Research methodology: Describes the application of the Business Process Management (BPM) Lifecycle, the literature review process, and the qualitative research methods used for conducting interviews.
C. Results: Presents the findings from the literature review and interviews, concluding with the development and step-by-step description of the Big Data Analytics Framework.
D. Discussion: Evaluates the research findings, correlates the project outcomes with the initial research goals, and addresses the limitations of the current study.
E. Conclusion: Synthesizes the core findings, confirming the value of the Big Data Analytics Framework and suggesting future research directions.
Keywords
Big Data, Project Management, Agile, Waterfall, CRISP-DM, Business Process Management, Data Analytics, Framework, Case Study, Innovation, Stakeholder Management, Hybrid Approach, Mining Industry, Technology Implementation, Process Automation
Frequently Asked Questions
What is the core focus of this research report?
The report investigates the project management approaches best suited for big data initiatives to address the current 50% failure rate and budget overruns common in the sector.
What are the central thematic fields covered in this document?
The core themes include the comparison of Agile and Waterfall methodologies, the role of CRISP-DM, and the practical challenges of integrating data analytics within large-scale corporate environments.
What is the primary objective of this work?
The objective is to understand best practices for managing big data projects and to develop a new, validated framework that helps organizations achieve their digital transformation goals.
Which scientific methodology was employed?
The researcher conducted a brief literature review of nine artefacts and performed five qualitative, semi-structured, face-to-face interviews with industry stakeholders.
What topics are discussed in the main part of the report?
The main section analyzes literature trends, presents interview findings, and outlines the seven-step Big Data Analytics Framework, providing justification for each phase.
Which keywords best characterize this work?
Key terms include Big Data, Project Management, Agile, Waterfall, CRISP-DM, and Data Analytics Framework.
Why is the CRISP-DM model still considered relevant in this framework?
The research concludes that while CRISP-DM is an older methodology, it provides the fundamental logical steps required for successful data analytics that many modern vendors continue to build upon.
How does the research reconcile the differences between Agile and Waterfall for big data?
The findings suggest a hybrid approach: While literature strongly recommends Agile, industry stakeholders often use Waterfall due to the nature of large assets, leading to the conclusion that a hybrid method is often more pragmatic.
- Arbeit zitieren
- Theres Mitscherling (Autor:in), 2018, Deriving a big data analytics framework. Approaching the project management process for big data initiatives, München, GRIN Verlag, https://www.grin.com/document/499625
