Excerpt
CONTENTS
ABSTRACT
LIST OF TABLES
LIST OF FIGURES
LIST OF APPENDICES
CHAPTER ONE: Introduction
1.1 Background
1.2 Aims
1.3 Objectives
1.4 Significance and Value
1.5 Structure of the Dissertation
CHAPTER TWO: Literature Review
2.1 Introduction
2.2 Night-time economy
2.2.1 The concept of night-time economy
2.2.2 Economic value of the night-time economy
2.2.3 Market segmentation
2.2.4 Student Nights
2.2.5 Leeds’ student nights market
2.3 Evaluation
2.3.1 Evaluation: Background
2.3.1 Types of Evaluation
2.3.2 Models of Evaluation
2.3.3 Post-event evaluation
2.3.3.1 Importance of Post-Event Evaluation
2.3.3.2 Conducting Post-Event Evaluation
2.4 Summary
CHAPTER THREE: Methodological process and methods
3.1 Introduction
3.2 Research Process
3.3 Understand Research Philosophy and Approach
3.3.1 Research Philosophy
3.3.2 Research Questions
3.3.3 Research Approach
3.3.4 Role of the Researcher
3.4 Formulate Research Design
3.5 Negotiate Access and Address Ethical Issues
3.5.1 Access
3.5.2 Ethics
3.6 Plan and Execute Data Collection
3.6.1 Sampling Methods
3.6.2 Participants
3.6.3. Data Collection
3.6.3.1 Internet Content Analysis
3.6.3.2 Questionnaire
3.6.3.3 Interviews
3.6.3.4 Justification
3.7 Limitations
3.8 Summary
CHAPTER FOUR: Analysis and Discussion
4.1 Introduction
4.2 Internet Content Analysis
4.2.1 Results
4.2.2 Discussion
4.3 Questionnaire
4.3.1 Structure
4.3.2 Results
4.3.3 Discussion
4.4 Answers to the Research Questions
4.4.1 RQ1: Student Nights’ Market
4.4.2 RQ2: Measuring Success
4.4.3 RQ3: Goal-attainment Method
4.5 Summary
CHAPTER FIVE: Conclusion and Recommendation
5.1 Introduction
5.2 Recommendation
5.2.1 Event Organisers
5.2.2 Improvements and Future Research
5.3 Conclusion
6. References
7. Appendices
LIST OF TABLES
Table 1 - Student Nights Organisers
LIST OF FIGURES
Figure 1 - Four Elements of Evaluation
Figure 2 – Research Process
Figure 3 - Importance of Evaluation
Figure 4 - Important Parameters in the Planning Stage
Figure 5 - Important Parameters after Staging an Event
LIST OF APPENDICES
Appendix 1: Declaration Form
Appendix 2: Informed Consent
Appendix 3: Participant Information Sheet
Appendix 4: Internet Content Analysis
Appendix 5: Questionnaire Results
ABSTRACT
In the events industry, post-event evaluation is becoming more and more recognised as a respected management tool for measuring success and a utensil for specifying whether objectives are accomplished. Yet, publications by academics and scholars have mostly overlooked the area which investigates the approaches organisers take for evaluating their events. To address this research gap, this study examines the evaluation approaches taken by student night organisers within the city of Leeds.
Student nights are defined as events taking place during late-night hours, being part of the night-time economy, which is a unique, unrepeatable and spatio-temporally restricted happening that specifically targets the student market of a city. The paper outlines that this is a relevant market segment for event organisers due to its size.
With an internet content analysis and an online questionnaire, student night organisers’ opinions on evaluation and their approaches to assess success were investigated.
With the empirical research, the dissertation shows that post-event evaluation theories, specifically goal-attainment models, are put partially into practice within the night-time economy. However, student night organisers in the city of Leeds do not take advantage of the full potential offered by the utilisation of summative event evaluation due to inconsistencies within the evaluation processes and frequencies. Further, the study identifies that organisations in the night-time economy measure success and assess performance when organising student nights mostly via economic objectives such as profit, cost and attendee numbers.
CHAPTER ONE: Introduction
1.1 Background
In the events industry, post-event evaluation is becoming more and more recognised as a respected management tool for measuring success and a utensil for specifying whether objectives are accomplished. Yet, publications by academics and scholars have mostly overlooked the area which investigates the approaches organisers take for evaluating their events. Instead the event evaluation research primarily covers subtopics such as event quality and process, customer satisfaction and the reasons for conducting evaluation (Williams and Bowdin 2007). Not only researchers but also organisations within the events industry neglect post-event evaluation (Blythe 2000). While Carlsen, Getz and Soutar (2001) mention that there has been significant development in the quality of event evaluation studies offering more breadth and comprehensiveness, the authors highlight the lack of homogeneity and the absence of standardised methods, which limit the comparability between different evaluation results. It is surprising that an absence of evaluation can be observed within the industry, as evaluation measures can help to determine the success or failure of a project. Hence, incorporating evaluation into an organisation’s overall process can prove to be beneficial (Watt 1998). When implementing evaluation, the total payoff can be great (Phillips, Breining and Phillips 2008). The academic literature agrees that the benefits resulting from conducting effective evaluation outweigh the expenses related to doing so (Wood 2009). This research will focus on the organisational point of view on the utilisation of goals, objectives and success indicators within post-event evaluation. For investigation, the Leeds student nights’ market was chosen.
1.2 Aims
The aim of the presented dissertation is to outline the student nights’ market in the city of Leeds. Further, the investigation of the utilisation of summative event evaluation within the setting of student nights in Leeds’ nightclubs and bars is an aim of this work. It intends to depict the event organisers’ perspective.
1.3 Objectives
In order to achieve the aims stated above, it is necessary to pursue and meet the following objectives:
- Outline the purpose of summative evaluation with focus on the events industry and in respect to the night-time economy.
- Test whether post-event evaluation theories are put into practice within the night-time economy.
- Examine and define the student nights’ market segment in the clubs and bars of the city of Leeds.
- Determine in how far student night organisers in Leeds take advantage of the potential offered by the utilisation of summative event evaluation.
- Identify how organisations in the night-time economy measure success and assess performance when organising student nights.
1.4 Significance and Value
Research on post-event evaluation approaches of organisations make a contribution to the academic field as it attempts to fill the research gap, which was identified by Williams and Bowdin (2007) and Getz (2010). By understanding the approaches, the success indicators and achievement standards important for student night organisers are established. With recommendations for refining methods and making approaches more sophisticated, practical advice to increase the level of success will be offered to student night organisers. The results from this empirical research will be an original contribution to the knowledge base of the field of event evaluation. Furthermore, the study investigates the student nights’ market in the city of Leeds and, thus, investigates a specific subsector of the events industry. By doing so, valuable information about the market segment is presented.
1.5 Structure of the Dissertation
In this dissertation, chapter one presents the introduction which is followed by the literature review. The literature review details on the night-time economy and student markets within the city of Leeds and their significance. By doing so, student nights and the student market segment are defined. Further, the second chapter offers a framework for understanding summative evaluation in the context of post-event evaluation. It focuses in particular on result-based evaluation models as these are predominant within the events sector. After establishing these foundations, chapter three presents the methodological processes of the presented dissertation. It outlines the applied research methods, data collection processes, and sampling as well as ethical implications that go along with primary research. Further chapter three also puts a focus on the limitations of this study. It then moves on to present and analyse the results of the empirical study. In the fourth chapter the implications of the results are also outlined. After the discussion of the results, the dissertation finishes with recommendations on the topic and a general conclusion.
CHAPTER TWO: Literature Review
2.1 Introduction
This chapter deals with the history of the night-time economy and displays its significance for the overall UK economy. Further, it defines the term ‘student night’ and outlines the student market within the city of Leeds relevant for student night organisers. In the following section, existing literature on evaluation theory, summative evaluation and post-event evaluation is reviewed. This secondary data collection provides background knowledge on the theories that will be drawn upon and delivers an overview of previous research in this field conducted by academics (Steward and Kamins 1993). Hence, chapter two of this dissertation provides the reader with the knowledge base needed to grasp the study by enhancing the understanding of the topic.
2.2 Night-time economy
This section of chapter two explains the concept of a night-time economy and highlights its economic value for the UK. It then proceeds to detail on the student market within the night-time economy, to present the market segmentation relevant for this study and defines the term ‘student nights’. The section then ties the information together and concludes by detailing on the student market in the city of Leeds.
2.2.1 The concept of night-time economy
The concept of a night-time economy was first introduced by the Italian politician Renato Nicolini, who was the councillor in charge of cultural policy in the time period from 1976 to 1985. He organised a summer programme of cultural events in the Italian capital Rome (Bianchini and Parkinson, 1993; Bianchini 1995). However, the utilisation of cultural policies to revive metropolitan nightlife was not an Italian invention. Many other European cities in countries such as Germany, France, Denmark, the Netherlands and Sweden followed similar policies in the 1970s and 1980s. They were introduced as politicians recognised the growing public demand for going out, entertainment and spending leisure time in city centres during the evening hours (Bianchini 1995). By emphasising the night-time economy, the lack of activity during evening and late-night hours in city centres was addressed and concepts to attract people back to the urban areas were introduced (Tiesdell and Slater 2006). Even though first concepts of the night-time economy were introduced in the 1970s and 1980s, it gained its popularity within the circles of scholars and policymakers during the 1990s. The timing of the field’s increasing popularity was linked to the extensive social impact of rave-party culture from the early 1990s in Europe, the embrace of the idea of night-time economy by policymakers and its heavy impact on city planning, and global recognition of the economic significance of the cultural industries. Night-time economy was mostly used to specifically support urban renewal, attract tourism and revive the cultural economy (Chew 2010). The night-time economy is defined as a hedonic space which is addressed to pleasure and relaxation and hence a part of leisure time. The space is occupied by all different types of members of society with varying levels of participation. At the same time, the late-night economy is a typical market space for buying and selling as well as creating profit (Maris 2009). To be more specific, the term night-time economy is defined as “a subset of the cultural economy composed of bars, dance clubs, karaoke establishments, rave parties, live-music venues, and other nightlife forms (while not including the commercial sex industries)” (Chew 2010, p. 3). While the definition by Chew (2010) focuses only on the core activities of the night-time economy such as pubs, nightclubs and restaurants, Evans (2012) also includes non-core activities such as accommodation, taxi services and food and drink supplies. City Councils within the UK also state that the night-time economy refers to bars, pubs, casinos, restaurants, takeaways and nightclubs that profit from visitors during late night hours (Plymouth City Council 2007; City of Edinburgh Council 2010). The night-time economy is also referred to as late night economy as well as evening economy, and is embodied by the notion of the 24-hour-city (Evans 2012). All core activities of the night-time economy can be referred to as nightclubs. Nightclubs are not only clubs and discos venues which offer dancing, but late-night bars, pubs and bars are also characterised as nightclubs (Kubacki et al. 2007). Hence, this goes hand in hand with the description of the core activities of the night-time economy.
The Licensing Act 2003 came into force on November 24, 2005. With this Act, a single integrated scheme for licensing premises, which deal with alcohol, provide entertainment and have flexible late night opening hours, was introduced. Hence, all core activities of the night-time economy are in need of premises licences according to the Licensing Act 2003 (Babb 2007). This means that all businesses operating a venue in the night-time economy have to have a Premises License according to the Licensing Act 2003.
Overall, the concept of a night-time economy is relatively new as it only gained popularity in the 1990s. It refers to businesses that operate at late night hours, which are now in need of a Premises License and thus can be identified easily.
2.2.2 Economic value of the night-time economy
The night-time economy is an important economic factor. However, it brings many negative and problematic aspects with it. The City of London noticed an increased number of concerns from the local community about noise, violence, and urination in streets related to the night-time economy (City of London 2012). Rubbish, noise and anti-social behaviour resulting from late-night activities are concerns of the City of Edinburgh (City of Edinburgh Council 2010). Similarly, Cardiff, Plymouth and Lancashire Councils recognise an increase of anti-social behaviour which is linked to the night-time economy (Plymouth City Council 2007; Cardiff Council 2012; Lightowlers et al. 2012). While some city authorities highlight negative aspects that are connected to a vibrant night-time economy such as littering, noise and fouling of streets, the night-time economy is accepted due to the economic opportunities arising from it. With a developed nightlife concept, cities attract residents and investors. By doing so, leisure venues are expanded, the number of jobs increased, greater numbers of tourists attracted and the vitality of urban areas improved (Evans 2012). Economically speaking, the night-time economy is contributing significantly to the UK’s overall economy. The nightlife sector is one of the key gauges of a prosperous overall economy and becoming more and more essential to the UK economy as a whole (Hollands and Chatterton 2002). In 2009, the night-time economy contributed £66 billion to the UK and employed 1.3 million people within its core activities. When combining the core and non-core activities, the late-night economy accounted for ten per cent of all UK employment, was composed of eight per cent of UK firms and generated six per cent of all UK turnover (Bevan and Turnham 2010). Within the UK, the British capital is dominant when it comes to the night-time economy. In London, core and non-core activities accounted for a total turnover of £10 billion in 22,000 firms which participate in the night-time economy (Evans 2012). When looking at the turnover from the year 2009, London is followed by Birmingham and Manchester. These two cities had a turnover of over £1 billion. Camden and Glasgow City complete the top five. Leeds is the sixth largest contributor to the UK’s night-time economy with a turnover of £873 million in 2009. In terms of firms that participate in the late-night economy, London is followed by Birmingham. Leeds claimed the third place within the UK, having 2,338 firms that were part of the core and non-core activities of the night-time economy in the year 2009 (Bevan and Turnham 2010). In short, even though the night-time economy brings negative factors with it, its economic value is widely accepted and recognised by academics and city authorities in the UK.
2.2.3 Market segmentation
Within the night-time economy of the UK, two aspects are highly emphasised: ‘youth’, the age group of 16-25 year-olds, and ‘alcohol’ (Tiesdell and Slater 2006). The night-time economies of many cities in the UK have boomed in recent years mostly by relying on the increasingly huge student populations (Geddes 2001). Graduates, students, young professionals and service workers as well as cultural intermediaries are a key force for the night-time and cultural economy (Hollands and Chatterton 2002). While the evening economies of British city centres are currently dominated by the 18-30 age groups (Geddes 2001), especially the age group from 18-24 is the main target group (Lightowlers et al. 2012; Mintel 2012). Within this group, university students are the most dominant. The majority of licensed premises in high streets of British towns and cities are directed towards a demographic clientele of 18-24 year olds. Within this group, students are forming a particular target for some of these venues (Roberts 2006). The student population is a sought after target as, despite of the current economic situation, the willingness to spend of this group has not changed; therefore, students represent a lucrative market for leisure operators (Mintel 2010). Hence, students are a profitable target market for the events industry within the night-time economy.
To clearly define this target group, market segmentation is needed. “Market segmentation is the division of a market into different groups of customers with distinctly similar needs and product/service requirements” (Baines, Fill and Page 2008; p. 217). In other words, a market is evaluated and then broken down into smaller parts, which are then more manageable. By doing so, resources, assets and efforts of an entity can be focused on a certain group or cluster of people or a specific target market instead of attempting to satisfy the needs of all individual market participants (Wind 1978; Middleton 1994). Hence, the overall goal of market segmentation is to group customers who have similar needs and comparable buying behaviour into segments (Perreault et al. 2000; Epetimehin 2011). To be valuable, a market segment has to be measurable, sustainable, and accessible as well as sufficiently different from another segment (Ahmed, Barber, and d’Astous 1998; Kotler and Armstrong 2008). Thus, the process of developing marketing concepts that suit the requirements of different target segments is facilitated and organisations can therefore concentrate on specific market participants, exempli gratia the most lucrative one (Court and Lupton 1997; Epetimehin 2011). Ideally, effective market segments have to meet four criteria: homogeneity within, heterogeneity to others, substantiality, and have to be operational (Perreault et al. 2000). Hence, the most common market segmentation concepts are demographic segmentation and psychographic segmentation. While demographic segmentation focuses on parameter such as age, sex, race, income, occupation, and socioeconomic status, psychographic segmentation or lifestyle segmentation goes further by also considering attitudes to life, beliefs and aspirations (Brassington and Pettitt 2006). In spite of the potential the college market offers to organisations, this market segment has not received a lot of attention (Field 1999; Noble, Haytko and Phillips 2007). A market segment that focuses on students clearly puts an emphasis on demographics. The typical core university student market is characterised by an 18-to-24-year-old person (Weese 2002). Thus, the student market segment is a subgroup of the overall 18-24 year olds (Wolburg and Pokrywczynski 2001). University students are a vital customer segment to many businesses as they tend to have a high level of disposable income which is spent on leisure services, entertainment activities as well as basic services and goods (Wolburg and Pokrywczynski 2001; University of Wisconsin 2007). Due to the sheer market size, the student market segment is one of the most coveted target groups for businesses (Wolburg and Pokrywczynski 2001; Noble, Haytko and Phillips 2007; Quintal, Phau and Sims 2009). This is also true for event organisers who operate within the night-time economy. The student market segment is homogeneous within, heterogeneous to other segments, substantial enough to make a profit, and operational as it clearly identifies the potential customer. Hence, it fulfils the parameters set by Perreault et al. (2000). In terms of the night-time economy, students spend a sizable share of their income on going out (Hollands 1995).
A study from RBS (2007) showed that students in Great Britain spend above £10.3 billion in housing and living costs with over £730 million of these expenses on going out. Within the overall group of 18-24 year olds, students are the most profitable segment. On average they spend about £6,500 on living costs annually and about £1,150 of these expenses are put towards entertainment (Campus Group 2011). If one excludes mature students, this target segment is largely characterised by young adults, in charge of their own finances and who have a relatively large proportion of leisure time within their schedules. Taking into consideration the large amount of free time and the vast amounts of spending on going out, it is not surprising that students are a substantial portion of the users of the night-time economy. This is especially true as they go out more frequently and more often than locals and other market segments (Hollands 1995). As a result of the lifestyle marketing in which the leisure industry specifically targets students, the student group is a protagonist within the night-time economy (Brabazon and Mallinder 2007).
To summarise, the student market segment is comprised of mainly 18-24 year olds who are attending a higher education institution. Further, members of this marketing segment have large amounts of leisure time and actively participate within the night-time economy. For these reasons, the students are a sought after market segment for organisations that operate within the night-time economy.
2.2.4 Student Nights
For the purposes of this paper, the night-time economy is essential. However, as the dissertation focuses on events within this particular economic sector, it is vital to define student nights. There are multiple definitions for events itself. Getz (1997, p. 4) defines it as follows: „every event is a unique blending of its duration, setting, management, and people” while Raj, Rashid and Walters (2009) simply refer to “happenings with objectives” (p. 11). Shone and Parry (2010) highlight eight characteristics of events: (a) uniqueness, (b) perishability, (c) intangibility, (c) ritual and ceremony, (d) ambience and service, (e) personal contact and interaction, (f) labour-intensiveness, and (g) fixed timescale. As one can see, there are various definitions and interpretations. Academic literature and various authors however agree largely on some aspects. An event is therefore generally defined as a unique, unrepeatable and spatio-temporally restricted happening (Wochnowski 1996; Holzbaur et al. 2002).
Based on these definitions, student nights are to be characterised in the following. Student nights are mentioned in academic literature for instance by Chatterton and Hollands who have researched youth culture, nightlife and urban change as part of a project funded by the UK Government's Economic and Social Research Council (Newcastle University 2002). The authors write:
„Student life leaves a visible mark on mainstream nightlife with mid-week evenings often designated as ‘students’ night’ by scores of competing pubs and clubs to avoid clashes with local young people” (Chatterton and Hollands 2002; p. 110).
However, academic literature lacks a clear definition of student nights. From the main aspects, a definition will be developed. Student nights are part of the night-time economy as they take place during late night hours and the early morning hours (Roberts 2006). Furthermore, they are staged in nightclubs, bars, pubs and music venues within city centres of a town (Hollands and Chatterton 2002). Thirdly, student nights target the student population within a city by having specific drink offers, entry fee reductions and the like for this particular group of people (Hollands et al. 2001; Kosnick 2010; Göcke 2012). For the purposes of this paper, a student night is hence defined as an event taking place during late-night hours, being part of the night-time economy, which is a unique, unrepeatable and spatio-temporally restricted happening that specifically targets the student market of a city.
2.2.5 Leeds’ student nights market
Having established the concept of a night-time economy, outlined the student market segment in general and defined student nights, these aspects have to be tied together. To bring all these pieces into the relevant context for this dissertation, the student night market within the city of Leeds is to be characterised.
As outlined above, Leeds has a vibrant night-time economy. Leeds is the sixth largest contributor to the UK’s night-time economy with a turnover of £873 million in 2009. In terms of firms that participate in the late-night economy, Leeds claimed the third place within the UK, having 2,338 firms that were part of the core and non-core activities of the night-time economy in 2009 (Bevan and Turnham 2010). Baron (2011) also highlights Leeds importance by stating that “Leeds has the second biggest night-time economy – after Westminster – in the UK.” When looking at the core activities, Leeds has a wide variety and high quality of nightlife to offer (Visit Leeds 2012; Pugsley 2012; Leeds-UK 2012; IQ Student Accommodation 2012). Over the years, Leeds has been home to some of the biggest nightclubs of the country and considered itself at the forefront of the clubbing scene (Pugsley 2012). The city is often described as the number one clubbing destination outside of London as it hosts one of the best nightlife scenes within the UK (Skiddle 2012; Leeds City Licensing Association 2012). This great reputation is a result of the bars, pubs and the thriving music scene within the city of Leeds (My life in Leeds 2012). The variety of these venues has a wide span: from hip and trendy bars and old fashioned drinking dens to late-night clubs (Leeds-UK 2012; Visit Leeds 2012) which suit the need of every student (Skiddle 2012). This variety is essential for a vibrant night-time economy as the quantity and range of venues operating during late-night hours has increased in lots of city centres of major towns within the UK (Roberts and Eldrige 2007).
Since 1992, a steady increase in the overall number of university students within Britain could be observed. There are about 2.5 million students pursuing higher education within the UK (Universities UK 2011), 1.27 million undergraduate students at UK universities in the academic year 2008/09 alone (Mintel 2010). Leeds is a major student city within the UK with an overall student population of over 200,000 (Marketing Leeds 2012); however, this estimate from Marketing Leeds is likely to refer to the Leeds City Region which embraces areas of West and North Yorkshire including other towns such as Bradford, Halifax and Harrogate and not only Leeds as a town itself. According to the Universities and Colleges Admissions Service, the British admission service for people applying to go to universities and colleges, Leeds has two major education institutions: University of Leeds and Leeds Metropolitan University. Each of these universities has about 30,000 students. The other educational entities in the city have less people enrolled. For instance Leeds Trinity University College has 3,300, Leeds College of Arts about 1,800 and Leeds College of Music around 700 students. Hence, it is safe to estimate the current student population in Leeds to be around 66,000 to 70,000 (UCAS 2012). In Leeds, the age group of 20-24 year-olds makes up the largest part of the population with circa 11.5 per cent. This is well above the national average of seven per cent and also above the eight per cent mark of the Yorkshire and the Humber region (Leeds City Council 2012a). Hence, the student population in the City of Leeds depicts an interesting market segment due to its sheer size.
A great number of British cities are dedicated to meeting the needs of students in educational matters, housing provision and entertainment terms. Nightlife venues offer students a variety of promotional discounts such as reduced admission fees and happy hours. In many towns and cities specific student bars, nightclubs and pubs exist with the sole purpose of catering to the exclusive needs of this target group (Hollands et al. 2001). The city of Leeds is no different and offers deals for students. Furthermore, it hosts many student-friendly venues, pubs and bars which operate within the night-time economy (BBC 2004). The RBS Student Living Index revealed that Leeds offers the best value and cost efficiency for students of all cities in the UK (RBS 2007).
Summing up, it is safe to say that the student market segment for night-time economy participants in Leeds is very lucrative due to its size. Leeds is a major student city. The large number of students tends to spend higher percentages of their income on going out and participate in the night-time economy more actively than other market segments (Hollands 1995). This might be the reason why businesses within the city of Leeds have already adapted their programmes and are catering to the needs of the large student population. This is no different for event organisers within the city.
2.3 Evaluation
This section of the literature review details on evaluation. It covers different types and models of evaluation. Further it deals with post-event evaluation in great detail. Hence, it provides the necessary background knowledge for understanding the theory this study is based upon.
2.3.1 Evaluation: Background
As evaluation is diverse, multiple definitions on this topic exist. The term ‘evaluate’ has its roots in the Old French words ‘value’ and ‘valoir’ as well as the Latin verb ‘válere’. All these have the same two meanings: ‘to work out the value of’ and ‘to be worth’. Not just in this historical origin but also in its present day use, evaluation has the double meaning of estimating the worth of and finding a numerical expression for. In the context of evaluation, worth has varying meanings (Mark, Greene and Shaw 2006). Hence, the definition depends on and is determined by the context (Komrey 2001). The American Evaluation Association (2012) states that “evaluation is a field that applies systematic inquiry to help improve programs, products and personnel, as well as the human actions associated with them.” By Farell et al. (2002) evaluation is defined as a method utilised to measure the value or worth of an activity. Another definition provided by Weiss (1972, quoted in Kaneko 1999, p. 433) is that evaluation measures “the effects of a program against the goals it set out to accomplish as a means of contributing to subsequent decision-making about the program and improving future programming.” Further, one of the oldest and yet still most prominent definitions on this topic highlights that evaluation has the meaning of determining whether predetermined goals have been met (Stufflebeam and Shinkfield 2007). While there are various other definitions from authors such as Springett (2001) and Tones (1998), most definitions agree that the main purpose of evaluation is to determine whether goals and objectives have been achieved and to assess the effects of a specific action. There are four main purposes for evaluation: (a) improve programmes and organisations, (b) utilise it as a surveillance mechanism, (c) assess a project’s merit, and (d) develop knowledge (Donaldson and Lipsey 2006). Green and South (2006) also establish four purposes: (a) accountability, (b) learning, (c) programme management and development, and (d) ethical obligation. No matter the underlying purpose, evaluation becomes more and more recognised and important in all industries (Van den Bussche et al. 2006); therefore also within the industry sector concerned with organising events.
2.3.1 Types of Evaluation
Depending on when evaluation takes place, there are different types of evaluation. Scholars distinguish between formative and summative evaluation (Komrey 2001). Formative evaluation is conducted while a process or programme is taking place. Hence, its goal is to optimise things while they are being executed by providing information for developing better service, ensuring quality and improving it (Schaub and Zenke 1997; Stufflebeam and Shinkfield 2007). Thus, formative evaluation can have consequences and effects on the evaluated subject as changes might be applied during the process (Komrey 2001). Some academics also separate formative evaluation in two steps: formative and process evaluation. In this case, formative evaluation focuses on programmes which are still under development, while process evaluation focuses on programmes which are already under way (The Health Communication Unit 2007). However, it is more common to simply distinguish between formative and summative evaluation (Freigang 2004). Contrary to formative evaluation, summative evaluation is a conclusive assessment at the end (Schaub and Zenke 1997), hence has a judging character (Komrey 2001). It aims to assess if a programme made a difference, which is an impact evaluation. Likewise, it seeks to evaluate a programme’s outcome by assessing if it met its stated objectives and goals (The Health Communication Unit 2007). By doing so a nominal-actual analysis is conducted (Freigang 2004), which determines accountability for failure and success in this retrospective assessment (Stufflebeam and Shinkfield 2007). To assess the success or failure and the achievements of a specific activity, summative evaluation is utilised (Green and South 2006) and the overall effectiveness and performance of a programme and whether it met its goals, expectations and objectives are assessed (Rossi, Lipsey and Freeman 2004).
2.3.2 Models of Evaluation
Numerous different concepts and approaches for conducting evaluation are suggested by academic evaluation literature (EPA 1998). Hansen (2005) distinguishes between six different models of evaluation: result models, explanatory process models, system models, economic models, actor models and programme theory models also known as theory-based models. Process models highlight on-going procedures and efforts. System models examine input, organisation, progression and outcome in terms of results. Economic models evaluate outcomes in terms of monetary expenses. Actor models are based on criteria for assessment which are established by the actors themselves. The programme theory model has its emphasis on gauging the validity of the programme theory on which the particular activity or organisation builds. The result models are applied in summative evaluation and, hence, are the ones relevant for this dissertation. Their focus is on the results of a specific programme, organisation or performance. Result models can be divided into goal-attainment model and effects model. The effects model is a goal free evaluation model with the intention to expose and interpret all consequences of the evaluated activity. The goal-attainment model on the other hand is the classic model in the academic literature on programme evaluation and organisational evaluation. It is based on predetermined goals and objectives. The goals that are assessed in this theoretical concept are objectives, performance targets and projected outcomes (Hansen 2005; Kahan 2008). As mentioned above, one of the oldest and wide-spread definitions of evaluation states that it means assessing whether predetermined objectives have been met (Stufflebeam and Shinkfield 2007), this clearly relates to the goal-attainment theory of the result models.
2.3.3 Post-event evaluation
Linking the evaluation models and types of evaluation to events, the area of post-event evaluation is to be detailed on as it is relevant for this study. In respect to the events industry, one can say that “event evaluation is the process of critically observing, measuring and monitoring the implementation of an event, in order to assess its outcomes accurately” (Allen et al. 2005, p. 449). Event evaluation can occur at any point throughout the event management process. Pre-event assessments most commonly take place during the stages where an event is researched and planned. Further, events can be monitored during its various implementation stages (Allen et al. 2005). These two steps relate to the aforementioned concept of formative evaluation (The Health Communication Unit 2007). Post-event evaluation is conducted after an event (Watt 1998). The main goals of this form of event evaluation are to answer the following two questions: (a) ‘Did the event meet its objectives?’ and (b) ‘What can be improved for the next event?’ (Shone and Parry 2010). Post-event evaluation “involves the gathering of statistics and data on an event and analysing them in relation to the event’s mission and objectives” (Allen et al. 2005, p. 451). Hence, it relates to summative evaluation and the goal-attainment model as detailed above (Hansen 2005). The majority of literature on events, which deals with evaluation, pertains to assessments of performance quality and customer satisfaction. A subject area of event evaluation which has been neglected and offers limited research is success factors (Getz 2010). Phillips, Breining and Phillips (2008) also recognise a lack of evaluation within the events industry even though evaluation practices are becoming more and more utilised. The most popular format for evaluation is post-event evaluation also known as summative evaluation (Allen, et al. 2005; Bowdin et al. 2006). Whilst academics as well as practitioners agree on the fact that evaluation is essential, how and what should be measured and assessed is still debated (O’Sullivan, Pickernell and Senyard 2009). Hence, this paper focuses on post-event evaluation and its utilisation within the industry.
2.3.3.1 Importance of Post-Event Evaluation
Event evaluation is a vital part of the overall event management process. By conducting event evaluation, insights are gained, valuable lessons are learnt and, thus, events are improved (Allen et al. 2005). Summative evaluation provides a clear record of performance. If it is not conducted, success and failure of an event are matter of perception and not measured and clearly defined (Tum, Norton and Wright 2006). As completing a retrospective assessment aids an organisation in determining an event’s strengths and weaknesses (Meyers 2012), if properly interpreted, used and implemented, it is a driver of continuous improvement (Allen et al. 2005). Research proposes that summative evaluation is essential for an event and should be completed regularly. In order to do this, it is of utmost importance to create gauges for the event evaluation. Such criterions can be founded on tangible and quantitative hard criteria and on intangible and qualitative soft criteria (Watt 1998; Crowther 2010). This goes hand in hand with the theory of the goal-attainment model of result-based evaluation as it assesses success according to predetermined goals (Hansen 2005). However, event evaluation is often neglected by organisers themselves after an event has been completed (Van der Wagen 2005). This is commonly due to cost and time constraints. An absence of awareness of significance as well as usefulness of evaluation also plays a role in the negligence (Robinson, Wale and Dickson 2010). Wood (2009) agrees that limited resources such as time and money are restraining factors and the main reason why companies neglect evaluation. Further, the author points out that objectives and evaluation are related to a vicious cycle. Events without clear objectives are hard to assess and the objectives for future events have to be based on evaluations conducted in the past. Blythe (2000) also highlights the confusion that comes with evaluation as
“having a reason for doing something does not always lead to formalising an objective, nor does having an objective necessarily mean having a formal system for evaluating its achievement” (p. 207).
Pugh and Wood (2004) point out that there is often inconsistency when carrying out post-event evaluation. This is an unwanted status quo as several positive learning effects can be attained from critiquing an event. It can help in making future events more successful as it points out performance weaknesses and establishes a benchmark for the future (Van der Wagen 2005; Robinson, Wale and Dickson 2010). Also, if no evaluation is done, events are staged with little to no information in regards to what they can do and achieve (Wood 2009). Further, improvements are then difficult to realise and the efficiency and effectiveness of existing strategies and methods is hard to measure if there is no or inconsistent evaluation (Pugh and Wood 2004). Hence, event evaluation ought to be integrated into the planning process to measure the success of a specific event (Watt 1998; Crowther 2010). Consequently, it is important to conduct event evaluation after each and every event and not only when staging big events (Watt 1998; Meyers 2012). Even though the methods, formats and formalities of event evaluations may vary, Watt (1998) emphasises that event evaluation is vital and must be carried out. This opinion is also advocated by Tum, Norton and Wright (2006) who state that it is vital to evaluate every event and every aspect of an event. Further, the authors stress post-event evaluation should be carried out soon after an event is competed to ensure accuracy of results. This outlines the importance of evaluation as well as the significance of the process of conducting post-event evaluation.
However, there is not only a problem with negligence and inconsistency of evaluation. Some authors also highlight that there are problems, even if evaluation is conducted by event organisers. Getz (2000) points out that the major focus is on economic factors, when evaluating events. This economic focus is especially true when assessing event impact within the course of evaluating an event. By emphasising economic aspects, other factors relevant to the overall evaluation of an event might be neglected (Williams and Bowdin 2007).
Summing up, academic literature recognises that evaluation is often not conducted due to the complexity of measuring all variables as well as a lack of financial means and time. Further problems due to inconsistency are also highlighted. However, the academic literature agrees that the benefits resulting from conducting effective evaluation outweighs the expenses related to doing so (Wood 2009) and points out the importance of evaluation for the events industry and all organisers operating in this sector.
2.3.3.2 Conducting Post-Event Evaluation
Summative event evaluation implicates the collection of data resulting from a specific event and their analysis with reference to its mission and aims (Bowdin et al. 2006). To measure an event’s effectiveness, objectives have to be set and evaluation methods have to be established (Blythe 2000). Evaluation has to be planned in advance as the predetermined event objectives guide the evaluation process (Van der Wagen 2005). Hence, objectives have to be chosen carefully. One also has to be cautious to distinguish between aims and objectives. An aim is a motive for carrying out a specific action (Blythe 2000). Objectives on the other hand imply specific outcomes and form targets which have to be achieved in a specified period of time (Howell and Badmin 1996; Blythe 2000). Sometimes, key performance indicators are implemented to convert the event objectives into quantifiable means which can be used to assess the success of an event (Allen et al. 2005). Indicators are “a specific statistical measure of performance. Key indicators are those performance measures considered to be the most important” (Tichelar 1997, p. 32). These goals, objectives and key performance indicators form criteria for the event evaluation. Tangible and quantitative gauges are usually referred to as hard criteria (Watt 1998; Tum, Norton and Wright 2006). They are also the most desired type of data as hard criteria are measurable and quantifiable. In contrary, soft data is usually more difficult to collect and analyse. Also, as hard data is more objectively based, it represents more common and credible standards for assessing an organisation’s performance (Phillips, Breining and Phillips 2008). Examples for such gauges are cost requirements, deadlines, performance specifications, specific quality standards, and resource constraints (Watt 1998). When establishing goals and objectives, they have to be SMART: specific, measurable, achievable/agreed-upon, realistic and time-based (Tum, Norton and Wright 2006; Day and Tosey 2011). Creating specific objectives is the initial measure as it is essential to establish what exactly has to be accomplished. Setting measurable goals refers to the fact that these goals have to be quantifiable to determine a specific level of accomplishment. Further, measurability establishes accountability as this leaves less room for interpretation. The term ‘achievable’ in this context means that objectives have to be realistic and reasonable, id est achievable with respect to the resources available. Moreover, objectives and goals have to be relevant. This can be achieved through prior validation of the link of projected results with the proposed objectives, and listing objective in their order of significance. The last aspect ‘time-based’ states that objectives have to have a deadline and, hence, have to be achieved by an agreed-upon specific point in the future (Jung 2007; MacLeod 2012). These SMART goals usually draw upon established concepts from goal-oriented theory, which relates back to the goal-attainment theory discussed before (Day and Tosey 2011). Each event has to be judged against its customised set of SMART objectives. If not all aspects of the SMART concept are fulfilled, the objectives are formulated incorrectly. If an event has specific, measurable, achievable, relevant and time-based goals and objectives, they can be utilised to gauge the level of success or failure (Watt 1998). For conduction event evaluation, Tum, Norton and Wright (2006) suggest a four-step-process as depicted in Figure 1 – Four Elements of Evaluation. The first step is to set standardised specifications. The second part is to gather feedback of the actual performance. Thirdly, the actual performance has to be measured against the standard specification. Lastly, a correction has to be done if there is a deviation from the specification. This is a continuous process and is not limited to one event, but can also lead from one event to the next.
Figure 1 - Four Elements of Evaluation
illustration not visible in this excerpt
(Adapted from Tum, Norton and Wright 2006; p. 243)
2.4 Summary
This chapter has summarised the essential background knowledge for this study. Firstly, in the course of detailing the night-time economy, it examined and defined the student nights’ market segment in the clubs and bars of the city of Leeds. Further, it outlined the purpose of summative evaluation with focus on the events industry and in respect to the night-time economy. Moreover, it explained post-event evaluation theories.
This chapter highlighted that there are no standardised methods for measuring success, failure and impact of events. In theory, the result-based model is predominant within the events industry. Further, the literature review points out, that intangible aspects are hard to assess via result-based model. Hence, non-profit and non-financial objectives are seldom put into place. Thus, this chapter builds a foundation on which the empirical research is based.
CHAPTER THREE: Methodological process and methods
3.1 Introduction
The aim of chapter three is to describe the methodological process and methods underlying this dissertation. This chapter critiques the philosophical stance this study is taking. Further it states the research questions and reflects why these are appropriate. Moreover, it provides an outline of the research methods selected and explains the chosen sampling methods and sampling size. Furthermore, the ethical implications connected to the empirical research are outlined.
3.2 Research Process
In most research projects, a multi-stage process is followed to complete the study. This process is referred to as the research process. The process depicted in Figure 2 – Research Process which was developed by Saunders, Lewis and Thornhill (2009) has been chosen as it is the most appropriate and suitable research process for this study compared to others such as Walliman (2011), Oliver (2010) or Stanley (2000).
Figure 2 – Research Process
illustration not visible in this excerpt
(Adapted from Saunders, Lewis and Thornhill 2009, p. 11)
[...]