Make the Most of Your Studies: A Case Study and Web Usability Study Using the Collaborative Virtual Learning Environment Moodle

Table of Contents

1 Introduction
1.1 Research Problem
1.2 Rationale
1.3 Aim and Objectives
1.4 Scope
1.5 Limitations and Implications
1.6 Originality and Value
1.7 Structure of this Dissertation

2 Methodology
2.1 Research Design
2.1.1 Single Intrinsic Case Study Approach
2.1.2 Sampling and Ethical Considerations
2.2 Data Collection
2.2.1 Usability Testing and Observation
2.2.2 Interviews
2.2.3 Questionnaires
2.2.4 Freedom of Information Requests
2.2.5 Additional Sources for Background Knowledge
2.3 Chapter Summary

3 Literature Review
3.1 Definitions
3.2 Moodle
3.3 Moodle Developments towards Moodle 2.0
3.4 Past Research
3.5 Online Communication Tools
3.6 Challenges and Opportunities Using e-Learning Approaches
3.7 European Legal Framework
3.8 Chapter Summary

4 Setting the Scene
4.1 Background and Infrastructure
4.2 Student Demographics
4.3 Moodle Implementation
4.4 Chapter Summary

5 Using Moodle as a central platform for?
5.1 Finding the Right Information, When and Where Needed
5.1.1 Information Dissemination Outside Moodle
5.1.2 Access to Moodle
5.1.3 Evaluation of Moodle’s Usability in ‘Business Essentials’
5.1.4 Recommended Re-Design and Setup for ‘Business Essentials’ .
5.1.5 Section Summary
5.2 Providing Interactive E-learning Content Using Moodle
5.2.1 Evaluation of Moodle’s Interactivity in ‘Business Essentials’
5.2.2 Identified Training Needs for University Staff Members
5.2.3 Example Functions and Features for ‘Business Essentials’
5.2.4 Section Summary
5.3 Challenges and Opportunities Outside Moodle
5.4 Evaluation of Selected Method and Methodology
5.5 Chapter Summary

6 Conclusion

7 Summary of Recommendations

References

Bibliography

Appendix Overview

Appendix 1 Permission Gatekeepers

Appendix 2 Permission Usability Testing

Appendix 3 Permission Interviews

Appendix 4 Invitation for Full-Time Students

Appendix 5 Invitation for ODL Students

Appendix 6 Online Setup in Moodle

Appendix 7 Usability Testing - Tasks

Appendix 8 Usability Testing - Evaluation Questions

Appendix 9 Performance Transcript - Student A

Appendix 10 Performance Transcript - Student B

Appendix 11 Performance Transcript - Student C

Appendix 12 Performance Transcript - ODL Team Leader

Appendix 13 Highlight Video - Academic Staff Member

Appendix 14 Interview with DELTA Staff Member

Appendix 15 Interview with Library Staff Member

Appendix 16 Interview with ODL Team Leader

Appendix 17 Interview with Academic Staff Member

Appendix 18 Index of Learning Styles (Results)

Appendix 19 Moodle Questionnaire (Results)

Appendix 20 Freedom of Information Request I

Appendix 21 Follow-up - Request I

Appendix 22 Freedom of Information Request II

Appendix 23 Follow-up - Request II

Appendix 24 Response to Follow-up Request II

List of figures

Figure 1.1: Knowledge Pyramid of The University Community

Figure 2.1: Applied Research Hierarchy

Figure 2.2: Screenshot: Example Report Screen in Business Essentials

Figure 3.1: Screenshot: Moodle Statistics (16 May 2009)

Figure 3.2: Herrmann Brain Dominance Instrument

Figure 4.1: Age Pattern in ‘Business Essentials’

Figure 4.2: Country of Origin in ‘Business Essentials’

Figure 4.3: Grade Pattern in ‘Business Essentials’

Figure 4.4: Overview Resits ‘in Business Essentials’

Figure 4.5: Screenshot: Databases for CampusMoodle Improvements

Figure 4.6: Screenshot: Submission form ‘CampusMoodle Known Issues’

Figure 5.1: Screenshot: RGU Homepage

Figure 5.2: Screenshot: RGU Intranet (Public View)

Figure 5.3: Screenshot: Student Portal ‘Home’

Figure 5.4: University's Recommended Login Procedure for Students

Figure 5.5: Current ‘Pre-Login Moodle Page’

Figure 5.6: Current Moodle Start Page

Figure 5.7: Current Module View ‘Business Essentials’

Figure 5.8: Example: Visual Lines

Figure 5.9: Example: Communicating Expectations

Figure 5.10: Short-term Solution for ‘Business Essentials’

Figure 5.11: Long-term Solution - Start Page

Figure 5.12: Long-term Solution - Module View

Figure 5.13: Example ‘Manage Release Dates’

Figure 5.14: Example Calendar and Upcoming Events

Figure 5.15: Example Forums (Overview)

Figure 5.16: Example Choice

Figure 5.17: Example Quiz

Figure 5.18: Example Workshop

Figure 5.19: Example Workshop ‘Specimen Assessment Form’

Figure 5.20: Example Wiki

Figure 5.21: Example Entry to Chat

Figure 5.22: Example Blocks as setup for in the RGU

Figure 5.23: Example Glossary

List of tables

Table 2.1: Methods of Judging Value in Research

Table 2.2: Overview Primary and Secondary Data Used

Table 2.3: Overview Sampling Regarding the Module ‘Business Essentials’

Table 2.4: Overview Interview Conduction

Table 4.1: Staff Statistics at The Robert Gordon University

Table 4.2: Overview of Relevant Tuition Fees for 2008/2009

Table 4.3: Funding in Business Essentials

Table 5.1: Examples Relevant Information Dissemination

Table 5.2: Responses Login Procedure

Table 5.3: Responses Regarding Frequency of Moodle Access

Table 5.4: Responses Regarding Moodle Access Online Time

Table 5.5: Responses Moodle Questionnaire Regarding Submission Dates

Table 5.6: Responses to Question 6 of Evaluation Questions

1 Introduction

This chapter provides an overview of the research undertaken. Following an outline in which the investigation took place, the overall aim and objectives are stated and the rationale justified. As this research might benefit differ- ent user groups, limitations and values are stated as appropriate. Finally, readers will be introduced to the structure this dissertation follows.

1.1 Research Problem

The Robert Gordon University is based in Aberdeen which is known as ‘The Oil Capital of Europe’. Indeed, the School of Engineering has a very strong link to the local oil and gas industry. The Energy Centre, as a separate initiative of The Robert Gordon University, was launched in January 2005 “to provide for industry a focal point for the development and deployment of all accredited learning material related to the oil and gas, and renewable industries” (The Robert Gordon University 2009).

A major shift from the in-house development ‘Virtual Campus’ was undertaken when rolling out the collaborative virtual learning environment Moodle across the whole University in September 2008. In particular, new ways of collaborative engagements between different users groups were seen. It was believed that new functionality provided and usability issues might impact on students’ learning experience.

Nielsen (2000, p. 9) justifies web usability as follows: “Usability rules the Web. Simply stated, if the customer can’t find a product, then he or she will not buy it.” This conforms to Krug’s first law of usability: “Don’t make me think!” (Krug 2000, p. 11). The parameters ‘knowledge’, ‘learning experi- ence’ or other ‘information services’ were seen as ‘products’ in this context, particularly when considering tuition fees to be paid prior to the relevant course. It was therefore believed, the University needs to operate to the same standards as a normal business to ensure high-quality delivery of its services to maintain its reputation.

Calhoun (2007, p. 176) adapted Choo’s illustration of “a highly collaborative model for knowledge creation and organizational growth” (Figure 1.1). A virtual learning environment (for formal and informal learning) should play an essential role in connecting ‘domain experts, information experts and IT experts’.

illustration not visible in this excerpt

Figure 1.1: Knowledge Pyramid of The University Community

(Source: CALHOUN 2007, p. 176)

Considering the concepts of information literacy, web usability and knowl- edge management as a starting point for sustainability regarding interactive e-learning, an exploration of how e-learning services were provided and embedded into the investigated module ‘Business Essentials (ENM203)’ at The Robert Gordon University was seen as essential. Also of interest was how functionality available was actually used by different user groups.

1.2 Rationale

Information literacy as a key element for independent learning within all disciplines “enables learners to engage critically with content and extend their investigations, become more self directed, and assume greater control over their own learning.” (Bundy 2004, p. 5). This was seen as comparable with the University’s overall strategy, given the following statement: “We expect our students to be active partners in the learning process” (The Robert Gordon University, year not given, p. 11-12).

As the concept of lifelong learning is one of the most important directions of the European Union, it is appropriate to undertake research within this field because “all member states shall submit to the Commission, by 30 June 2010 and 30 June 2015 respectively, reports on the implementation and the impact of the Lifelong Learning Programme.” (Decision No 1720/2006/EC, Article 15 No 4).

Moreover, an evaluation of the recent implemented system ‘Moodle’ and its usage after two academic terms was seen as appropriate.

Finally, professional personal development, supporting existing interest in human-machine-processes within an interactive e-learning context as well as interest in open-source software, were researcher’s personal drives for undertaking this study.

1.3 Aim and Objectives

The overall aim of this study was to draw up recommendations on how the collaborative virtual learning environment ‘Moodle’ could be integrated and embedded more effectively in the studies of postgraduate Oil & Gas related courses at The Robert Gordon University. ‘Studies’ in this context includes formal and informal learning as the focus was on the overall user experience on Moodle. Therefore, the focal point was on postgraduate engineering students and University staff members attached to the module ‘Business Essentials (ENM203)’ at The Robert Gordon University.

Six objectives were set to achieve the stated aim:

1. To examine the latest e-learning developments, general issues and defi- nitions as well as investigating existing studies on web usability, other virtual learning environments and programming activities towards Moodle 2.0;
2. To analyse and synthesize the learning style of sampled full-time and distance learning students attached to the module ‘Business Essentials (ENM203)’ at The Robert Gordon University;
3. To investigate how the system ‘Moodle’ is accessed and used by sam- pled students from the module ‘Business Essentials’ and University staff at The Robert Gordon University;
4. To identify areas of improvement to address students’ and academic in- formation needs for supporting the overall ‘student learning experience’ and ‘teaching experience’;

5. To establish conditions under which the use of the collaborative virtual learning environment could impact positively on students’ learning experience;

6. To draw up short-term and long-term recommendations for future de- velopment.

1.4 Scope

Main focus of this study was the module ‘Business Essentials (ENM203)’ which aims “to develop an understanding of safety and environmental proc- esses and legislations relevant to the Oil and Gas Industry” and “to develop behavioural skills and an initial understanding of team working and project engineering processes.” (The Robert Gordon University 2008) This module is embedded in most of the Oil & Gas related postgraduate courses offered by the Energy Centre such as MSc in Oil and Gas Engineering, MSc in Petro- leum Production Engineering and MSc in Drilling & Well Engineering which have full accreditation by the professional body, namely the Energy Insti- tute (The Robert Gordon University 2009a). Access to relevant e-learning services and information for the full-time and distance learning delivery mode were provided for this research.

1.5 Limitations and Implications

As mentioned previously, this study mainly investigated specific settings, processes and participants from The Robert Gordon University regarding one specific module, which was part of the University’s Masters Energy Pro- gramme. Data collected showed that a high number of students have a non- European educational background. Therefore, the findings of this study might not be transferable to students from a British educational background. Although invited to this study, only a small number of distance learning stu- dents took part in this research (section 2.2,Table 2.3). Consequently, key findings relating to student preferences were mainly developed from data provided by full-time students. Finally, access to programming code or spe- cific settings was not requested for this study. Proposed recommendations might be limited by current configuration settings.

1.6 Originality and Value

As this study was undertaken using a single intrinsic case study approach, key findings and recommendations were tailored to the Energy Centre’s needs in the first instance. Where applicable and appropriate, however, The Robert Gordon University as a learning organisation could adapt these for other schools and departments.

Navigation recommendations were partly based on the latest developments towards Moodle 2.0, which were discussed within the Moodle community. For that reason, this study may be of interest to core Moodle developers. Indicated challenges and opportunities might be of interest to other administrators and users of the Moodle community.

Finally, this study might be of interest to other researchers or practitioners undertaking an investigation into web usability or the use of virtual learning environments.

1.7 Structure of this Dissertation

Following this introduction, the chosen research design, explanations and justifications of selected approaches and data collection techniques are pro- vided in the ‘Methodology’ (chapter 2). This study’s research context is qualified by providing an overview of current research activities and devel- opments in the ‘Literature Review’ (chapter 3). Insights of the University’s current infrastructure, relevant student demographics and Moodle imple- mentation are analysed, evaluated and presented in the separate chapter ‘Setting the Scene’ (chapter 4). Key findings drawn from selected data collection techniques are further detailed and evaluated in main analysis and discussion chapter (5 Using Moodle as a central platform for…?). Based on these findings, areas of improvements are pointed out and recommenda- tions made as appropriate. Also part of this analysis is an evaluation of the chosen research design, followed by suggestions for future research activities (section 5.4). Following the in-depth analysis, key findings are summarised and ‘Conclusions’ linked back to the original aim and objectives formulated in section 1.3 (Conclusions, chapter 6). For future sustainability a ‘Summary of Recommendations’ is provided at the end of this study sug- gesting overall objectives to be set by The Robert Gordon University (chap- ter 7).

2 Methodology

The main focus of this chapter is to provide an insight of the research ap- proach taken for this study. Therefore, explanations and justifications for the selected methods and methodology are provided. In order to achieve triangulation different data collection techniques were carried out. The evaluation of the chosen research design, however, is detailed in section 5.4 as part of the analysis.

2.1 Research Design

Research design is formed by ‘research methodology’ which is the general philosophy that guides the research and ‘research methods’ which are tools used for gathering data (Dawson 2007, p. 24). Structuring the research for delivering evidence to answer the research problem was the purpose of conducting the research design (McGivern 2006, p. 79).

Depending on the aims and objectives qualitative, quantitative or a combination of both approaches can be chosen. Pickard (2007, p. 19) adapted an overview of criteria for judging research from Lincoln and Guba:

illustration not visible in this excerpt

Table 2.1: Methods of Judging Value in Research

(Source: PICKARD, A., 2007, p. 19)

Table 2.1 shows the main focus of different methodological approaches that apply to practitioner research as well as to student research. This framework provides criteria as to how a researcher can demonstrate the value of an investigation to ensure credibility.

Within this qualitative research, different quantitative and qualitative data collection techniques (section 2.2) were applied to achieve triangulation (Pickard 2007, p. 168). However, quantitative data was used for qualitative judgements only. Therefore, a decision against statistical analysis was made. Table 2.2 demonstrates the collection of primary and secondary data used for this study. Data analysis was performed as the research progressed to continually refine and reorganise data in the light of emerging results (Daw- son 2007, p. 115).

illustration not visible in this excerpt

Table 2.2: Overview Primary and Secondary Data Used

2.1.1 Single Intrinsic Case Study Approach

The content of this study required an “in-depth investigation of one or more examples of a current social phenomenon, utilizing a variety of sources of data” rather than generalisation (Jupp 2006, p. 20). The qualitative approach conforms to Wilson’s suggestion of “uncovering the determining factors of behaviour” (Wilson 2006, p. 667).

Therefore, the main purposes were understanding, reconstruction and the offer to transfer findings. Interpretivist research was carried out by using a single intrinsic case study investigation. According to Pickard this type of case study is carried out “for no other purpose than to give us a better un- derstanding of the case; the case is studied as much for its ordinariness as for any peculiarities.” (Pickard 2007, p. 86). The in-depth approach was used to observe characteristics of an individual unit, in this case the Energy Centre of The Robert Gordon University (Blaxter, Hughes and Tight 2006, p. 71)

Pickard summarises that “Using case studies is the most appropriate re- search method when the purpose of the research requires holistic, in-depth investigation of a phenomenon or a situation from the perspective of all stakeholders involved.” (Pickard 2007, p. 93). Students and University staff members were the primary stakeholders investigated. Applying Pickard’s research hierarchy to this study, a graphical display can be given as follows:

illustration not visible in this excerpt

Figure 2.1: Applied Research Hierarchy

(PICKARD, A., 2007. p. XV, applied according to the research of this study)

2.1.2 Sampling and Ethical Considerations

“Sampling is about selecting, without bias and with as much precision as resources allow, the ‘items’ or elements from which or from whom we wish to collect data.” (McGivern, p. 309).

Walliman (2006, p.75) explains the process of sampling as the selection of a small group of people from a large population. Generally, two types of sam- pling can be used for selecting participants - ‘probability sampling’ or ‘pur- posive sampling’. The first approach aims to provide an equal chance for all people within the research population being selected (Dawson 2007, p. 55).

Pickard (2007, p. 88) points out, “qualitative case study research always uses purposive sampling to identify information-rich sources within the case.” For that reason, participants, key informants and gate keepers were identified in preparation of different data collection techniques as well as for ethical considerations. Table 2.3 details the sampling overview used in this study (main scope: Module ‘Business Essentials’).

Permission for conducting this study prior to all data collection techniques was given by identified gate keepers as follows (Appendix 1):

- Head of Energy Centre;
- Head of School (School of Engineering);
- Discipline Leader for Well Construction and Module Developer for ‘Business Essentials’;
- Open and Distance Learning Team Leader.

illustration not visible in this excerpt^{1 2 2}

Table 2.3: Overview Sampling Regarding the Module ‘Business Essentials’

Participants were made aware of this study and/or were asked to sign a consent form as indicated in Table 2.3 prior to the relevant data collection technique. All participants made minimal use of limiting their permission level (Appendix 9 to Appendix 17), although the public use of this study was explained. Depending on the type of data, particularly from the usability testing, stricter limitation was seen as appropriate to protect participant’s identity. Consequently, the research was undertaken in compliance with current British legislation, namely the eight data protection principles (Data Protection Act 1998).

It should be underlined that all potential participants attached to the module ‘Business Essentials’ received a formal invitation and were able to familiarise themselves further using online information provided (Appendix 6). Furthermore, additional face-to-face explanations were given as required and where appropriate. The strong involvement of University staff members was a key success factor for this study as quick decisions could be made as appropriate (e.g. embedding data collection into tutorials).

2.2 Data Collection

Jupp (2006, p. 57) defines data as “observations about the social world” which are collected with the overall aim to analyse them after their collection. These ‘observations in the widest sense’ include quantitative as well as qualitative data (Jupp 2006, p. 58). After considering the research problem and focus (section 1.1), the most appropriate data collection techniques that ‘fit for purpose’ were selected (Pickard 2007, p. 167).

2.2.1 Usability Testing and Observation

According to Pickard (2007, p. 227) “the purpose of user testing is to gather data on how users interact with a system and how well the system responds to user behaviour.” In contrast, ‘observations’ are designed to investigate “how people behave and interact in particular situations.”

(Pickard 2007, p. 201). By deciding on usability testing for evaluating an online environment, the ‘Observer effect’ of participants’ altering behaviour was taken in account (McGivern 2006, p. 180). To overcome this disadvan- tage of observation, unstructured interview questions were used in conjunc- tion with, investigating participants’ reasons for behaviour (McGivern 2006, p. 268). However, this study was primarily undertaken to evaluate the system rather than the user (Pickard 2007, p. 230).

Dumas and Redish (1999, p. 22) establish five typical characteristics for usability testing, which can be summarised as follows:

- The improvement of a product is the primary goal.
- Real users are chosen as participants.
- Real tasks are given and performed.
- Participant observation and recording is carried out.
- Following a data analysis, real problems are identified and areas of improvement can be recommended.

Piloting was carried out using a postgraduate student from The Robert Gordon University (a first time user of Moodle) after the researcher’s walkthrough. As a result significant changes to the original setup were made and the wording for usability task 6 was changed. Students and University staff were asked to perform the same tasks.

However, the wording from the following student tasks differed slightly for staff members (Appendix 7):

1. Open a web browser and log into Moodle.
2. Go to Module “Business Essentials (ENM 203)”.
3. Go to a Forum of your choice.
4. Go to the topic you experienced the most difficulties with (so far) and confirm via voice or circle / mark the topic using the mouse.
5. Go to a place where you would expect to find deadlines for submissions (“due dates”) and confirm via voice or circle / mark the topic using the mouse.
6. Please search for (additional) information about the topic “Team Work & Behavioural Skills” using any online environment. Please open a new web browser, if required.
7. Logout from Moodle and close your web browser.

Evaluation questions (Appendix 8) were adapted using the ‘Post Test Interview’ questions provided from the Web Accessibility Initiative (W3C Web Accessibility Initiative 2003). Both, usability tasks and evaluation questions were uploaded and visible to full-time students and distance learners at all times (see Appendix 6).

According to Nielsen (2000a) “the cost-benefit analysis of user testing provides the optimal ratio around three or five users, depending on the style of testing.” Therefore, five participants were recruited - three full-time students and two University staff members - in order to discover major performance differences (Table 2.3).

Prior to each experiment, additional information regarding usability testing, the tasks and evaluation questions were given as appropriate. Participants were asked to sign a consent form (Appendix 2). Details regarding the spe- cific setup can be found in each Performance Transcript (Appendix 9 to Ap- pendix 12). The software TechSmith Morae (Version 3) was used for re- cording and analysing participants’ data as well as the production of a high- light video (Appendix 13).

It is important to point out that a specific laboratory situation was installed in order to discover the participants’ navigation techniques when approaching or accessing Moodle in the first instance. Therefore, the browser settings were changed as follows:

- Opening the browser offers a “blank page”;
- No favourites were stored;
- No browsing history was stored.

As expected, all participants responded differently to the experimental setup which uncovered their general online behaviour and habits.

Key findings, pointing out navigation and consistency issues, are summa- rised in section 5.1.3 and taken in account for the design of the recom- mended layouts (section 5.1.5, Figure 5.10, Figure 5.11, Figure 5.12).

2.2.2 Interviews

Interviews are one way to “access what was in, and on, the interviewee’s mind.” (Pickard 2007, p. 172, quoting Stenhouse). Jupp (2006, p. 157) defines interviews as “a method of data collection, information or opinion gathering that specifically involves asking a series of questions.” According to McGivern (2006, p. 315), consideration was given to the fact that attitudes and behaviour may be affected by a variety of intervening factors (e.g. physical and social environment).

The researcher identified four key informants at The Robert Gordon Univer- sity with whom semi-structured face-to-face interviews were conducted (Table 2.4):

illustration not visible in this excerpt

Table 2.4: Overview Interview Conduction

Prior to each interview, an email including the approved research proposal and interview consent form was sent out to the key informants (Appendix 3). All interviews were held and digitally recorded within the relevant de- partment at The Robert Gordon University. A professional recording and transcription set was used for creating interview transcripts. Including markings for potential quotes, these transcripts were used as part of mem- ber checking which Pickard summarises as follows: “Those involved are given the opportunity to read their case reports and encouraged to com- ment on the contents, adding their own interpretations.” (Pickard 2007, p. 91). In some cases participants provided additional explanations on how statements should be understood. The discussion and evaluation of interviews by using quotes is carried out in various sections in chapter 4 and 5. The amount of information gathered provided a general insight into The Robert Gordon’s internal processes and responsibilities which were considered when recommendations were formulated.

2.2.3 Questionnaires

Jupp (2000, p. 252) defines a questionnaire as “a set of carefully designed questions given in exactly the same form to a group of people in order to collect data about some topic(s) in which the researcher is interested.”

Two online questionnaires were embedded within this study to reach a large number of students that were geographically dispersed. Both questionnaires were designed to obtain qualitative data regarding participants’ learning style, preferences and online behaviour. After piloting, access was given via the general online study setup (Appendix 6). In addition, permission was granted to embed this data collection technique into the tutorials of full- time students to maximise participation. Responses were analysed elec- tronically.

Index of Learning Styles Questionnaire:

Soloman and Felder (year not given) developed the ‘Index of Learning Styles Questionnaire’⁴ which was embedded into this study. The voluntary questionnaire was a discussion starting point for the topic ‘Team Work & Behavioural Skills’ in the investigated module ‘Business Essentials’. Accord- ing to feedback from the academic staff member who embedded this exer- cise into the tutorials, it was necessary to provide an explanation of dis- played results as students experienced difficulties in its interpretation, al- though interpretation explanations were provided by Soloman and Felder at the end of the questionnaire. Key findings are stated in section 4.2 and em- bedded in recommendations made as the analysis progresses in chapter 5. Results are detailed in Appendix 18.

Moodle Questionnaire:

A questionnaire which allowed students to evaluate the system “Moodle” itself and its embedment within the studies was developed. After careful evaluation, the online survey provider “SurveyGizmo” was used to create the questionnaire because of its high level of web accessibility and security safeguards on its server location. Also, many different types of questions could be integrated and analysed (OSU Web Accessibility Center 2008).

Students were asked to answer 20 short mandatory questions covering the areas ‘Accessing Moodle’ (Questions 1 to 3), ‘Moodle Features’ (Questions 4 to 11), ‘Behaviours and Views’ (Questions 12 to 15) and ‘Demographics’ (Questions 16 to 20). Additionally, a comment box was provided at the end which was completed with valuable, qualitative feedback by 25 out of 53 respondents. Key findings of this 10-to-15 minute questionnaire, particular regarding the learning and teaching style, are embedded throughout the discussion and recommendations made in the main analysis and discussion chapter (chapter 5). Used questionnaire design and analysis performed are detailed in Appendix 19.

2.2.4 Freedom of Information Requests

“A person who requests information from a Scottish public authority which holds it is entitled to be given it by the authority.” (Freedom of Information (Scotland) Act 2002). The law came into force on January 2005. Therefore, the data collection technique ‘Freedom of Information Request’ might not be established in research yet. However, it was identified as the most appro- priate one for receiving anonymous and objective data for qualitative judgements. Referring to section 38 of the Freedom of Information (Scot- land) Act (section ‘Personal Information’), no conflict with the Data Protec- tion Act 1998 was recognised regarding this study. This was particularly im- portant for providing information about the general student population in the investigated module since its creation (section 4.2).

The same reasoning was applied when requesting documentation about the piloting of Moodle, as the involvement of students and different University departments was expected. By choosing this data collection technique, the balance between detailed information and data protection was guaranteed as all relevant files provided could be reviewed by the University’s Records Manager prior to its distribution within 20 working days.

After following online guidance of the ‘Scottish Information Commissioner’ (year not given), the University’s Records Manager was sent the approved research proposal and asked if any data protection concerns might be raised prior to both requests.

Freedom of Information Request I:

The overall purpose of the first Freedom of Information Request was to find out how the ‘Piloting’ was conducted prior to Moodle’s implementation (Appendix 20). As only five (short) electronic files were received, a followup request was undertaken (Appendix 21). Key findings are discussed and evaluated in section 4.3.

Freedom of Information Request II:

The overall purpose of the second Freedom of Information Request (Appendix 22) was to retrieve demographic data from which a broad picture of the overall student population within the investigated module since its creation in 2006 could be developed in section 4.2. Moreover, the collected parameters ‘Gender’, ‘Age’, ‘Country of Origin’, ‘Funding’, ‘Final Grade’ and ‘Re-sits’ were seen as valuable insight into the stakeholder group ‘Students’ when discussing and evaluating other findings throughout chapter 5.

Clarification of unknown parameters entered in researcher’s prepared Excel Spreadsheet was required (Appendix 23). The response to this follow-up request provided guidance on how data and abbreviations used should be understood (Appendix 24).

2.2.5 Additional Sources for Background Knowledge

Additional data was collected using Moodle’s build-in functionality (e.g. report options as displayed in Figure 2.2). Where appropriate, administrative staff were asked to provide additional background information regarding specific procedures (e.g. student induction or timetables).

illustration not visible in this excerpt

Figure 2.2: Screenshot: Example Report Screen in Business Essentials

For gaining relevant background knowledge the researcher created a user account at the official Moodle Community⁵ which allowed following and con- tributing to discussions in various forums. Moreover, the researcher at- tended ‘MoodleMoot UK 09’: The fifth Annual UK conference for users, ad- ministrators and developers of Moodle at Loughborough University. This conference mainly addressed educational needs. In contrast, attending a knowledge management conference in Germany helped the researcher to understand how Moodle can be used within a corporate environment.

Key findings related to this data collection is embedded are embedded in chapter 5, dealing mainly with the evaluation of Moodle and its future de- velopment.

2.3 Chapter Summary

This chapter provided a detailed insight into the theoretical framework of this study. Selected research design was examined and justified. Within the qualitative research approach, the decision for a single intrinsic case study approach was made. Different data collection techniques were introduced by explaining their purpose and use within this study context: namely usability testing and observation, interviews, questionnaires and Freedom of Infor- mation Requests as primary research and literature review and background knowledge gained from relevant conferences as secondary research. Data collection was presented by referencing to the relevant section where data was used and this raw data is detailed in the appendix. Overall, it was pointed out that quantitative data collected was used for qualitative judge- ments only. Reference to an evaluation of the selected research design was made (section 5.4).

3 Literature Review

McMenemy (2007, p. 100) reviewed Ranganathan’s ‘Five Laws of Library Science’ with respect to its relevance in the 21st century. These five state- ments provide a framework of how an information service should work. At present, the most important one might be the Fourth law which states: ‘Save the time of the reader’. McMenemy concludes “Ranganathan’s five laws continue to give us a blueprint for our professional values that is as relevant as it was in 1931.” This is seen as in line with usability expert Krug’s professional experience that users do not read web pages, but only scan them. Reasons indicated were: the overall time factor, the fact that there is no “need to read everything” and users are used to other media such as newspapers or books (Krug 2000, p. 23).

Therefore, the literature review was undertaken for two purposes. On the one hand the purpose was to provide an overview of latest e-learning developments, general issues and definitions as well as investigating existing studies on web usability, other virtual learning environments and programming activities towards Moodle 2.0. On the other hand, knowledge gained by secondary research was directly applied when evaluating the current layout in Moodle and creating a re-design.

3.1 Definitions

The Joint Information Systems Committee (JISC) defines e-Learning as “learning facilitated and supported through the use of information and communications technology. It can cover a spectrum of activities from the use of technology to support learning as part of a ‘blended’ approach (a combination of traditional and e-learning approaches), to learning that is delivered entirely online. Whatever the technology, however, learning is the vital element.” (JISC 2007).

Regarding the ‘online supporting system’ within the literature many different terms are used to describe a similar phenomenon, depending on the purpose of the system itself. The researcher listed a few terms only:

- ‘Virtual learning environment (VLE)’ (Catherall 2005)
- ‘Collaborative virtual learning environment (CVLE)’ (Akar et. al 2004)
- ‘Networked collaborative e-learning’ (McConnell 2006)
- ‘E-learning systems’ (Levy 2006)
- ‘Content management system (CMS)’ (Cole 2005)
- ‘Learning management system (LMS)’ (Rice IV 2006)
- ‘Course management system (CMS)’ (Rosato et. al 2007)
- ‘Managed learning environments (MLE)’ (Joint Information Systems Committee 2007a).

Even within the Moodle Community no specific term seemed to be agreed as the following statement was made at the Community’s homepage:

“Moodle is a Course Management System (CMS), also known as a Learning Management System (LMS) or a Virtual Learning Environment (VLE). It is a Free web application that educators can use to create effective online learning sites.” (Moodle.org, year not given).

3.1 Definitions

Within this study the term ‘collaborative virtual learning environment’ is used, based on the following reasons: Firstly, it is argued that Moodle features available can enhance collaborative learning. Secondly, the researcher agreed with the following statement: “All of us are potential teachers as well as learners - in a true collaborative environment we are both.”(MoodleDocs 2009) Thirdly, the University’s ‘Department for the Enhancement of Learning, Teaching and Assessment’ (DELTA) sees the main purpose of Moodle as providing a collaborative virtual learning environment for students (The Robert Gordon University 2009b).

The term ‘Information Literacy’ was also seen as important when investigat- ing virtual learning related behaviours. Andretta (2005, p. 32) quotes the UK’s professional body CILIP (Chartered Institute of Library and Information Professionals) “Information literacy is never defined properly.” Reffell (2003, p. 125) explains very precisely that IT skills are not enough; in particular he refers to the European Computing Driving Licence (ECDL). “For students in higher education we also felt that the uncritical approach to IT presented by ECDL was inappropriate in terms of developing a broader understanding of IT and information issues.” The element of ‘critical evaluation of information’ seems to be agreed by researchers across different countries (e.g. Andretta 2005; Bundy 2004; Lee, Theng and Goh 2005; Ellis 1997). With its many facets, information literacy is also known by different names such as library orientation, bibliographic instruction, user education and information skills training (IL Information Literacy 2008).

CILIP formed the ‘CSG Information Literacy Group’ and established a defini- tion for ‘Information Literacy’ as follows (CILIP 2007): “Information literacy is knowing when and why you need information, where to find it, and how to evaluate, use and communicate it in an ethical manner. This definition implies several skills. We believe that the skills (or competencies) that are required to be information literate require an understanding of:

- a need for information
- the resources available
- how to find information
- the need to evaluate results
- how to work with or exploit results
- ethics and responsibility of use
- how to communicate or share your findings
- how to manage your findings.”

‘Information Architecture’ is defined as “the art and science of organizing and labeling websites, intranets, online communities and software to sup- port usability” (The Information Architecture Institute 2007). Rosenfeld and Morville (2002, p. 4) also specify that the facilitation of task completion and intuitive access to content is involved which underlines the previous focus on usability.

Dumas and Redish (1999, p. 4) explain that “usability means that the peo-ple who use the product can do so quickly and easy to accomplish their own tasks.” Two international standards are relevant for defining usability and human-centred design (Usability.gov, year not given):

1. “[Usability refers to] the extent to which a product can be used by speci- fied users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of user. - ISO 9241-11”

2. “Human-centered design is characterized by: the active involvement of users and a clear understanding of user and task requirements; an appropriate allocation of function between users and technology; the iteration of design solutions; multi-disciplinary design. - ISO 13407”

3.2 Moodle

The term ‘Moodle’ has in fact two meanings: Firstly, it stands for ‘Modular Object Orientated Developmental Learning Environment’, and secondly as a verb “to let the mind or body wander and do something creative but without particular purpose” (Cole 2005, p. xiii) or to describe “the process of lazily meandering through something, doing things as it occurs to you to do them, an enjoyable tinkering that often leads to insight and creativity.” (Rice 2006, p. 5). This type of programming (modular object orientated) allows to pre- sent information in blocks, referred as ‘Moodle blocks’ in this dissertation.

Created by Martin Dougiamas, the open source software under GNU Public License has become very popular over the past few years. In fact, the community sites offer statistics which provide an insight of 55,379 regis- tered sites in 210 countries worldwide, whereby 4011 sites were registered in the United Kingdom, including The Open University, for instance (Figure 3.1). Additionally, 10042 sites requested privacy and were not listed publically. (Moodle.org 2009 and 2009a, Stand: 16 May 2009).

This popularity was also recognised by the Joint Information Systems Com- mittee (JISC 2009) which is “funded by the UK HE and FE funding bodies to provide world-class leadership in the innovative use of ICT to support edu- cation and research”. For instance, as the most recent projects at JISC the projects ‘Moodle Repository Create, Upload, Tag and Embed (MR-CUTE)’, ‘MUVEs⁶, Moodle and Microblogging⁷ (M3)’ and ‘Copyright Licensing Applica- tions using SWORD for Moodle’ can be indentified (JISC 2009a, 2009b, 2009c).

illustration not visible in this excerpt

Figure 3.1: Screenshot: Moodle Statistics⁸ (16 May 2009)

At the very end of this study, the IMS Global Learning Consortium (2009) awarded Moodle with two ‘Global Learning Impact Awards’ as follows:

1. “Best Learning Platform to Repository Integration: MrCute - Moodle Repository System - UK,

2. Best Interoperability Innovation: Moodle Simple Learning Tools for Interoperability Consumer - Spain.”

‘SLOODLE’⁹ (year not given), as another open source project, that demonstrates an integration of the multi-user virtual learning environment ‘Second Life’ into Moodle. Particularly, the second project mentioned from JISC already explores future possibilities.

3.3 Moodle Developments towards Moodle 2.0

The full roadmap of current developments towards Moodle 2.0 is detailed in the MoodleDocs.¹⁰ Generally, it is suggested that the development will sup- port a knowledge sharing community via one single access point. Based on this study’s analysis, some specific developments which might be of interest to The Robert Gordon University are highlighted here (MoodleDocs 2009a):

1. Use of Repositories;
2. Use of ePortfolios;
3. Navigation improvements combined with a general usability review;
4. Conditional Activities;
5. Improvement / Creation of activities (Wiki, Quiz, Blog);
6. Secure RSS Feeds;
7. Community hubs;
8. Turnitin Integration (stated as “hopefully”).

At the MoodleMootUK 2009, the question when Moodle 2.0 will be released, was answered by core developers. It is hoped to release this version by the end of 2009. However, developers pointed out that there needs to be an upgrade to Moodle 1.9 first as the direct upgrade from older Moodle version will not be supported. Developers explained that this was a ‘lesson learned’ from previous releases where major changes had been made and upgrading caused unexpected errors.

Julian Ridden, known in the Moodle Community as the ‘Moodleman’, launched an alpha version of Moodle 2.0 online which allows users to explore and feedback prior its release¹¹. It should also be noted that various Moodle tutorials on how to use Moodle features, are channelled using YouTube in order to make them publically available.

3.4 Past Research

Although a wide range of literature regarding e-Learning itself can be consulted, very little research has been done regarding (collaborative) virtual learning environments. It was found that the main focus in current research relied on quantitative approaches when investigating information literacy. However, findings from these studies supported this study.

Major research within the UK was undertaken in the JUBILEE project (JISC User Behaviour in Information seeking: Longitudinal Evaluation of Electronic information services) or the JUSTEIS project (JISC User Surveys: Trends in Electronic Information Services). One of the main conclusions from these projects is that “the main trend (apart from the rise in popularity of Google) appears to be the way that academic staff support for use of electronic information may increasingly be formalised through the VLE” and that “interactive tutorial material seems more important than simple provision of a range of resources.” (Urquhart 2003, p. 179).

In 2002, a ‘Study of managed learning environments’ was undertaken by JISC in association with UCISA (Universities and Colleges Information Sys- tems Association). It was found that “significant levels of MLE development activity across all institutions” had taken place (JISC 2007b). According to this study the virtual learning environment ‘Blackboard’ was the most com- mon one (with 43 %), followed by ‘WebCT’ (34 %) within higher education compared the most common virtual learning environment within the United Kingdom such as WebCT, Blackboard, FirstClass, Learnwise, Bodington (non-commercial), Teknical Virtual Campus and Moodle (open-source) to provide a neutral overview rather than stating a preference.

User studies of the system, particularly with the main focus on the student, can rarely be found. The SOLE project (Students’ Online Learning Experience) used interviews and log diaries as main data collection techniques to investigate the student perspective regarding the use of a virtual learning environment (Institute for Learning and Research Technology 2006). The Centre for Research in Library and Information Management (CERLIM) also undertook usability studies and employed screen capture software when dealing with disabled students. (Craven and Booth 2006).

Regarding the system Moodle itself, the Consortium for School Networking (CoSN) investigated the implementation of Moodle within school in the United States of America. It was found that “implementation models range from formal development programs to a grass roots “build as you teach” approach”. (Consortium for School Networking 2008, p. 6). Another Ameri- can study was undertaken by Rosato, Dodds and Laughlin (2007). The re- searchers conducted usability testing on the virtual learning environments WebCT, Sakai and Moodle from the perspective of a student using the sys- tem for the first time. Satisfaction was also measured and the researchers stated in their analysis: “Based on the data, it appears that users either really liked Moodle or hated it.” (Rosato, Dodds and Laughlin 2007, p. 10).

3.5 Online Communication Tools

Technology and the availability of data and information nowadays have changed daily processes in life on a private, public and economic level. Ar- genti (2006, p. 357), referring to an American study carried out in 2005, compares figures from 1996 where “15 % of Americans had Internet ac- cess” whereas 10 years later “more than 60 % have Internet access”.

[...]

---

¹ DELTA: Department for the Enhancement of Learning, Teaching and Assessment

² Distance Learning Students may also referred to ODL students (Open and Distance Learning)

³ FT/PT: Full-time / Part-time

⁴ See online: http://www.engr.ncsu.edu/learningstyles/ilsweb.html

⁵ Moodle Community: www.moodle.org

⁶ MUVE = MultiUser Virtual Environment, in this case Second Life

⁷ In this case: Twitter

⁸ Updates available on: http://moodle.org/stats/

⁹ SLOODE: Simulation Linked Object Oriented Dynamic Learning Environment

¹⁰ Updates on: http://docs.moodle.org/en/Roadmap

¹¹ Alpha version of Moodle 2.0 available: http://playpen.moodle.com.au/moodle20/