Excerpt
List of contents
Abstract
Chapter 1 Introduction
Rationale for the study
Chapter 2 Theory
2.1 Review of the literature.
The scholarship of learning.
The scholarship of teaching and learning
Formative student evaluations.
Summative student evaluations
Analysing feedback
2.2 Theories and questions guiding thesis
Nine steps of instruction
Learning outcomes
Instructional materials
A rational for nonstandardized instruments in course assessment
Conducting course assessment with informally developed instruments
Hypotheses
Chapter 3 Methods
Participants
Materials
Statistics procedures and design
Chapter 4 Results
Factor analysis
Comparison of the courses
Regression
Chapter 5 Discussion and conclusion
Implication for theory
Implication for practice.
Limitations
Future directions
References
Appendices
Appendix A
Abstract
In the present study a newly developed feedback form for college course evaluation (Feedback Form for the Excellence Courses: FFEC) is tested on a sample of participants of the courses of Psychology of Excellence in Business and Education (N = 200). FFEC addressed both issues of learning outcomes of a course and instructional qualities of teachers consisted of 5 dimensions: animating knowledge, promoting self-regulated learning, stimulating intellectual atmosphere, structuring the class, and establishing communication with students. Aside from its prospective practical purposes, FFEC showed very good psychometric properties, with regard to the internal consistency of subscales and overall factorial validity. As to concurrent construct validity, multiple regression analysis showed that subscales of FFEC accounted for 46% of variance in the overall course rating. Finally, the suggestions for the future use of FFEC are discussed.
Keywords: Course evaluation, instructor quality, learning outcomes, instructional materials, teaching effectiveness.
Chapter 1
Introduction
Rationale for the Study
Colleges and universities have become increasingly concerned with evaluating the quality of their programs and courses, and becoming more responsive to the needs of the academic community. One such method of evaluation involves soliciting the feedback of its constituents, the students, in rating the quality of the instruction in the classroom. These rating instruments take a variety of forms. Marsh and Roach (1993), however, classified them into the four major groups, depending on their main task: (a) providing instructors with feedback on their impact on the classroom; (b) providing students with the opinions of their peers, for help in deciding whether or not to take a particular class with a particular instructor; and (c) providing administrators with ratings of instructors to be used in decisions of tenure and promotion, and to a more limited extent, as validation of teaching quality in presentations to outside accrediting agencies. A fourth task that is growing in importance is the use of ratings as an overall outcome description for on-going research on the teaching process (Marsh, 1984; Marsh & Roche, 1993).
Researchers have devoted considerable time to examining the use of the various instruments, discussing the final product, and analyzing the different correlations among measures. With considerable consistency, the research has teased out the elements of instructors’ behaviour that students judge to be most important (Marsh & Roche, 1993). With varying labels and weights depending upon the individual studies, these include such factors as learning value, instructor enthusiasm, organization and clarity of explanation, individual rapport, group interaction, ability to stimulate student interest and learning, breadth of coverage of material, and overall fairness in examinations, grading, assignments, and workload difficulty (Marsh & Roche, 1993).
The search for quality in higher education teaching seems to be now a regular activity in every university, which tends to achieve even more significant position in the future. Student ratings are widespread and a common tool for evaluating faculty. When asked, most faculty members approve of the use of student ratings of instruction for teaching improvement.
In the present study at Ludwig-Maximilians-Universität München (LMU), the quality of Psychology of Excellence in Business and Education courses in Winter Semester 2005/2006 is being analysed. The purpose of this master thesis will be both to check on students’ perceptions of teaching qualities and to develope a valid and reliable feedback-form. The long-run intention is to improve the overall quality of the program by indicating what students understand by teaching effectiveness. This would consequently provide the teachers with helpful information on the course design.
There is plenty of controversy over what exactly constitutes "effective teaching", but at the same time, there is also a considerable body of teaching research and lore, universally accepted and useful. The goal in the following section is to pass on the results of that research and some of that lore — about how to organize and deliver a lecture, ask stimulating questions, create provoking atmosphere, lead discussions, deal with classroom management problems, write exams, and the like. In the beginning, some general principles of effective teaching are considered along with research evidence that supports it.
The first seven of these principles come from a 1987 article entitled, appropriately enough, "Seven Principles for Good Practice in Undergraduate Education" (Chickering & Gamson, 1987; Chickering & Gamson, 1991). Although focused on undergraduate teaching, these principles can be applied equally well to teaching psychology at any level.
1. Encourage Student-Faculty Contact
Frequent and fruitful contact between students and faculty inside and outside the classroom is important for establishing and maintaining students' motivation and involvement in the learning process. When students know that their teacher cares about their progress, they are more likely to persist at learning tasks, even when the tasks are difficult. Further, the experience of getting to know at least a few teachers as individuals can enhance students' intellectual and emotional commitment to learning.
2. Encourage Cooperation Among Students
Learning is enhanced when it occurs as part of a team effort. Like effective work, effective learning can often come through social collaboration rather than isolated effort in a competitive atmosphere. Collaborative learning also offers the advantage of helping students, develop social responsibility while interacting with others whose ideas and thoughts expand their own thinking and deepen their own understanding.
3. Encourage Active Learning
"Learning is not a spectator sport" (Chickering & Gamson, 1991, p.66). Students learn, retain, and understand more course material when they actively engage it—by talking about it, writing about it, questioning it, debating it, applying it, and relating it to what they already know-rather than sitting passively as information washes over them in lectures, videos, or other pre-packaged formats.
4. Give Prompt Feedback
To focus their efforts to learn, students need feedback about what they know and what they don't know. This feedback should be frequent enough to guide students' efforts; it can take the form of pre-tests on course material as well as all sorts of quizzes, examinations, papers, projects, and other assignments that help students reflect on how far they have come in accomplishing their learning goals and how far they still have to go.
5. Emphasize "Time on Task"
Laboratory studies indicate that some forms of learning can take place without conscious attention, but in psychology courses there is no substitute for paying attention and devoting time to the learning task. Teachers can play a vital role in promoting adequate "time on task" by focusing most of every class session on the material to be learned, not on trivial or irrelevant discussions. They can also help students learn to use time wisely, both in terms of efficient and effective study skills and in relation to time management in general. Whether this assistance comes through personal advice or referral to campus counselling facilities, it can offer a vital aid to learning for those who need it most.
6. Communicate High Expectations
Expect a lot from your students and you are likely to get it. A teacher's high expectations can maximize the performance of all students-from those who are bright and motivated to those who are less well prepared or initially less eager to exert themselves. When students experience their teacher's high expectations as realistic, if challenging, their expectations of themselves tend to rise accordingly (see Treisman, 1985).
7. Respect Diverse Talents and Ways of Learning
Like other people, students differ among themselves in many ways, including what they know, what talents they have, and even their favourite ways to learn. A student who shines during class discussion might be all thumbs in a lab session. Students who are used to dealing on their own with practical matters in the workplace might be less adept at, or patient with, assignments that focus on abstract theories or group problem solving. If possible, give students the opportunity to show their talents and learn in ways that work for them, but don't hesitate to challenge them to learn in new ways that do not come so easily.
The next three principles of effective teaching come from the Center for Teaching Excellence, (formerly the Division of Instructional Development) at the University of Illinois, Urbana-Champaign (Center for Teaching Excellence, 1999).
1. Be Organized and Prepared
The organization and planning that goes into your psychology course help determine what students will learn and how easily they will learn it. A well-planned and well-organized course also conveys to students that you care about teaching and about them. The perception of a caring teacher, in turn, is associated with higher student ratings of teacher performance.
2. Communicate Enthusiasm
Effective and highly rated teachers also tend to communicate enthusiasm, even love, for psychology, and for the teaching enterprise. There are many ways to get this message across. Lively lectures, fascinating demonstrations, and other dramatic classroom activities certainly convey enthusiasm for the teaching enterprise, but so do highquality, carefully chosen presentations and assignments, challenging exams, well-conceived grading systems, and the like. When students sense their teacher's passion for the course is authentic, the effect can be contagious.
3. Be Fair and Ethical
Fairness and the highest ethical standards—in presenting material, in dealing with students, and in evaluating them—is fundamental to effective, high quality teaching. A related goal is to assure that students deal fairly with one another in the classroom. Students thrive in learning situations where the teacher's integrity and their confidence in that integrity govern all aspects of the course. High ethical standards should be evident in lectures that make all students feel welcome, in even-handed grading, in unbiased consideration of students' requests, and in the avoidance of even the appearance of impropriety in faculty-student relationships. Discrimination, bias, or self-interested abuses of power have no place in teaching (American Psychological Association, 2002).
Chapter 2
Theory
2.1 Review of the literature
The basic principles of effective teaching what Boyer (1990) called the scholarship of teaching. include (a) knowledge of the content of psychology courses, (b) understanding and applying pedagogical techniques that are capable of communicating that knowledge to students, and (c) striving to promote the kind of active learning that encourages students to be critical, creative thinkers who will use what they learned in class to their advantage long after graduation, and who will see the value of lifelong education. In other words, effective teaching not only involves transforming and extending knowledge, it pushes teachers in creative new directions (Boyer, 1990).
The Scholarship of Learning
The scholarship of teaching, itself, arises in part from the scholarship of learning (Cambridge, 2001), which seeks to apply what scientists in psychology and other disciplines have discovered about the learning process and how teachers can facilitate that process. Many of the teaching techniques described later have evolved from this scholarship of learning. Psychologists' long-standing involvement in the scholarship of learning can be seen not only in the vast psychological literature on the topic, but also in the fact that learning theorists, from early behaviourists to the cognitive theorists of today, have had a keen interest in how their research can be applied to education. In fact, no discipline has been more involved than psychology in the study of teaching and learning (Huber & Morreale, 2002; Nummedal, Benson, & Chew, 2002): Optimal teaching was the promise of the grand learning theories that dominated the early years of psychology as a discipline and the field's most prominent theorists, from Thorndike to Skinner, were concerned about the application of their theories to education In addition, the "cognitive revolution" of the 1960s initiated research that led to a better understanding about the ways in which information is processed (Skinner, 1954, 1968). Another line of research focused on the cognitive development of college students and both lines of research continue to ground scholarly work on teaching and learning in psychology (Huber & Morreale, 2002, p. 13). There is a long history of interest among some psychologists in promoting excellence in the teaching of psychology. For example, the 1899 convention of the American Psychological Association (APA) included a session on the teaching of psychology, and the second of APA's special-interest divisions, formed in 1945, was the division on the Teaching of Psychology (Nummedal et al., 2002). It is now known as the Society for the Teaching of Psychology (STP). In 1990, the APA created its education directorate with the charge to encourage research on the teaching of psychology across all subfields. In the late 1990s, the STP Task Force on Defining Scholarship in Psychology established evaluative criteria for the scholarship of teaching. They included a high level of disciplinary expertise, innovation, replication by others, documentation and exposure to peer review, and significant or impact content (Nummedal et al., 2002).
The Scholarship of Teaching and Learning
The result of all this activity in psychology, education, and related disciplines has been the development of a scholarship of teaching and learning (SoTL), an applied research enterprise with the central aim of improving the quality of teaching and learning in college classrooms and departments and disseminating it beyond these settings (DeAngelis, 2003; McKinney, 2003). Through SoTL research, instructors are encouraged to be more critical examiners of their own teaching and to share their findings with other instructors in their discipline and across disciplines. They are also encouraged to see their teaching activities as potentially valuable sources of scholarship.
This scholarship can take many forms, ranging from simple critical observation of classroom patterns, to use of classroom data to try out new classroom interventions, to research that compares testing methods to see which best fosters learning The basic notion is that the teaching and learning in one's own discipline or profession is itself a kind of living laboratory, a setting in which all sorts of intriguing, researchable questions quite specific to the discipline or profession develop. Since those of us in academia are engaged in teaching, evaluation and course design all the time, it becomes a natural and very attractive place to conduct a lot of our research. (DeAngelis, 2003, pp. 54-55)
As an example of SoTL in psychology, consider a problem that almost all psychology instructors encounter, namely students who arrive harbouring a whole range of culturally transmitted misconceptions about behaviour and mental processes (Nummedal et al., 2002). Do these misconceptions—which typically include the "facts" that schizophrenics have split personalities, that negative reinforcement is a form of punishment, and that people only use 10% of their brain cells—create proactive inhibition that interferes with students' learning of empirically supported conclusions about psychology? One psychology teacher believed that they do, but rather than simply bemoaning this possibility, he chose to contribute to the scholarship of teaching and learning by documenting the degree to which his own students' misconceptions about psychology affected their learning in his classes (Cerbin, 2000). Later, he conducted research into the effects of a problem-based learning intervention aimed at encouraging students to evaluate and revise their preconceptions. His work allowed him to assess the actual severity of a perceived problem, led to a test of methods for dealing with the problem that could be of value to other psychology teachers, and helped to guide his classroom teaching. SoTL is being encouraged by organizations such as the Carnegie Foundation for the Advancement of Teaching and the American Association for Higher Education. The Carnegie Scholars program brings together faculty from a variety of academic fields to design and conduct research on issues in teaching and learning in their fields. The Carnegie Academy for the Scholarship of Teaching and Learning (CASTL) supports the Teaching Academy Campus Program, which works with institutions of all types to foster and support the scholarship of teaching and learning. CASTL has also reached out to scholarly and professional organizations with small grants and other assistance aimed at helping to increase information sharing about teaching and learning.
Now it is time to consider how one can evaluate the outcome of teaching efforts, identify areas of weakness, and develop a plan for making improvements in those areas. These considerations are important not only because they can affect one’s prospects for salary increases, retention, promotion, and tenure, but also because teaching is more satisfying and enjoyable when one has found ways to do it to the best of one’s ability. Let's first identify several sources and varieties of evaluative information about teaching, and then look at some of the ways in which one can most profitably use that information to help one becomes a better teacher. Remember that the process of regularly gathering, carefully considering, and constantly adjusting to teaching evaluations is not just for new teachers. No matter how experienced they are, all teachers can always find ways to do better in one area or another. In general, only those who have stopped caring about the quality of their teaching have stopped seeking or attending to feedback about their performance.
One can get evaluative feedback on performance as a teacher from four main sources: students, colleagues in department, faculty development activities, and one own experience. The timing of the feedback can make it either formative or summative. Summative feedback comes at the end of the course, or perhaps the end of the academic year, and serves to “grade” teaching and suggests improvements. Formative feedback, in contrast, is gathered during a course and serves to tell about what one is doing well, and not so well, while there is still time to adjust the way him/her is handling that course. Although policies and traditions vary at different institutions, most teaching evaluations tend to be summative, and tend to come in the form of end-of-course evaluations provided by students. Collecting summative feedback might be voluntary on campus, but do not be surprised if one find that it is mandatory, especially for faculty who are not tenured or at the highest academic rank. Although most of one’s attention (and concern) might be focused on the summative evaluations that affect administrators' decisions about him/her, one should not forget to gather formative evaluations, too. Formative evaluations can alert one to problems, or the need for changes in teaching style or methods that will ultimately affect summative evaluations.
Given what is said earlier in this text about the basic principles of good teaching, the value of creating an inclusive classroom atmosphere, and the importance of establishing productive relationships with students, it should come as no surprise that teachers who try their best to teach well, and who care about their teaching, tend to get the best evaluative feedback from their students (Cashin, 1995; Junn, 1994; Lowman, 1995, 1998; Marsh & Roche, 1997; McKeachie, 1997a, 1997b; Murray, 1997; Seldin, 1999a).
When students in one study were asked to describe the best college courses they had taken, they reported a comfortable classroom atmosphere and interesting course content as the most important components (Levy & Peters, 2002). They also said that instructors in those courses "had a sense of humor and were entertaining; were excited about the material ... (had) a caring attitude toward students, and ... were approachable" (Levy & Peters, 2002, p. 47). These teachers were lauded for blending lecturing with a variety of other more active teaching techniques. By contrast, in two retrospective surveys, former students portrayed their worst teachers as having shown no enthusiasm for their subject matter, having been detached and aloof, and having provided students little encouragement, support, or help (Carson, 1999; McKinney 2001). These teachers were described as arbitrary, unfair, contradictory, and autocratic, not only in their grading systems, but in relation to individual students. They were said to have belittled and embarrassed some students, sometimes by making sexist or racist comments, while showing favoritism toward others. They were remembered, too, as showing scant concern for how they presented material, and especially for disorganized, confusing lectures that were delivered in a monotone voice and came straight from the textbook. Also included among the "worst" teachers were those who were entertaining and lenient, but who were perceived as holding their students, and themselves, to a low standard. These teachers were described as giving "Mickey Mouse" assignments, and offering "light and painless" classes in which lectures were aimed at the least able students and from which little was learned (Carson, 1999). Respondents in both surveys said that their worst teachers made them feel some combination of fear, anger, and frustration; caused them to lose interest in the course content; and taught them little or nothing (Carson, 1999; McKinney, 2001; see Table 1).
Table1
Seven Deadly Sins of Teaching
1. Arrogance: Presenting yourself as superior in status and knowledge, always presuming oneself to be right, and failing to consider other viewpoints.
2. Dullness: Boring your students with tedious, unrelenting lectures or other unchanging routines of teaching; offering no opportunities for active learning.
3. Rigidity: Teaching so as to convey the authoritarian idea that you know what is best for those over whom you have power.
4. Insensitivity: Saying things that show you do not empathize with your students and/or failing to respond to students' signals of interest, confusion, or concern.
5. Vanity: Engaging in self-promotion or self-congratulation in class, including by overemphasizing one's own work or point of view.
6. Self-indulgence: Allowing oneself to meet only minimum standards as a teacher; exhibiting laziness, lack of effort, and disinterest in the course and the students.
7. Hypocrisy: This one appears in many forms, but one of the most common is seen in teachers who decry their students' lack of motivation to learn, while themselves displaying a lack of motivation to teach.
Note. These were described by Eble (1983). Teachers who commit these sins tend to receive unfavorable summative evaluations from their students.
Formative Student Evaluations
As mentioned in earlier pages, one can prevent these negative reactions simply by letting students know one care about them, and by staying in close touch with them throughout the course. This communication process includes giving students plenty of opportunities to let him/her know what they think of teaching style, the course organization, and the like. One might not be able to address some of their criticisms (e.g., "This class meets too early in the morning"), but others might be easy to fix (e.g., "We could hear well if the hallway door was shut."). Whatever the case, one can be sure that students will appreciate the chance to provide formative evaluations, and to hear more about why one has chosen to teach the course. In fact, asking for formative evaluations can impress students enough to elevate summative evaluations, even if it was impossible or inappropriate to make all the changes they requested!
Collecting formative evaluations can even affect students' impressions of faculty in general by countering the prevailing view that teachers ignore the evaluations they are forced to collect at the end of the term (Sojka, Gupta, & Deiter-Schmelz, 2002). The Table 2 below is a sample form used in gathering information of teaching style, the course organization, and the like.
Table 2
Sample Forms for Use in Gathering Formative Student Evaluations
Circle response to each question below.
How would you rate the instructor's overall teaching effectiveness?
illustration not visible in this excerpt
How would you rate the overall quality of this course?
illustration not visible in this excerpt
Most of the time, I find this class interesting.
illustration not visible in this excerpt
How would you rate the pace at which material is covered in class?
illustration not visible in this excerpt
It is easy to remain attentive during lectures.
illustration not visible in this excerpt
The instructor is friendly toward students.
illustration not visible in this excerpt
The instructor is well-prepared for class.
illustration not visible in this excerpt
I find the lecture notes useful.
illustration not visible in this excerpt
I find the Web page useful.
illustration not visible in this excerpt
Use the following scale to indicate how much you have enjoyed each of the topics below.
illustration not visible in this excerpt
__________Research Methods
__________Development
__________Psychological Aspects
__________Sensation & Perception
Open-ended comment section
What aspects of this course do you like? What do you think the instructor is doing well?
What aspects do not you like? What would you like to see the instructor do differently?
Stephen Brookfield’s critical incident questionnaire
1. At what moment in the class this week did you feel most engaged with what was happening?
2. At what moment in the class this week did you feel most distanced from what was happening?
3. What action that anyone (teacher or student) took in class this week did you find most affirming and helpful?
4. What action that anyone (teacher or student) took in class this week did you find most puzzling and confusing?
5. What about the class this week surprised you the most? (This could be
something about your own reactions to what went on, or something that someone did, or anything else that occurs to you.)
Note. From Brookfield, S. (1996). Brookfield's Questions. The National Teaching and Learning Forum, 5, 8.
To increase the chances that one will get candid and thoughtful responses, let students know that their comments should be made anonymously, and reinforce that message by appointing a student to collect the forms and place them in a large envelope that he or she seals and delivers. (One can leave the room while the students complete them). In addition, emphasize the fact that one is asking for evaluations because one wants to use the feedback to improve this course as well as teaching skills in general.
Summative Student Evaluations
As one might expect, the same set of teacher behaviours that shape students' evaluations during a course tend to affect their ratings when the course is over. Accordingly, if one has gathered formative student evaluations throughout the course, he or she will probably not be surprised by the summative evaluations, which were received at the end. Nevertheless, some psychology teachers misinterpret the meaning of summative student evaluations. In particular, and regardless of their academic rank or tenure status, many believe that students give higher ratings to instructors who teach less demanding courses or grade more leniently (Sojka et al., 2002). Some of these faculty members appear to employ a self-serving cognitive bias, in that they treat the poor ratings they might receive as the misguided comments of the disgruntled or the ungrateful while dismissing their colleagues' more favorable ratings as the inevitable result of offering an easy course in an entertaining manner. In short, whereas most students would like to see their summative evaluations given more weight in the salary, promotion, and tenure decisions that administrators make about their teachers, many of those teachers see student ratings as having little or no value. In fact, there are psychology faculty—including some renowned for scientific research on other topics—who argue that it is impossible to accurately measure the quality of teaching because no criteria for "good teaching" have ever been, or ever will be, established.
It is not argued that student ratings alone define teaching effectiveness, but a considerable body of research does show that summative student evaluations can have value, not only in guiding administrative decisions but also in helping teachers to teach better. For one thing, the interrater reliability of students' evaluations of their teachers has been found to range from .74 in classes of 10 to 25 students to .95 in classes of 50 students or more (Marsh & Roche, 1997). Further, high summative student evaluations are strongly associated with desirable teacher characteristics such as enthusiasm, energy, and interest in teaching the course, but not with teachers' gender, age, ethnicity, teaching experience, or research productivity (Cashin, 1995). These evaluations are also not significantly related to potentially confounding factors such as the level or type of course taught, the time of day classes meet, class size, or students' age, gender, year in school, grade-point average, personality traits, and prior interest in the subject matter (Cashin, 1995; Marsh & Roche, 1997). Summative evaluations are related to the amount of work assigned and to the leniency of the grading system, but the direction of the correlation are opposite to what critics of student ratings might predict. As suggested by the student survey data mentioned earlier, teachers who employ more stringent grading standards and assign heavier student workloads (up to a reasonable limit) tend to receive higher summative student evaluations than those who are more lenient and less demanding (Cashin, 1988; Centra, 1993; Marsh & Roche, 1997, 2000; Watkins, 1994). One series of studies did find negative correlations between workload and student ratings (Gillmore & Greenwald, 1994; Greenwald, 1997; Greenwald & Gillmore, 1997a, 1997b), but those studies appear to suffer from methodological flaws that severely compromise the conclusions that can be drawn from them (Marsh & Roche, 2000). Although students enjoy an entertaining instructor, they can easily discriminate between style and content, and they recognize that style alone is not enough to justify high overall rating. In fact, students appear to value a course more, and to be more satisfied with it, when teachers require them to work hard to learn challenging material (McKeachie, 1997a; Marsh & Roche, 2000). Finally, student evaluations are positively correlated with a variety of commonly agreed-on indicators of good teaching, including the amount of material students learn in the course (Lowman, 1998; Marsh & Roche, 1997). The validity of student ratings is also suggested by the fact that these ratings tend to improve after teachers engage in structured efforts to improve their teaching (Marsh & Roche, 1997). Like students who never see the teacher comments that earned them a particular grade on a paper, it is hard to know what one is doing right and what one is doing wrong as a teacher if one get no end-of-term feedback about his/her teaching and course. Sometimes, it was not given the summative information one needs to evaluate the teaching in enough detail to guide good decisions about what changes to make. For example, many summative feedback forms ask students to provide only global ratings of the instructor (from, say, "good" to "poor") on just a few dimensions, such as "quality of teaching" or "personality" (see Table 3). These forms are quick and easy to complete, but are not particularly informative because they do not provide much detailed feedback, and they do not address enough of the many dimensions on which a teacher's performance can and should be evaluated (Marsh & Dunkin, 1997; Marsh & Bailey, 1993; Marsh & Roche, 1997, 2000; Seldin, 1999a). One had better use more elaborate forms that allow students to rate him/ her on a wide variety of dimensions and that allow him/her to insert dimensions in which one has a particular interest (e.g., "value of classroom demonstrations").
Table 3
illustration not visible in this excerpt
Table 3. Example of a global summative student evaluation form. This form is easy for students to complete, but actually conveys relatively little information to teachers, or to those who must evaluate their performance. Global forms that focus on too few dimensions can also create frustration in students who, at the end of a course, might like to offer more substantial and detailed feedback.
There are many of these more elaborate forms available. Some of them provide space for students to summarize in their own words what they think of the teacher and the course. Reading statistical summaries of students' ratings, combined with the experience of reading students' written comments, gives one two useful categories of evaluative information. The first is an aggregated profile of strengths and weaknesses, a profile that can be plotted as a graph, and compared from one term to the next, or from one course to the next, to evaluate the effectiveness of one’s efforts to improve various aspects of teaching. The second is a more immediate and personal portrait of students' reactions to one and the course, a portrait that often contains telling and useful elements. One can get the most out of written summative remarks by creating a two-by-two grid to analyze them. Then consult with experienced colleagues or experts about what changes one can or should make in teaching style and methods in response to the quantitative and qualitative summative feedback.
Analyzing Feedback
Reading student evaluations can be a bit depressing because even the best teachers of psychology leave some students dissatisfied, and because new teachers, especially, tend to take positive comments for granted and brood about the negative ones. To help oneself view comments in an objective and rational way, create a two-by-two table, and label its four cells as follows: Positive Comments, Negative Comments, Suggestions for Improvement, and Factors Beyond one’ s Control. A student's comment that "lectures are interesting" would go in the Positive Comments cell. "The quizzes are difficult" could go in either the Negative Comments or the Positive Comments column, depending on the goals. If one deliberately gives difficult quizzes to better prepare students for tests, this would be a positive comment, but it should prompt one to tell the class why the quiz questions are difficult. "I hate early morning classes" goes into Factors Beyond one’s Control, but let the students know that one is aware of how they feel, and perhaps plan to include a few more active learning events to keep everyone involved. "Put an outline on the board" would go in the Suggestions for Improvement cell. As already mentioned, one won't want to, or might be unable to, follow every student recommendation and correct every perceived fault, but after one has read and analyzed evaluations, takes a few minutes in class to thank students for their comments, discuss their feedback, and explain any changes one will (or won't) be making. Before deciding what to change, tally the number of students who made various comments and suggestions. If only one person claims the lecture’s pace is too slow, the problem probably lies with that student, so during the classroom discussion of feedback, one might offer to discuss the problem with the person who made that comment. Similarly, if a few students find the pace too fast, and a few others think it is too slow, one is probably teaching at about the right pace. Again, however, mention that one realizes that not everyone is satisfied with the teaching tempo and offer individual help to those who are struggling. Of greater concern, are comments suggesting that students are nearly unanimous in perceiving lectures as confusing, boring, or overwhelmingly detailed? Such feedback should be a signal for him/her to consider how to address the problem. When in doubt about how to respond to student feedback, discuss it with an experienced colleague or someone at the campus instructional development office. These consultations can be extremely helpful in guiding changes that significantly improve teaching effectiveness (Marsh, 1987; Marsh & Dunkin, 1997).
2.2 Theories and questions guiding thesis
Theory that guides the thesis is Gagné’s theory. Based on this theory, researcher created questionnaires. These questionnaires focus on assessing the quality of Psychology of Excellence in Business and Education courses within four key subscales: instructor quality, learning outcomes, instructional materials, and course difficulty. Gagné’s nine events (steps) of instruction were presented first as instructor quality subscale. Learning outcomes were following. Course difficulty and instructional material subscales were added as additional subscales. Researcher also mentioned a rational for forming nonstandardized instruments in course assessment and conducting course assessment with these instruments.
In The Conditions of Learning, Gagné acknowledges that he was considering the question "What factors really can make a difference to instruction?". His model proposed that the conditions of learning—some internal and some external to the learner—that affect the process of learning make up the events of learning. When deliberately planned, those events constitute instruction. "Thus it is reasonable to define instruction as being made up of events external to the learner which are designed to promote learning" (Gagné, 1977).
Gagné’s model proposed nine events of instruction. These events are specific functions of communication behaviours that he identified as components of instruction. Gagné divided these nine events into two groups: the first five represent communication behaviours that occur before the acquisition of information. The last four occur after acquisition has developed. The nine instructional events are applied for group, tutorial, and individual delivery are provided. According to Gagné, the events of instruction are labels that "serve to relate the internal processes to the external events that constitute instruction; that is they provide names for the total set of events (internal and external) that must be considered to take place during each phase of learning" (Gagné, 1974). Gagné’s model recommends these nine events be considered to facilitate and maximize instruction (see Figure 1).
Figure 1
illustration not visible in this excerpt
The processes involved in an act of learning are, to a large extent, activated internally. That is to say, the output of any one structure (or result of any one kind of processing) becomes an input for the next, as Figure 1 indicates. However, these processes may also be influenced by external events, and this is what makes instruction possible. Selective perception, for example, may obviously be affected by particular, arrangements of external stimuli. The features of a picture or text organized by perception may be influenced by highlighting, underlining, bold printing, and other measures of this general sort. Similarly, the particular kind of semantic encoding that is done in learning may be specified or suggested by meaningful information provided externally.
From these reflections on the implications or learning theory, one can derive a definition. Typically, instruction consists of a set of events external to the learner designed to support the internal processes of learning (Gagne´, 1985). The events of instruction are designed to make it possible for learners to proceed from “where they are” to the achievement of the capability identified as the target objective (see Table 4). In some instances, these events occur as a natural result of the learner’s interaction with the particular materials of the lesson. Mostly, however, the events of instruction must be deliberately arranged by an instructional designer or teacher. The exact form of these events is not something that can be specified in general for all lessons, but rather must be decided for each learning objective. The particular communications chosen to fit each set of circumstances, however, should be designed to have the desired effect in supporting learning processes.
Table 4. Events of instruction and their relation to processes of learning
illustration not visible in this excerpt
The functions served by the various events of instruction in an act of learning are listed in Table 4, in the approximate order in which they are typically employed (Gagne, 1985). The initial event of gaining attention is one that supports the learning event of reception of the stimuli and the patterns of neural impulses they produce. Before proceeding further, another instructional event is designed to prepare the learner for the remaining sequence. This is event number 2, informing the learner of the objective, which is presumed to set in motion a process of executive control by means of which the learner selects particular strategies appropriate to the learning task and its expected outcome. Event number 3 is also preparatory to learning and refers to the retrieval of items of prior learning that may need to be incorporated in the capability being newly learned. Events 4 through 9 are each related to the learning processes.
It should be realized that these events of instruction do not invariably occur in this exact order, although this is their most probable order. Even more important, by no means are all of these events provided for every lesson. Their role is to stimulate internal information processes, not to replace them. Sometimes, one or more of the events may already be obvious to the learner and, therefore, may not be needed. Also, one or more of these events are frequently provided by learners themselves, particularly when they are experienced self-learners. In designing instruction, the list of instructional events usually becomes a checklist. In using the checklist the designer asks, “Do theses learners need support at this stage for learning this task?”
1. Gain attention
Present a problem or a new situation. Use an "interest device" that grabs the learner's attention. This can be thought of as a "teaser" (the short segment shown in a television show right before the opening credits that is designed to keep you watching and listening). The ideal is to grab the learners' attention so that they will watch and listen, while you present the learning point. You can use such devices as: Storytelling, Demonstrations, Presenting a problem to be solved Doing something the wrong way (the instruction would then show how to do it the right way), Why it is important
[...]