Length Matters. How the Length of Self-Generated Notes Used in an Exam Impacts Students’ Performance, Retention, and Anxiety

Length Matters

Compared to Closed-Book, Open-Book and single page Cheat-Sheet examination forms, authorizing the use of Limited Notes on a social studies exam improves middle schoolers' performance and retention while lowering test anxiety.

Discussions about whether Open-Book (OB) or Closed-Book (CB) examination forms are superior have persisted for over 70 years (Stalnaker & Stalnaker, 1934). Today, however, the topic of exam forms exceedingly is relevant. In the midst of the Covid-19 pandemic, educators are forced to consider new evaluation forms because social distancing regulations have made traditional in-presence examinations less viable. The traditional exam form is CB, in which students are allowed no material during the exam, while OB exams characteristically allow students the unlimited use of notes, textbooks, and other reference materials (Eilertsen & Valdermo, 2000). Arguments have been made, that traditional CB exams rely on memorization and recognition of simple facts and should thus be discarded for a newer method, such as OB, that demands an understanding of the nature of the knowledge itself rather than pure recall (Stalnaker & Stalnaker, 1934; Eilertsen & Valdermo, 2000). However, some researchers, such as Kalish (1958), assert that OB examinations would dissuade students from spending as much time studying as they previously would have for a more traditional exam which could therefore harm performance and retention.

This debate, however, is hardly two-sided. In fact, modern studies typically investigate three options: Open-Book, Closed-Book, and Cheat-Sheet (CS) exams (Gharib et al. 2012). Erbe (2010) defines a cheat sheet, or crib sheet, as a single piece of paper with notes on or answers to a test that is used to cheat on or prepare for an exam. She also proposes that the authorized use of a cheat sheet inan exam can reduce student test anxiety and helps facilitate deeper learning of the material. However, empirical studies concerning these three test forms have shown mixed results. For instance, Skidmore and Aagaard (2003) found that undergraduates who were allowed a cheat sheet performed better on average than those students who took a traditional CB test. At the same time, Dickson and Miller (2005) found no significant performance effects regarding the use of a cheat sheet. Gharib et al. (2012) compared all three methods and found the best performances from undergraduates in the OB condition, then CS, and worst in the CB condition. This paper proposes that a fourth test option should be considered: Limited Notes (LN).

It is the intent of this paper to explore the heretofore uninvestigated effects of authorized notes in an exam between the length of one page or unlimited pages. The length of four pages, front and back, is suggested here as an ideal length of self-generated notes in an exam because students should be able to fit all high priority information into these four pages without including too much unnecessary detail. By limiting the students to only four pages, students may not be as inclined as they may be in an unlimited condition to include all potentially relevant detail, causing their notes to become much more difficult to study and navigate. They will have to strategically decide which information is important enough to include in their notes. However, by allowing them this many pages, they should be able to accomplish this task without having to intentionally leave out important information due to lack of space. Furthermore, Gharib et al. (2012) suggests that the ideal examination form is different according to the course subject or discipline. They mention that studying for a psychology exam may be more focused on the memorization and understanding of facts while study strategies for a mathematics exam may be more reliant on the correct usage of various formulas. It is conceivable that the ideal length of notes also varies with the subject of the course. Many previous CB and CS studies focused on one of the two above mentioned disciplines (Gharib et al., 2012). This study suggests investigating the effect of the authorized use of four pages in notes on a social studies exam because it is a core class in the curriculum of the participants (middle schoolers) that closest resembles psychology. Thus, middle school students who use Limited Notes on an exam will demonstrate superior performance and retention as well as lowered anxiety during a social studies examination as students in the Closed-Book, Cheat-Sheet or Open-Book condition.

Performance

Research into the impact of allowing students a single authorized note page on an exam began as early as 1979. Dorsel and Cundiff investigated this, focusing on two primary hypotheses. The first was the Coding Hypothesis, which states that the preparation of a cheat sheet facilitates efficient coding of the course material and therefore increases students' performance on a subsequent exam, regardless of whether they are allowed their cheat sheet or not. Alternatively, the Dependency Hypothesis states that students may rely too heavily on a cheat sheet and use it as a crutch, causing them to perform worse in non-cheat sheet exams. Their results showed that the preparation of a cheat sheet can be beneficial to performance as long as the students do not become dependent on it. For instance, students who prepared a cheat sheet but were unaware that they would not be allowed to use it in the exam scored significantly lower than all other conditions on the test, suggesting that the Dependency Hypothesis is true in this case. However, when students were aware that they would or would not be allowed to use their prepared cheat sheet on the test, they performed better. Students that scored the best were those who were aware that they would not be allowed to use their cheat sheet, but prepared one anyway, suggesting that the Coding Hypothesis may also hold value. The[re was, however, no significant performance differences between students in the CB condition and the CS condition when they were aware that they would be allowed to use it (Dorsel & Cundiff, 1979). In a more recent study, Dickson and Miller (2005) also found no significant performance effects between students who participated in CS or traditional CB exams. However, there are a variety of studies which did observe a significant increase in average grades on an exam in students in the CS condition as opposed to CB (Skidmore & Aagaard, 2003; Rice et al., 2017; Gharib & Phillips 2012; Gharib et al., 2012). Both Gharib and Phillips (2012) and Gharib et al. (2012) included an OB condition in their study along with CB and CS. In both studies, undergraduates in the OB condition showed the best performance followed by CS and finally CB.

There is some research that provides evidence that university students perform better on examinations in which they are allowed an unrestricted amount of notes and reference materials than on those where they are allowed a single page of notes or none at all (Agarwal et al., 2008; Gharib & Phillips, 2012; Gharib et al., 2012). Each of these studies provided evidence that students achieved higher average grades on an OB exam compared to either CB or CS. However, according to their recent meta-analysis which specifically compared OB and CB exam performance from 30 different studies, Durning et al. (2016) found no significant increase in initial performance on OB compared to CB exams. Some potential mechanisms that could explain these results include reported time spent studying, type of exam, lack of experience with open-question exam types, as well as quality of reference materials used in the exam (Song & Thuente, 2015; Durning et al., 2016).

Preparing notes for a subsequent exam can be both helpful and harmful for a student's study strategy. As mentioned above, preparing self-generated notes can help code relevant information into memory, but it can also be harmful if the student becomes dependent on the material (Dorsel & Cundiff, 1979). This dependency can manifest when students justify spending less time preparing and studying for an exam in which they believe they will have access to reference material. In the 20 studies included in an analysis of exam preparation, students often reported studying less for OB than CB exams and even reported attending class less frequently if they knew the exam would be in OB format (Durning et al., 2016). It is conceivable that, had students spent as much time preparing for a OB as they would have a CB exam, their performance on an OB exam would have been superior to that of on a CB one. By introducing a Limited Notes condition in which students are authorized only four pages of self-generated notes, it could encourage students to spend more time preparing and reviewing for the exam. When preparing for an OB exam, students may rely too much on their ability to search for answers in their textbooks and notes. They may be inclined to bring all of the reference material available to them and spend less time studying or summarizing said material. The act of constructing quality LN alone increases study time and effort. For instance, students in the LN condition would be forced to strategically search out and summarize the most important and relevant course material because their space is limited. Their space is not so limited, however, that they have little of their own notes to review when studying. Loftman (1975) found that time spent reviewing was a better indicator for latter performance than overall time spent studying. It is comprehensible that students in the LN condition would spend the most time reviewing their notes because they are not reliant on preconstructed materials like textbooks, as seen in OB condition, while also having enough space to viably cover and review for all relevant information on the test, unlike in the CS condition.

One of the advantages of offering OB, CS, or LN exams, is that instead of operationalizing student ability as rote learning and memorization skills as is characteristic to CB tests, examiners are freer to construct questions which rely on complex understanding of the course material as well as its practical applications (Erbe, 2010; Song & Thuente, 2015). This question format is more representative of real-world knowledge application and thus may be a predictor of students' future abilities (Boniface, 1985). Included in Durning et al.'s (2016) meta­analysis were many studies that used multiple choice tests as a measure of performance. However, Song and Thuente (2015) mention that the use of cheat sheets and reference materials does not lend themselves to multiple choice tests because they do not measure complex thought application and reasoning skills thar are facilitated by the use of notes in an exam. In fact, the less comprehensive studying characteristic to an OB exam may harm students' ability to recall simple facts (Song & Thuente, 2015). While LN exams also lend themselves more towards the practical application of knowledge, the increased time spend studying while combing over information to include/exclude from the final four pages of notes may also help students with their recall, making this the superior form to CS and OB exams.

While time spent studying is positively correlated with exam performance (Durning et al., 2016), the time spent searching through notes during the test is negatively correlated with exam success. Boniface (1985) found that those students in an OB exam achieved higher scores when they allocated more exam time towards writing and fewer towards searching through notes and texts. On average, students spent 33 percent of their time searching through their notes and texts. However, if students were only allowed four pages of self-generated notes, they would likely spend a fraction of that time searching their notes, because they would be more cognizant of what material they have brought with them and where they have organized it. Students in a CS condition would likely spend the least amount of time searching their notes; however, this is because they are only allowed a single page. This negates the study and coding benefits of LN and therefore likely mitigates the benefits of fewer time spent searching notes in the exam.

Another further reason that students may not perform at the optimal level when taking OB or CS exams is their lack of experience with such exam types. Out of all of the studies included in their meta-analysis, only 30% reported having some (11%) or significant (19%) prior experience with OB exams. Furthermore, all besides two of these were performed with university level students, most often undergraduates (Durning et al., 2016). This is significant because most students at the university level are already familiar with the structure of a CB exam and focus their learning strategies on recall and memorization. In fact, during an implementation of OB learning strategies in upper secondary education students in Norway, one student mentioned, “If we had started this OB three years ago, we would've been used to it by now. Now it's too late!” (Eilertsen & Valdermo, 2000, p. 94). For this reason, the proposed study below will suggest investigating the impacts of the use of LN on an exam in first year middle schoolers (6th grade). This will hopefully reduce the participants' familiarity with using strict recall and rote learning as a study and test strategy as is common in traditional CB exams (ie. Eilertsen & Valdermo, 2000). By implementing new exam forms such as OB, CS, and LN in middle schoolers, the potential performance and retention benefits may be increased further due to the lack of rigidity of study strategy.

Another moderator of performance is the quality of the self-prepared notes. Song and Thuente (2015) found three significant implications of quality regarding cheat sheets. First, there was a large discrepancy in the quality of cheat sheets between those students who scored within the top half of the exam and those who scored in the bottom half. Secondly, the quality of the students' cheat sheet was highly correlated with their subsequent grades on the exam. Thirdly, an improvement in quality of cheat sheets over time was followed with an improvement in exam performance. To this regard, the following study proposal suggests that students should be randomly allocated to their conditions. With random allocation and a large enough sample population, the variance found in the quality of the cheat sheets should be similar for each condition as well as being representative of a real-world classroom. Additionally, this third implication further lends credence to the concept of implementing LN exam forms as early as possible (i.e. middle schoolers) because as students practice writing their own reference material and increase the quality of their notes over time, their exam performance should also increase.

These are the mechanisms through which the implementation of LNon examinations should improve student performance. First, by introducing LN in the first year of middle school it should allow students to become familiar with a learning strategy that doesn't focus purely on memorization. As they become accustomed to applying their knowledge and reasoning in exams, their performance should increase over time, making this form superior to CB. Also, using LN may be a way for students to implement both traditional knowledge memorization as well as deductive reasoning skills in their exams. Furthermore, they will likely also learn more efficient ways to construct their LN and increase the quality of their notes over their academic careers. This should also lead to an increase in performance on subsequent exams. Not only would the implementation of LN on exams increase the quality of students' notes but would also increase the quality of their studying. Students need to be critical and strategic about what they include in their LN and will thus spend more time carefully including only the most important information. This provides an added benefit when students study their LN before an exam because reviewing also leads to better grades, making this the superior form to OB and CS. The following study suggests a method to quantitatively measure whether students perform better on LN, CS,OBor CB examinations.

[...]