Methodological issues in exploring students’ ideas about elementary astronomy

Introduction

The purpose of this study is to investigate the methods and the questions or the tasks used for researching students’ ideas in the domain of elementary cosmology mostly in primary education. Astronomy seems to be fertile and attractive. This is mostly because, in many ways, the scientific perspective of the Earth and relative concepts is contrary to intuition or common-sense and contrary to appearance. For instance, the Sun seems to move on the horizon during daylight hours, but actually it is the Earth that spins on its axes giving that impression. Young students’ developing understanding of the Earth and relative phenomena provides a fascinating natural experiment in the sources of young children's developing scientific knowledge. Most studies in this area involve children’s understanding of the shape of the Earth, the force of gravity and simple astronomical phenomena such as the day/night cycle.

In the last forty years, students’ understanding and ideas of scientific phenomena have been of considerable interest to researches in the field of science education. In Mouly’s (1978) words, the aim of a research is to promote progress and enable people to resolve their conflicts. In a similar vein, the purpose of the researches in science education is to ensure maturity and progression of education where lacks exist (Cohen and others, 2000). Nevertheless, in some cases the findings of such studies are not consistent and as a result there is disagreement about whether a lack exists or not. Several researchers have claimed that this inconsistency is related with the research approaches using different methods and reaching different conclusions.

Generally speaking, the word method concerns those techniques of eliciting answers to predetermined questions, measuring or recording information, describing a phenomenon and carrying out experiments (Cohen and others, 2000). In this study, the term will be referred to the range of approaches used in educational research to collect data and not for data analysis. This information has been used for interpretation, inference and explanation of children’s ideas. The methods for analysing the data will not be examined here, albeit there is a significant interrelationship between the data collected and the tools or approaches used for their interpretation. Therefore, even this not being the purpose of this study, some weaknesses of the methods used will be mentioned.

As Kaplan (1973) suggested the purpose of methodology is not to understand the findings and the products of a research, but to perceive the way of enquiring. Thus, the present study will examine the methods and the type of questioning used for researching young children’s understanding in the domain of basic astronomical concepts.

In the following sections, methodological issues related to the aforementioned research problem will be considered. The next part examines the research methods used for examining children’s ideas and understanding whereas the third part comprises a review of the particular approaches and techniques applied for investigating young students’ knowledge about basic elementary astronomy. The last part involves general comments and conclusions.

Questioning types: weaknesses and strengths

There are two main methods for carrying out investigations in this field of educational research (students’ ideas about basic astronomical events); qualitative or quantitative research methods with a long debate being established about the relative value of quantitative and qualitative inquiry (Patton, 1990). Both approaches have arisen from different research needs and were used to support different theories about students’ ideas in this particular domain of research.

In recent years, education research has moved away from the numerical approach (quantitative methodologies) and emphasis has been given on qualitative methodology. Studying three respected science education journals, Devatak et al (2009) concluded that qualitative research approaches are more preferred than other methods. The opposite tendency is noticed concerning the researches of science education carried out in the area of astronomy.

A debate is established about the methods used to approach this kind of research problem in this particular area of science education. The disputation has basically arisen from the inconsistency of the results produced by different researches. Not to mention that, in most cases, the underlying theory for the origination of children’s ideas as well as the way that investigators analyse data differ a lot. In other words, different needs have indicated which methodology could be better to be followed with the critics of a theory choosing the opposite methodology from the supporters. In the present study, attention has been paid to the research questions or tasks that have been previously used in many studies. In agreement with the researchers who claimed that there is a relationship between the research questions and the results, Panagiotaki (2003) indicated that different testing methods of student’s conceptions and understandings evoke different responses in this domain.

There is a wide range of types of questions used to examine young students’ ideas about elementary astronomy. Yet, two were the most commonly used. Open ended approaches based upon qualitative methods and closed, derived from quantitative methodology. In the following paragraphs, the different tools (techniques) of questioning will be examined.

Open-ended questioning.

This approach simply refers to asking a question to a participant of a survey in which the latter answers with no restriction. If the question is included in a questionnaire, it is put and then a space is left for an answer without constraints. The collected data in this case are in a verbal (oral or written) and pictorial form. In order to identify children’s ideas about basic cosmological notions, both kinds of data were obtained in open ended questions. As an approach, this type of questioning has distinctive features of qualitative methods (Cohen and others, 2000).

This method can lead to obtain more in depth information that could not be easily revealed by using quantitative methods. As Strauss and Corbin (1990) have argued, it can be used for a better understanding of a phenomenon and for gaining new insights about already known things. Many investigators in science education explained that they used such an approach so as to gain an in-depth comprehension of their participants’ views in as a natural setting as possible. Cronbach (1975) stated that quantitative methods do not have this ability of taking a full account of the interaction that takes place in social contexts. Having the features of qualitative methodology, an open-ended question is very attractive technique for small study samples offering information that might have been concealed in a questionnaire differently designed. Moreover, such questions provide the quality of the response of being honest, authentic and rich (Cohen and others, 2000). That is to say that respondents have the freedom to express their answer in the way they want. They do not have to try to understand what researchers mean with the question and then respond accordingly. In other type of questioning, like forced-choice questions examined below, a participant should choose a response even if his/her own perspective is not enlisted. In this sense, the respondent should understand the question and interpret it according to the options provided whereas in open questions the answer is free of such constraints. Of course, this could be an advantage of open-ended questioning but a disadvantage as well.

This disadvantage is the heart of the problem of questioning using a set of printed or written questions, in general. Different respondents might read and interpret the same words differently, thereby rendering the answers ambiguous (Cohen and others, 2000). Furthermore, the respondent might provide irrelevant answers which can be avoided if some instructions were given to him/her during the procedure of completing the questionnaire.

In addition, open- endedness has problems related with data handling. Word-based data are susceptible to combinations and aggregation (Cohen and others, 2000). This might be the reason for being relatively rare to find explicit discussions about how to analyze data gathered using such a methodology (Burgess, 1985). It is difficult, but not impossible, to transform the data into numbers, however even when the researcher manages to convert views into numbers, he or she might use principles of quantitative methodology making the method not being purely qualitative. This could have an impact on the process of collecting the data because the investigator, especially in the case of interviews, would encourage participants to give answers that could be converted into numbers. To illustrate, researchers who use open-ended questions as their research instrument and aim at using numerical approaches to analyze them, might phrase their questions in such a way to obtain a response that can be transformed to numbers. But, this type of questioning does not have the features of open-endedness and does not allow the participant to respond in his/her own way. Further, in interview settings, researchers might ask the same question until they get a “numerical answer” and thus, leading their sample towards a desired response. This way, the open questions might “lose” the strength of being free of limitations. However, statistical and numerical approaches are quite rarely used because, as Bryman (2004) pointed out, the aim in this case is to search for codes (phrases or concepts/notions) in the analyzed data.

Moreover, open-ended questions make it difficult to compare between the responses (Cohen and others, 2000). This is because articulate and inarticulate respondents may reply differently and therefore, there will be very few things in common to compare. At last, it should be added that completing an open-ended questionnaire is more time consuming than a closed questionnaire and this might affect the participation of respondents.

Close choice questions

Closed type of questioning recommends the range of answers from which respondents have to choose. For researching children’s ideas about Earth and relative phenomena (day/night alteration, seasons etc.) multiple choice questions and rank order questions were used. In general terms, this type of questions can be answered quickly, straightforwardly and are not related to respondents’ articulation (Wilson and McLean, 1994). In addition, all participants in such surveys are constrained to respond in the same way making the analysis of the answers quick and easy. This method is based upon the quantitative approach providing numerical data which are precise due to the form of answers. In contrast to open-ended types where vast quantities of wordbased information should be considered, closed questions enable comparisons to be made across different groups of respondents in the study sample (Oppenheim, 1992). As Johnson and Onwuegbuzie (2004) have mentioned, one of the strengths of closed questions, and of quantitative methods in general, is that they are suitable for validating already expressed theories and for testing hypotheses. Also, closed type questions could be used to generate frequencies of answers and tendencies of students’ perspectives.

On the other hand, closed questions do not allow any “freedom” to the respondent. To illustrate, ranking tasks and multiple choice questions do not enable the participants to write an answer in their own terms and use their own words to describe their ideas. Additionally, respondents cannot add any comments, remarks and explanations to the options provided and there is always the risk that the possible answers might not be exhaustive and satisfactory for the respondents (Oppenheim, 1992).

Another weakness, which has to do with the limitation of the available answers, is the danger of missing out interesting “phenomena”. If there is a discrepancy between the participant’s point of view and the categories of responses designed by the researcher, the former will try to answer as best as he/she could based on the options. This way, the mismatch of the perspectives will be concealed.

Many of the researchers in this particular domain of students’ knowledge about cosmology have used this technique of questioning to test theories expressed by researchers who have used qualitative methods. Based upon the advantages of this methodology, they claimed that the use of closed questions has the potential to reduce some of the problems of open-ended questioning.

As mentioned above, multiple choice questions and rank ordering tasks were used in science education research in the area of a child’s ideas about basic cosmology.

Multiple choice questions

In this type of questions the researcher designs the range of responses so as to record accurately the possible answers given by the participant. The latter has to reply according to given statements. The development and the use of multiple choice tests have the potential to contribute valuably to the area of children’s misconceptions and can be helpful in identifying their ideas about scientific concepts (Treagust, 1988).

However, this method cannot reveal in-depth details of their understanding. As a quantitative method, it has the benefits mentioned above but researchers should be very careful when they design the categories offering an exhaustive range of possible answers and avoiding any overlaps of the possible answers (Cohen and others, 2000). Moreover, it is important for respondents to be informed whether they could choose more than one response (multiple answer mode) or only one (single answer mode). It is always a problem of this approach that the list of answers provided could not comprise all the possible answers and thus a respondent might not have the opportunity to choose his/her perspective.

Rank ordering tasks/questions

In this technique, a list of statements or categories is provided, and the respondent is asked to indicate an order of some sort. Rank ordering type of questioning has many similarities with multiple choice questions described above. It has the characteristic of identifying options from which the participants can choose what is more suitable for them and rank it appropriately. However, it differs from multiple questioning because it requires the identification of priorities (Cohen and others, 2000). It could be argued that this enables a relative degree of “freedom” in terms of priority and preference but the respondent has to rank the categories that are predetermined by the researcher. Thus, the participant is constrained to express his/her perspective using the investigator’s list of ideas. Further, even if the experimenter has carefully designed the categories allowing possible fine distinctions to be made, the identification of priorities might not be easily perceived or understood by the respondent (Cohen and others, 2000).

In the area of science education research about astronomical events held by young children, most of the questioning methods described above were used in research interview settings.

The research interview has been defined as a conversation between the interviewer and the participant of the research. The interviewer aims at obtaining research-relevant data but more importantly he/she is capable focusing the interviewee on content specified by the objectives of the research (Cannel and Kahn, 1968). In this sense, it differs from a questionnaire addressing the problem mentioned above. An interview allows a greater depth of the questions posed due to the continuous interactions of the interviewer and the respondent. It has, also the potential of a higher response rate because from the part of the participant, he/she can become motivated and more involved providing more details about his/her ideas in comparison to a questionnaire (Oppenheim, 1992).

An interview differs from a questionnaire in the process of gathering the data and in the way that responses are recorded. The former involves the collection of data through oral interaction between the participant and the interviewer whereas in a questionnaire, respondents should read the question and record their perspective on their own. The latter might be problematic because respondents may be reluctant to read and write for several reasons (Cohen and others, 2000). Especially, from my own experience, young children are unwilling to write their answers and they would prefer to reply verbally on a question. Moreover, if the students are at a very early age, their ability of reading and writing might not be adequate to complete a questionnaire.

Nevertheless, the interaction taking place during the interview, except from advantages, it is a source of disadvantages as well (Borg, 1963). Some of them were mentioned above and this is something that I had confronted as a researcher. In addition, as Cohen and others (2000) pointed out, the basic weakness of this technique concerns any possible subjectivity and bias on the part of the researcher that could affect the responses of the interviewee. Another disadvantage is related with ethical issues. From my own experience, it is more difficult for consent to be achieved so as to conduct interviews with young children than asking them to complete a questionnaire. In a similar research in Greek students’ ideas about day/night cycle and seasons the gatekeepers (head-teacher and parents) were too reluctant about letting their children to take part in interviews. The situation becomes much more complicated if research requires the use of video and tape record methods. Therefore, if we accept the axiom mentioned in Homan’s (2001) article that there is an obligation to inform and obtain the consent of the gatekeepers, who are responsible for withholding or giving the access to a researcher, our chosen methodology might be affected.

Despite the comparison of these two approaches, interviews and written questions, the former has many things in common with a questionnaire. According to the methods of questioning used, an interview could have the same strengths and weaknesses previously discussed. Indeed, examining young students’ ideas about basic astronomical events, researchers have used structured interviews where both open-ended questions (standardized open-ended interviews) and closed questions (closed quantitative questions) were asked. Despite the opportunity of asking the interviewee for clarification and elaboration, the characteristics of open-ended and closed questions continue to exist in interview settings. In the section below, the use of such methods and techniques will be examined by critically reviewing the methodology chosen by several researchers in this particular domain of science education.

A review of the methods used in several researches

Surveys about children’s ideas and understandings about the Earth and relative phenomena provides insights into the structure of their notions, the origin of their ideas and the way scientific knowledge is acquired (Panagiotaki and others, 2006). This way, stakeholders could be informed about any lack of knowledge or any difficulties in understanding basic astronomy and take the appropriate decisions to facilitate the acquisition process. However, the disagreement of the findings does not allow the generation of general conclusions. Thus, different findings could lead to very different implications for teaching.

[...]