WHY RESEARCH IS OF VALUE
How can educators, parents, and students obtain the information they need? Many ways of obtaining information, of course, exist. One can consult experts, review books and articles, question or observe colleagues with relevant experience, examine one’s own past experience, or even rely on intuition. All these approaches suggest possible ways to proceed, but the answers they provide are not always reliable. Experts may be mistaken; source documents may contain no insights of value; colleagues may have no experience in the matter; and one’s own experience or intuition may be irrelevant or misunderstood.
This is why a knowledge of scientific research methodology can be of value. The scientific method provides us with another way of obtaining information—information that is as accurate and reliable as we can get. Let us compare it, therefore, with some of the other ways of knowing.
WAYS OF KNOWING
SENSORY EXPERIENCE
We see, we hear, we smell, we taste, we touch. Most of us have seen fi reworks on the Fourth of July, heard the whine of a jet airplane’s engines overhead, smelled a rose, tasted chocolate ice cream, and felt the wetness of a rainy day. The information we take in from the world through our senses is the most immediate way we have of knowing something.
Using sensory experience as a means of obtaining information, the director of the gifted-student program mentioned above, for example, might visit an advanced placement English class to see and hear what happens during a week or two of the semester. Sensory data, to be sure, can be refined. Seeing the temperature
on an outdoor thermometer can refine our knowledge of how cold it is; a top-quality stereo system can help us hear Beethoven’s Fifth Symphony with greater clarity;
similarly, smell, taste, and touch can all be enhanced, and usually need to be. Many experiments in sensory perception have revealed that we are not always wise to trust
our senses too completely. Our senses can (and often do) deceive us: The gunshot we hear becomes a car backfiring; the water we see in the road ahead is but a mirage; the chicken we thought we tasted turns out to be rabbit.
Sensory knowledge is undependable; it is also incomplete. The data we take in through our senses do not account for all (or even most) of what we seem to feel is the
range of human knowing. To obtain reliable knowledge, therefore, we cannot rely on our senses alone but must check what we think we know with other sources.
AGREEMENT WITH OTHERS
One such source is the opinions of others. Not only can we share our sensations with others, we can also check on the accuracy and authenticity of these sensations: Does this soup taste salty to you? Isn’t that John over there? Did you hear someone cry for help? Smells like mustard, doesn’t it? Obviously, there is a great advantage to checking with others about whether they see or hear what we do. It can help us discard what is untrue and manage our lives more intelligently by focusing on what is true. If, while hiking in the country, I do not hear the sound of an approaching automobile but several of my companions do and alert me to it, I can proceed with caution.
All of us frequently discount our own sensations when others report that we are missing something or “seeing” things incorrectly. Using agreement with others as a means of obtaining information, the tenth-grade biology teacher in Atlanta, for example, might check with her colleagues to see if they fi nd discussions more effective than lectures in motivating their students to learn. The problem with such common knowledge is that it, too, can be wrong. A majority vote of a committee is no guarantee of the truth. My friends might be wrong about the presence of an approaching automobile, or the automobile they hear may be moving away from rather than toward us. Two groups of eyewitnesses to an accident may disagree as to which driver was at fault. Hence, we need to consider some additional ways to obtain reliable knowledge.
EXPERT OPINION
Perhaps there are particular individuals we should consult—experts in their fi eld, people who know a great deal about what we are interested in fi nding out. We are likely to believe a noted heart specialist, for example, if he says that Uncle Charlie has a bad heart. Surely, a person with a PhD in economics knows more than most of us do about what makes the economy tick. And shouldn’t we believe our family dentist if she tells us that back molar has to be pulled? To use expert opinion as a means of obtaining information, perhaps the physical education teacher in Tulsa should ask a noted authority in the physical education fi eld whether ability in one sport correlates with ability in another.
Well, maybe. It depends on the credentials of the experts and the nature of the question about which they are being consulted. Experts, like all of us, can be mistaken. For all their study and training, what experts know is still based primarily on what they have learned from reading and thinking, from listening to and observing others, and from their own experience. No expert, however, has studied or experienced all there is to know in a given fi eld, and thus even an expert can never be totally sure. All any expert can do is give us an opinion based on what he or she knows, and no matter how much this is, it is never all there is to know.
Let us consider, then, another way of knowing: logic.
LOGIC
We also know things logically. Our intellect—our capability to reason things out—allows us to use sensory data to develop a new kind of knowledge. Consider the famous syllogism:
All human beings are mortal.
Sally is a human being.
Therefore, Sally is mortal.
To assert the fi rst statement (called the major premise), we need only generalize from our experience about the mortality of individuals. We have never experienced anyone who was not mortal, so we state that all human beings are. The second statement (called the minor premise) is based entirely on sensory experience. We come in contact with Sally and classify her as a human being. We don’t have to rely on our senses, then, to know that the third statement (called the conclusion) must be true. Logic tells us it is. As long as the first two statements are true, the third statement must be true.
Take the case of the counselor in Philadelphia who is asked to advise a student on how to improve her study habits. Using logic, she might present the following argument:
Students who take notes on a regular basis in class find that their grades improve.
If you take notes on a regular basis, then your grades should improve as well.
This is not all there is to logical reasoning, of course, but it is enough to give you an idea of another way of knowing. There is a fundamental danger in logical reasoning, however: It is only when the major and minor premises of a syllogism are both true that the conclusion is guaranteed to be true. If either of the premises is false, the conclusion may or may not be true.
There is still another way of knowing to consider: the method of science.
THE SCIENTIFIC METHOD
When many people hear the word science, they think of things like white lab coats, laboratories, test tubes, or space exploration. Scientists are people who know a lot, and the term science suggests a tremendous body of knowledge. What we are interested in here, however, is science as a method of knowing.
It is the scientific method that is important to researchers.
What is this method? Essentially it involves testing ideas in the public arena. Almost all of us humans are capable of making connections—of seeing relationships and associations—among the sensory information we experience. Most of us then identify these connections as “facts”—items of knowledge about the world in which we live. We may speculate, for example, that our students may be less attentive in class when we lecture than when we engage them in discussion. A physician may guess that people who sleep between six and eight hours each night will be less anxious than those who sleep more or less than that amount. A counselor may feel that students read less than they used to because they spend most of their free time watching television.
But in each of these cases, we do not really know if our belief is true. What we are dealing with are only guesses or hunches, or as scientists would say, hypotheses. What we must do now is put each of these guesses or hunches to a rigorous test to see if it holds up under more controlled conditions. To investigate our speculation on attentiveness scientifi cally, we can observe carefully and systematically how attentive our students are when we lecture and when we hold a class discussion. The physician can count the number of hours individuals sleep, then measure and compare their anxiety levels. The counselor can compare the reading habits of students who watch different amounts of television.
Such investigations, however, do not constitute science unless they are made public. This means that all aspects of the investigation are described in sufficient detail so that the study can be repeated by anyone who questions the results—provided, of course, that those interested possess the necessary competence and resources. Private procedures, speculations, and conclusions are not scientific until they are made public. There is nothing very mysterious, then, about how scientists work in their quest for reliable knowledge.
In reality, many of us proceed this way when we try to reach an intelligent decision about a problem that is bothering us. These procedures can be boiled down to five distinct steps.
1. First, there is a problem of some sort—some disturbance in our lives that disrupts the normal or desirable state of affairs. Something is bothering us. For most of us who are not scientists, it may be a tension of some sort, a disruption in our normal routine. Examples would be if our students are not as attentive as we wish or if we have diffi culty making friends. To the professional scientist, it may be an unexplained discrepancy in one’s fi eld of knowledge, a gap to be closed. Or it could be that we want to understand the practice of human sacrifi ce in terms of its historical significance.
2. Second, steps are taken to defi ne more precisely the problem or the questions to be answered, to become clearer about exactly what the purpose of the study is. For example, we must think through what we mean by student attentiveness and why we consider it insufficient; the scientist must clarify what is meant by human sacrifice (e.g., how does it differ from murder?).
3. Third, we attempt to determine what kinds of information would solve the problem. Generally speaking, there are two possibilities: study what is already known or carry out a piece of research. As you will see, the fi rst is a prerequisite for the second; the second is a major focus of this text. In preparation, we must be familiar with a wide range of possibilities for obtaining information, so as to get firsthand information on the problem. For example, the teacher might consider giving a questionnaire to students or having someone observe during class. The scientist might decide to examine historical accounts or spend time in societies where the practice of human sacrifice exists (or has until recently). Spelling out the details of information gathering is a major aspect of planning a research study.
4. Fourth, we must decide, as far as it is possible, how we will organize the information that we obtain. It is not uncommon, in both daily life and research, to discover that we cannot make sense of all the information we possess (sometimes referred to as information overload). Anyone attempting to understand another society while living in it has probably experienced this phenomenon. Our scientist will surely encounter this problem, but so will our teacher unless she has fi gured out how to handle the questionnaire and/or observational information that is obtained.
5. Fifth, after the information has been collected and analyzed, it must be interpreted. While this step may seem straightforward at fi rst, this is seldom the case. As you will see, one of the most important parts of research is to avoid kidding ourselves. The teacher may conclude that her students are inattentive because they dislike lectures, but she may be misinterpreting the information. The scientist may conclude that human sacrifice is or was a means of trying to control nature, but this also may be incorrect.
In many studies, there are several possible explanations for a problem or phenomenon. These are called hypotheses and may occur at any stage of an investigation. Some researchers state a hypothesis (e.g., “Students are less attentive during lectures than during discussions”) right at the beginning of a study. In other cases, hypotheses emerge as a study progresses, sometimes even when the information that has been collected is being analyzed and interpreted. The scientist might find that instances of sacrifi ce seemed to be more common after such societies made contact with other cultures, suggesting a hypothesis such as: “Sacrifice is more likely when traditional practices are threatened.”
We want to stress two crucial features of scientific research: freedom of thought and public procedures. At every step, it is crucial that the researcher be as open as humanly possible to alternative ways of focusing and clarifying the problem, collecting and analyzing information, and interpreting results. Further, the process must be as public as possible. It is not a private game to be played by a group of insiders. The value of scientific research is that it can be replicated (i.e., repeated) by anyone interested in doing so.
The general order of the scientifi c method, then, is as follows:
Identifying a problem or question
Clarifying the problem
Determining the information needed and how to obtain it
Organizing the information
Interpreting the results
In short, the essence of all research originates in curiosity—a desire to fi nd out how and why things happen, including why people do the things they do, as well as whether or not certain ways of doing things work better than others.
A common misperception of science fosters the idea that there are fi xed, once-and-for-all answers to particular questions. This contributes to a common, but unfortunate, tendency to accept, and rigidly adhere to, oversimplified solutions to very complex problems. While certainty is appealing, it is contradictory to a fundamental premise of science: All conclusions are to be viewed as tentative and subject to change, should new ideas and new evidence warrant revision. It is particularly important for educational researchers to keep this in mind, since the demand for final answers from parents, administrators, teachers, and politicians can often be intense.
For many years, there has been a strong tendency in Western culture to value scientific information over all other kinds. In recent years, the limitations of this view have become increasingly recognized and discussed. In education, we would argue that other ways of knowing, in addition to the scientific, should at least be considered. As we have seen, there are many ways to collect information about the world around us.
How can educators, parents, and students obtain the information they need? Many ways of obtaining information, of course, exist. One can consult experts, review books and articles, question or observe colleagues with relevant experience, examine one’s own past experience, or even rely on intuition. All these approaches suggest possible ways to proceed, but the answers they provide are not always reliable. Experts may be mistaken; source documents may contain no insights of value; colleagues may have no experience in the matter; and one’s own experience or intuition may be irrelevant or misunderstood.
This is why a knowledge of scientific research methodology can be of value. The scientific method provides us with another way of obtaining information—information that is as accurate and reliable as we can get. Let us compare it, therefore, with some of the other ways of knowing.
WAYS OF KNOWING
SENSORY EXPERIENCE
We see, we hear, we smell, we taste, we touch. Most of us have seen fi reworks on the Fourth of July, heard the whine of a jet airplane’s engines overhead, smelled a rose, tasted chocolate ice cream, and felt the wetness of a rainy day. The information we take in from the world through our senses is the most immediate way we have of knowing something.
Using sensory experience as a means of obtaining information, the director of the gifted-student program mentioned above, for example, might visit an advanced placement English class to see and hear what happens during a week or two of the semester. Sensory data, to be sure, can be refined. Seeing the temperature
on an outdoor thermometer can refine our knowledge of how cold it is; a top-quality stereo system can help us hear Beethoven’s Fifth Symphony with greater clarity;
similarly, smell, taste, and touch can all be enhanced, and usually need to be. Many experiments in sensory perception have revealed that we are not always wise to trust
our senses too completely. Our senses can (and often do) deceive us: The gunshot we hear becomes a car backfiring; the water we see in the road ahead is but a mirage; the chicken we thought we tasted turns out to be rabbit.
Sensory knowledge is undependable; it is also incomplete. The data we take in through our senses do not account for all (or even most) of what we seem to feel is the
range of human knowing. To obtain reliable knowledge, therefore, we cannot rely on our senses alone but must check what we think we know with other sources.
AGREEMENT WITH OTHERS
One such source is the opinions of others. Not only can we share our sensations with others, we can also check on the accuracy and authenticity of these sensations: Does this soup taste salty to you? Isn’t that John over there? Did you hear someone cry for help? Smells like mustard, doesn’t it? Obviously, there is a great advantage to checking with others about whether they see or hear what we do. It can help us discard what is untrue and manage our lives more intelligently by focusing on what is true. If, while hiking in the country, I do not hear the sound of an approaching automobile but several of my companions do and alert me to it, I can proceed with caution.
All of us frequently discount our own sensations when others report that we are missing something or “seeing” things incorrectly. Using agreement with others as a means of obtaining information, the tenth-grade biology teacher in Atlanta, for example, might check with her colleagues to see if they fi nd discussions more effective than lectures in motivating their students to learn. The problem with such common knowledge is that it, too, can be wrong. A majority vote of a committee is no guarantee of the truth. My friends might be wrong about the presence of an approaching automobile, or the automobile they hear may be moving away from rather than toward us. Two groups of eyewitnesses to an accident may disagree as to which driver was at fault. Hence, we need to consider some additional ways to obtain reliable knowledge.
EXPERT OPINION
Perhaps there are particular individuals we should consult—experts in their fi eld, people who know a great deal about what we are interested in fi nding out. We are likely to believe a noted heart specialist, for example, if he says that Uncle Charlie has a bad heart. Surely, a person with a PhD in economics knows more than most of us do about what makes the economy tick. And shouldn’t we believe our family dentist if she tells us that back molar has to be pulled? To use expert opinion as a means of obtaining information, perhaps the physical education teacher in Tulsa should ask a noted authority in the physical education fi eld whether ability in one sport correlates with ability in another.
Well, maybe. It depends on the credentials of the experts and the nature of the question about which they are being consulted. Experts, like all of us, can be mistaken. For all their study and training, what experts know is still based primarily on what they have learned from reading and thinking, from listening to and observing others, and from their own experience. No expert, however, has studied or experienced all there is to know in a given fi eld, and thus even an expert can never be totally sure. All any expert can do is give us an opinion based on what he or she knows, and no matter how much this is, it is never all there is to know.
Let us consider, then, another way of knowing: logic.
LOGIC
We also know things logically. Our intellect—our capability to reason things out—allows us to use sensory data to develop a new kind of knowledge. Consider the famous syllogism:
All human beings are mortal.
Sally is a human being.
Therefore, Sally is mortal.
To assert the fi rst statement (called the major premise), we need only generalize from our experience about the mortality of individuals. We have never experienced anyone who was not mortal, so we state that all human beings are. The second statement (called the minor premise) is based entirely on sensory experience. We come in contact with Sally and classify her as a human being. We don’t have to rely on our senses, then, to know that the third statement (called the conclusion) must be true. Logic tells us it is. As long as the first two statements are true, the third statement must be true.
Take the case of the counselor in Philadelphia who is asked to advise a student on how to improve her study habits. Using logic, she might present the following argument:
Students who take notes on a regular basis in class find that their grades improve.
If you take notes on a regular basis, then your grades should improve as well.
This is not all there is to logical reasoning, of course, but it is enough to give you an idea of another way of knowing. There is a fundamental danger in logical reasoning, however: It is only when the major and minor premises of a syllogism are both true that the conclusion is guaranteed to be true. If either of the premises is false, the conclusion may or may not be true.
There is still another way of knowing to consider: the method of science.
THE SCIENTIFIC METHOD
When many people hear the word science, they think of things like white lab coats, laboratories, test tubes, or space exploration. Scientists are people who know a lot, and the term science suggests a tremendous body of knowledge. What we are interested in here, however, is science as a method of knowing.
It is the scientific method that is important to researchers.
What is this method? Essentially it involves testing ideas in the public arena. Almost all of us humans are capable of making connections—of seeing relationships and associations—among the sensory information we experience. Most of us then identify these connections as “facts”—items of knowledge about the world in which we live. We may speculate, for example, that our students may be less attentive in class when we lecture than when we engage them in discussion. A physician may guess that people who sleep between six and eight hours each night will be less anxious than those who sleep more or less than that amount. A counselor may feel that students read less than they used to because they spend most of their free time watching television.
But in each of these cases, we do not really know if our belief is true. What we are dealing with are only guesses or hunches, or as scientists would say, hypotheses. What we must do now is put each of these guesses or hunches to a rigorous test to see if it holds up under more controlled conditions. To investigate our speculation on attentiveness scientifi cally, we can observe carefully and systematically how attentive our students are when we lecture and when we hold a class discussion. The physician can count the number of hours individuals sleep, then measure and compare their anxiety levels. The counselor can compare the reading habits of students who watch different amounts of television.
Such investigations, however, do not constitute science unless they are made public. This means that all aspects of the investigation are described in sufficient detail so that the study can be repeated by anyone who questions the results—provided, of course, that those interested possess the necessary competence and resources. Private procedures, speculations, and conclusions are not scientific until they are made public. There is nothing very mysterious, then, about how scientists work in their quest for reliable knowledge.
In reality, many of us proceed this way when we try to reach an intelligent decision about a problem that is bothering us. These procedures can be boiled down to five distinct steps.
1. First, there is a problem of some sort—some disturbance in our lives that disrupts the normal or desirable state of affairs. Something is bothering us. For most of us who are not scientists, it may be a tension of some sort, a disruption in our normal routine. Examples would be if our students are not as attentive as we wish or if we have diffi culty making friends. To the professional scientist, it may be an unexplained discrepancy in one’s fi eld of knowledge, a gap to be closed. Or it could be that we want to understand the practice of human sacrifi ce in terms of its historical significance.
2. Second, steps are taken to defi ne more precisely the problem or the questions to be answered, to become clearer about exactly what the purpose of the study is. For example, we must think through what we mean by student attentiveness and why we consider it insufficient; the scientist must clarify what is meant by human sacrifice (e.g., how does it differ from murder?).
3. Third, we attempt to determine what kinds of information would solve the problem. Generally speaking, there are two possibilities: study what is already known or carry out a piece of research. As you will see, the fi rst is a prerequisite for the second; the second is a major focus of this text. In preparation, we must be familiar with a wide range of possibilities for obtaining information, so as to get firsthand information on the problem. For example, the teacher might consider giving a questionnaire to students or having someone observe during class. The scientist might decide to examine historical accounts or spend time in societies where the practice of human sacrifice exists (or has until recently). Spelling out the details of information gathering is a major aspect of planning a research study.
4. Fourth, we must decide, as far as it is possible, how we will organize the information that we obtain. It is not uncommon, in both daily life and research, to discover that we cannot make sense of all the information we possess (sometimes referred to as information overload). Anyone attempting to understand another society while living in it has probably experienced this phenomenon. Our scientist will surely encounter this problem, but so will our teacher unless she has fi gured out how to handle the questionnaire and/or observational information that is obtained.
5. Fifth, after the information has been collected and analyzed, it must be interpreted. While this step may seem straightforward at fi rst, this is seldom the case. As you will see, one of the most important parts of research is to avoid kidding ourselves. The teacher may conclude that her students are inattentive because they dislike lectures, but she may be misinterpreting the information. The scientist may conclude that human sacrifice is or was a means of trying to control nature, but this also may be incorrect.
In many studies, there are several possible explanations for a problem or phenomenon. These are called hypotheses and may occur at any stage of an investigation. Some researchers state a hypothesis (e.g., “Students are less attentive during lectures than during discussions”) right at the beginning of a study. In other cases, hypotheses emerge as a study progresses, sometimes even when the information that has been collected is being analyzed and interpreted. The scientist might find that instances of sacrifi ce seemed to be more common after such societies made contact with other cultures, suggesting a hypothesis such as: “Sacrifice is more likely when traditional practices are threatened.”
We want to stress two crucial features of scientific research: freedom of thought and public procedures. At every step, it is crucial that the researcher be as open as humanly possible to alternative ways of focusing and clarifying the problem, collecting and analyzing information, and interpreting results. Further, the process must be as public as possible. It is not a private game to be played by a group of insiders. The value of scientific research is that it can be replicated (i.e., repeated) by anyone interested in doing so.
The general order of the scientifi c method, then, is as follows:
Identifying a problem or question
Clarifying the problem
Determining the information needed and how to obtain it
Organizing the information
Interpreting the results
In short, the essence of all research originates in curiosity—a desire to fi nd out how and why things happen, including why people do the things they do, as well as whether or not certain ways of doing things work better than others.
A common misperception of science fosters the idea that there are fi xed, once-and-for-all answers to particular questions. This contributes to a common, but unfortunate, tendency to accept, and rigidly adhere to, oversimplified solutions to very complex problems. While certainty is appealing, it is contradictory to a fundamental premise of science: All conclusions are to be viewed as tentative and subject to change, should new ideas and new evidence warrant revision. It is particularly important for educational researchers to keep this in mind, since the demand for final answers from parents, administrators, teachers, and politicians can often be intense.
For many years, there has been a strong tendency in Western culture to value scientific information over all other kinds. In recent years, the limitations of this view have become increasingly recognized and discussed. In education, we would argue that other ways of knowing, in addition to the scientific, should at least be considered. As we have seen, there are many ways to collect information about the world around us.