EDF600 – Research Methods

Lola Aagaard’s Notes

Chapter 1 

I. How do you know what you know?  Or understand what you understand? 

            A. Authoritative declaration / opinion of experts – you accepted what someone else told you or what you read or heard somewhere.  You didn’t check into the details, but just accepted the information as fact.  This can be trouble, depending on your source

                        1. It is against the rules to chew gum at Rodburn Elementary School.  How do I know?  Somebody told me so.

2. One of the major exports of Costa Rica is bananas.  How do I know?  I read it somewhere.

3. Instead of a human baby, a pregnant woman recently was delivered of an alien child!  How do I know?  I saw it on the cover of the Enquirer! 

            B. Experience / insight – you come up with an understanding of something based solely on your previous experience.

                        1. The very qualities that made me fall in love with my husband are the ones that drive me crazy after 20 + years of marriage.  How do I know?  It was an insight that came to me several years ago when I was analyzing an argument we had!

                        2. For a teacher to be lenient with deadlines for some students is worse for them than being strict about turning in work.  How do I know?  From four years of looking at the grades of my undergrads – some kids take advantage of every soft spot I’ve got, then turn about half of the work (terribly done!) all in a rush at the end of the semester.  I didn’t do them any favors by letting them get away with that. 

            C. Divine inspiration / supernatural revelation – the source of much of our religious knowledge and understanding around the world.  “It came to me in a dream...”  “I heard a voice in my head, saying...”  Again, knowing the actual source is crucial! 

            D. Research – purposely doing something to find the answers to questions or to clarify issues.  There are many varieties (I classify them a bit differently than your text-book does): 

                        1. Library-type research – looking up stuff in books, magazines, now the Internet.  This type is most-often used in the typical “research paper” of school infamy. 

                        2. Investigative journalism-type research – to write terrific descriptive articles, journalists do a lot of research, both in the library and prowling around talking to people and digging up documents, etc.

                                    Why did that last superintendent resign so unexpectedly?  Digging around through budgets or other documents and talking to various people may bring that information to light.

                         3. Evaluation – you want to know if a specific program or treatment or “something” is working or not. 

The science teachers district-wide started using a new hands-on science kit – has it made a difference in kids’ science test scores? 

                        4. Action research – you are facing a practical problem in your job setting so you do some library research and collect some data locally to see if you can figure out a way to solve the problem. 

The parents you most need to see are not coming to parent-teacher conferences or open house night – why not?  You look through some articles and decide to survey parents about some particular issues that seemed to be important in the literature you read (but you survey them at a ball game, or somewhere that there will be a lot of parents present!).  After you identify some of the issues that are keeping them away, you implement some changes in the hopes of getting more to attend. 

5. Academic research – this category of research is differentiated by the primary role of theory.  You collect, analyze, and interpret data and then connect the results to theory of some sort.  The purpose of theory is to describe and explain phenomena.  In academic research you move beyond the local setting and up into the realm of theory.

If in looking into why the superintendent resigned, you find that he had been promoted up through the ranks of the organization, you might look at literature (library research) that deals with theories regarding the rather predictable consequences of promoting from within and see if they tie in with what you found in your nosing around.  If you relate your specific situation to some sort of theory, the piece of work becomes academic research.

If with the parent attendance study you linked the issues that were important to your students’ parents to existing theory and results from other studies in the same general area, you would be moving it from the realm of action research to academic research.

In some academic research, there is no concern for applying the results to any other setting, but they will still develop a theory from or apply theory to what was found in the study.

Research can be used to develop, support, or refute theories (they are NEVER proven or disproven, so don’t use those terms, please).           

Specific types:

            a. survey

            b. correlational

            c. causal comparative

            d. quasi-experimental

            e. experimental

            f. historical

            g. ethnographic

            h. phenomenological

            i. (grounded theory)                    

 II. Types of academic research – academic research comes in many varieties.  Which type you choose to do depends mostly on your philosophical beliefs!  That will determine what questions are of interest to you, what you will accept as data, and how you will approach the whole endeavor.

             A. Ontology – your view of the world

                        How do you see the world?  Is it something like a giant jigsaw puzzle waiting to be solved?  (Pieces are waiting to be discovered and when we find them [through tedious analysis] we’ll be a little closer to seeing the picture on the box-top.)

Or is the world more like a riddle?  (Riddles may have more than one answer that makes sense, although one might be the most well-known.)  

1. Jigsaw puzzle / truth-seeking -- This is the traditional view of natural scientists – they are interested in finding and describing the fundamental elements and how they interact together to make the world run the way it does.  Researchers in many other disciplines, however, have adopted this view.  All of them are after “the truth” and their research styles are generally very strict and controlled (think white lab coats), because better procedure gives them more of a chance at discovering truth.

                        This is basically a deductive approach – their theorizing starts with the general (their idea of the picture on the boxtop) and gets applied to the particular (if the picture is of a flower garden, then this piece would be part of the pathway).  Pieces that don’t fit the overall picture are useless (outliers, nonsignificant).  Sometimes their idea of what the overall picture is changes as a result of research – if enough pieces that they try don’t seem to fit, then maybe the picture is NOT of a flower garden but of a downtown city scene.  But they still hold only one view at a time – they don’t deal with multiple views of the world.

                        If a truth-seeking researcher finds that his/her study supports an existing theory, he/she will want to “generalize” the results of that individual study to a larger group of people than were in the sample – to be able to say, “What I found is true of all groups like the one I studied.”  This is saying, essentially, “I found a puzzle piece and I know where it fits in the big picture.”  It is the desire to generalize that is responsible for the strict rules associated with some types of academic research – they have to be careful about uncontrolled influences that might give them false results.

                        Example of this type of research in education (from one of the articles at the back of the chapter):  If motivation of students is important to their successful achievement, then if I motivate group A and not Group B in a particular way, I should see differences in their test scores.

                        Other terms for this type of philosophy that you might run across include:  conventional, objective, rationalistic, positivist, logical-positivist, and convergent. 

                        2. Riddle / perspective-seeking – On the other hand, there are people who view the world more as a riddle to be solved.  What is the answer to this one?

                        “Why did the chicken cross the road?”  I know three different answers to that:

a. the time-honored traditional answer:  “to get to the other side”

b. a regional variation (from Oklahoma): “to show the armadillo it could be done.” 

c. from a particular time and place (L.A. in the late 1980s): “it was running from a sniper.”  (There was a rash of freeway sniper incidents during that time in southern CA.)

Which answer is “the truth?”  There isn’t really one answer that you can say is “the right one,” because each is valid for it’s own time and place – it depends on the perspective of the people telling the riddle.   Researchers of this philosophy are comfortable with multiple views of the world – they are not after a single “truth.” 

Think of movie reviewing – one reviewer gives Lord of the Rings a thumbs up and one gives it a thumbs down.  Which is right?  Is the film good or bad?  There isn’t any overall “truth” to discover here – it is all a matter of the perspective of the person watching the film.

This is much more of an inductive process – the researcher starts with specific data and very broad initial questions (“What’s going on here?”) and develops theory or relates it to theory AFTER the data collection.  There are not the strict controls on study design in research done under this philosophy because perspective-seekers are not so interested in generalizing their results to people outside the study.  (Although they sometimes imply how their results could be applied outside their study setting – see Alan Peshkin’s books.)

There are, however, guidelines to follow to be sure that you actually understand the perspectives of the people in your study.  Are they telling you the truth?  Have you pushed your ideas onto them?  Do you really understand the significance of something they said or did?  Whether or not what they found in this study applies in any other setting is left up to the readers of the report.

Other labels for this philosophy include constructive, subjective, naturalistic, and divergent. 

You may see the world one way for some questions and another way for others – that’s perfectly natural.  Few people are one way all the time, plus there are in-between positions! 

            B. Epistemology (the nature of knowledge, or...what you accept as data) – there are two basic types of data – quantitative and qualitative. 

                        1. Quantitative – numerical data – counts of things (how many parents are in favor of having students wear uniforms to school?), measurements of things (ability, attitude, anxiety, and other things that don’t start with A).

                        Truth-seeking researchers often use quantitative data.  When used under that philosophy, the quantitative data are generally analyzed using statistical techniques – you compare averages (in order to limit the complexity of the data) rather than dealing with individual scores.  And generally you estimate the probability of having made a mistake in your conclusions and only report the study results if that probability is very low.              Numerical data are generally displayed in tables in the report, with some minimal written description of what to look for in the table.  The types of studies that typically use quantitative data include survey, correlational, causal-comparative, and experimental. 

                        Perspective-seeking researchers don’t use quantitative data as often, but it does happen.  Generally they don’t analyze them statistically, but might put in tables or graphs, using the numbers to describe some aspect of their study.           

                        2. Qualitative – word data – documents, interviews, notes on observations.  These are complex sources and perspective-seeking researchers revel in the complexity!  They will simplify it to the extent of finding similarities (categories) across individuals, and they may count how many individuals talked about that category, and then relate the categories to theory, but they keep the complexity, as well.  Large sections of quoted material from interviews will be included in the reports of qualitative research, to give the reader a good sense of what kinds of comments were classified in the various categories.  Typical types of research that use qualitative data are historical, ethnographic, and phenomenological.

                        Truth-seeking researchers dealing with qualitative data often try to reduce it to numbers that they can then use in statistical procedures.  (I took a stat class titled, “Qualitative Data Analysis.”)  

                        Quant./Qual. Example:  Diary entries of high school students. 

                        To convert these qualitative data to quantitative, you could skim through several diaries, counting the number of times particular topics were mentioned (teachers, the opposite sex, class content, personal embarrassment, etc.).  That would tell you which topics were written about the most:

                                        Topic               Average mentions

                                    Opposite sex               5 times per page

                                    Teachers                     1 time every 2 pages

                                    Class content             1 time every 5 pages 

            But this doesn’t tell you WHAT the teens said about their classes, the opposite sex, or their teachers.  For that you would have to read all of the words in the diaries and use the words as data, giving examples of actual comments as illustrations of more detailed categories.                    

                        3. When would you collect which type of data?  That depends on three things:

                                    a. what your research questions are – do you have questions

about things that could be measured and simplified with statistics?  Or are the questions that are interesting to you not going to be answered by numbers?

b. whether you wish to generalize (and whom you are trying to convince) – generalization is only well-accepted when it is based on experimental studies, or at least randomly selected participants.

c. how much time you have to compete the study – qualitative takes a LOT more time to collect, analyze, and write up.

                       There are also multi-method studies that combine qualitative and quantitative for a “best of both worlds” approach.

             C. Axiology (or specifically, ideology) – whether politics is an overt part of your research.  Some researchers just accept the status quo, and do research as if there is nothing influential or changeable in the socio-political situation within which they are working.  Others are overtly challenging, questioning, and wishing to reform the existing socio-political system in place where they are doing their research.

                        Example:  adult illiteracy.  Why is there so much illiteracy in Eastern KY?  Suppose two researchers look at the data and attempt to find underlying causes for the illiteracy among adults in the region.  What they found was that some of the illiterate are undiagnosed LD, others are unmotivated underachievers, others had dysfunctional families who didn’t keep them in school in the early years, and some were combinations of these things.  So far no political agenda is apparent – it shows up in the interpretation (and sometimes the original research questions).

                        Researcher A says the data mean that schools need better diagnostic procedures for learning disabilities and underachievement, increased emphasis on motivation for underachievers, and perhaps counseling intervention for dysfunctional families. (Status quo)

                        Researcher B says that illiterate adults are part of this country’s underclass, and that the failure of the schools to recognize learning disabilities or to provide proper motivation is simply part of the deliberate plan by the ruling upper class for maintaining their elite position in society.  The solution to illiteracy, thus, is empowerment of the lower class so that they can recognize the domination of the elite over all others in society and can unite against it! (Reform)

                        Both researchers collected and analyzed the same data in the same way and their world views were the same.  What differed were their ideologies.  There are a number of reform-minded strands of theory and research in education:
                        critical theory, radical theory, feminist research, gay/lesbian research, social class research (some), and research regarding various ethnicities (some).  All believe that the social order needs to be changed in some fundamental ways and this belief is reflected in their research.  The role of the political system in whatever situation they are studying is often included as part of the analysis. Sometimes ideology dictates methods, as some neo-Marxists view quantitative methods as a tool of the ruling class they refuse to use them.

            KERA had some real political reform qualities!  Think about the anti-nepotism provisions, school governance changes (SBDM), and “all children can learn” – these were aimed at changing who was employed at schools, who had a say in how they were run, and who got more attention in the classroom.  Those are all elements of the social order.