EDF 600 – Research Methods
Lola Aagaard’s Notes on Chapter 2
Selecting and Defining a Research Topic
I. Selecting a topic -- For many students beginning
the research process, the most difficult issue is the selection of a research
topic. The entire process
sometimes goes like this for truth-seeking quantitative research:
a. Determining general topic/problem/question
b.
Small literature review, talking to people
c.
Determining specific topic/problem/question
d.
More extensive literature review
e.
Determining hypothesis
f.
Collection of data
g.
Analysis of data
h.
Interpretation of results, conclusions
i.
Connection back to theory, generalizations
(In perspective-seeking, qualitative research, some of the
collection and analysis of data may come before a final problem statement, and
the review of the literature is often left until last.)
Those first 3 steps are the most important in some ways,
because it’s the same as with computers, garbage in—garbage out.
If you aren’t really sure what you’re doing when you begin the
research project, then you won’t know what to say when you’re finished with
it. You need to have quality
questions.
A. What is a quality
topic/problem/question?
1. One that is
of interest to you – very important!
2. One that you
can actually research – can it be answered by the collection and analysis of
data? And do you have the time and
resources (money, expertise, etc.) to do the necessary research?
3. Does it have
theoretical or practical significance? This
is often called the “So what?” aspect of a research problem.
You have to be able to explain why it is important to find the answer to
the question. Might it support or
contradict a particular theory? Or
does it, as in action research, have practical significance – will it help
solve problems you are having or to help you improve the quality of your work?
B. Reconstructed logic
vs. Logic in use --
Having said all of that, I should
tell that you that often research is not done in the order I’ve given up
above. Many times someone comes up
with a question, gathers and analyzes the data, then goes back to fit it into a
theory that they’ve found from looking at the literature after the fact.
Other times a researcher might stumble across a goldmine of data – KDE
has tons of it, for instance – and then develop a study based on those data.
But when they write it up, they put it into the “accepted” order,
with the literature review right up front, even if it was done last. This is the difference between what someone called
“reconstructed logic” and “logic in use.”
How you actually get things done is “logic in use” and it can be
extremely messy – everything may have gone wrong and you made a multitude of
goofs and then had to somehow salvage the study from disaster.
But when you write it up for a conference or journal or you thesis, you
use “reconstructed logic” and put it all down in the right order as though
everything had gone perfectly! This
is not considered unethical, by the way – it is standard practice.
C. Where do you get ideas for a
research topic/problem/question?
1. Read the
literature –
a. You
can find out what questions have already been answered, so you don’t waste
your time going over old ground –
if you’re interested in looking at whether peer groups are more of a factor in
the lives of adolescents or preschoolers, that question has already been
exhaustively addressed in previous research.
b. You
can find out what still needs to be done ---
i. Suggestions for future
research are often included at the end of articles or conference presentation
papers. A study of whether
assigning homework is associated with higher student test performance might have
only looked at the hours of homework or number of assignments.
The authors suggested that future researchers look at the type of
assignment rather than just how many students did or how long it took them.
ii. Replication of an
existing study (repeating the exact same study, but with a different type of
subject or in a different setting) is often very useful.
If the published studies all deal with high school students’ learning
of social studies, you can repeat the study with middle school or upper
elementary students. Or you can
switch it to GED students who aren’t in a regular high school, or do it with
rural students if the ones in the journal article were urban.
Or use a subpopulation – the study dealt with urban high schools
students, but your interest is in minority girls, so replicate the study using
just minority female urban high school students.
iii. Apply the theory,
perspective, explanation to a new situation – you can take their theory of how
students best learn social studies and see if it works for science instruction
or for adults taking professional development classes.
iv. If you find studies
on the same topic that disagree with each other, you can do a third study and
see which side your results support.
v. If you read a study
that does not ring true with your experience, then you can conduct your own and
see how it comes out.
(Of course, you get your theory connections
from the literature, but you also find out how other people conducted
their research, what measures or surveys they used – these are great
sources for ideas for you stuff.)
2. Attend professional conferences to
hear what the current fads and theories are.
Find out what the state, regional, and national organizations are for
your field. And not just the
general teachers’ or administrators’ or counselors’ association, but
the subject-specific associations, as well – science, reading, math,
social studies, etc. Most have
great practical sessions, as well as some research / theoretical sessions.
3. Ask an expert – either at a
conference or over e-mail or one of your content-area professors (not me!).
Someone who has been reading the literature regularly will be able to
give you an idea of what the top questions are.
4. Ask an administrator in your school /
district. Most principals
and central office staff are full of questions to which they would like to
know the answers, but they don’t have time to collect the data.
Often the questions deal with parent attitudes, but sometimes they
are about teacher attitudes or practices or student response to particular
things. SBDM councils might be
another source of ideas.
5. Use your own experience and that of
your colleagues – you must have curiosities about things that go on in the
classroom or your workplace. How
would students respond to an integrated social studies/science project on
which they collaborate with other children around the world via e-mail? (There are many such projects out there, by the way.
One was called the Green River project – students all over the
world who lived anywhere near any community whose name translated to
something like Green River (Rio Verde) sent each other regular newsletters
about what was happening in their community – what the weather was like,
flora and fauna, customs, school work, etc.)
Would kids who have a practice test at the beginning of the week in
spelling do better on the Friday test than kids who did NOT have a practice
test? Are the children in your
classroom who have the most trouble understanding long division (or literary
metaphor) at a different cognitive stage than the ones who are flying
through it?
II.
Stating the Research Topic/Problem/Question
A. Some necessary terms
1. Variables – things that vary (differ) from one person to
another
a.
Independent variable – grouping variable, generally not measured but
already existing or assigned (gender, treatment/control group, age group)
b. Dependent
variable – measured variable, value “depends” on the independent
variable. The outcome variable:
Achievement, attitude, anxiety, etc.
c.
Confounding variables – other things that aren’t being studied in your
research project but may influence the outcome and goof up your results.
(If you’re studying the outcome of a new approach to a science unit on
dinosaurs and in the middle of the study a special is broadcast on TV and some
of your students watch it and others don’t – some of their knowledge at the
end of the unit may have come from the TV special, not your unit, and if you
don’t take that into account in your analysis it will “confound” your
results.)
2. Operational definition
– statement of how you are defining your variables. If one variable is political affiliation, how are you
determining that? It makes a big
difference! By looking at voter
registrations or how they actually voted in the last presidential election?
Nearly all of OK registered Democrat, but in national elections they
voted Republican. My husband,
although he votes Libertarian, registers Democrat.
I vote Libertarian and register Independent.
My mother votes Libertarian and registers Republican.
How many categories are you allowing?
Repub, Dem, other? Repub,
Dem, Independent, other? Repub,
Dem, Libertarian, Socialist, Communist, Green Party, Garden Party?
How are you measuring anxiety? Hooking
up a meter to check for sweaty palms and racing heartbeat? Having a counselor interview the person and give a rating?
Having the person fill out a questionnaire?
What about achievement? On a
teacher-made test? The CATS?
The CTBS? Your decision
about this should be made clear in your operational definitions.
B. Stating the Topic/Problem --
A well-written topic/problem statement indicates:
1. the variables
of interest to the researcher and
2. the
relationship between them, as well as
3. the type of
people in the sample.
For instance:
“The purpose of this study is to investigate the effect of positive
reinforcement on the quality of English compositions written by high school
seniors.”
What are the variables of concern in that
problem statement? Which is
independent and which dependent? How
would you supply one and measure the other?
Topic/Problem statements start you on the process
of operationally defining you variables and thinking about method – how are
you going to measure things? How
will you get your groups?
The research question or problem should be presented
along with enough background to provide justification for the study and it
potential significance (answer the “So what?” question).
For example (from your textbook):
“The purpose of this study is to compare the
effectiveness of salaried paraprofessionals and parent volunteers with
respect to assisting the reading achievement of first grade children.”
This statement might be preceded or followed by a
discussion concerning the following things:
1.
the role of paraprofessionals;
2. increased use of paraprofessionals by schools;
3. the expense
involved; and
4 the
search for alternatives, such as parent volunteers (or senior citizens).
(And even though that example is practical in the extreme,
there is plenty of theory out there about the benefit of having parents present
in the school and classroom – the results could be set into that theoretical
context very easily.)
C. Where does the topic/problem statement come in the
research paper? Sometimes it is right up front, after a few
introductory paragraphs. Other times it comes after the literature review.
In your papers for class, I’d like it after the introductory
paragraphs (where a few sources might be cited, but it isn’t your main
literature review). (But I can be
persuaded otherwise, if in writing it up you feel it flows better to have it
after the lit. revew!)
III. Hypotheses – A research hypothesis is a
follow-up to your problem statement. In
the research hypothesis you indicate how you think the results of your research
will come out – you predict the outcome of your study.
(Hypotheses are for quantitative truth-seeking studies – they aren’t
generally used at all in perspective-seeking studies.)
There are several different kinds of hypotheses.
A. Types of hypotheses
1.
Non-directional research – Non-directional research hypotheses
simply state that there will be a difference (or relationship) of some kind.
No predictions about what kind of difference or relationship are made.
If you are not sure which way your results
will come out and there isn’t enough theory in the area to make a good
prediction, you can state your research hypothesis non-directionally.
For instance, for the
paraprofessional vs. parent volunteer study above, your non-directional research
hypothesis might be something like:
“There is a significant difference between the reading achievement
scores of 1st grade
children assisted by classroom paraprofessionals and 1st grade
children assisted by parent
volunteers.”
You haven’t said in which
direction the difference will be; you have made no prediction about which group
of children will get better reading scores.
You have just stated that you believe there will be a difference of some
kind.
2. Directional
research – a directional hypothesis specifies in which direction the
difference will be, or precisely what kind of relationship will be found.
If you have a good
reason (from your lit review, say) for believing that parent volunteers will be
better or worse than paraprofessionals in assisting children to learn to read,
then you could say so in a directional hypothesis:
“First grade children assisted by parent volunteers have higher reading
achievement scores
than 1st grade children assisted by classroom paraprofessionals.”
3. Statistical
or Null -- A statistical (or null) hypothesis always states that there will
be no difference – that is what you are testing when you run statistics.
“There will be no difference between the reading achievement scores of 1st
grade children
assisted by classroom paraprofessionals and 1st grade children
assisted by parent
volunteers.”
If you find a
significant difference with your statistics, then you have failed to confirm the
null and you have confirmed (or supported) the alternative, which is your
research hypothesis (that there was a difference of some kind).
(If your
research hypothesis was directional rather than non-directional than you
interpret your statistical tests somewhat differently.
We won’t get into that now except to say that if you are predicting the
direction of your results, you increase your chances of finding them
statistically significant, assuming you were correct on the direction!)
(If you are
actually predicting that there will be no difference or no relationship among
the variables in your study, then you have an even more interesting statistical
problem. It can be done, but it is
unusual.)
B. Criteria for good hypotheses – a good hypothesis must:
1. State:
a. who the participants in the study are;
b. the independent variable (or one correlational variable)
c. the dependent variable (or the other correlational variable)
2. Have a basis in sound rationale – it must be reasonable.
3. Be testable, meaning the
collection and analysis of data will support
them or fail to support them
Look at this:
"High school
students who participate in peer counseling have less absenteeism than high
school students who receive individual counseling."
Identify the participants, the independent variable, and
the dependent variable. Is it
directional or non-directional? What
would be the null form?